Zuckerberg and Fridman Showcase a Major VR Communication Breakthrough

Podcast host Lex Fridman, interviewing Mark Zuckerberg in VR, expressed his astonishment, stating that it was the most incredible sight he had ever witnessed. They were using Meta’s upcoming photorealistic avatars, set to launch on Quest, which are so realistic that they transcend the uncanny valley.

Meta’s New VR Avatars: Bridging the Gap to Immersive Virtual Interactions

Mark Zuckerberg has faced considerable online criticism for the less-than-impressive appearance of Meta’s previous VR avatars. However, these latest avatars seem to represent an enormous leap forward, offering a glimpse into a future where VR interactions and meetings will truly provide the sensation of being in the same room as others.

Here’s the current process: The photorealistic avatars are currently part of a research initiative that involved both Zuck and Fridman spending hours in a studio. During this time, their facial expressions were captured using various cameras, which were then used to create avatar heads that can be controlled based on their motions.

In this instance, they used Meta Quest Pro VR headsets, but the technology will perform equally effectively on the more affordable Meta Quest 3. Several built-in cameras on the headset monitor your facial expressions, with a focus on tracking eye and mouth movements.

The data from these movements, combined with audio captured by the headset’s microphones, forms a compact data stream that can be transmitted over long distances almost instantly. This data is then used to animate an avatar, which is already stored on the conversation partner’s headset.

Fridman was clearly amazed by the effect and remarked, “The realism here is truly astonishing. I believe this represents the future of how people can connect with one another in a profoundly meaningful way on the internet… It’s fascinating how such intimate conversations can be conducted remotely. I typically avoid remote podcasts for this reason, but this technology changes everything.”

Streamlining Avatar Capture: A Future of Faster and More Efficient Smartphone Scanning

The present method for capturing avatars will soon undergo significant simplification, requiring only 2-3 minutes on a smartphone. Zuckerberg mentioned, “Currently, we might be collecting more facial expressions than necessary during the scanning process, as we are still fine-tuning how to streamline it and extrapolate from the existing scans.”

He continued, “Our objective, and we already have a project in progress for this, is to conduct a swift scan where you use your mobile phone to move it in front of your face for a few minutes, utter a few sentences, and make various facial expressions.”

Demonstrating some of the most advanced technology for capturing and transmitting facial expressions during an interview involving two individuals known for their reserved demeanor was an intriguing choice, and Fridman acknowledged the irony in this.

Challenging Monotone Perceptions: Embracing Nuanced Expressions in the Metaverse

He remarked, “The meme will probably be that the two most monotone people are in the metaverse together. But I actually think that adds to the challenge. What’s truly impressive here are the nuances in expressions and the role of the eyes. People often say I’m monotone and devoid of emotion, but that’s not the case. My expression of emotions tends to be more subtle, often conveyed through my eyes. One thing I’ve observed is how even subtle movements at the corners of the eyes can convey emotions like happiness or boredom.”

While these photorealistic avatars are well-suited for certain intimate face-to-face interactions, they are versatile in their construction. They can be used for various purposes, such as assuming someone else’s appearance, exaggerating or toning down expressions, or even departing from photorealism altogether to adopt different avatars.

This conversation is truly captivating and offers a surprising glimpse into the near future, making it well worth an hour of your time. You can explore it below!

Read the original article on: New Atlas

Read more: 2023 Industrial Design Trends to Watch