Mark Zuckerberg is close to his “Holy Grail”: this will be the technology that promises to change the world

iProUP participated in Inside the labs, a round table with Zuckerberg and his team of researchers to find out what the future of technology will be

The search for “new realities” is already a clear objective for technology companies (and not so much). Expanding the frontiers of business, entertainment and social life is part of his short-term vision and a strategy that will mark the future of all industries.

Virtual reality today is already, forgive the redundancy, a reality. VR devices can be obtained anywhere in the world and offer all kinds of experiences: from recreational and playful within video games, to educational, work and social.

And this is accompanied by a vision of the metaverse that, in a very short time, has become a very recurring word in the business vocabulary, although many still have doubts about how it will be developed and applied in the “real world”.

The person responsible for this last point is mark zuckerberg, founder and CEO of Meta. At the end of 2021, Facebook changed its name and its creator revealed what the future of the company would be: the metaverse development.

Mark Zuckerberg is clear that the future of computing will go through VR.

Almost a year after this announcement, iProUP participated in Inside the labsa panel discussion with Zuckerberg and his team of VR researchers to learn what will be the future of immersive technologies.

After the “Holy Grail” of virtual reality

One of the problems posed by virtual reality is the how we perceive things around us. We have to bring that sense of reality into these virtual worlds and recreate the physical sensations of being with someone, because visual experiences alone are not enough,” Zuckerberg said.

Today’s VR headsets offer amazing 3D visual experiences, but still differ in many ways from what we see in the real world. For example, they have a lower resolution than that offered by notebooks, televisions and phones; furthermore, the lenses distort the wearer’s vision and cannot be worn for long periods.

iProUP shared a Zoom with the founder of Meta and his team to learn about the future of immersive technology

iProUP shared a Zoom with the founder of Meta and his team to learn about the future of immersive technology

These problems were identified by Goal and that is why he has been working for a long time (even many years before changing his name), to achieve it.

The objective? “A lightweight display that’s so advanced it can deliver what our eyes need to function naturally, so they feel like we’re seeing the real world in VR. This is known as the ‘Visual Turing Test’ and passing it is considered the Holy Grail of display research“, explained the CEO of Meta.

Although Zuckerberg assured that “there is still a long way to go to recreate this realism,” he explained that his Reality Labs Research (RL Research) Display Systems Research (DSR) team is working on several technologies to pass the visual Turing test:

  • “Varifocal” technology: ensures correct focus and allows clear and comfortable vision at arm’s length for long periods of time.
  • Distortion Correction: Helps resolve optical errors, such as warping and color fringing, introduced by viewing optics.
  • Resolution that approaches or exceeds human vision 20/20 or 6/6.
  • High Dynamic Range (HDR) technology that expands color gamut, brightness, and contrast in VR.

Additionally, DSR is developing prototypes that explore the form factor of headsets and how to minimize the size, weight, and power units of current VR headsets to make room for these visuals.

Michael AbrashChief Scientist at Reality Labs, commented: “Real-life human experience is key to creating truly immersive VR. For that we have to create increasingly complex equipment, yet compact to use for long periods of time.” “.

All the prototypes with which Meta has been working over the years

All the prototypes with which Meta has been working over the years

The history of evolution

The DSR team has worked to meet these challenges with an extensive series of prototypes. Each of them is designed to push the boundaries of technology and designand undergoes rigorous user studies to assess progress toward passing the visual Turing test.

Douglas LangmanSenior Director of Display Systems Research at Reality Labs Research, commented: “In 2015 we started. Since then, we have explored many different ways to bring better visual experiences. They compete in size, power, price, but they also have to be compact. Every element has to be compatible, not just small“.

In 2017, the team experienced its first major breakthrough with varifocal technology with a research prototype called Half Dome Zero. They used this prototype to conduct a user study, the first of its kind, which validated that the varifocal it would be essential to offer more visual comfort in the future VR.

Tryouts for the Meta team started in 2015

Tryouts for the Meta team started in 2015

Since this pivotal result, the team has continued to apply this same rigorous process of creating prototypes across the DSR portfoliopushing the limits of retinal resolution, distortion, and high dynamic range.

The way our eyes work is different in virtual reality. The objects do not have the same depth and the way of focusing is different. That’s why working with the resolution is not enoughZuckerberg assured.

The four versions of Half Dome

The four versions of Half Dome

Simultaneously, the DSR team developed 3 prototypes to delve into each of the key technologies for VR.

butterscotch

It was designed to demonstrate the experience of the retina resolution in virtual reality, which is the “gold standard” for any product with a display. Devices like televisions and mobile phones have long since surpassed the 60 pixels per degree (ppd) benchmark, allowing them to represent the 20/20 or 6/6 line on an eye chart.

In this regard, VR lags behind because the immersive FOV spreads the available pixels over a larger area, reducing resolution. This limits perceived realism and the ability to present fine text, which is critical to passing the visual Turing test.

butterscotch asd

Butterscotch was designed to demonstrate the retina resolution experience in virtual reality.

To create the experience of a resolution close to retinathe Butterscotch team reduced the field of view to half that of Quest 2 and developed a new hybrid lens that would fully resolve a higher resolution.

However, the resulting prototype does not it was not even close marketable, was heavy and bulkybut it did a great job of showing the difference higher resolution makes to the VR experience

star burst

This is the viewer prototype DSR HDR VR. High Dynamic Range (HDR) is the technology that is most linked to a greater sense of realism and depth.

The key metric to achieve a better HDR is the nits, or the amount of light an object emits. Research has shown that the ideal number for peak brightness is 10,000 nits, and the TV industry has made strides in introducing HDR displays that come close to this figure.

In VR, the maximum nits are 100 in Quest 2. Getting past that while maintaining a usable form factor is a big challenge.

Starbust, another of Meta's prototypes

Starbust, another of Meta’s prototypes

The Starburst prototype is bulky, heavy and wired; that’s why people hold it like binoculars.

But the result produces a full range of luminosity that is often seen indoors or at night. Starburst reaches 20,000 nits, making it one of the brightest HDR displays ever built, and one of the few in 3D, an important step in setting user preferences for realistic light rendering in VR.

holocake 2

Finally, the Holocake 2 is the thinnest and lightest VR headset Meta has ever made.. Based on the original holographic optics prototype, which looked like a pair of sunglasses, but lacked mechanical and electrical components and had significantly lower optical performance, Holocake 2 is a fully functional headset, capable of running any PC VR title. .

To achieve the ultra-compact format, the Holocake 2 team had to significantly reduce the size of the optics and, at the same time, make the most of the space.

Holocake 2 is the company's most compact functional prototype

Holocake 2 is the company’s most compact functional prototype

The solution was two-fold: first, to use polarization-based optical folding (or pancake optics) to reduce the gap between the display panel and the lens; second, reducing the thickness of the lens itself by replacing a conventional curved lens with a thin, flat holographic lens.

The creation of the holographic lens was a novel approach to reducing the form factor that represented a remarkable step forward for VR display systems.

“This is our first attempt at a fully functional viewer that takes advantage of holographic optics and we believe that further miniaturization of the scope is still possible,” said Meta’s founder.

Uniting the technologies

Taking into account all their advances, the team developed a conceptual design that can mark a before and after in the future of virtual reality.

Mirror Lake is a concept design with a form factor similar to that of a ski goggle that will integrate almost all of the advanced visual technologies that DSR has been incubating for the past seven years, including varifocal and eye tracking, in a compact, lightweight, low-cost format. consumption. It demonstrates what a state-of-the-art complete display system could look like.

The concept of Mirror Lake could be a before and after for the development of this type of technology

The concept of Mirror Lake could be a before and after for the development of this type of technology

“One of the reasons this is so exciting to work on is because it is a genuinely new technology. We’re not just refining types of displays that have been around for decades, we’re exploring new territory so that the physical and virtual worlds can coexist.Zuckerberg assured.

The company’s goal is to bring all these technologies together, integrating the visual elements necessary to pass Turing’s visual test in a light, compact and low consumption format; Y Mirror Lake is one of the possible paths to that goal..

Augmented, mixed and virtual reality will be essential technologies, especially to improve our social interactions. And we are making it happen: we are the company most committed to VR and we are working with different laboratories and companies to define what the next 10 years of social experiences and computing will look like.“, concluded the CEO of Meta.

We wish to give thanks to the writer of this short article for this amazing web content

Mark Zuckerberg is close to his “Holy Grail”: this will be the technology that promises to change the world