3D Lightfield, AR, and VR: Together in the Multiverse
By: David Fattal
Tags: AR, Light Field, 3d, Metaverse, VR

There’s no doubt in my mind that VR is an amazing, immersive technology. You drop on a headset and — just like that — you’re whisked away to another world! But what about everyone around you? Your friends, your family — they get left out of these cool moments. It’s a missed opportunity for co-workers, customers, and clients that can’t share in these unique experiences. People keep talking about this interconnectedness as a “Metaverse,” but it’s more than that. It’s a multiverse begging to connect us all, through a huge variety of device types.

The idea behind Leia’s 3D Lightfield technology — and the Lume Pad, for that matter — is to bring us together. We get to have a shared experience. Something immersive, but without the hurdles of difficult set-ups and extra hardware to lug around or strap on. That’s something we sometimes forget about in Silicon Valley: The human element. Our technology is about removing the wires and barriers between the person who is standing right next to you.

Immersion in a familiar device

3D Lightfield was designed to work with familiar devices such as phones or tablets, that everyone knows how to use. You turn them on, touch the screen, and they just work. Best of all, you have already incorporated them into your daily life. They are so familiar that they have become an extension of yourself.

Leia’s 3D Lightfield technology does not get in the way of that familiarity. Our first device, the Lume Pad, is a powerful tablet that extends the full (2D) Android ecosystem with intuitive 3D apps, games, and experiences — all brought closer to life with added depth, textures, and lights. Maybe you want to share photos or videos in 3D while around the house. Engage in immersive 3D gaming or learning. Or experience popular video streaming sites in real-time converted 3D. The Lume Pad just works. Better yet, it doesn’t sacrifice performance on all the regular 2D apps you rely upon to get your job done.

The Future: A 3D window to the digital world

Technically, every time you Slack, Zoom, or otherwise team up online, you’re tapping into the Metaverse concept. Being able to enter the same chat room to connect in a shared virtual space is the future. The question is how you want to connect with it.

3D Lightfield displays offer a window-like experience of the digital world. A different view that brings the Metaverse to you in a unique way. We do it with hardware you’re already familiar with that requires little effort on the end user’s part.

Some can strap on a VR head-mounted display, look through an AR app, or watch Ariana Grande perform to a packed virtual crowd in Fortnite. These are great glimpses of an already-interconnected world. How you look into that world — and how it impacts you — varies. Scott Stein at CNET recently went into this at length. Here’s one of several quotes that stood out to me: “I’ve been waiting for all of these VR headsets, smartwatches, phones, tablets, AR devices to work better together, seamlessly, like they’re interconnected.”

I couldn’t agree more.

It’s the things we share, the content we watch, and the games that we play that connect us. That’s why we have created the tools for developers and content creators to seamlessly render their 3D, AR, or VR creations onto our screens.

Scott goes on, “Facebook’s upcoming social platform, Horizon, is an example of this: The avatar-based app will work in VR, but Facebook intends for it to work in AR as well, and on regular laptops and phones. Microsoft’s AltSpaceVR is already like this.”

At Leia, we imagine a way to join your friends in the Metaverse that’s as easy as picking up your mobile device. A not-too-distant future where you hold up a Leia-enabled phone or tablet and the 3D objects in that virtual world could hover on the screen. Better still, it all shines with a photorealistic, accurately light-shaded 180-degree view. You turn, and the on-board gyros could move your perspective within the world. With ARCore (which the Lume Pad already supports) you get the full 6DoF experience: wander through space, just like you’d do with AR-ready mobile devices — but in immersive 3D.

The best part of the 3D Lightfield window into the Metaverse? No extra hardware to hang on your head or over your eyes! So it’s a shared experience with people around you. And it’s your face, recorded in 3D and rendered realistically in the digital space. It lets you embed a real-time, photo-realistic recording of yourself (showing your true emotions, live), further bridging the gap between the real and virtual worlds.

While it’s still too early to say more, our lab teams are working on ways to experience the real-world in 3D through the camera lens with all types of 3D digital filters and effects to create the next level of video chat engagement (Creepy as some of these filters might be!)

While we are working hard on the future, how about trying LeiaTube, our latest app on Lume Pad that instantly converts 2D or VR 360 videos into gorgeous Lightfield experiences? Or following along with esports games in a Lightfield spectator mode? Or journey across the 1,200 beautifully immersive learning scenes that the Mozaik3D app has to offer. It’s all possible today.

So, use your AR apps, your VR headsets — or a Lume Pad! We see multiple windows to this multiversal Metaverse. With Leia’s Lightfield screens, though, we also envision a way to experience this bold new virtual world together, with your friends and family right by your side.