Prototyping with React VR
Tags: Product, React, Reactvr, VR, Prototyping

Why React

One of React’s biggest innovations is that it enables developers to describe a system, such as the UI of a web or mobile app, as a set of declarative components. The power of this declarative approach is that the description of the UI is decoupled from its implementation, allowing authors to build custom “renderers” that target more platforms than just web browsers, such as hardware, terminal applications, music synthesizers, and Sketch.app.

React as a paradigm is perfect for wrapping the complexity of underlying platform APIs and providing consistent and smooth tools to the developers using them. Jon Gold, “Painting with Code” http://airbnb.design/painting-with-code

React Native Under the Hood

Because React VR is built on top of React Native, let’s start with a look at how it works under the hood. React Native is built on a renderer that controls native UI on iOS and Android. The React application code runs in a JavaScript virtual machine in a background thread on the mobile device, leaving the main thread free to render the native UI. React Native provides a bridge for communication between the native layer and the JavaScript layer of the app. When the React components in your application are rendered, the React Native renderer serializes all UI changes that need to happen into a JSON-based format and sends this payload asynchronously across the bridge. The native layer receives and deserializes this payload, updating the native UI accordingly.

Diagram 1: React Native architecture.

Over the past year at Airbnb, we’ve invested heavily in React Native because we recognize the power of being able to share knowledge, engineers, and code across platforms. In November, we launched our new Experiences platform, which is largely written in React Native on our iOS and Android apps, and we formed a full-time React Native Infrastructure team to continue this investment.

Learn once, write anywhere. Tom Occhino, React Native: Bringing modern web techniques to mobile

React VR is Built on React Native

React VR’s architecture mirrors that of React Native, with the React application code running in a background thread — in this case, a Web Worker in the web browser. When the application’s React components are rendered, React VR utilizes the React Native bridge to serialize any necessary UI changes and pass them to the main thread, which in this case is the browser’s main JavaScript runtime. Here, React VR utilizes a library from Oculus called OVRUI to translate the payload of pending UI updates into Three.js commands, rendering a 3D scene using WebGL.

Diagram 2: React VR is built directly on top of React Native

Finally, React VR utilizes WebVR’s new navigator.getVRDisplays() API to send the 3D scene to the user’s head mounted display, such as an Oculus Rift, HTC Vive or Samsung Gear VR. WebVR, a new standard being spearheaded by Mozilla, is supported in recent builds of major web browsers. Check out webvr.info for the latest information on browser support.

Because React VR implements a lot of the same public APIs that React Native implements, we have access to the same vast ecosystem of patterns, libraries, and tools. It will feel familiar for any developer who has built React or React Native apps. We were able to get a VR prototype up and running quickly; in no time at all, we scaffolded a basic React application, set up Redux, and began hitting our production JSON API for sample data.

With hot module reloading and Chrome Dev tools debugging, we could iterate nearly as fast as in React web and React Native development, which allowed us to throw a bunch of UI ideas at the proverbial wall to see what would stick.

Using React (JavaScript) has turned out to be a bigger win for VR app development than I expected — UI dev is several x faster than Unity. John Carmack, Oculus CTO and original creator of Quake.

Flexbox in VR

React VR inherits React Native’s flexbox-based layout and styling engine, with a few tweaks to allow transforms in 3 dimensions.

Flexbox support for React Native is provided by Yoga, a cross-platform layout engine created by Facebook to simplify mobile development by translating flexbox directives into layout measurements. Because it’s written in C, Yoga (née css-layout) can be embedded natively in Objective-C and Java mobile apps. React VR also uses Yoga for flexbox layout. “But how?” you ask, “It’s written in C!” The React VR team has accomplished this by using Emscripten to cross-compile the Yoga C code into JavaScript. How cool is that?

This is a powerful feature of React VR: developers can use the same styling and layout system across web, React Native, and VR, which opens the doors to directly sharing layout styles across these platforms.

Sharing Primitives

Like React Native, React VR provides a set of basic primitives used to construct UI— <View> , <Text> , <Image> , Stylesheet —in addition to adding of its own VR-specific primitives, such as <Pano> , <Box> , among others. This allowed us to drop in some of our existing React Native components into VR, rendered on a 2D surface.

This is hugely exciting because we’ve built our UI component system upon react-primitives , a library we developed for sharing React components across platforms by providing the basic <View> , <Text> , <Image> , etc. primitives for a variety of platforms, including web, native, and Sketch.app (via our react-sketchapp project).

This means we can use the buttons, rows, icons and more directly in VR, keeping Airbnb design language consistent without having to rewrite it all from scratch.

Check out our engineer Leland Richardson’s talk at React Europe for a more in-depth look at the promise of react-primitives , below.

Airbnb engineer Leland Richardson’s talk at React Europe: “React as a Platform: A path towards a truly cross-platform UI”

<CylindricalLayer />

As you could imagine, placing 2D content onto a flat plane in 3D space often falls short of an optimal viewing experience. Currently, many VR apps solve this by rendering 2D UI onto a cylindrical plane curved in front of the viewer, giving it a “2.5D” feel.

A screenshot of Oculus home captured in Gear VR, showing of its use of a cylindrical layer for displaying 2D content for a “2.5D” feel