How to display 3D humanoid avatar with React
Tags: javascript, react, threejs, vrm
Published: 2019-12-01 15:28:14

## Introduction

3DCG and VR technology are used in various places and become familiar with us. And the same phenomenon occurs on web browser. Today I introduce VRM and how to display it with React and @pixiv/three-vrm.

### What is VRM?

VRM is a file format for handling 3D humanoid avatar (3D model) data for VR applications. If you have an avatar that conforms to VRM, you can enjoy various applications that require a 3D avatar. 

### What is @pixiv/three-vrm?

{% github pixiv/three-vrm no-readme %}

@pixiv/three-vrm is a JavaScript library to use VRM on Three.js. This enables to render VRM on web applications like [VRoid Hub](https://hub.vroid.com/).

![VRoid Hub](https://i.gyazo.com/e544321e8d12b5c0011e794bf9b043e3.gif)

## Prepare VRM

At first, you need to download VRM from [VRoid Hub](https://hub.vroid.com/).

1. Search VRM models by tags.
2. Select your favorite model.
3. Move to model page and download by clicking "Use this model"

## Set up project

```console
$ npx create-react-app three-vrm-sample
$ cd three-vrm-sample/
$ yarn add @pixiv/three-vrm three react-three-fiber
```

```html
<!DOCTYPE html>
<html>
  <head>
    <title>@pixiv/three-vrm sample</title>
    <style>
      html,
      body {
        background-color: #000;
        color: #fff;
        margin: 0;
        width: 100vw;
        height: 100vh;
      }

      #root {
        width: 100%;
        height: 100%;
      }
    </style>
  </head>
  <body>
    <div id="root"></div>
  </body>
</html>

```

```jsx
import React from 'react'
import ReactDOM from 'react-dom'

const App = () => null

ReactDOM.render(<App />, document.getElementById('root'))

```

## Add VRM Loader

We can load VRM with `GLTFLoader` because VRM is similar format to GLTF.

```js
import { VRM } from '@pixiv/three-vrm'
import { useRef, useState } from 'react'
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader'

const useVrm = () => {
  const { current: loader } = useRef(new GLTFLoader())
  const [vrm, setVrm] = useState(null)

  const loadVrm = url => {
    loader.load(url, async gltf => {
      const vrm = await VRM.from(gltf)
      setVrm(vrm)
    })
  }

  return { vrm, loadVrm }
}
```

## Display VRM with react-three-fiber

[react-three-fiber](https://github.com/react-spring/react-three-fiber) is a React renderer for Three.js. You can use Three.js declaratively with it. I use the following three elements this time.

- `<Canvas>`: Wrapper element for `react-three-fiber` elements
- `<spotLight>`: Light element to illuminate objects
- `<primitive>`: 3D object element

When you input a VRM file, `handleFileChange()` create object url and load VRM.

```jsx
import React from 'react'
import { Canvas } from 'react-three-fiber'
import * as THREE from 'three'

const App = () => {
  const { vrm, loadVrm } = useVrm()

  const handleFileChange = event => {
    const url = URL.createObjectURL(event.target.files[0])
    loadVrm(url)
  }

  return (
    <>
      <input type="file" accept=".vrm" onChange={handleFileChange} />
      <Canvas>
        <spotLight position={[0, 0, 50]} />
        {vrm && <primitive object={vrm.scene} />}
      </Canvas>
    </>
  )
}
```

Result:

![Result](https://i.gyazo.com/6d6b2f16f3f9213c06bbb5d5de7aa965.png)

## Improve appearance

VRM model in the result is small and facing the other side. You want to see it closer and its face, don't you?


### 1. Create a new camera from `THREE.PerspectiveCamera` and set its position.

> Note:
> `useThree` gives you access to all the basic THREE objects like gl, scene, camera, clock and so on...

```jsx
import React, { useEffect, useRef } from 'react'
import { useThree, Canvas } from 'react-three-fiber'
import * as THREE from 'three'

const App = () => {
  const { aspect } = useThree()
  const { current: camera } = useRef(new THREE.PerspectiveCamera(30, aspect, 0.01, 20))
  const { vrm, loadVrm } = useVrm()

  const handleFileChange = event => {
    const url = URL.createObjectURL(event.target.files[0])
    loadVrm(url)
  }

  // Set camera position
  useEffect(() => {
    camera.position.set(0, 0.6, 4)
  }, [camera])

  return (
    <>
      <input type="file" accept=".vrm" onChange={handleFileChange} />
      <Canvas camera={camera}>
        <spotLight position={[0, 0, 50]} />
        {vrm && <primitive object={vrm.scene} />}
      </Canvas>
    </>
  )
}
```

### 2. Rotate and look at camera

Assign `camera` to `vrm.lookAt.target` and rotate vrm 180°.

```jsx
import { VRM } from '@pixiv/three-vrm'
import { useEffect, useRef, useState } from 'react'
import { useThree } from 'react-three-fiber'
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader'

const useVrm = () => {
  const { camera } = useThree()
  const { current: loader } = useRef(new GLTFLoader())
  const [vrm, setVrm] = useState(null)

  const loadVrm = url => {
    loader.load(url, async gltf => {
      const vrm = await VRM.from(gltf)
      vrm.scene.rotation.y = Math.PI
      setVrm(vrm)
    })
  }

  // Look at camera
  useEffect(() => {
    if (!vrm || !vrm.lookAt) return
    vrm.lookAt.target = camera
  }, [camera, vrm])

  return { vrm, loadVrm }
}
```

Final Code:

```jsx
import { VRM } from '@pixiv/three-vrm'
import ReactDOM from 'react-dom'
import React, { useEffect, useRef, useState } from 'react'
import { useThree, Canvas } from 'react-three-fiber'
import * as THREE from 'three'
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader'

const useVrm = () => {
  const { camera } = useThree()
  const { current: loader } = useRef(new GLTFLoader())
  const [vrm, setVrm] = useState(null)

  const loadVrm = url => {
    loader.load(url, async gltf => {
      const vrm = await VRM.from(gltf)
      vrm.scene.rotation.y = Math.PI
      setVrm(vrm)
    })
  }

  // Look at camera
  useEffect(() => {
    if (!vrm || !vrm.lookAt) return
    vrm.lookAt.target = camera
  }, [camera, vrm])

  return { vrm, loadVrm }
}

const App = () => {
  const { aspect } = useThree()
  const { current: camera } = useRef(new THREE.PerspectiveCamera(30, aspect, 0.01, 20))
  const { vrm, loadVrm } = useVrm()

  const handleFileChange = event => {
    const url = URL.createObjectURL(event.target.files[0])
    loadVrm(url)
  }

  // Set camera position
  useEffect(() => {
    camera.position.set(0, 0.6, 4)
  }, [camera])

  return (
    <>
      <input type="file" accept=".vrm" onChange={handleFileChange} />
      <Canvas camera={camera}>
        <spotLight position={[0, 0, 50]} />
        {vrm && <primitive object={vrm.scene} />}
      </Canvas>
    </>
  )
}

ReactDOM.render(<App />, document.getElementById('root'))

```

Result:

![Result](https://i.gyazo.com/50331ad877ae474a778221768a2d584e.png)

Looks good 👍

## Conclusion

VRM will be used more widely in the future. I hope this article will help you when you need to use VRM with React.

@pixiv/three-vrm has more features, so if you are interested, please read the documentation and try it out.

If you have any problems or questions, please write comment or reply to my Twitter account.

Sample Repository: {% github saitoeku3/three-vrm-sample no-readme %}