From Desktop 3d Apps to Web 3d Apps using Blender and React
Tags: react, threejs, javascript, webdev
Published: 2022-04-18 08:43:55

In this tutorial, I will walk you thru the steps to create a 3d react application with some interactivity so in the final you will have something like [this](https://cra-fiber-threejs.vercel.app/) 

- What is Blender? - Simply Explained
- Create React App
- Install dependencies
- Export blender asset
- Compress asset
- Convert asset to `JSX` component
- Integrate new component
- Enhanced component and functionality
    - Adding some style
    - Install dependency
    - Edit React Components
- Resources
- Appendix

## Before you start

You will need to have the following installed or configured and know at least the basics of using them before proceeding.

- NodeJS installed (preferable > 12)
- Basic Knowledge in React
- Previous use of `create-react-app`
- Not mandatory, but some basic knowledge of using blender 3d app to understand the concept of mesh and material

## What is Blender? Simply Explained
This tutorial is not a blender tutorial, so that it will be a short explanation.
Blender is a free, open-source 3D creation suite. With a strong foundation of modeling capabilities, there's also robustly texturing, rigging, animation, lighting, and other tools for complete 3D creation.

![Blender Program](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fois6la34f49b0hrlut7.png)
 

![Spring - Blender Open Movie Blender](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/q30yak8vp0mk7a3qbopo.png)
Source: Spring - Blender Open Movie Blender, Animation Studio via [YouTube](https://www.youtube.com/watch?v=WhWc3b3KhnY)

## Create React App

```bash
npx create-react-app cra-fiber-threejs
npm run start
```

If everything works successfully, you can navigate to: http://localhost:3000/, and you will see a React App

## Install dependencies

- Install `gltf-pipeline`; this will help you to optimize our glTF, meaning smaller for the web; this is installed globally 

```bash
npm install -g gltf-pipeline
```

- Install @react-three dependencies for our project, navigate to `cra-fiber-threejs` folder and run

```bash
npm i @react-three/drei
npm i @react-three/fiber
```

## Export blender asset
- Open blender program with you're created, 3d model
- if you have installed blender and created a 3d modeling, in case you didn't, take a look in the optional step
![Exporting Menu Blender](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/06jxz8ad3b7hci2tavax.png)

_Optional_
- If you have installed blender but didn't create any model, [here](https://github.com/OmerHerera/cra-fiber-threejs/blob/STEP_01/assets/Shoe.blend) you have the one I'm using in the tutorial
- If you didn't install blender and want the compressed `glb` file [here](https://github.com/OmerHerera/cra-fiber-threejs/blob/STEP_01/assets/shoe.glb), you can download it.

## Compress asset
- The file we exported from the previous step, some times are significant, and they are not optimized for the web, so we need to compress it
- Navigate where you saved the `.glb` file (from the previous step) and run the following command:

```bash
gltf-pipeline -i <input file glb> -o <output glb> --draco.compressionLevel=10
e.g:
gltf-pipeline -i shoe.glb -o ShoeModelDraco.glb --draco.compressionLevel=10
```

## Convert asset to `JSX` component

To start interacting with our 3d model, we need to convert it to a JSX component using gltfjsx. You can read more [here](https://github.com/pmndrs/gltfjsx).  `gltfjsx` - Turns GLTFs into JSX components)
- Navigate where you saved the .glb file from the previous step and run the following command:

```bash
npx gltfjsx <outputed glb from previus step>
e.g. npx gltfjsx ShoeModelDraco.glb
```

- The output will be a `js` file with content similar to:
```jsx
/*
Auto-generated by: https://github.com/pmndrs/gltfjsx
*/

import React, { useRef } from 'react'
import { useGLTF } from '@react-three/drei'

export default function Model({ ...props }) {
  const group = useRef()
  const { nodes, materials } = useGLTF('/ShoeModelDraco.glb')
  return (
    <group ref={group} {...props} dispose={null}>
      <mesh geometry={nodes.shoe.geometry} material={materials.laces} />
      <mesh geometry={nodes.shoe_1.geometry} material={materials.mesh} />
      <mesh geometry={nodes.shoe_2.geometry} material={materials.caps} />
      <mesh geometry={nodes.shoe_3.geometry} material={materials.inner} />
      <mesh geometry={nodes.shoe_4.geometry} material={materials.sole} />
      <mesh geometry={nodes.shoe_5.geometry} material={materials.stripes} />
      <mesh geometry={nodes.shoe_6.geometry} material={materials.band} />
      <mesh geometry={nodes.shoe_7.geometry} material={materials.patch} />
    </group>
  )
}

useGLTF.preload('/ShoeModelDraco.glb')
```

- The output It's a React component with all the meshed/materials mapped ready to work
- If you worked with blender, you can see that it has mapped all its meshes objects and all its materials
- This component can now be dropped into your scene. It is asynchronous and, therefore, must be wrapped into `<Suspense>`
 which gives you complete control over intermediary loading-fallbacks and error handling.


## Integrate new component
- Go the project you created using `create-react-app`
- Copy your new file created in step "Convert asset to `JSX` component" e.g. ShoeModelDraco.js to `src/` folder
- Create a new file for your new component and called it `BlenderScene.js`, this file will include for the simplicity 
also some logic and the Scene components, in a real application you will want to separate them in different files/components, copy the following code:
```jsx
import React, { Suspense } from 'react';
import { Canvas } from "@react-three/fiber"
import { ContactShadows, Environment, OrbitControls } from "@react-three/drei"
import Model from './ShoeModelDraco'
function Scene() {
  return (
    <div className='scene'>
      <Canvas shadows dpr={[1, 2]} camera={{ position: [0, 0, 4], fov: 50 }}>
        <ambientLight intensity={0.3} />
        <spotLight intensity={0.5} angle={0.1} penumbra={1} position={[10, 15, 10]} castShadow />
        <Suspense fallback={null}>
          <Model />
          <Environment preset="city" />
        <ContactShadows rotateX={Math.PI / 2} position={[0, -0.8, 0]} opacity={0.25} width={10} />
        </Suspense>
        <OrbitControls minPolarAngle={Math.PI / 2} maxPolarAngle={Math.PI / 2} enableZoom={false} enablePan={false} />
      </Canvas>
    </div>
  )
}
function BlenderScene() {
  return (
    <>
      <Scene />
    </>
    
  );
}

export default BlenderScene;
```

- Copy into the public folder the `.glb` output file from step "Export blender asset," in my case: `ShoeModelDraco.glb`

- Use the `BlenderScene` component you just created, open the `App.js` file, and import it something like:
```jsx
import './App.css';
import BlenderScene from './BlenderScene';

function App() {
  return (
    <BlenderScene /> 
  );
}

export default App;
```

- If everything runs successfully, you should see your 3d model in the browser, something like this:

![React component](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fgvri96eq49b5otrwumg.png) 

- The only interactivity that you have it's that you can spin the 3d model, and that's it, 
- In the following steps, we will:
    - Add more fun/complex interactivity 
    - Display nicer in the browser
    - In the resources part, you can find a [link](https://github.com/OmerHerera/cra-fiber-threejs/tree/STEP_01) for the branch with the code until this step

## Enhanced component and functionality
If you are reading here, kudos 💪🏼.
You are almost done 🥵; you have your 3d model in the browser 🎉, but you saw, it's not very interesting and boring; let's start adding cool stuff 😎.

*Disclaimer: The following code it's not production-ready, and I did some hacks and also not best practices when writing the components*

### Adding some style
- Open the `App.css` file and add in the end of it the following:
```css
#root {
  position: relative;
  margin: 0;
  padding: 0;
  overflow: hidden;
  outline: none;
  width: 100vw;
  height: 100vh;
}
.scene {
    height: 500px;
    padding: 100px;
  
}
```

### Install dependency
- We will install `react-colorful`, a tiny color picker component for React and Preact apps. We will use it for choosing colors

```bash
npm i react-colorful
```

### Edit React Components
 
- Open `ShoeModelDraco.js` file and copy the following code
- We add functionality to work with the mouse when the user clicks on our model
- We add state to know which part of the model was selected
```jsx
    /*
Auto-generated by: https://github.com/pmndrs/gltfjsx
*/

import React, { useRef } from 'react'
import { useGLTF } from '@react-three/drei'
import { useFrame } from '@react-three/fiber'
export default function Model({ props, currentState, setCurrentState, setHover }) {
  const group = useRef()
  const { nodes, materials } = useGLTF('/ShoeModelDraco.glb');
  // Animate model
  useFrame(() => {
    const t = performance.now() / 1000
    group.current.rotation.z = -0.2 - (1 + Math.sin(t / 1.5)) / 20
    group.current.rotation.x = Math.cos(t / 4) / 8
    group.current.rotation.y = Math.sin(t / 4) / 8
    group.current.position.y = (1 + Math.sin(t / 1.5)) / 10
  })
  return (
    <>
      <group
      ref={group} {...props}
      dispose={null}
      onPointerOver={(e) => {
        e.stopPropagation();
        setHover(e.object.material.name);
      }}
      onPointerOut={(e) => {
        e.intersections.length === 0 && setHover(null);
      }}
      onPointerMissed={() => {
        setCurrentState(null);
      }}
      onClick={(e) => {
        e.stopPropagation();
        setCurrentState(e.object.material.name);
      }}>
      <mesh receiveShadow castShadow geometry={nodes.shoe.geometry} material={materials.laces} material-color={currentState.items.laces} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_1.geometry} material={materials.mesh} material-color={currentState.items.mesh} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_2.geometry} material={materials.caps} material-color={currentState.items.caps} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_3.geometry} material={materials.inner} material-color={currentState.items.inner} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_4.geometry} material={materials.sole} material-color={currentState.items.sole} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_5.geometry} material={materials.stripes} material-color={currentState.items.stripes} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_6.geometry} material={materials.band} material-color={currentState.items.band} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_7.geometry} material={materials.patch} material-color={currentState.items.patch} />
      </group>
    </>
  )
}

useGLTF.preload('/ShoeModelDraco.glb')
```


- Open `BlenderScene.js` file and copy the following code
- We add state in order to know which part of the model was 
selected
- Added work with the picker component
- Added animation to the model, floating illusion
```jsx
import React, { useState, useEffect, Suspense } from 'react';
import { Canvas } from "@react-three/fiber"
import { ContactShadows, Environment, OrbitControls } from "@react-three/drei"
import { HexColorPicker } from 'react-colorful'
import Model from './ShoeModelDraco'
function Scene() {
  // Cursor showing current color
  const [state, setState] = useState({
    current: null,
    items: {
      laces: "#ffffff",
      mesh: "#ffffff",
      caps: "#ffffff",
      inner: "#ffffff",
      sole: "#ffffff",
      stripes: "#ffffff",
      band: "#ffffff",
      patch: "#ffffff",
    },
  });
  const [hovered, setHover] = useState(null)
  useEffect(() => {
    const cursor = `<svg width="64" height="64" fill="none" xmlns="http://www.w3.org/2000/svg"><g clip-path="url(#clip0)"><path fill="rgba(255, 255, 255, 0.5)" d="M29.5 54C43.031 54 54 43.031 54 29.5S43.031 5 29.5 5 5 15.969 5 29.5 15.969 54 29.5 54z" stroke="#000"/><g filter="url(#filter0_d)"><path d="M29.5 47C39.165 47 47 39.165 47 29.5S39.165 12 29.5 12 12 19.835 12 29.5 19.835 47 29.5 47z" fill="${state.items[hovered]}"/></g><path d="M2 2l11 2.947L4.947 13 2 2z" fill="#000"/><text fill="#000" style="white-space:pre" font-family="Inter var, sans-serif" font-size="10" letter-spacing="-.01em"><tspan x="35" y="63">${hovered}</tspan></text></g><defs><clipPath id="clip0"><path fill="#fff" d="M0 0h64v64H0z"/></clipPath><filter id="filter0_d" x="6" y="8" width="47" height="47" filterUnits="userSpaceOnUse" color-interpolation-filters="sRGB"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feColorMatrix in="SourceAlpha" values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 127 0"/><feOffset dy="2"/><feGaussianBlur stdDeviation="3"/><feColorMatrix values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0"/><feBlend in2="BackgroundImageFix" result="effect1_dropShadow"/><feBlend in="SourceGraphic" in2="effect1_dropShadow" result="shape"/></filter></defs></svg>`
    const auto = `<svg width="64" height="64" fill="none" xmlns="http://www.w3.org/2000/svg"><path fill="rgba(255, 255, 255, 0.5)" d="M29.5 54C43.031 54 54 43.031 54 29.5S43.031 5 29.5 5 5 15.969 5 29.5 15.969 54 29.5 54z" stroke="#000"/><path d="M2 2l11 2.947L4.947 13 2 2z" fill="#000"/></svg>`
    if (hovered) {
      document.body.style.cursor = `url('data:image/svg+xml;base64,${btoa(cursor)}'), auto`
      return () => (document.body.style.cursor = `url('data:image/svg+xml;base64,${btoa(auto)}'), auto`)
    }
  }, [hovered])
  
  function Picker() {
    return (
      <div style={
        {
          display: state.current ? "block" : "none",
          position: "absolute",
          top: "50px",
          left: "50px",

       }
      }>
        <HexColorPicker
          className="picker"
          color={state.items[state.current]}
          onChange={(color) => {
            let items = state.items;
            items[state.current] =  color
          }}
        />
        <h1>{state.current}</h1>
      </div>
    )
  }
  return (
    <div className='scene'>
      <Canvas shadows dpr={[1, 2]} camera={{ position: [0, 0, 4], fov: 50 }}>
        <ambientLight intensity={0.3} />
        <spotLight intensity={0.5} angle={0.1} penumbra={1} position={[10, 15, 10]} castShadow />
        <Suspense fallback={null}>
          <Model
            currentState={ state }
            setCurrentState={(curState) => {
              setState({
                ...state,
                current: curState
              })
            }}
            setHover={ setHover}
          />
          <Environment preset="city" />
        <ContactShadows rotateX={Math.PI / 2} position={[0, -0.8, 0]} opacity={0.25} width={10} />
        </Suspense>
        <OrbitControls minPolarAngle={Math.PI / 2} maxPolarAngle={Math.PI / 2} enableZoom={false} enablePan={false} />
      </Canvas>
      <Picker />
    </div>
  )
}
function BlenderScene() {
  return (
    <>
      <Scene />
    </>
    
  );
}

export default BlenderScene;
```
- If everything works successfully, you should see something like this:
![Final Result](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8o3psx8363jr2zotxt1y.png)
 
- In the resources part, you can find a [link](https://github.com/OmerHerera/cra-fiber-threejs/tree/STEP_02) for the branch with the code until this step

- Live Working example [here](https://cra-fiber-threejs.vercel.app/)

## Resources 
- [blender](https://www.blender.org/)

- [gltfjsx](https://github.com/pmndrs/gltfjsx)

- [react-colorful](https://www.npmjs.com/package/react-colorful)

- [STEP_01](https://github.com/OmerHerera/cra-fiber-threejs/tree/STEP_01)

- [STEP_02](https://github.com/OmerHerera/cra-fiber-threejs/tree/STEP_02)

## Appendix
- Blender
    - Blender is the free and open-source 3D creation suite. It supports the entirety of the 3D pipeline—modeling, rigging, animation, simulation, rendering, compositing, and motion tracking, even video editing, and game creation; more in [here](https://www.blender.org/about/)
- glTF files 
    - Graphics Language Transmission Format or GL Transmission Format, more [here](https://en.wikipedia.org/wiki/GlTF)
- gltf-pipeline
    - Content pipeline tools for optimizing glTF, more [here](https://github.com/CesiumGS/gltf-pipeline)