Unity MR Part 12: Improve Scene
Tags: unity3d, mixedreality, tutorial, beginners
Published: 2024-01-11 08:51:30

👀 Stumbled here on accident? Start with the [introduction](https://dev.to/taikonauten/part-0-introduction-56fl)!

📚 The aim of this article is to enhance the visual and interactive elements of our scene. We will focus on several key improvements:

1. **Graphics Enhancement**: We'll enhance the graphics by configuring the Universal Render Pipeline (URP) assets.
2. **Remove Debug Plane Material**: The debug plane material, used for initial testing, will be removed for a cleaner look.
3. **Door Shadow Effects**: We'll set up the scene so that the door casts shadows on planes that will now be invisible, adding depth and realism.
4. **Create a 'Behind the Door' Scene**: We will develop a scene that is only revealed when the door is open, adding an element of surprise and exploration.

This article incorporates some advanced techniques that can significantly elevate your MR project. If you encounter any challenges or need further clarification, feel free to send us a message. Additionally, you can refer to the final project in our GitHub repository dedicated to this article series.

---

ℹ️ If you find yourself facing any difficulties, remember that you can always refer to or download the code from our accompanying [GitHub repository](https://github.com/taikonauten/unity-mr-article-series/tree/main/12_ImproveScene)

---

## Light Estimation

ℹ️ Light estimation for MR devices like the Meta Quest 3 involves assessing the real-world lighting conditions and applying that information to the virtual environment to create a more immersive and realistic experience.

Regrettably, as indicated by Unity in their forums, this feature is not currently supported.

---

## Instantiate the door facing the user

Currently, the door Prefab is instantiated with a consistent rotation. Let's modify the `MRArticleSeriesController` Script so that the door faces the user upon instantiation. To do this, locate the `OnButtonPressedRightAsync` method in the `MRArticleSeriesController` Script and update it as follows.

```csharp
private async void OnButtonPressedRightAsync(InputAction.CallbackContext context)
{
    Debug.Log("MRArticleSeriesController -> OnButtonPressedRightAsync()");

    if (doorInstance != null) {
        Debug.Log("MRArticleSeriesController -> OnButtonPressedRightAsync(): Door already instantiated");

        return;
    }

    if (rayInteractor.TryGetCurrent3DRaycastHit(out RaycastHit hit))
    {
        Pose pose = new(hit.point, Quaternion.identity);
        Result<ARAnchor> result = await anchorManager.TryAddAnchorAsync(pose);

        result.TryGetResult(out ARAnchor anchor);

        if (anchor != null) {
            // Instantiate the door Prefab
            doorInstance = Instantiate(door, hit.point, Quaternion.identity);

            // Unity recommends parenting your content to the anchor.
            doorInstance.transform.parent = anchor.transform;

            // Make the door face the user after instantiating
            doorInstance.transform.LookAt(new Vector3(
                Camera.main.transform.position.x,
                doorInstance.transform.position.y,
                Camera.main.transform.position.z
            ));
        }
    }
}
```

As you can see, we have added a call to `doorInstance.transform.LookAt` in the script. After making this addition, save the file and return to the Unity Editor to apply these changes.

ℹ️ For more information on the `LookAt` function and how it works, you can refer to the Unity documentation. This resource provides detailed insights into how `LookAt` can be used to orient objects in your scene: [Unity - Scripting API: Transform.LookAt](https://docs.unity3d.com/2023.2/Documentation/ScriptReference/Transform.LookAt.html).

---

## Update AR Default Plane

In this step, we will modify the material of the `AR Default Plane` to make it transparent while still allowing it to receive shadows. This adjustment enables us to see the shadows cast by the door on our floor. Implementing this change enhances the realism of our MR experience, as it allows virtual objects like the door to interact more naturally with the environment.

Go to **Window → Package Manager** and install the package `https://github.com/Cyanilux/URP_ShaderGraphCustomLighting.git` with `Install package from git URL`:


![Install a package via git](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t6orrodzo5xmj7hy3t5f.png)
<figcaption>Install a package via git
</figcaption>
---

Once you have completed the installation, close the Package Manager. Now, edit the `AR Default Plane` by double-clicking it in the `Project` window.


![Replacing the default material with the ShadowReceiver material](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/129vbnmzc158tvhk0f18.png)
<figcaption>Replacing the default material with the ShadowReceiver material</figcaption>
---

Replace the default Material with `ShadowReceiver` as seen in the above screenshot. That’s all for the `AR Default Plane`.

If you can’t find `ShadowReceiver` via search you can also drag and drop the material from **Packages/com.cyanilux.shadergraph-customlighting/Examples/ShadowReceiver.mat.**

---

## Optimize Lightning

In this step, we will focus on optimizing the `Directional Light` in our scene. You have the option to adjust the values as shown in the upcoming screenshot, or alternatively, you can modify the settings according to your own preferences and the specific requirements


![Optimizing the Directional Light](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vezmwqtkpoakh16t6ynb.png)
<figcaption>Optimizing the Directional Light</figcaption>
---

## Optimize URP

We are currently set to the `Balanced` quality level for the Android build. Let's switch to using the `High Fidelity` quality setting by default, as illustrated in the upcoming screenshot. This change will enhance the visual quality of our application, offering a more detailed and immersive experience on Android devices.


![Set High Fidelity as the default quality on Android](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7jtkn85lnmhc0r5u0297.png)
<figcaption>Set High Fidelity as the default quality on Android</figcaption>
---

Next, we must disable `HDR` in our URP asset. `HDR` is not supported in this context and will cause the application to crash if enabled. You can locate the URP asset at **Assets/Settings/URP-HighFidelity.asset**. The HDR option is available as the first option under the Quality section.


![Disabling HDR in our URP asset](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2y5w1vzqyr68ohl8fbxa.png)
<figcaption>Disabling HDR in our URP asset</figcaption>
---

The `High Fidelity` settings utilize the `Renderer Features` `Screen Space Ambient Occlusion`, which significantly impacts performance. To optimize, you can either disable this feature or remove it entirely —both approaches are effective. You can find this setting in the asset located at **Assets/Settings/URP-HighFidelity-Renderer.asset**.


![Disabling Screen Space Ambient Occlusion from the High Fidelity-Renderer](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ydn03fgjqz5b83fpwjx5.png)
<figcaption>Disabling Screen Space Ambient Occlusion from the High Fidelity-Renderer</figcaption>
---

## The behind the door

Let's create a straightforward effect that simulates a scene visible only when the door is open and the user looks through it. This approach will add an intriguing layer of interactivity to our project, enhancing the user experience by revealing a hidden scene.

Edit our door prefab by double-clicking `Assets/Prefabs/Door` in the Project window.

1. Disable the `Door` mesh through the inspector so we can look through the door.
2. Add a plane into the root of the prefab via **Create → 3D → Plane.**
3. Use the following Transform values as follows:
    

|     | **X** | **Y** | **Z** |
| --- | --- | --- | --- |
| **Position** | 0   | 1.025 | 0   |
| **Rotation** | 90  | 0   | 0   |
| **Scale** | 0.1 | 1   | 0.205 |

The prefab should now resemble the appearance displayed in the upcoming screenshot.


![Creating a Plane inside our door prefab](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/otju9w0jjjq5p68p80zz.png)
<figcaption>Creating a Plane inside our door prefab</figcaption>
---

As you will notice, this method effectively creates the illusion where the "inside" of the door is only visible from the front. To observe this effect, take a look at the door from the back in the Scene view, as illustrated in the upcoming screenshot.


![The “invisible” plane as seen from the back of the door](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/906fsgu0ibspvxi37kvo.png)
<figcaption>The “invisible” plane as seen from the back of the door</figcaption>
---

Now create a new `Render Texture` in **Assets/Materials** and name it `DoorPlaneRenderTexture`. Edit the `Render Texture` as follows:


![DoorPlaneRenderTexture (Render Texture)](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/613doduq8bzlc0ibdbf0.png)
<figcaption>DoorPlaneRenderTexture (Render Texture)</figcaption>
---

As you can observe, we have set the `Size` to 1024x2048, which correlates with our plane's approximate dimensions of a 1:2 ratio. This size selection ensures that the texture or material applied to the plane is properly scaled and aligned, reflecting the plane's physical proportions in the virtual environment.

Now create a new `Material` in **Assets/Materials** and name it `DoorPlaneRenderTexture`. Edit the `Material` as follows:


![DoorPlaneRenderTexture (Material)](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4fu8s4iqn9kz2z996hfm.png)
<figcaption>DoorPlaneRenderTexture (Material)</figcaption>
---

Now, choose the `DoorPlaneRenderTexture` as the `Base Map` as seen in the next screenshot.


![Choosing the DoorPlaneRenderTexture as the Base Map](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vuyuzyxqr9tuju3sz9q9.png)
<figcaption>Choosing the DoorPlaneRenderTexture as the Base Map</figcaption>
---

For the final step, assign the `DoorPlaneRenderTexture` material to the `Mesh Renderer` of the `Plane`.


![Assign the DoorPlaneRenderTexture material to the Planes Mesh Renderer](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/q60d93q9yhusdtoj0v83.png)
<figcaption>Assign the DoorPlaneRenderTexture material to the Planes Mesh Renderer</figcaption>
---

Now, let's create a simple scene that will be rendered onto the plane. Before we start, make sure to return to the `SampleScene` and exit the prefab editor.

1. In your hierarchy, create an empty GameObject. You can do this by selecting `Create Empty`. Name this GameObject `DoorPlaneScene`.
2. Ensure the position of `DoorPlaneScene` is set to **X: 0, Y: 0, Z: 0**, unless it's already positioned there.
3. Create another empty GameObject as a child of `DoorPlaneScene` and name it `CameraOffset` . Set the **Y** position of `CameraOffset` to **1.1176** in its Transform properties.
4. To the `CameraOffset` GameObject, add a Camera component. This camera will capture the scene for rendering onto the plane.
5. Add another empty GameObject under `DoorPlaneScene` and name it `Scene`. This GameObject will hold the elements of your rendered scene.
6. Inside `Scene`, create a Cube (**3D Object → Cube**). Position this Cube with its **Z** coordinate set to **5**.
    

These steps set up a basic scene structure, with a camera positioned to capture the scene, and a simple Cube as a visual element. The scene will be rendered from the perspective of the added camera.

The result looks as follows:


![Hierarchy after adding the scene which will be rendered onto our plane](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/z7bg2uhzrkep3g5yj7yi.png)
<figcaption>Hierarchy after adding the scene which will be rendered onto our plane</figcaption>
---

We aim for our new `Camera` to render only the contents of the `Scene` GameObject. To achieve this, navigate to **Layers → Edit Layers...**, as illustrated in the upcoming screenshot. Here, add a new user layer and name it `DoorPlaneScene`. This step is crucial for ensuring that the camera specifically captures the elements within the `Scene`, isolating its view to this particular part of your project for targeted rendering.


![Adding a new user layer DoorPlaneScene](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/n1r8jotz9edpzt3f73d5.png)
<figcaption>Adding a new user layer DoorPlaneScene</figcaption>
---

Now, select the `DoorPlaneScene` GameObject in the hierarchy and change its layer to `DoorPlaneScene`:


![Selecting the DoorPlaneScene layer for our DoorPlaneScene GameObject](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/keh68qsiq7z8j9pquu54.png)
<figcaption>Selecting the DoorPlaneScene layer for our DoorPlaneScene GameObject</figcaption>
---

Confirm by also changing the layer for all children:


![Changing the layer for all children](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w0ekx9ed070t8ktvh56c.png)
<figcaption>Changing the layer for all children</figcaption>
---

Lets configure the new `Camera`. Select our `Main Camera` in the hierarchy and copy the component values as seen in the next screenshot.


![Copy the component values from Main Camera](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/08fyf236xzhoih7qf7k5.png)
<figcaption>Copy the component values from Main Camera</figcaption>
---

Now select the new `Camera` again and paste the component values:


![Paste component values to the new Camera](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3uoarnm5s4wf4v1eu9t0.png)
<figcaption>Paste component values to the new Camera</figcaption>
---

Now, let's configure some additional settings for the new `Camera`:

1. Set the tag of the Camera to `Untagged`.
2. Under the Rendering tab, change the `Culling Mask` to include only the `DoorPlaneScene` and `Ignore Raycast`. This ensures the camera renders only the objects in the `DoorPlaneScene` layer.
3. Assign `DoorPlaneRenderTexture` as the `Output Texture`. This means the camera's view will be rendered to this texture.
4. Set the `Far` value under `Clipping Planes` to **250**. This determines how far the camera can see.
5. Choose `Skybox` as the `Background Type` under the `Environment` tab.
    

After making these changes, your camera should be set up as shown in the following screenshot.


![The configured new Camera](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gwqe9w0faf03gmy40jwc.png)
<figcaption>The configured new Camera</figcaption>
---

Finally, create a prefab from the `DoorPlaneScene` and then remove it from the hierarchy, following the same process we used for the `Reticle` in the `Raycasts` article.

Our plan is to incorporate the `DoorPlaneScene` into our Door prefab. This way, `DoorPlaneScene` will only be rendered when necessary. To do this, edit the `Door` prefab and drag and drop the `DoorPlaneScene` into the root of the prefab. This action is depicted in the upcoming screenshot and ensures that the scene is efficiently integrated with the Door prefab.


![Adding the DoorPlaneScene to our Door prefab](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vi4o4g0jwurps893c3eo.png)
<figcaption>Adding the DoorPlaneScene to our Door prefab</figcaption>
---

Save the prefab and return to the `SampleScene`.

Select the `Main Camera` again and set the `Culling Mask` from `Everthing` to: `Default`, `TransparentFX`, `Ignore Raycast` `Water` and `UI`.


![Selecting the Culling Mask for the Main Camera](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9o8lci5k2ffccluwggba.png)
<figcaption>Selecting the Culling Mask for the Main Camera</figcaption>
---

Drag and drop the prefab `Door` into your scene to investigate the changes we made. The plane now should look as follows:


![Plain rendering the scene we just created](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2457i59dupmr8ack8jrb.png)
<figcaption>Plain rendering the scene we just created</figcaption>
---

As you can observe, we now have our new scene projected onto the plane inside the door.

Next, re-enable the `Door` mesh, which we had disabled in a previous step. However, there's still an issue to address: the scene is static and doesn't adapt to the user's movements. Let's resolve this.

Create a new script and name it `MimicCamera`, then attach it to our newly added `Camera`. The script will be structured as follows:

```csharp
using System.Collections;
using System.Collections.Generic;
using UnityEngine;

namespace Taikonauten.Unity.ArticleSeries
{
    public class MimicCamera : MonoBehaviour
    {
        public float offsetDistance = 30.0f;
        [SerializeField] private GameObject sceneCamera;
        private Camera mainCamera;
        public Transform DoorTransform { get; set; } = null;

        void Awake() {
            mainCamera = Camera.main;
        }

        void Update()
        {
            if (DoorTransform == null) {
                return;
            }

            // Calculate the direction and position offset from the door to the main camera
            Vector3 directionToCamera = mainCamera.transform.position - DoorTransform.position;
            Vector3 sceneCameraPosition = DoorTransform.position + directionToCamera.normalized * offsetDistance;

            sceneCameraPosition.y = sceneCameraPosition.y > 2f ? 2f : sceneCameraPosition.y;

            // Update the position and rotation of the scene camera
            sceneCamera.transform.position = sceneCameraPosition;
            sceneCamera.transform.rotation = Quaternion.LookRotation(DoorTransform.position - sceneCameraPosition);
        }
    }
}

```

Select the required values for the `MimicCamera` script as seen in the following screenshot:


![Select the required values for the MimicCamera script](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xvci914o8ww4bv46ujn2.png)
<figcaption>Select the required values for the MimicCamera script</figcaption>
---

We still need to assign the `DoorTransform` when the `Door` is instantiated. To do this, edit the `MRArticleSeriesController` script and add a private field for the `MimicCamera` script. This modification is shown on line 1 in the following code snippet.

Afterward, assign the `DoorTransform` following our `LookAt` method call. If you need guidance, you can also refer to the code in the `12_ImproveScene` project for a clearer understanding [12_ImproveScene](https://github.com/taikonauten/unity-mr-article-series/tree/main/12_ImproveScene).

```csharp
...

// Make the door face the user after instantiating
doorInstance.transform.LookAt(new Vector3(
    Camera.main.transform.position.x,
    doorInstance.transform.position.y,
    Camera.main.transform.position.z
));

doorInstance.GetComponentInChildren<MimicCamera>().DoorTransform = doorInstance.transform;

...
```

---

## Adding a complex scene behind the door

For the final step, we'll upgrade from the simple cube to a more complex scene. In our example, we utilized the `Free Low Poly Nature Forest` asset, which you can find here: [Free Low Poly Nature Forest](https://assetstore.unity.com/packages/3d/environments/landscapes/free-low-poly-nature-forest-205742). Add the asset to your project.

Find the '`Demo_01` scene from the package within the **Assets/Pure Poly/Free Low Poly Nature Pack/Scenes** directory. Then, drag and drop it into your hierarchy, as demonstrated in the upcoming screenshot.


![Drop the Demo_01 scene into our hierarchy](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cbu9wpnfyxzmv2ojj0fx.png)
<figcaption>Drop the Demo_01 scene into our hierarchy</figcaption>
---

If the `Demo_01` scene looks as follows, we need to replace the shader that is used for the material.


![Demo_01 with an invalid material](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/agu2zwk5vsf11dugb758.png)
<figcaption>Demo_01 with an invalid material</figcaption>
---

Open the material located here: **Assets/Pure Poly/Free Low Poly Nature Pack/Materials/PP\_Standard\_Material**. Edit the material as follows:


![Fixing the invalid material](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e63fdawuv2jcwuqlqqic.png)
<figcaption>Fixing the invalid material</figcaption>
---

1. Set the shader to `Universal Render Pipeline/Lit`.
2. As the `Base Map` choose the file `PP_Color_Palette`.
3. Set `Smoothness` to **0** for the `Metallic Map`.
    

Copy the `Free_Forest` GameObject, then edit our `DoorPlaneScene` prefab. Within the prefab, replace the cube with the `Free_Forest` GameObject, as illustrated in the next screenshot:


![Replacing the cube with the Free_Forest GameObject in our DoorPlaneScene prefab](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/om4ou6sjezi2h2uqjh62.png)
<figcaption>Replacing the cube with the Free_Forest GameObject in our DoorPlaneScene prefab</figcaption>
---

Set the `Transform` of the `Free_Forest` GameObject as follows:

|     | **X** | **Y** | **Z** |
| --- | --- | --- | --- |
| **Position** | 0   | \-3.7 | 0   |
| **Rotation** | 0   | \-135 | 0   |
| **Scale** | 1   | 1   | 1   |


![Set the Transform of the Free_Forest GameObject](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/73tbd1frhf4xotcy2kar.png)
<figcaption>Set the Transform of the Free_Forest GameObject</figcaption>
---

Also ensure, that the layer of `Free_Forest` is set to `DoorPlaneScene`:

![Setting the Free_Forest layer to DoorPlaneScene](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0u6uu7p88y1itluexivc.png)
<figcaption>Setting the Free_Forest layer to DoorPlaneScene</figcaption>
---

Save the prefab and return to your scene. Now remove the `Demo_01` scene from your hierarchy:


![Remove the Demo_01 scene from your hierarchy](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vydp885fiqm87fsxyl06.png)
<figcaption>Remove the Demo_01 scene from your hierarchy</figcaption>
---

This enhancement significantly elevates the visual appeal and immersion of the scene, demonstrating the versatility and potential of using varied assets in your MR environment.

ℹ️ If you find yourself facing any difficulties, remember that you can always refer to or download the code from our accompanying GitHub repository. [12_ImproveScene](https://github.com/taikonauten/unity-mr-article-series/tree/main/12_ImproveScene).

---

## Testing the app

We are now ready to test the final version of our app. Here's what you need to do:

1. Choose `Build and Run` in Unity.
2. Once the app is running, press the trigger on the left controller.
3. You should see a label displaying the message "...Listening...".
4. Say the phrase “open the door”.
5. After a short delay, the animation of the door opening should begin.
6. Feel free to move around and explore what lies behind the door.
    

This test will allow you to experience the full functionality of the app, from voice command recognition to the dynamic opening of the door and the revealing of the scene behind it. Enjoy the immersive experience of exploring the new environment you've created!

👏

{% embed https://player.vimeo.com/video/901578624?h=752a78b014 %}
<figcaption>Video of the app where the door opens when the phrase “open the door“ is recognized</figcaption>
---