Unity MR Part 7: Raycasts
Tags: unity3d, mixedreality, tutorial, beginners
Published: 2024-01-11 08:51:03

👀 Stumbled here on accident? Start with the [introduction](https://dev.to/taikonauten/part-0-introduction-56fl)!

📚 This article will guide you through the concept of `Raycasts` in MR, explaining their significance and functionality. Additionally, it will provide a comprehensive tutorial on implementing `Raycasts` within the Unity platform.

---

ℹ️ If you find yourself facing any difficulties, remember that you can always refer to or download the code from our accompanying [GitHub repository](https://github.com/taikonauten/unity-mr-article-series/tree/main/07_Raycasts)

---

The aim of this article is to conduct hit tests on planes we created previously, using our controller. We'll then, in the next article, place an Anchor at the identified position. This process is fundamental for accurately positioning virtual objects in relation to the real world.

ℹ️ `Raycasts` in Unity and MR are used to determine if and where a virtual ray intersects with objects in the game world or MR environment. They're crucial for interaction, allowing the detection of objects, surfaces, or points in 3D space. In MR, `Raycasts` can be used for selecting or interacting with virtual objects overlaid on the real world.

---

## Creating a custom Reticle

ℹ️ In AR, a reticle is a visual indicator, often a small pointer or dot, used to help users accurately target and interact with virtual elements overlaid onto the real world. It provides a focal point for interaction, indicating where actions like tapping, selecting, or placing virtual objects will occur. Reticles are especially useful in AR experiences to improve precision and user experience in environments where direct touch interactions are not possible.

To create a custom `Reticle`, we'll utilize the asset provided in the link. Download the file directly into your Assets folder. For the purposes of this article, we'll organize it in a subfolder named **3DModels**. Alternatively, you can download the asset to a different folder and then drag and drop it into the **3DModels** folder within Unity.

⬇️ [arrow.fbx](https://github.com/taikonauten/unity-mr-article-series/blob/main/blog/arrow.fbx)

Let's begin by creating a visual indicator, a `Reticle`, which will appear when we hover over any of the planes managed by the `AR Plane Manager`.

1.  First, create a new empty GameObject via **Create → Empty** in your hierarchy and name it 'Reticle'.
2.  Next, drag and drop this GameObject into your Prefabs folder, then remove the `Reticle` GameObject from the hierarchy.
3.  Double-click the prefab `Reticle` you just created to start editing it.
4.  Make sure the `Reticle` GameObject's position is set to **X: 0, Y: 0, Z: 0**.
5.  Then, drag and drop the **.fbx** file from your **3DModels** folder onto the `Reticle` GameObject in your hierarchy view.
6.  Make sure the `arrow` GameObject's position is set to **X: 0, Y: 0, Z: 0**.
7.  Finally, adjust the position, rotation, and scale of the Cone and Cylinder to suit your needs.
    

| **Cone** | **X** | **Y** | **Z** |
| --- | --- | --- | --- |
| **Position** | 0   | 0   | \-0.1 |
| **Rotation** | 90  | 180 | 0   |
| **Scale** | 0.2 | 0.2 | 0.2 |

| **Cylinder** | **X** | **Y** | **Z** |
| --- | --- | --- | --- |
| **Position** | 0   | 0   | \-0.3 |
| **Rotation** | 90  | 180 | 0   |
| **Scale** | 0.2 | 0.2 | 0.2 |

With the specified values, you will achieve the Prefab configuration as shown in the upcoming screenshot. If you are designing a custom Reticle, remember to modify only the position, rotation, and scale of the Mesh components. It's important to avoid altering these values for the `Reticle` GameObject itself to maintain its intended functionality and alignment.


![Custom Reticle after updating Position, Rotation and Scale](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5tlnbphva8qb24deuy4s.png)
<figcaption>Custom Reticle after updating Position, Rotation and Scale</figcaption>
---

## Updating our AR Default Plane

We aim for the `Reticle` to be visible only when hovering over one of our `AR Default Plane`s. To achieve this, modify the `AR Default Plane` Prefab (by double clicking the `AR Default Plane` in the **Assets/Prefabs** folder), add the `XR Simple Interactable` Script and select the newly created `Reticle` Prefab as the `Custom Reticle`.


![Setting the custom Reticle on the AR Default Plane](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/m2k4v074ixhdevftqrb8.png)
<figcaption>Setting the custom Reticle on the AR Default Plane</figcaption>
---

This Prefab will now be instantiated at the position where our ray intersects with any of the planes registered in our `AR Plane Manager`. When you run your app with these settings, you'll observe that the custom `Reticle` is now accurately indicating the plane being hovered over.


![The custom Reticle in MR](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ypuekxctud1cy1p8h5z4.png)
<figcaption>The custom Reticle in MR</figcaption>
---

## Improve the appearance of the XR Interactor Line Visual

In this step, we'll adjust the color gradients for `Valid`, `Invalid`, and `Blocked` states in the `XR Interactor Line Visual`. This modification will change the line's color based on interaction: it will display one color when hovering over a valid target (in our scenario, detected planes) and another color when no valid target is hovered over (anything other than planes). This visual cue enhances the user's understanding of interaction possibilities within the AR environment.

You can use the following color gradients as seen in the upcoming screenshots or use your own custom gradients.


![XR Interactor Line Visual with updated color gradients](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8tiwztmvcavtrdlctmlm.png)
<figcaption>XR Interactor Line Visual with updated color gradients</figcaption>
---

The `Valid` color gradients with a starting color of `#00A0FF`.


![Valid Color Gradient](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/oe5bm3wuz8e7votjlkqu.png)
<figcaption>Valid Color Gradient</figcaption>
---

The `Invalid` color gradients with a starting color of `#FFFFFF`.


![Invalid Color Gradient](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9edo7hqd0p42rwd8h2fq.png)
<figcaption>Invalid Color Gradient</figcaption>
---

The `Blocked` color gradients with a starting color of `#FFEB04`.


![Blocked Color Gradient](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/n0wc3tci5x7ssk84pxxq.png)
<figcaption>Blocked Color Gradient</figcaption>
---

Ensure that you also apply the same values to the `Right Controller`.

That concludes the adjustments to the color gradients of the `XR Interactor Line Visual`. We'll examine the outcome of these changes at the end of this article, providing a comprehensive view of how these modifications enhance the visual feedback in our application.

---

## Improve appearance of the Reticle

For the final visual enhancement, let's create a new Material for the custom `Reticle`. Go ahead and make a new Material in your **Assets/Materials** folder and name it ‘Reticle’. You can either use the following suggested values or select your own based on your specific needs. This step will ensure that the custom `Reticle` has a distinctive and appropriate appearance in your application.


![The new Reticle Material](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v0hxdslgtcicr88cag9b.png)
<figcaption>The new Reticle Material</figcaption>
---

![Base Map color for the Reticle Material](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ydwe1bphodv8i3kxx92c.png)
<figcaption>Base Map color for the Reticle Material</figcaption>
---

Edit the `Reticle` Prefab by double-clicking it in **Assets/Prefabs**. Now, drag and drop the new Material onto both meshes within the `Reticle` Prefab (Cone and Cylinder). Once you've done this, your `Reticle` Prefab should resemble the appearance shown in the upcoming screenshot.

![Retacle Prefab with the updated Material](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/u2ks4rha5m0a25ut2scp.png)
<figcaption>Retacle Prefab with the updated Material</figcaption>
---


![Retacle Prefab with the updated Material in our app](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w6kdv7nl5d2232gg389w.png)
<figcaption>Retacle Prefab with the updated Material in our app</figcaption>
---

## Reacting to a OnButtonPressed Event and check if the Ray hits a Plane

For the final step in this article we dive a bit into code. Create a Script named `MRArticleSeriesController.cs` in your **Assets/Scripts** folder and use the following code.

```csharp
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.InputSystem;
using UnityEngine.XR.Interaction.Toolkit;

namespace Taikonauten.Unity.ArticleSeries
{
    public class MRArticleSeriesController : MonoBehaviour
    {
        public ActionBasedController controller;
        public InputActionReference buttonAction;
        public XRRayInteractor rayInteractor;

        void OnEnable()
        {
            Debug.Log("MRArticleSeriesController -> OnEnable()");
            buttonAction.action.performed += OnButtonPressed;
        }

        void OnDisable()
        {
            Debug.Log("MRArticleSeriesController -> OnDisable()");
            buttonAction.action.performed -= OnButtonPressed;
        }

        private void OnButtonPressed(InputAction.CallbackContext context)
        {
            Debug.Log("MRArticleSeriesController -> OnButtonPressed()");

            if (rayInteractor.TryGetCurrent3DRaycastHit(out RaycastHit hit))
            {
                Debug.Log(hit);
            }
        }
    }
}

```

Drag and drop the script onto your `XR Origin (XR Rig)`. Then, as shown in the upcoming screenshot, select an `ActionBasedController`, `InputActionReference`, and `XRRayInteractor`. This step integrates the script with the XR Rig, ensuring that the controller, input actions, and ray interactions are properly configured.


![Adding the MRArticleSeriesController script to our XR Origin (XR Rig) GameObject](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g69iutxmlf0aqxwqm09q.png)
<figcaption>Adding the MRArticleSeriesController script to our XR Origin (XR Rig) GameObject</figcaption>
---

Let's proceed to build and deploy the app onto our headset to see the results. Navigate to **File → Build Settings** and, as depicted in the upcoming screenshot, enable the `Development Build` option under the `Android` section.


![Enable Development Build in Build Settings](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jcryfro3avg4bhu5js7l.png)
<figcaption>Enable Development Build in Build Settings</figcaption>
---

To ensure proper functionality, it's necessary to disable `Vulkan` in the `Graphics APIs` in the Player section. Navigate to **Edit -> Project Settings**, and as illustrated in the following screenshot, remove `Vulkan` from the list of enabled Graphics APIs. This adjustment is crucial for compatibility.

As of Unity version `2023.2.3f` and `Unity OpenXR: Meta 1.0.1`, creating a development build with Vulkan enabled leads to crashes. This known issue highlights the importance of avoiding Vulkan in your development builds with these specific versions to ensure stability and prevent unexpected crashes.


![Removing the Vulkan Graphics API from Graphics APIs](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/11h5m44gf8lu2o94irjc.png)
<figcaption>Removing the Vulkan Graphics API from Graphics APIs</figcaption>
---

Select `Build and Run` to test the application. Since we have configured only the Right Controller, which suffices for our purposes, you should observe the following console output when the `Trigger` button is pressed.

To view the output from the build on your headset, you need to choose the connected Oculus Quest in your Console window.


![Selecting the Meta Quest 3 in Console](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b10exali4zh4kmzizpsy.png)
<figcaption>Selecting the Meta Quest 3 in Console</figcaption>
---

As demonstrated in the upcoming screenshot, the `Debug.Log` statement generates the expected output. This visual confirmation in the screenshot effectively showcases that the application is functioning as intended, aligning with our anticipated results.

1.  AndroidPlayer "Oculus\_Quest" Player -> OnButtonPressed()
2.  AndroidPlayer "Oculus\_Quest" UnityEngine.RaycastHit
    

![The debug logs after pressing the Trigger button of the right controller](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fcr50yefkiqbi7pu4ol2.png)
<figcaption>The debug logs after pressing the Trigger button of the right controller</figcaption>
---

## Next article

In our next article, we'll be focusing on an essential aspect of AR development: `Planes` and the `AR Plane Manager` in Unity. Planes are fundamental to understanding and interacting with the real world in AR applications. We will explore how the `AR Plane Manager` detects and manages these planes, allowing for the placement of virtual objects on flat surfaces like floors, tables, or walls.