SteamVR Overlay with Unity: Controller Input
Tags: unity3d, steamvr, openvr, vr
Published: 2024-06-18 15:01:34

In this part, we will make the watch hidden by default and show it in a few seconds by pressing a controller button.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yj6umot4u9eizsgfkdjw.gif)

There are various ways to get controller input with Unity. This time, we will use [OpenVR Input API (SteamVR Input)](https://github.com/ValveSoftware/openvr/wiki/SteamVR-Input).

## Prerequisite setting

Open the SteamVR Settings.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t32iq53hn1p5nsfw9xd1.png)

Toggle **Advanced Settings**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/549oreg9jk784t4773m2.png)

Toggle **Developer > Enable debugging options in the input binding user interface** to **On**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rigizcffoij3wiw2xntj.png)

## Create action manifest

SteamVR Input uses abstracted actions instead of direct access to the physical buttons.
The developer predefines actions and may create default mappings for each controller. Users can assign physical buttons to the actions in the SteamVR Input setting window.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ffx83rfwp4ccllvbjtm7.png)

First, create the action manifest that defines actions in JSON format. We use the Unity SteamVR Plugin GUI tool to generate the action manifest easily.

### Generate action manifest

Select Unity menu **Window > SteamVR Input**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/m2jqigt1n9ewfr3h6ojt.png)

It asks you whether you want to use a sample action manifest file.
This time we will make up from the first so select No.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pifrl2hh0aou8fgewio9.png)

Input the action sets name to **“Watch”**, and select the below dropdown to **per hand**. An application can have multiple action sets.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/stzacsek8vkgr7kpaddb.png)

**Per hand** means it can map different actions to the buttons of the left and right-hand controller separately.
**Mirrored** means it can map actions to only one controller and the other buttons are mirrored to the mapping.

Click the **NewAction** name that is in the **In** box of the **Actions** section.
The **“Action Details”** is shown to the right. Change the **Name** to **“WakeUp”**.
Click the **“Save and generate”** button at the left bottom to generate an action manifest file.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s51rgvet3nafw0461u62.png)

It will generate a new action manifest file as **StreamingAssets/SteamVR/actions.json**.

```json
{
  "actions": [
    {
      "name": "/actions/Watch/in/WakeUp",
      "type": "boolean"
    }
  ],
  "action_sets": [
    {
      "name": "/actions/Watch",
      "usage": "leftright"
    }
  ],
  "default_bindings": [],
  "localization": []
}
```

## Create default binding

Click the right bottom **Open binding UI** button.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tn2re0jybmn5y4meaww9.png)

If any VR controller is not shown, check your HMD is recognized from SteamVR.

Click the **“Create New Binding”**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t03057rs5el9ociegwnt.png)

Set the Y button to show the watch for a few seconds. If your controller doesn’t have the Y button, use another button instead.

Click the **+** button to the right of the **Y Button**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eh80pb6voosgjzxl61lt.png)

Select **BUTTON**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2l31lthf3uhv8rmlx71j.png)

Select **None** to the right of the Click, then select **wakeup** action.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jz5krz7saktu7qsi2s3n.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3m058df3manpz6fztkdw.png)

Click the left bottom check icon to save changes.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ridp4mr8ad3584mf78rr.png)

Click the right bottom **Replace Default Binding**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w4xv1v4sobexu5hxp2ss.png)

Click **Save**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/swt8f3p74vd2v7k3y8tt.png)

At this point, a new default binding file is added to the action manifest.

`actions.json`
```json
{
   "action_sets" : [
      {
         "name" : "/actions/Watch",
         "usage" : "single"
      }
   ],
   "actions" : [
      {
         "name" : "/actions/Watch/in/WakeUp",
         "type" : "boolean"
      }
   ],
   "default_bindings" : [
      {
+        "binding_url" : "application_generated_unity_retryoverlay_exe_binding_oculus_touch.json",
+        "controller_type" : "oculus_touch"
      }
   ],
   "localization" : []
}
```

The default binding file is generated into the same directory to actions.json. These files will be included in the build for users.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ffrm5hluq2zw6gcz99qn.png)

Click the **<- BACK** button at the upper left, then you see the new setting is active.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/julogqv193yaue21jetr.png)

Now we’ve made the default binding. Close the SteamVR Input window.

Also, close the Unity SteamVR Input window.
At this time, it asks you to save the change, so click **Close**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hwnfj46azm4q8cskgigt.png)

In this tutorial environment, clicking the Save button overwrites the action manifest as an empty **default_bindings**.
(There is no problem with clicking Save when we open the binding setting window next time.)

---

### How about other controller bindings?

You can create other controller’s default bindings by clicking the controller name on the right side.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/txjz55r3v1x0suwza0yh.png)

Other controllers’ default bindings are added to default_bindings param of action manifest.

```diff
{
    "action_sets" : [
       {
          "name" : "/actions/NewSet",
          "usage" : "leftright"
       }
    ],
    "actions" : [
       {
          "name" : "/actions/NewSet/in/NewAction",
          "type" : "boolean"
       }
    ],
    "default_bindings" : [
      {
         "binding_url" : "application_generated_unity_steamvr_inputbindingtest_exe_binding_oculus_touch.json",
         "controller_type" : "oculus_touch"
      },
+     {
+        "binding_url" : "application_generated_unity_steamvr_inputbindingtest_exe_binding_vive_controller.json",
+        "controller_type" : "vive_controller"
+     }
   ],
   "localization" : []
}
```

You don’t necessarily have to create other controller default bindings because SteamVR automatically remaps actions for other controllers if at least one default binding exists.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9e7al8ucfkf8vctwfcow.png)

---

### Create script

Create a new script `InputController.cs` into `Scripts` folder.
We will add the controller input related code to this file.

On hierarchy, **right click > Create Empty** to create an empty object, change the object name to **InputController**.
Drag `InputController.cs` from the project window to the **InputController** object.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9o5rzzptco92fysdyp4l.png)

### Initialize and cleanup OpenVR

Copy the below code to **InputController.cs**.

```cs
using System;
using UnityEngine;
using Valve.VR; 

public class InputController : MonoBehaviour
{
    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();
    }

    private void Destroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }
}
```

## Set action manifest path

Set the action manifest path with [SetActionManifestPath()](https://valvesoftware.github.io/steamvr_unity_plugin/api/Valve.VR.CVRInput.html#Valve_VR_CVRInput_SetActionManifestPath_System_String_) at launch. (read the [wiki](https://github.com/ValveSoftware/openvr/wiki/SteamVR-Input#api-documentation) for details)
The action manifest is generated as **StreamingAssets/SteamVR/actions.json** so we set this.

```diff
using System;
using UnityEngine;
using Valve.VR; 

public class InputController : MonoBehaviour
{
    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();

+       var error = OpenVR.Input.SetActionManifestPath(Application.streamingAssetsPath + "/SteamVR/actions.json");
+       if (error != EVRInputError.None)
+       {
+           throw new Exception("Failed to set action manifest path: " + error);
+       }
    }

    private void Destroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }
}
```

## Get action set handle

An application can have some action sets that have multiple actions. We use an action set handle to determine which action set is used.
Get action set handle with [GetActionSetHandle()](https://valvesoftware.github.io/steamvr_unity_plugin/api/Valve.VR.CVRInput.html#Valve_VR_CVRInput_GetActionSetHandle_System_String_System_UInt64__).

The action set is set with a string **/actions/[action_set_name]**.
This time, it’s **/actions/Watch**.

```diff
public class InputController : MonoBehaviour
{
+   ulong actionSetHandle = 0;

    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();

        var error = OpenVR.Input.SetActionManifestPath(Application.streamingAssetsPath + "/SteamVR/actions.json");
        if (error != EVRInputError.None)
        {
            throw new Exception("Failed to set action manifest path: " + error);
        }

+       error = OpenVR.Input.GetActionSetHandle("/actions/Watch", ref actionSetHandle);
+       if (error != EVRInputError.None)
+       {
+           throw new Exception("Failed to get action set /actions/Watch: " + error);
+       }
    }

    private void Destroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }
}
```

## Get action handle

Next, get the action handle with [GetActionHandle()](https://valvesoftware.github.io/steamvr_unity_plugin/api/Valve.VR.CVRInput.html#Valve_VR_CVRInput_GetActionHandle_System_String_System_UInt64__).
Action is set with **/actions/[action_set_name]/in/[action_name]**.
This time, action is **/actions/Watch/in/Wakeup**.

```diff
public class InputController : MonoBehaviour
{
    ulong actionSetHandle = 0;
+   ulong actionHandle = 0;

    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();

        var error = OpenVR.Input.SetActionManifestPath(Application.streamingAssetsPath + "/SteamVR/actions.json");
        if (error != EVRInputError.None)
        {
            throw new Exception("Faild to set action manifest path: " + error);
        }

        error = OpenVR.Input.GetActionSetHandle("/actions/Watch", ref actionSetHandle);
        if (error != EVRInputError.None)
        {
            throw new Exception("Failed to get action set /actions/Watch: " + error);
        }

+       error = OpenVR.Input.GetActionHandle($"/actions/Watch/in/WakeUp", ref actionHandle);
+       if (error != EVRInputError.None)
+       {
+           throw new Exception("Faild to get action /actions/Watch/in/WakeUp: " + error);
+       }
    }

    private void Destroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }
}
```

## Update action state

Update each action state at the first of each frame.

### Prepare action set to update

We use an action set to specify which actions will be updated.
Pass target action sets as an array of [VRActiveActionSet_t](https://valvesoftware.github.io/steamvr_unity_plugin/api/Valve.VR.VRActiveActionSet_t.html) type.
This time, the action set is only * so we make a single-element array `VRActionActiveSet_t[]`.

```diff
private void Start()
{
    OpenVRUtil.System.InitOpenVR();

    var error = OpenVR.Input.SetActionManifestPath(Application.streamingAssetsPath + "/SteamVR/actions.json");
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to set action manifest path: " + error);
    }

    error = OpenVR.Input.GetActionSetHandle("/actions/Watch", ref actionSetHandle);
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to get action set /actions/Watch: " + error);
    }

    error = OpenVR.Input.GetActionHandle($"/actions/Watch/in/WakeUp", ref actionHandle);
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to get action /actions/Watch/in/WakeUp: " + error);
    }
}

+ private void Update()
+ {
+     // Action set list to update
+     var actionSetList = new VRActiveActionSet_t[]
+     {
+         // This time, Watch action only.
+         new VRActiveActionSet_t()
+         {
+             // Pass Watch action set handle
+             ulActionSet =  actionSetHandle,
+             ulRestrictedToDevice = OpenVR.k_ulInvalidInputValueHandle,
+         }
+     };
+ }

...
```

The second argument `activeActionSize` is the byte size of `VRActiveActionSet_t` the struct.

## Get action value

We have updated the action state. Next, we will get the action value.
The function used to retrieve action value varies depending on the action type.

### GetDigitalActionData()

This gets an on/off value likes the button is pushed or not.

### GetAnalogActionData()

This gets analog values like thumb stick direction or how much trigger is pulled.

### GetPoseActionData()

This gets a pose like controller position and rotation.

This time, we want to get the button on/off value so use `GetDigitalActionData()`.
Get the action value with WakeUp action handle we prepared.

```diff
private void Update()
{
    var actionSetList = new VRActiveActionSet_t[]
    {
        new VRActiveActionSet_t()
        {
            ulActionSet =  actionSetHandle,
            ulRestrictedToDevice = OpenVR.k_ulInvalidInputValueHandle,
        }
    };
    
    var activeActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VRActiveActionSet_t));
    var error = OpenVR.Input.UpdateActionState(actionSetList, activeActionSize);
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to update action state: " + error);
    }
    
+   var result = new InputDigitalActionData_t();
+   var digitalActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(InputDigitalActionData_t));
+   error = OpenVR.Input.GetDigitalActionData(actionHandle, ref result, digitalActionSize, OpenVR.k_ulInvalidInputValueHandle);
+   if (error != EVRInputError.None)
+   {
+       throw new Exception("Failed to get WakeUp action data: " + error);
+   }
}
```

The 3rd argument `digitalActionSize` is the byte size of `InputDigitalActionData_t` structure.
The 4th argument `ulRestrictToDevice` is generally not used so set `OpenVR.k_ulInvalidInputValueHandle`.

The returned value type is [InputDigitalActionData_t](https://valvesoftware.github.io/steamvr_unity_plugin/api/Valve.VR.InputDigitalActionData_t.html).

```cs
struct InputDigitalActionData_t
{
    bool bActive;
    VRInputValueHandle_t activeOrigin;
    bool bState;
    bool bChanged;
    float fUpdateTime;
};
```

The action on/off state is set to `bState`.
`bChanged` is `true` at the frame where the action state is changed.

Let’s detect the action.

```diff
private void Update()
{
    var actionSetList = new VRActiveActionSet_t[]
    {
        new VRActiveActionSet_t()
        {
            ulActionSet =  actionSetHandle,
            ulRestrictedToDevice = OpenVR.k_ulInvalidInputValueHandle,
        }
    };
    
    var activeActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VRActiveActionSet_t));
    var error = OpenVR.Input.UpdateActionState(actionSetList, activeActionSize);
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to update action state: " + error);
    }
    
    var result = new InputDigitalActionData_t();
    var digitalActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(InputDigitalActionData_t));
    error = OpenVR.Input.GetDigitalActionData(actionHandle, ref result, digitalActionSize, OpenVR.k_ulInvalidInputValueHandle);
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to get WakeUp action data: " + error);
    }

+   if (result.bState && result.bChanged)
+   {
+       Debug.Log("WakeUp is executed");
+   }
}
```

Run the program and push the Y button, then the message is logged to the Unity console.

## Create event handler

### Notify to Unity Event

Add `UnityEvent` into **InputController.cs** to notify WakeUp action, then call it with `Invoke()`.

```diff
using System;
using UnityEngine;
using Valve.VR;
+ using using UnityEngine.Events;

public class InputController : MonoBehaviour
{
+   public UnityEvent OnWakeUp; 
    
    private ulong actionSetHandle = 0;
    private ulong actionHandle = 0;

    // code omit.

    private void Update()
    {
        var actionSetList = new VRActiveActionSet_t[]
        {
            new VRActiveActionSet_t()
            {
                ulActionSet =  actionSetHandle,
                ulRestrictedToDevice = OpenVR.k_ulInvalidInputValueHandle,
            }
        };
        
        var activeActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VRActiveActionSet_t));
        var error = OpenVR.Input.UpdateActionState(actionSetList, activeActionSize);
        if (error != EVRInputError.None)
        {
            throw new Exception("Failed to update action state: " + error);
        }
        
        var result = new InputDigitalActionData_t();
        var digitalActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(InputDigitalActionData_t));
        error = OpenVR.Input.GetDigitalActionData(actionHandle, ref result, digitalActionSize, OpenVR.k_ulInvalidInputValueHandle);
        if (error != EVRInputError.None)
        {
            throw new Exception("Failed to get WakeUp action data: " + error);
        }

        if (result.bState && result.bChanged)
        {
-           Debug.Log("WakeUp is executed");
+           OnWakeUp.Invoke();
        }
    }
```

### Attach event handler

Add method into `WatchOverlay.cs` that will be called when `WakeUp` action is executed.

```diff
public class WatchOverlay : MonoBehaviour
{

    ...

+   public void OnWakeUp()
+   {
+       // Show watch.
+   }
}
```

In the hierarchy, open `InputController` inspector, then set `WatchOverlay.OnWakeUp()` to `OnWakeUp()` field.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/92wrvf65j3rp9p6raomq.png)

## Add code to show/hide the watch

Update **WatchOverlay.cs** to show the watch only when `WakeUp` action is executed.

### Hide watch by default

```diff
private void Start()
{
    OpenVRUtil.System.InitOpenVR();
    overlayHandle = Overlay.CreateOverlay("WatchOverlayKey", "WatchOverlay");
    
    Overlay.FlipOverlayVertical(overlayHandle);
    Overlay.SetOverlaySize(overlayHandle, size);
-   Overlay.ShowOverlay(overlayHandle);
}
```

### Show watch by WakeUp action

Show overlay when the WakeUp action is executed.

```diff
public void OnWakeUp()
{
+   Overlay.ShowOverlay(overlayHandle);
}
```

### Add hide method

Add `HideOverlay()` to **OpenVRUtil.cs**.

```diff
public static void ShowOverlay(ulong handle)
{
    var error = OpenVR.Overlay.ShowOverlay(handle);
    if (error != EVROverlayError.None)
    {
        throw new Exception("Failed to show overlay: " + error);
    }
}

+ public static void HideOverlay(ulong handle)
+ {
+     var error = OpenVR.Overlay.HideOverlay(handle);
+     if (error != EVROverlayError.None)
+     {
+         throw new Exception("Failed to hide overlay: " + error);
+     }
+ }

public static void SetOverlayFromFile(ulong handle, string path)
{
    var error = OpenVR.Overlay.SetOverlayFromFile(handle, path);
    if (error != EVROverlayError.None)
    {
        throw new Exception("Failed to draw image file: " + error);
    }
}
```

### Hide watch after three seconds

This time, we use Unity Coroutine to wait three seconds.

`WatchOverlay.cs`
```diff
using System;
+ using System.Collections;
using UnityEngine;
using Valve.VR;
using OpenVRUtil;

public class WatchOverlay : MonoBehaviour
{
    public Camera camera;
    public RenderTexture renderTexture;
    public ETrackedControllerRole targetHand = ETrackedControllerRole.RightHand;
    private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;
+   private Coroutine sleepCoroutine;

    ...

    public void OnWakeUp()
    {
        Overlay.ShowOverlay(overlayHandle);
+       if (sleepCoroutine != null)
+       {
+           StopCoroutine(sleepCoroutine);
+       }
+       sleepCoroutine = StartCoroutine(Sleep());
    }

+   private IEnumerator Sleep()
+   {
+       yield return new WaitForSeconds(3);
+       Overlay.HideOverlay(overlayHandle);
+   }
}
```

Run the program, and check the watch is shown for three seconds when the Y button is pushed.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kr69g2zdqhm0pupvl5w3.gif)

## Complete 🎉

The watch application is completed.
This tutorial ends here, so the final code organization is up to you.

Look back at what we touched on in this tutorial.

- Initialize OpenVR
- Create overlay
- Draw image file
- Change overlay size and position
- Follow devices
- Draw camera output
- Create dashboard overlay
- Process events
- Controller input

We learned the basics of overlay application development.
OpenVR has many more APIs, so how about trying to make original tools?

The next page is additional information for more details.