SteamVR Overlay with Unity: Change Size and Position
Tags: unity3d, steamvr, openvr, vr
Published: 2024-06-16 19:04:04

## Change overlay size
Pass width [SetOverlayWidthInMeters()](https://valvesoftware.github.io/steamvr_unity_plugin/api/Valve.VR.CVROverlay.html#Valve_VR_CVROverlay_SetOverlayWidthInMeters_System_UInt64_System_Single_) to set an overlay size (read the wiki for details). The width unit is meters. Height is automatically calculated based on the image aspect ratio. The default width is 1 m.
Change the overlay width to 0.5 m.

```diff
private void Start()
{        
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    SetOverlayFromFile(overlayHandle, filePath);

+   var error = OpenVR.Overlay.SetOverlayWidthInMeters(overlayHandle, 0.5f);
+   if (error != EVROverlayError.None)
+   {
+       throw new Exception("Failed to set overlay size: " + error);
+   }

    ShowOverlay(overlayHandle);
}
```

Run the program, the overlay should be shown as half size.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a99wgyw6zay79ph10lhz.png)

## Set overlay absolute position

Let’s display the overlay at the absolute position in the VR space with [SetOverlayTransformAbsolute()](https://valvesoftware.github.io/steamvr_unity_plugin/api/Valve.VR.CVROverlay.html#Valve_VR_CVROverlay_SetOverlayTransformAbsolute_System_UInt64_Valve_VR_ETrackingUniverseOrigin_Valve_VR_HmdMatrix34_t__). (read the [wiki](https://github.com/ValveSoftware/openvr/wiki/IVROverlay::SetOverlayTransformAbsolute) for details)

### Prepare position and rotation

We will put the overlay at the absolute position that moved to 2 m in the Y-axis (upper direction), and 3 m in the Z-axis (forward direction). Also, rotate it 45 degrees around the Z-axis.

First, prepare the position and rotation with `Vector3` and `Quaternion`.

```diff
private void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    SetOverlayFromFile(overlayHandle, filePath);
    
    var error = OpenVR.Overlay.SetOverlayWidthInMeters(overlayHandle, 0.5f);
    if (error != EVROverlayError.None)
    {
        throw new Exception("Failed to set overlay size: " + error);
    }

+   var position = new Vector3(0, 2, 3);
+   var rotation = Quaternion.Euler(0, 0, 45);
    
    ShowOverlay(overlayHandle);
}
```

### Create transformation matrix

Overlay position is calculated by multiplying an origin and transformation matrix.
The origin is the base position like the center of the play area that is defined in [ETrackingUniverseOrigin](https://valvesoftware.github.io/steamvr_unity_plugin/api/Valve.VR.ETrackingUniverseOrigin.html).
The transformation matrix is represented as [HmdMatrix34_t](https://valvesoftware.github.io/steamvr_unity_plugin/api/Valve.VR.HmdMatrix34_t.html) type.

```diff
private void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    SetOverlayFromFile(overlayHandle, filePath);
    
    var error = OpenVR.Overlay.SetOverlayWidthInMeters(overlayHandle, 0.5f);
    if (error != EVROverlayError.None)
    {
        throw new Exception("Failed to set overlay size: " + error);
    }

    var position = new Vector3(0, 2, 3);
    var rotation = Quaternion.Euler(0, 0, 45);
+   var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
+   var matrix = rigidTransform.ToHmdMatrix34();

    ShowOverlay(overlayHandle);
}
```

### Change overlay position
Pass the transformation matrix to `SetOverlayTransformAbsolute()` to change the overlay position.

```diff
private void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    SetOverlayFromFile(overlayHandle, filePath);
    
    var error = OpenVR.Overlay.SetOverlayWidthInMeters(overlayHandle, 0.5f);
    if (error != EVROverlayError.None)
    {
        throw new Exception("Failed to set overlay size: " + error);
    }

    var position = new Vector3(0, 2, 3);
    var rotation = Quaternion.Euler(0, 0, 45);
    var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
    var matrix = rigidTransform.ToHmdMatrix34();
+   error = OpenVR.Overlay.SetOverlayTransformAbsolute(overlayHandle, ETrackingUniverseOrigin.TrackingUniverseStanding, ref matrix);
+   if (error != EVROverlayError.None)
+   {
+       throw new Exception("Failed to set overlay position: " + error);
+   }

    ShowOverlay(overlayHandle);
    ...
```

### Arguments of SetOverlayTransformAbsolute()

`SetOverlayTransformAbsolute()` has three arguments.

The 1st is the overlay handle.

The 2nd `ETrackingUniverseOrigin.TrackingUniverseStanding` means that the floor center of the play area is the origin of the transform.
Otherwise, `ETrackingUniverseOrigin.TrackingUniverseSeated` uses the position that the user has reset as the seated position.

The 3rd, `ref matrix` is a reference to the transformation matrix.

## Check the program

Run the program. The overlay position should be changed.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/y54t7zmt2k63603k8w8x.png)
*Move upward to 2 m, forward to 3 m, and rotate 45 degrees in the Z-axis from the play area origin*

---

### Optional: Left-handed and Right-handed system

Unity uses a left-handed system, and OpenVR uses a right-handed system.
In OpenVR, +y is up, +x is right, and -z is forward.
For now, we don’t need to pay attention to the difference because the `SteamVR_Utils.RigidTransform` internally converts the two systems.
However, when you make the transformation matrix manually, be careful that the Z-axis and the rotation directions are reversed.

---

## Organize code
### Set size
Move the size setting code into `SetOverlaySize()`.

```diff
public class WatchOverlay : MonoBehaviour
{
    ...

    private void Start()
    {
        InitOpenVR();
        overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");
        
        var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
        SetOverlayFromFile(overlayHandle, filePath);

-       var error = OpenVR.Overlay.SetOverlayWidthInMeters(overlayHandle, 0.5f);
-       if (error != EVROverlayError.None)
-       {
-           throw new Exception("Failed to set overlay size: " + error);
-       }
+       SetOverlaySize(overlayHandle, 0.5f);

        var position = new Vector3(0, 2, 3);
        var rotation = Quaternion.Euler(0, 0, 45);
        var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
        var matrix = rigidTransform.ToHmdMatrix34();
        error = OpenVR.Overlay.SetOverlayTransformAbsolute(overlayHandle, ETrackingUniverseOrigin.TrackingUniverseStanding, ref matrix);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to set overlay position: " + error);
        }
    
        ShowOverlay(overlayHandle);
    }

    ...

+   private void SetOverlaySize(ulong handle, float size)
+   {
+       var error = OpenVR.Overlay.SetOverlayWidthInMeters(handle, size);
+       if (error != EVROverlayError.None)
+       {
+           throw new Exception("Failed to set overlay size: " + error);
+       }
+   }
}
```

### Set position
Move the position setting into `SetOverlayTransformAbsolute()`.

```diff
public class WatchOverlay : MonoBehaviour
{
    ...

    private void Start()
    {
        InitOpenVR();
        overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");
        
        var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
        SetOverlayFromFile(overlayHandle, filePath);

        SetOverlaySize(overlayHandle, 0.5f);

        var position = new Vector3(0, 2, 3);
        var rotation = Quaternion.Euler(0, 0, 45);
-       var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
-       var matrix = rigidTransform.ToHmdMatrix34();
-       error = OpenVR.Overlay.SetOverlayTransformAbsolute(overlayHandle, ETrackingUniverseOrigin.TrackingUniverseStanding, ref matrix);
-       if (error != EVROverlayError.None)
-       {
-           throw new Exception("Failed to set overlay position: " + error);
-       }
+       SetOverlayTransformAbsolute(overlayHandle, position, rotation);

        ShowOverlay(overlayHandle);
    }

    ...

+   private void SetOverlayTransformAbsolute(ulong handle, Vector3 position, Quaternion rotation)
+   {
+       var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
+       var matrix = rigidTransform.ToHmdMatrix34();
+       var error = OpenVR.Overlay.SetOverlayTransformAbsolute(handle, ETrackingUniverseOrigin.TrackingUniverseStanding, ref matrix);
+       if (error != EVROverlayError.None)
+       {
+           throw new Exception("Failed to set overlay position: " + error);
+       }
+   }
}
```

## Final code

```cs
using UnityEngine;
using Valve.VR;
using System;

public class WatchOverlay : MonoBehaviour
{
    private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;

    private void Start()
    {        
        InitOpenVR();
        overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

        var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
        SetOverlayFromFile(overlayHandle, filePath);

        SetOverlaySize(overlayHandle, 0.5f);

        var position = new Vector3(0, 2, 3);
        var rotation = Quaternion.Euler(0, 0, 45);
        SetOverlayTransformAbsolute(overlayHandle, position, rotation);

        ShowOverlay(overlayHandle);
    }
    
    private void OnApplicationQuit()
    {
        DestroyOverlay(overlayHandle);
    }

    private void OnDestroy()
    {
        ShutdownOpenVR();
    }

    private void InitOpenVR()
    {
        if (OpenVR.System != null) return;

        var error = EVRInitError.None;
        OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
        if (error != EVRInitError.None)
        {
            throw new Exception("Failed to initialize OpenVR: " + error);
        }
    }

    private void ShutdownOpenVR()
    {
        if (OpenVR.System != null)
        {
            OpenVR.Shutdown();
        }
    }
    
    private ulong CreateOverlay(string key, string name)
    {
        var handle = OpenVR.k_ulOverlayHandleInvalid;
        var error = OpenVR.Overlay.CreateOverlay(key, name, ref handle);
        if (error != EVROverlayError.None)
        { 
            throw new Exception("Failed to create overlay: " + error);
        }
        return handle;
    }

    private void DestroyOverlay(ulong handle)
    {
        if (handle != OpenVR.k_ulOverlayHandleInvalid)
        {
            var error = OpenVR.Overlay.DestroyOverlay(handle);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to dispose overlay: " + error);
            }
        }
    }

    private void SetOverlayFromFile(ulong handle, string path)
    {
        var error = OpenVR.Overlay.SetOverlayFromFile(handle, path);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to draw image file: " + error);
        }
    }

    private void ShowOverlay(ulong handle)
    {
        var error = OpenVR.Overlay.ShowOverlay(handle);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to show overlay: " + error);
        }
    }

    private void SetOverlaySize(ulong handle, float size)
    {
        var error = OpenVR.Overlay.SetOverlayWidthInMeters(handle, size);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to set overlay size: " + error);
        }
    }

    private void SetOverlayTransformAbsolute(ulong handle, Vector3 position, Quaternion rotation)
    {
        var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
        var matrix = rigidTransform.ToHmdMatrix34();
        var error = OpenVR.Overlay.SetOverlayTransformAbsolute(handle, ETrackingUniverseOrigin.TrackingUniverseStanding, ref matrix);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to set overlay position: " + error);
        }
    }
}
```

Here, we have set the overlay size and position. But we need to attach the overlay to controllers instead of the absolute position to create a watch application.
Next part, we will make the overlay follow devices such as HMD or controllers.