[Unity] Fast Pixel Reading (Part 1): Render Textures & Compute Shaders
Tags: unity3d, gamedev
Published: 2023-05-24 19:44:55

## Intro

For a project, I needed to call *ReadPixels* every single frame, i.e. in the Update function.

As it turns out, this is a very costly operation, because it involves communicating information from the GPU to the CPU.

The following post details how I improved the speed at which I get information from the GPU to the CPU from **0.257ms** to **0.075ms**, an improvement of 70%.

## Foreword

For this post I replicate my findings from a Unity project I built from scratch so that it can be replicated easily if wanted.

The following are the steps I took, starting in Unity Hub.
I also include simple performance evaluations at every step.

With "baseline" I shall refer to the fps count level measured in my "3D (URD)" Unity scene, before having done anything to it after creating it.

## Testing

For evaluating performance I use the following script, which logs the average fps count over the first 10 seconds of running the game.

```cs
using UnityEngine;

public class AverageFPSCounter : MonoBehaviour
{
    private float startTime;
    private int frameCount;

    private void Start()
    {
        startTime = Time.time;
        frameCount = 0;
    }

    private void Update()
    {
        frameCount++;
        
        if (Time.time - startTime >= 10f)
        {
            float averageFPS = frameCount / (Time.time - startTime);
            Debug.Log("Average FPS over 10 seconds: " + averageFPS);

            // Optionally, you can stop the script from updating further.
            enabled = false;
        }
    }
}

```

## Setup

1. Create a new Unity project using the "3D (URP)" template. (I'm using Unity version 2021.3.16f1)
2. Create a new "Camera" game object.
3. Remove the "Audio Listener" component.

Running the game now with 2 active cameras in the scene naturally tanks performance by quite a bit.

Performance Evaluation: **Baseline drop of 43%**

## RenderTexture

In my case I just want to read the color of pixels from a separate camera, which does not need to be rendered to the player, which is why I just render it into a *RenderTexture*, wich is a lot more efficient.

1. Create a *RenderTexture*.
2. Assign it to the "Output Texture" of the secondary camera.

Performance Evaluation: **Baseline drop of 11%** 

## ReadPixel

I created a simple script that reads the color of the first pixel from the *RenderTexture*, and does this every single frame.

```cs
using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class PixelReader : MonoBehaviour
{
    [SerializeField] private RenderTexture _renderTexture;
    private Texture2D texture2D;

    private void Start()
    {
        texture2D = new Texture2D(_renderTexture.width, _renderTexture.height, TextureFormat.RGBA32, false);
    }

    private void Update()
    {
        ReadFirstPixel();
    }

    private void ReadFirstPixel()
    {
        // Copy the RenderTexture's pixels to the Texture2D
        RenderTexture.active = _renderTexture;
        texture2D.ReadPixels(new Rect(0, 0, _renderTexture.width, _renderTexture.height), 0, 0);
        texture2D.Apply();

        // Get the first pixel's color
        Color firstPixelColor = texture2D.GetPixel(0, 0);
    }
}

```

1. Attach the script to the secondary camera.
2. Drag previously created *RenderTexture* into the visible "Render Texture" field.

Performance evaluation: **Baseline drop of 23%**

## Line-by-Line Code Performance Evaluation

Executing the *ReadFirstPixel* function 100 times, the average runtime for it is **0.257ms**.

I also made separate evaluations for all lines within the *ReadFirstPixel* function. The number in brackets next to the line number indicates the average runtime for the line across 100 runs.

Line 1 (0.011ms):
```cs
RenderTexture.active = _renderTexture;
```

Line 2 (**0.213ms**):
```cs
texture2D.ReadPixels(new Rect(0, 0, _renderTexture.width, _renderTexture.height), 0, 0);
```

Line 3 (0.064ms):
```cs
texture2D.Apply();
```

Line 4 (0.007ms):
```cs
Color firstPixelColor = texture2D.GetPixel(0, 0);
```

Conclusion: As expected, *ReadPixels* is indeed the function that takes the longest to execute. Let's try to optimize this!

## Compute Shader

Instead of using *ReadPixels*, we will be creating a *Compute Shader* to transfer the color data from the GPU to the CPU in a faster way.

The following is my Compute Shader.

```
#pragma kernel ReadFirstPixel

// Input texture
Texture2D<float4> inputTexture;

// Output buffer
#pragma bind_buffer(name: outputBuffer, binding: 0)

RWStructuredBuffer<float4> outputBuffer;

[numthreads(1, 1, 1)]
void ReadFirstPixel(uint3 id : SV_DispatchThreadID)
{
    // Read the color of the first pixel
    float4 color = inputTexture.Load(int3(0, 0, 0));

    // Store the color in the output buffer
    outputBuffer[0] = color;
}
```

The following is my new version of the *PixelReader* script, which now uses the compute shader.

```cs
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System;

public class PixelReader : MonoBehaviour
{
    [SerializeField] private RenderTexture inputTexture;
    private Texture2D texture2D;

    [SerializeField] private ComputeShader ReadFirstPixel;
    private ComputeBuffer outputBuffer;
    private int kernelID;

    private void Start()
    {
        texture2D = new Texture2D(inputTexture.width, inputTexture.height, TextureFormat.RGBA32, false);

        // Create the output buffer
        outputBuffer = new ComputeBuffer(1, sizeof(float) * 4);

        // Get the kernel ID of the ReadFirstPixel function
        kernelID = ReadFirstPixel.FindKernel("ReadFirstPixel");

        // Set the input texture and output buffer to the shader
        ReadFirstPixel.SetTexture(kernelID, "inputTexture", inputTexture);
        ReadFirstPixel.SetBuffer(kernelID, "outputBuffer", outputBuffer);
    }

    private void Update()
    {
        FastReadPixel();
    }

    private void FastReadPixel()
    {
        // Dispatch the compute shader
        ReadFirstPixel.Dispatch(kernelID, 1, 1, 1);

        // Read the result from the output buffer
        float[] outputArray = new float[4];
        outputBuffer.GetData(outputArray);

        // Extract the color components from the output array
        Color color = new Color(outputArray[0], outputArray[1], outputArray[2], outputArray[3]);
    }

    void OnDestroy()
    {
        // Release the output buffer
        outputBuffer.Release();
    }
}

```
Performance evaluation: **Baseline drop of 25%**

## Line-by-Line Code Performance Evaluation

Executing the new *FastReadPixel* function 100 times, the average runtime for it is **0.075ms**.

I also made separate evaluations for all lines within the *FastReadPixel* function. The number in brackets next to the line number indicates the average runtime for the line across 100 runs.

Line 1 (0.005ms):
```cs
ReadFirstPixel.Dispatch(kernelID, 1, 1, 1);
```

Line 2 (0.001ms):
```cs
float[] outputArray = new float[4];
```

Line 3 (**0.073ms**):
```cs
outputBuffer.GetData(outputArray);
```

Line 4 (0.001ms):
```cs
Color color = new Color(outputArray[0], outputArray[1], outputArray[2], outputArray[3]);
```