SIMD aggregate performance
Tags: csharp, dotnet, simd
Published: 2023-09-22 22:24:59

## Foreword 🍓
Dotnet provides several classes, some under the <u>System.Runtime.Intrinsics</u> namespace that allow hardware to execute instructions in <u>parallel</u>.

```csharp
using System.Runtime.Intrinsics;

Vector512 v512;
Vector256 v256;
Vector128 v128;
```

The number <u>suffix </u>(512, 256, 128) indicates the size in bits of the vector that the hardware can process in parallel.

This has positive impact in operations that performs aggregates, specially in a loop with large arrays.

To know if the hardware allows this type of registers we can consult the static read-only property <u>IsHardwareAccelerated</u>

```csharp
if (Vector256.IsHardwareAccelerated)
{
    _is256 = true;
    ...
}
```

The above code will test if our hardware supports 256 bit vector operations through JIT intrinsics.

## Exploring 🧠

Suppose we want to simultaneously calculate the maximum and minimum of a sequence of integers using <u>Vector256</u>.

The process will consist of creating a loop in which we will move forward taking 256-bit chunks and updating the maximum and minimum

```csharp
(T Min, T Max) MinMax256<T>(ReadOnlySpan<T> source) 
    where T : struct, INumber<T>
{

}
```

First we initialize some variables to hold the current element, the last element, and the last size wise element (the <u>to </u>variable)

```csharp
ref T current = ref MemoryMarshal.GetReference(source);
ref T last = ref Unsafe.Add(ref current, source.Length);
ref T to = ref Unsafe.Add(ref last, -Vector256<T>.Count);

Vector256<T> minElement = Vector256.LoadUnsafe(ref current);
Vector256<T> maxElement = minElement;
```

Then we start the loop. Inside, we load data in 256 bit chunks calling <u>Vector256.LoadUsafe</u>

```csharp
 while (Unsafe.IsAddressLessThan(ref current, ref to))
 {
     Vector256<T> tempElement = Vector256.LoadUnsafe(ref current);
     minElement = Vector256.Min(minElement, tempElement);
     maxElement = Vector256.Max(maxElement, tempElement);
     current = ref Unsafe.Add(ref current, Vector256<T>.Count);
 }
```
We use the static Min and Max methods of <u>Vector256</u>and store the value in minElement and maxElement.

Finally, we increment the position counter (current) by adding 256 bits to the pointer.

Once we have exceeded the established size, we have to calculate the maximum and minimum individually

```csharp
T min = minElement[0];
T max = maxElement[0];

for (int i = 1; i < Vector256<T>.Count; i++)
{
    T tempMin = minElement[i];
    if (tempMin < min)
    {
        min = tempMin;
    }
    T tempMax = maxElement[i];
    if (tempMax > max)
    {
        max = tempMax;
    }
}
```

After that we calculate the remaining elements if any:

```csharp
while (Unsafe.IsAddressLessThan(ref current, ref last))
{
    if (current < min)
    {
        min = current;
    }
    if (current > max)
    {
        max = current;
    }
    current = ref Unsafe.Add(ref current, 1);
}
```

And that's all, we return the results:

```csharp
return (min, max);
```

## Benchmark 🔥

A quick test with BenchmarkDotnet calculating the maximum and minimum of an array of 10_000 integers reveals a performance improvement of <u>x146 </u>with Vector256 support.

> 💡 Ryzen 7 1700, 1 CPU 
> .NET SDK=8.0.100-rc.1.23455.8

| Method | Mean (ns) |
|--------|------|
| 🐢 MinMaxLinq  .NET Framework 4.8 | 118,675.226 |
| ⚡ <u>MinMaxSimd  .NET 8.0</u> | <u>808.150</u>|

## Farewell
All the code with a more elavorated example is hosted in github. Be happy and love your family 💖
{% embed https://github.com/NetDefender/SimdIteration %}


## References

{% embed https://learn.microsoft.com/es-es/dotnet/api/system.runtime.intrinsics?view=net-8.0 %}

{% embed https://github.com/dotnet/runtime/issues/79005 %}

{% embed https://devblogs.microsoft.com/dotnet/hardware-intrinsics-in-net-core/ %}