9 "Black Magics" and "Tricks" in C#
Tags: dotnet, csharp
Published: 2023-07-27 04:11:20

# 9 "Black Magics" and "Tricks" in C#

We know that `C#` is an extremely advanced language due to its "syntactic sugar". These "sugar" sometimes are **too convenient** that some people might think they are written hard in the `C#` compiler, irrational—somewhat like "black magic".

So, let's see if these **advanced** features in `C#` are hard-written by the compiler ("black magic"), or extendable ("tricks") "duck typing".

I'll list a directory first. You can try to make a judgment based on this directory, whether it is "black magic" (hard-written by compiler) or "duck typing" (custom "tricks"):
1. `LINQ` operations, with `IEnumerable<T>` type;
2. `async/await`, with `Task`/`ValueTask` types;
3. Expression trees, with `Expression<T>` type;
4. Interpolated strings, with `FormattableString` type;
5. `yield return`, with `IEnumerable<T>` type;
6. `foreach` loop, with `IEnumerable<T>` type;
7. `using` keyword, with `IDisposable` interface;
8. `T?`, with `Nullable<T>` type;
9. Generic operations of `Index/Range` for any type.

## 1. `LINQ` operations, with `IEnumerable<T>` type
Not "black magic", but "duck typing".

`LINQ` is a new feature released with `C# 3.0`, which can operate on data conveniently. It's been 12 years now. Although some functions need to be enhanced, it's still much more convenient compared to other languages.

As I mentioned in my last blog, `LINQ` does not necessarily have to be based on `IEnumerable<T>`. Just define a type and implement the required `LINQ` expressions. The `LINQ's` `select` keyword will call the `.Select` method. You can use the following "tricks" to implement the effect of "transferring flowers and trees":
```csharp
void Main()
{
    var query = 
        from i in new F()
        select 3;
        
    Console.WriteLine(string.Join(",", query)); // 0,1,2,3,4
}

class F
{
    public IEnumerable<int> Select<R>(Func<int, R> t)
    {
        for (var i = 0; i < 5; ++i)
        {
            yield return i;
        }
    }
}
```

## 2. `async/await`, with `Task`/`ValueTask` types
Not "black magic", but "duck typing".

`async/await` was released with `C# 5.0`, which can do asynchronous programming very conveniently. Its essence is a state machine.

The essence of `async/await` is to find a method named `GetAwaiter()` under a type. This method must return a class inheriting from `INotifyCompletion` or `ICriticalNotifyCompletion`, which also needs to implement the `GetResult()` method and `IsComplete` property.

This is explained in the `C#` language specification. Calling `await t` will execute in the following order:
1. First call the `t.GetAwaiter()` method to get the waiter `a`;
2. Call `a.IsCompleted` to get a boolean type `b`;
3. If `b=true`, then immediately execute `a.GetResult()`, to get the running result;
4. If `b=false`, then depending on the situation:
   1. If `a` does not implement `ICriticalNotifyCompletion`, execute `(a as INotifyCompletion).OnCompleted(action)`
   2. If `a` has implemented `ICriticalNotifyCompletion`, execute `(a as ICriticalNotifyCompletion).OnCompleted(action)`
   3. The execution is suspended afterward, and returns to the state machine after `OnCompleted` is completed;

If you are interested, you can visit the specific specification explanation on `Github`: [https://github.com/dotnet/csharplang/blob/master/spec/expressions.md#runtime-evaluation-of-await-expressions](https://github.com/dotnet/csharplang/blob/master/spec/expressions.md#runtime-evaluation-of-await-expressions)

Normally `Task.Delay()` is based on a `thread pool timer`. You can implement a single-threaded `TaskEx.Delay()` with the following "tricks":
```csharp
static Action Tick = null;

void Main()
{
    Start();
    while (true)
    {
        if (Tick != null) Tick();
        Thread.Sleep(1);
    }
}

async void Start()
{
    Console.WriteLine("Start execution");
    for (int i = 1; i <= 4; ++i)
    {
        Console.WriteLine($"The {i}th time, time: {DateTime.Now.ToString("HH:mm:ss")} - Thread number: {Thread.CurrentThread.ManagedThreadId}");
        await TaskEx.Delay(1000);
    }
    Console.WriteLine("Execution completed");
}

class TaskEx
{
    public static MyDelay Delay(int ms) => new MyDelay(ms);
}

class MyDelay : INotifyCompletion
{
    private readonly double _start;
    private readonly int _ms;
    
    public MyDelay(int ms)
    {
        _start = Util.ElapsedTime.TotalMilliseconds;
        _ms = ms;
    }
    
    internal MyDelay GetAwaiter() => this;
    
    public void OnCompleted(Action continuation)
    {
        Tick += Check;
        
        void Check()
        {
            if (Util.ElapsedTime.TotalMilliseconds - _start > _ms)
            {
                continuation();
                Tick -= Check;
            }
        }
    }

    public void GetResult() {}
    
    public bool IsCompleted => false;
}
```

The running effect is as follows:
```
Start execution
The 1th time, time: 17:38:03 - Thread number: 1
The 2th time, time: 17:38:04 - Thread number: 1
The 3th time, time: 17:38:05 - Thread number: 1
The 4th time, time: 17:38:06 - Thread number: 1
Execution completed
```

> Note that it is not necessary to use `TaskCompletionSource<T>` to create a user-defined `async/await`.

## 3. Expression trees, with `Expression<T>` type
It's "black magic". There is no "room for operation", only when the type is `Expression<T>`, it will be created as expression trees.

`Expression trees` were released with `C# 3.0` along with `LINQ`, which is a far-sighted "black magic".

For the following code:
```csharp
Expression<Func<int>> g3 = () => 3;
```

It will be translated by the compiler as:
```csharp
Expression<Func<int>> g3 = Expression.Lambda<Func<int>>(
    Expression.Constant(3, typeof(int)), 
    Array.Empty<ParameterExpression>());
```

## 4. Interpolated strings, with `FormattableString` type
It's "black magic". There is no "room for operation".

`Interpolated strings` were released with `C# 6.0`. Many languages have provided similar features before that.

Only when the type is `FormattableString`, it will produce different compilation results. As in the following code:
```csharp
FormattableString x1 = $"Hello {42}";
string x2 = $"Hello {42}";
```

The compiler generates the following results:
```csharp
FormattableString x1 = FormattableStringFactory.Create("Hello {0}", 42);
string x2 = string.Format("Hello {0}", 42);
```

Please note that it essentially calls `FormattableStringFactory.Create` to create a type.

## 5. `yield return`, with `IEnumerable<T>` type;
It's "black magic", but there are additional notes.

`yield return` can be used not only for `IEnumerable<T>`, but also for `IEnumerable`, `IEnumerator<T>`, `IEnumerator`.

Therefore, if you want to simulate the behavior of `generator<T>` in `C++`/`Java` with `C#`, it will be simpler:
```csharp
var seq = GetNumbers();
seq.MoveNext();
Console.WriteLine(seq.Current); // 0
seq.MoveNext();
Console.WriteLine(seq.Current); // 1
seq.MoveNext();
Console.WriteLine(seq.Current); // 2
seq.MoveNext();
Console.WriteLine(seq.Current); // 3
seq.MoveNext();
Console.WriteLine(seq.Current); // 4

IEnumerator<int> GetNumbers()
{
    for (var i = 0; i < 5; ++i)
        yield return i;
}
```

`yield return`—"iterator" was released with `C# 2.0`.

## 6. `foreach` loop, with `IEnumerable<T>` type
It's "duck typing". There is a "room for operation".

`foreach` does not necessarily have to be used with the `IEnumerable<T>` type, as long as the object has a `GetEnumerator()` method:
```csharp
void Main()
{
    foreach (var i in new F())
    {
        Console.Write(i + ", "); // 1, 2, 3, 4, 5, 
    }
}

class F
{
    public IEnumerator<int> GetEnumerator()
    {
        for (var i = 0; i < 5; ++i)
        {
            yield return i;
        }
    }
}
```

Besides, if the object has implemented `GetAsyncEnumerator()`, `await foreach` can also be used for asynchronous looping:
```csharp
async Task Main()
{
    await foreach (var i in new F())
    {
        Console.Write(i + ", "); // 1, 2, 3, 4, 5, 
    }
}

class F
{
    public async IAsyncEnumerator<int> GetAsyncEnumerator()
    {
        for (var i = 0; i < 5; ++i)
        {
            await Task.Delay(1);
            yield return i;
        }
    }
}
```

`await foreach` was released with `C# 8.0` along with `asynchronous stream`. For details, you can refer to my previous article "Code demonstration of new functions in each version of C#".

## 7. `using` keyword, with `IDisposable` interface
Yes, and no.

For `reference types` and normal `value types` with `using` keyword, it **must** be based on `IDisposable` interface.

But `ref struct` and `IAsyncDisposable` tell a different story. Since `ref struct` does not allow arbitrary movement, and reference types—managed heap, will allow memory movement, so `ref struct` is not allowed to have any relationship with `reference types`. This relationship includes inheriting `interface`—because `interface` is also a `reference type`.

But the need to release resources still exists. What should we do? "Duck typing" comes. You can manually write a `Dispose()` method, without inheriting any interfaces:
```csharp
void S1Demo()
{
    using S1 s1 = new S1();
}

ref struct S1
{
    public void Dispose()
    {
        Console.WriteLine("Normal release");
    }
}
```

The same principle holds if using the `IAsyncDisposable` interface:
```csharp
async Task S2Demo()
{
    await using S2 s2 = new S2();
}

struct S2 : IAsyncDisposable
{
    public async ValueTask DisposeAsync()
    {
        await Task.Delay(1);
        Console.WriteLine("Async release");
    }
}
```

## 8. `T?`, with `Nullable<T>` type
It's "black magic". Only `Nullable<T>` can accept `T?`. `Nullable<T>` is a `value type`, yet it can directly accept `null` values (normally, `value types` do not allow `null` values).

Example code is as follows:
```csharp
int? t1 = null;
Nullable<int> t2 = null;
int t3 = null; // Error CS0037: Cannot convert null to 'int' because it is a non-nullable value type
```

Generated code is as follows (`int?` and `Nullable<int>` are exactly the same, skipping the code that failed to compile):
```cil
IL_0000: nop
IL_0001: ldloca.s 0
IL_0003: initobj valuetype [System.Runtime]System.Nullable`1<int32>
IL_0009: ldloca.s 1
IL_000b: initobj valuetype [System.Runtime]System.Nullable`1<int32>
IL_0011: ret
```

## 9. Generic operations of `Index/Range` for any type
There is "black magic", and there is "duck typing"—there is room for operation.

`Index/Range` was released with `C# 8.0`, which can conveniently operate on an index position and take out the corresponding value like `Python`. Things that required calling `Substring` and other complex operations are now very simple.

```csharp
string url = "https://www.super-cool.com/product/7705a33a-4d2c-455d-a42c-c95e6ac8ee99/summary";
string productId = url[35..url.LastIndexOf("/")];
Console.WriteLine(productId);
```

Generated code is as follows:
```csharp
string url = "https://www.super-cool.com/product/7705a33a-4d2c-455d-a42c-c95e6ac8ee99/amd-r7-3800x";
int num = 35;
int length = url.LastIndexOf("/") - num;
string productId = url.Substring(num, length);
Console.WriteLine(productId); // 7705a33a-4d2c-455d-a42c-c95e6ac8ee99
```

As you can see, the `C#` compiler ignored `Index/Range` and translated it directly into calling `Substring`.

But arrays are different:
```csharp
var range = new[] { 1, 2, 3, 4, 5 }[1..3];
Console.WriteLine(string.Join(", ", range)); // 2, 3
```

Generated code is as follows:
```csharp
int[] range = RuntimeHelpers.GetSubArray<int>(new int[5]
{
    1,
    2,
    3,
    4,
    5
}, new Range(1, 3));
Console.WriteLine(string.Join<int>(", ", range));
```

It can be seen that it really created a `Range` type and then called `RuntimeHelpers.GetSubArray<int>`, which is completely "black magic".

But it's also "duck typing". As long as the code implements the `Length` property and `Slice(int, int)` method, you can call `Index/Range`:
```csharp
var range2 = new F()[2..];
Console.WriteLine(range2); // 2 -> -2

class F
{
    public int Length { get; set; }
    public IEnumerable<int> Slice(int start, int end)
    {
        yield return start;
        yield return end;
    }
}
```

Generated code is as follows:
```csharp
F f = new F();
int length2 = f.Length;
length = 2;
num = length2 - length;
string range2 = f.Slice(length, num);
Console.WriteLine(range2);
```

# Conclusion
As you can see, there are indeed a lot of "black magic" in `C#`, but there are also a lot of "duck typing", and there is a great "room for operation" for "tricks".

> It is rumored that `C# 9.0` will add "duck typing" tuples—"Type Classes". At that time, the "room for operation" will definitely be much larger than it is now, looking forward to it!