Peeking into LINQ DistinctBy source code
Tags: beginners, csharp, tutorial, dotnet
Published: 2022-08-01 15:06:31

_I originally published this post on [my blog](https://canro91.github.io/2022/07/11/LinqDistinctBySourceCode/) a couple of weeks ago. It's part of a post series about LINQ._

"You should be ready, willing, and able to read the source code of your dependencies." That's a piece of advice I found and shared in a past edition of my [Monday Links](https://canro91.github.io/2022/01/17/MondayLinks/).

Inspired by that advice, I decided to look into the LINQ DistinctBy source code. Let's see what's inside the new LINQ DistinctyBy method.

## What LINQ DistinctBy method does?

**DistinctBy returns the objects containing unique values based on one of their properties. It works on collections of complex objects, not just on plain values.**

DistinctBy is one of the [new LINQ methods introduced in .NET 6](https://canro91.github.io/2022/06/27/NET6LinqMethods/).

The next code sample shows how to find unique movies by release year.

```csharp
var movies = new List<Movie>
{
    new Movie("Schindler's List", 1993, 8.9f),
    new Movie("The Lord of the Rings: The Return of the King", 2003, 8.9f),
    new Movie("Pulp Fiction", 1994, 8.8f),
    new Movie("Forrest Gump", 1994, 8.7f),
    new Movie("Inception", 2010, 8.7f)
};

// Here we use the DistinctBy method with the ReleaseYear property
var distinctByReleaseYear = movies.DistinctBy(movie => movie.ReleaseYear);
//                                 ^^^^^^^^^^

foreach (var movie in distinctByReleaseYear)
{
    Console.WriteLine($"{movie.Name}: [{movie.ReleaseYear}]");
}

// Output:
// Schindler's List: [1993]
// The Lord of the Rings: The Return of the King: [2003]
// Pulp Fiction: [1994]
// Inception: [2010]

record Movie(string Name, int ReleaseYear, float Score);
```

Notice we used `DistinctBy()`on a list of movies. We didn't use it on a list of released years to then find one movie for each unique release year found.

<figure>
<img src="https://source.unsplash.com/BovnwxDCing/600x400" alt="Hungry stray cat" />

<figcaption>Wow! He was curious to peek into the source code of that pie. Photo by <a href="https://unsplash.com/@xrcj?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Bing Han</a> on <a href="https://unsplash.com/s/photos/curious?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></figcaption>
</figure>

## LINQ DistinctBy source code

This is the source code for the DistinctBy method. [[Source]](https://github.com/dotnet/runtime/blob/main/src/libraries/System.Linq/src/System/Linq/Distinct.cs#L48)

```csharp
public static IEnumerable<TSource> DistinctBy<TSource, TKey>(this IEnumerable<TSource> source, Func<TSource, TKey> keySelector, IEqualityComparer<TKey>? comparer)
{
    if (source is null)
    {
        ThrowHelper.ThrowArgumentNullException(ExceptionArgument.source);
    }
    if (keySelector is null)
    {
        ThrowHelper.ThrowArgumentNullException(ExceptionArgument.keySelector);
    }

    // Step 1
    return DistinctByIterator(source, keySelector, comparer);
}

private static IEnumerable<TSource> DistinctByIterator<TSource, TKey>(IEnumerable<TSource> source, Func<TSource, TKey> keySelector, IEqualityComparer<TKey>? comparer)
{
    // Step 2
    using IEnumerator<TSource> enumerator = source.GetEnumerator();

    // Step 3
    if (enumerator.MoveNext())
    {
        // Step 4
        var set = new HashSet<TKey>(DefaultInternalSetCapacity, comparer);
        do
        {
            // Step 5
            TSource element = enumerator.Current;
            if (set.Add(keySelector(element)))
            {
                yield return element;
            }
        }
        // Step 6
        while (enumerator.MoveNext());
    }
}
```

Well, it doesn't look that complicated. Let's go through it.

### 1. Iterating over the input collection

First, `DistinctBy()` starts by checking its parameters and calling `DistinctByIterator()`. This is a common pattern in other LINQ methods. Check parameters in one method and then call a child iterator method to do the actual logic. (See Step 1. in the above code sample)

Then, the `DistinctByIterator()` initializes the underling enumerator of the input collection with a `using` declaration. The `IEnumerable` type has a `GetEnumerator()` method. (See Step 2.)

The `IEnumerator` type has a `MoveNext()` method to advance the enumerator to the next position and a `Current` property to hold the element at the current position.

If a collection is empty or if the iterator reaches the end of the collection, `MoveNext()` returns `false`. And, when `MoveNext()` returns `true`, `Current` gets updated with the element at that position. [[Source]](https://docs.microsoft.com/en-us/dotnet/api/system.collections.ienumerator?view=net-6.0)

Then, to start reading the input collection, the iterator is placed at the initial position of the collection calling `MoveNext()`. (See Step 3.) This first `if` avoids allocating memory by creating a set in the next step if the collection is empty.

### 2. Finding unique elements

After that, `DistinctByIterator()` creates a set with a default capacity and an optional comparer. This set keeps track of the unique keys already found. (See Step 4.)

The next step is to read the current element and add its key to the set. (See 5.)

If a set doesn't already contain the same element, `Add()` returns `true` and adds it to the set. Otherwise, it returns `false`. And, when the set exceeds its capacity, the set gets resized. [[Source]](https://docs.microsoft.com/en-us/dotnet/api/system.collections.generic.hashset-1.add?view=net-6.0#System_Collections_Generic_HashSet_1_Add__0_)

If the current element's key was added to the set, the element is returned with the `yield return` keywords. This way, `DistinctByIterator()` returns one element at a time.

Step 5 is wrapped inside a `do-while` loop. It runs until the enumerator reaches the end of the collection. (See Step 6.)

Voilà! That's the DistinctBy source code. Simple but effective. Not that intimidating, after all. The trick was to use a set. It's a good exercise to read the source code of standard libraries to pick conventions and patterns.

***

_Hey! I'm Cesar, a software engineer and lifelong learner. If you want to learn more about this subject, check my course [Getting Started with LINQ](https://bit.ly/csarag-linq-course) on Educative where I cover other LINQ methods in depth._

_Happy coding!_