Evaluating “ReadLine using System.IO.Pipelines” Performance in C#
Tags: benchmark, csharp, dotnetcore, dotnet
Published: 2020-08-16 16:52:12

---
title: Evaluating “ReadLine using System.IO.Pipelines” Performance in C#
published: true
date: 2020-08-16 16:10:33 UTC
tags: benchmark,csharp,dotnetcore,dotnet
canonical_url: https://medium.com/@joni2nja/evaluating-readline-using-system-io-pipelines-performance-in-c-69e9ed658920
series: Evaluating “ReadLine using System.IO.Pipelines” Performance in C#
---

#### Read string line by line using System.IO.Pipelines API in C#

![](https://cdn-images-1.medium.com/max/1024/1*9i0fZ9otnPpsWN_C8YoH2w.png)

A typical approach for reading a string line by line from a file would probably be using [ReadLine](https://stackoverflow.com/questions/8037070/whats-the-fastest-way-to-read-a-text-file-line-by-line) APIs. I have heard all good things about [System.IO.Pipelines](https://devblogs.microsoft.com/dotnet/system-io-pipelines-high-performance-io-in-net/), which was born from the work of the .NET Core team to make Kestrel one of the [fastest web servers](https://www.techempower.com/benchmarks/#section=data-r19&hw=ph&test=plaintext).

{% tweet 1266143330477453314 %}

Although it is probably better suited for network socket IO, I was wondering how easy and how does it perform compared to the traditional ReadLine APIs? So I decided to give it a spin by creating simple benchmarks using [BenchmarkDotNet](https://benchmarkdotnet.org/) as part of my learning journey.

But, for this post, instead of using a file as the source of the stream, I decided to use in-memory, presumably more-stable-for-benchmark MemoryStream. Feel free to modify it to use a file as the stream source and try it yourself.

{% gist https://gist.github.com/jo-ninja/b4f9be0bcd09afe8842931c4f85fc430 %}

The code for ReadLine is so dead simple; nothing fancy here.

{% gist https://gist.github.com/jo-ninja/142e91286fab2458749af0c867081b72 %}

_(My apologies for the possibly misleading method name_ _ReadLineUsingStringReaderAsync; it doesn’t use_ _StringReader class at all)_.

You will probably spot that unfamiliar C# 9.0 [Pattern Combinators](https://github.com/dotnet/csharplang/blob/master/proposals/csharp-9.0/patterns3.md#pattern-combinators) is not null; Yes, I’m using the latest (at the time of writing) .NET 5 Preview 7 (5.0.0-preview.7) and the preview version of Visual Studio (Version 16.8.0 Preview 1.0). Don’t blame me, I love bleeding edge stuff!😎

_See how_ _is not null is transformed_ [_here_](https://sharplab.io/#v2:EYLgtghgzgLgpgJwD4AEBMBGAsAKBQZgAJ1CBhQgb10JuKJQBZCBZACgEpLraeUMAGQlEIBeQgCJxAbm48aASwBmhVsPnCAdgHsYhDQFcANoc5Ucci4XkbdAD1GEMM85Z62A1O+euAvrMJ+OD5AA)_._

To simplify the processing of a ReadOnlySequence<T>, here I used SequenceReader<T>; taken from the official sample [here](https://docs.microsoft.com/en-us/dotnet/standard/io/buffers#sequencereadert), modified it slightly to make it follow a similar pattern while (... is not null) as _ReadLineUsingStringReaderAsync_ does.

{% gist https://gist.github.com/jo-ninja/57536c7e815e633769311cf24bbe7854 %}

And finally, here is the result: 40 benchmarks in 15 minutes on my machine.

![](https://cdn-images-1.medium.com/max/1024/1*1FetdhtL7yBqLEQP2sRFyw.png)

Here is the gist version:

{% gist https://gist.github.com/jo-ninja/0b5d2ab6eff454e5078cb49e2b18ace7 %}

Legends:

```
LineNumber : Value of the 'LineNumber' parameter
LineCharMultiplier : Value of the 'LineCharMultiplier' parameter
Mean : Arithmetic mean of all measurements
Error : Half of 99.9% confidence interval
StdDev : Standard deviation of all measurements
Ratio : Mean of the ratio distribution ([Current]/[Baseline])
RatioSD : Standard deviation of the ratio distribution ([Current]/[Baseline])
Gen 0 : GC Generation 0 collects per 1000 operations
Gen 1 : GC Generation 1 collects per 1000 operations
Gen 2 : GC Generation 2 collects per 1000 operations
Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
Code Size : Native code size of the disassembled method(s)
1 ms : 1 Millisecond (0.001 sec)
```

You can find the source code in my [GitHub repository](https://github.com/jo-ninja/ReadLinesBenchmarks).

### Conclusion

- Pipelines versions are better in terms of memory usage (using less memory).
- In terms of speed, it is surprisingly slower than the _ordinary_ ReadLine version given the string length ≤ 80 (perhaps I am doing it wrong? Let me know! I am still learning!). It is starting to shine, getting faster and faster if the string length ≥ 90. (270% 🚀 faster for string length = 1000).
- Less GC pressure (a good thing) for Pipelines versions (Gen 0, Gen 1).
- The amount of code to write for the Pipelines version is longer.

[Update] See [part 2](https://dev.to/joni2nja/evaluating-readline-using-system-io-pipelines-performance-in-c-part-2-pmf) of this series for a better version! (Spoiler: faster on every test!)

_DISCLAIMER: Your mileage may vary. As with all performance work, each of the scenarios chosen for your application should be measured, measured and measured. There is no silver bullet._

#### **Resources:**

- [System.IO.Pipelines: High performance IO in .NET](https://devblogs.microsoft.com/dotnet/system-io-pipelines-high-performance-io-in-net/)
- [System.IO.Pipelines in .NET](https://docs.microsoft.com/en-us/dotnet/standard/io/pipelines)
- [Work with Buffers in .NET](https://docs.microsoft.com/en-us/dotnet/standard/io/buffers#sequencereadert)
- [An Introduction to SequenceReader by Steve Gordon](https://www.stevejgordon.co.uk/an-introduction-to-sequencereader)