LInQ in Javascript: LInQer!
Tags: programming, javascript, unittests, github
Published: 2020-01-20 12:50:55

---
title: LInQ in Javascript: LInQer!
published: true
date: 2020-01-04 23:08:30 UTC
tags: programming,javascript,unit tests,github
canonical_url: https://siderite.dev/blog/linq-in-javascript-linqer/
---

Original post at [https://siderite.dev/blog/linq-in-javascript-linqer](https://siderite.dev/blog/linq-in-javascript-linqer)

Update: LInQer is published on NPM and you can use it in Node.js as well, using 
```
const Linqer = require('@siderite/linqer');
```

Update: LInQer can now be found published at [https://siderite.github.io/LInQer/LInQer.min.js](https://siderite.github.io/LInQer/LInQer.min.js "Main minimized file") and [https://siderite.github.io/LInQer/LInQer.extra.min.js](https://siderite.github.io/LInQer/LInQer.extra.min.js "Extra functions like shuffle and binarySearch") and you can freely use it in your web sites. Source code on [GitHub](https://github.com/Siderite/LInQer "The C# Language Integrated Queries ported for Javascript for amazing performance").

Update: All the methods in Enumerable are now implemented, and some extra ones as well. Optimizations have been developed for operations like orderBy().take() and .count().

![Performance gains of over 2000%](https://siderite.dev/Posts/files/LInQer_637137761100896733.jpg)  This entire thing started from [a blog post](https://www.wix.engineering/post/breaking-chains-with-pipelines-in-modern-javascript "Breaking Chains with Pipelines in Modern JavaScript") that explained something that seems obvious in retrospect, but you have to actually think about it to realise it: Array functions in Javascript may seem similar to [LInQ](https://docs.microsoft.com/en-us/dotnet/csharp/linq/ "Language Integrated Query (LINQ)") methods in C#, but they work with arrays! Every time you filter or you map something you create a new array! The post suggested creating functions that would use the modern Javascript features of [iterators](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols "Iteration protocols") and [generator functions](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Generator "The function\* declaration (function keyword followed by an asterisk) defines a generator function, which returns a Generator object.") and supplant the standard Array functions. It was brilliant!

  And then the post abruptly veered and swerved into a tree. The author suggested we add the pipe operator to Javascript, just like they have in functional programming languages, so that we can use those static functions with hash characters as placeholders and... ugh! It was horrid! So I decided to solve the problem in a way that was more compatible with my own sensibilities: make it work just like LInQ (or at least like the [Java streams](https://winterbe.com/posts/2014/07/31/java8-stream-tutorial-examples/ "They are not integrated in the language, so they need a wrapper, too: toStream()")).

This is how LInQer was born. You can find the sources on GitHub at [/Siderite/LInQer](https://github.com/Siderite/LInQer "The C# Language Integrated Queries ported for Javascript for amazing performance"). The library introduces a class called Enumerable, which can wrap any iterable (meaning arrays, but also generator functions or anything that supports [for...of](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/for...of "The for...of statement creates a loop iterating over iterable objects, including: built-in String, Array, array-like objects (e.g., arguments or NodeList), TypedArray, Map, Set, and user-defined iterables. It invokes a custom iteration hook with statements to be executed for the value of each distinct property of the object.")), then expose the methods that the [Enumerable](https://docs.microsoft.com/en-us/dotnet/api/system.linq.enumerable "Provides a set of static methods for querying objects that implement IEnumerable<T>") class in .NET exposes. There is even a [unit test](https://api.qunitjs.com/ "qUnit, baby! Old school!") file called tests.html. Open it in the browser and see the supported methods. As you can see in the image of the blog post, the same logic (filtering, mapping and slicing a 10 million item array) took 2000ms using standard Array functions and 150ms using the Enumerable class.

"But wait! It says there that Enumerable exposes static methods! What exactly did you improve?", you will ask. In .NET we have something called "[extension methods](https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/extension-methods "Extension Methods (C# Programming Guide)")", which are indeed static methods that the language understands apply to specific classes or interfaces and allows you to call them like you would instance methods. So a method that looks like `public static int Sum(this IEnumerable<int> enumerable)` can be used statically `Enumerable.Sum(arr)` or like an instance method `arr.Sum()`. It's syntactic sugar. But enough about C#, in Javascript I've implemented Enumerable as a wrapper, as Javascript doesn't know about interfaces or static extension methods. This class then exposes prototype functions that work the same way as in LInQ.

Here is where the magic happens:

```
Enumerable.prototype = {
  [Symbol.iterator]() {
    const iterator = this._src[Symbol.iterator].bind(this._src);
    return iterator();
  },
```

In other words I wrap \_src (the original iterable object) in my Enumerable by exposing the same iterator for both! Now both initial object and its wrapper can be used in for...of loops. The difference is that the wrapper now exposes all that juicy functionality. Let's see a simple example: select (which is the logical equivalent of Array.map):

```
select(op) {
  _ensureFunction(op);
  const gen = function* () {
    for (const item of this) {
      yield op(item);
    }
  }.bind(this);
  return new Enumerable(gen());
},
```

Here I am constructing a generator function which I bind to the Enumerable instance so that inside it 'this' is always that instance. Then I am returning the wrapper over the generator. In the function I am yielding the result of the op function on each item. Because of how generator functions work, only while iterating will it need to do its work, meaning that if I use the Enumerable into a for...of loop and breaking after three items, the op function will only be applied on those items, not on all of them. In order to use the Enumerable object in regular code, that works with arrays, you just use `.toArray` and you are done!

Let's see the code in the tests used to compare the standard Array function use with Enumerable:

```
// this is the large array we are using in both code blocks:
  const largeArray = Array(10000000).fill(10);

// Standard Array functions
  const someCalculation = largeArray
                             .filter(x=>x===10)
                             .map(x=>'v'+x)
                             .slice(100,110);
// Enumerable
  const someCalculation = Enumerable.from(largeArray)
                             .where(x=>x===10)
                             .select(x=>'v'+x)
                             .skip(100)
                             .take(10)
                             .toArray();
```

There are differences from the C# Enumerable, of course. Some things don't make sense, like `thenBy` after `orderBy`. It can be done, but it's complicated and in my career I've only used thenBy twice! `toDictionary`, `toHashSet`, `toLookup`, `toList` have no meaning in Javascript. Use instead `toMap`, `toSet`, `toObject`, `toArray`. Last, but not least... `join`. Join is so complicated to use in LInQ that I almost never used it. Also, joining is usually done in the database, so I rarely needed it. I didn't see a point in implementing it. Cast and AsEnumerable also didn't make sense. Stuff like that.

But don't fret. Any function that you want to add to this can simply be added by your own code into `Enumerable.prototype`! So if you really need something custom, it's easy to add without modifying the original library. Also, I have no idea how kids use and pack libraries nowadays in Javascript. I am testing all this in the browser. If you want it with modules, npm, Node.js... you can do it yourself! :D

There is one disadvantage for the prototype approach and the reason why the initial article suggested to use standalone functions: tree-shaking, the fancy word used to express the automatic elimination of unused code. But I think there is a solution for that, one that I won't implement since I believe the library is small enough and minified and compressed it will be very small indeed. The solution would involve separating each of the LINQ methods (or categories of methods, like first, firstOrDefault, last and lastOrDefault, for example) in different files. Then you can use only what you need and the files would attach the functions to the Enumerable prototype.

In fact, there are a lot of LINQ methods I have never had use for, stuff like intersect and except or append and prepend. It makes sense to create a core Enumerable class and then add other stuff only when required. But as I said, it's small enough to not matter.

Bottom line, I hope you find this library useful or at least it inspires you as it did me when I've read the original blog post from Dan Shappir.

P.S. I am not the only one that had this idea, you might also want to look at [linq.js](https://github.com/mihaifm/linq "This is a javascript implementation of the .NET LINQ library. It contains all the original .NET methods plus a few additions.") which uses a completely different approach, using regular expressions. I think the closest to what LINQ should be is [Manipula](https://github.com/litichevskiydv/manipula "Implementation of LINQ"), that uses custom iterators for each operation rather than use the same class. Another possibility is [linq-collections](https://github.com/isc30/linq-collections "Strongly typed Linq and Collections implementation for Javascript and TypeScript (ECMAScript 5) "), which does the work via TypeScript, apparently. Also [linq.es6](https://github.com/schmuli/linq.es6 "An implemenation of LINQ using ES6 generators and iterators")... I am sure there are more examples if one looks closely. Feel free to let me know if you did or know of similar work and also please give me whatever feedback you see fit. I would like this library to be useful, not just a code example for my blog.