Wormhole - Data Collection Cheat Sheet and Library in 4 Languages
Tags: showdev
Published: 2019-02-25 21:09:02

---
title: Wormhole - Data Collection Cheat Sheet and Library in 4 Languages
published: true
description: Hoping to make language transitions easier
tags: #showdev
cover_image: https://thepracticaldev.s3.amazonaws.com/i/tga6v2qk8xycoifkbbj9.jpg
---

During the [Advent of Code](https://adventofcode.org) this past year, I was trying to enhance my knowledge of Elixir, as well as just functional programming in general. There were times where I found a function that most other languages I used didn't have an analogous function (a glaringly obvious one being [`Enum.reduce_while`](https://hexdocs.pm/elixir/Enum.html#reduce_while/3)), and other times where I was writing functions I'd used often in other languages (Clojure's [`frequencies`](http://clojuredocs.org/clojure.core/frequencies) would've been mighty handy!). I finally decided to bite the bullet and try to create a list of collection manipulation functions I used often in other languages, implement them in others I've learned/currently trying to learn, and discover new functions that I wouldn't want to live without!

# What Is Wormhole?

It's this, really:
{% github jdsteinhauser/wormhole %}

I've got an addiction. I love learning new languages. With learning new languages, you end up finding functions, classes, and concepts that you wish that you had in other languages. Sometimes, those functions are named different things and it gets confusing when you switch between languages. I end up doing a lot of data collection manipulation, and so I decided to start with what I knew best and branch out from there!

# What Functions Am I Looking For?

For a non-exhaustive list, I wanted to have at least the following:
- Collection basics: `map`, `filter`, `reduce`, and `scan`
- Chunking data: `chunk`, `chunk_by`
- Common stats: `min_by`, `max_by`, `group_by`, `frequencies`
- Other hella useful things: `reduce_while`, `juxt`, `identity`

# What Languages Am I Targeting?

For now, I have filled in my perceived gaps in functions in C#, Clojure, and Elixir. I have an F# solution that I'll be comfortable with early this week, and I've started looking at a comprehensive list of Ruby functions as well. After that... well, I'm not entirely sure! I think I'm going to go through Rust, JavaScript, Java, and possibly Kotlin and Python 3 to see what other handy things I can implement across all those languages.

# Will These Be Deployed to Package Managers?

Yes... but not right now. I need to get the documentation to a suitable state. I've pulled down several packages before but I've never pushed mine up to any! I'm sure that will end up being a blog post in and of itself.

# Current Cheat Sheet

Here's a summary of the languages I've targeted so far, with documentation links to each function that either already exists, or that I've implemented in Wormhole.

|Function|C#|F#|Clojure|Elixir|
|--------|--|--|-------|------|
|`map`   |[`Enumerable.Select`](https://docs.microsoft.com/en-us/dotnet/api/system.linq.enumerable.select?view=netcore-2.2)|[`Seq.map`](https://msdn.microsoft.com/en-us/visualfsharpdocs/conceptual/seq.map%5b't,'u%5d-function-%5bfsharp%5d)|[`clojure.core/map`](http://clojuredocs.org/clojure.core/map)|[`Enum.map/2`](https://hexdocs.pm/elixir/Enum.html#map/2), [`Stream.map/2`](https://hexdocs.pm/elixir/Stream.html#map/2)|
|`filter`|[`Enumerable.Where`](https://docs.microsoft.com/en-us/dotnet/api/system.linq.enumerable.where?view=netcore-2.2)|[`Seq.filter`](https://msdn.microsoft.com/en-us/visualfsharpdocs/conceptual/seq.filter%5b't%5d-function-%5bfsharp%5d)|[`clojure.core/filter`](http://clojuredocs.org/clojure.core/filter)|[`Enum.filter/2`](https://hexdocs.pm/elixir/Enum.html#filter/2), [`Stream.filter/2`](https://hexdocs.pm/elixir/Stream.html#filter/2)|
|`reduce`|[`Enumerable.Aggregate`](https://docs.microsoft.com/en-us/dotnet/api/system.linq.enumerable.aggregate?view=netcore-2.2)|[`Seq.reduce`](https://msdn.microsoft.com/en-us/visualfsharpdocs/conceptual/seq.reduce%5b't%5d-function-%5bfsharp%5d)|[`clojure.core/reduce`](http://clojuredocs.org/clojure.core/reduce)|[`Enum.reduce`](https://hexdocs.pm/elixir/Enum.html#reduce/2)|
|`reduce_while`|`ReduceWhile`|`reduceWhile`|`reduce-while`|[`Enum.reduce_while/3`](https://hexdocs.pm/elixir/Enum.html#reduce_while/3)|
|`scan`|`Scan`|[`Seq.scan`](https://msdn.microsoft.com/en-us/visualfsharpdocs/conceptual/seq.scan%5b't,'state%5d-function-%5bfsharp%5d)|[`clojure.core/reductions`](http://clojuredocs.org/clojure.core/reductions)|[`Enum.scan`](https://hexdocs.pm/elixir/Enum.html#scan/2), [`Stream.scan`](https://hexdocs.pm/elixir/Stream.html#scan/2)|
|`chunk`|`Chunk`|`chunk`*|[`clojure.core/partition`](http://clojuredocs.org/clojure.core/partition)|[`Enum.chunk_every/4`](https://hexdocs.pm/elixir/Enum.html#chunk_every/4), [`Stream.chunk_every/4`](https://hexdocs.pm/elixir/Stream.html#chunk_every/4)|
|`chunk_by`|`ChunkBy`|`chunkBy`|[`clojure.core/partition-by`](http://clojuredocs.org/clojure.core/partition-by)|[`Enum.chunk_by/2`](https://hexdocs.pm/elixir/Enum.html#chunk_by/2), [`Stream.chunk_by/2`](https://hexdocs.pm/elixir/Stream.html#chunk_every/4)|
|`juxt`|`Juxt`|`juxt`, `juxt2`, `juxt3`|[`clojure.core/juxt`](http://clojuredocs.org/clojure.core/juxt)|`Wormhole.juxt/1`|
|`min_by`|`MinBy`|[`Seq.minBy`](https://msdn.microsoft.com/en-us/visualfsharpdocs/conceptual/seq.minby%5b't,'u%5d-function-%5bfsharp%5d)|`min-by`|[`Enum.min_by/3`](https://hexdocs.pm/elixir/Enum.html#min_by/3)|
|`max_by`|`MaxBy`|[`Seq.maxBy`](https://msdn.microsoft.com/en-us/visualfsharpdocs/conceptual/seq.maxby%5b't,'u%5d-function-%5bfsharp%5d)|`max-by`|[`Enum.max_by/3`](https://hexdocs.pm/elixir/Enum.html#max_by/3)|
|`frequencies`|`Frequencies`|`freqs`|[`clojure.core/frequencies`](http://clojuredocs.org/clojure.core/frequencies)|`Wormhole.freqs/1`|
|`group_by`|[`Enumerable.GroupBy`](https://docs.microsoft.com/en-us/dotnet/api/system.linq.enumerable.groupby?view=netcore-2.2)|[`Seq.groupBy`](https://msdn.microsoft.com/en-us/visualfsharpdocs/conceptual/seq.groupby%5B't%2C'key%5D-function-%5Bfsharp%5D)|[`clojure.core/group-by`](http://clojuredocs.org/clojure.core/group-by)|[`Enum.group_by/3`](https://hexdocs.pm/elixir/Enum.html#group_by/3)|
|`identity`|`Identity`|[`Operators.id`](https://msdn.microsoft.com/en-us/visualfsharpdocs/conceptual/operators.id%5B't%5D-function-%5Bfsharp%5D)|[`clojure.core/identity`](http://clojuredocs.org/clojure.core/identity)|`Wormhole.identity/1`|

* F# contains a [`Seq.windowed`](https://msdn.microsoft.com/en-us/visualfsharpdocs/conceptual/seq.windowed%5B't%5D-function-%5Bfsharp%5D) function, but it only moves the chunk one element at a time.

# Why Is This Stuff Useful?

Well, some of the functions are either self-explanatory or already written about in several other articles. I'll cover some of the lesser known ones and why I personally found them useful.

## Chunking

I've written about `chunk` and `chunk_by` before, but in case you missed it, check out my previous article!

{% link https://dev.to/jdsteinhauser/alright-break-it-up-using-partitionchunk-h15 %}

## Reduce While

I'll admit that this is possibly a not-so-often used case. Sometimes you don't want to reduce an entire sequence - just up to a certain point. Unfortunately, `reduce` is typically all or nothing. That doesn't really work when you have a potentially infinite series of data. However, Elixir's `reduce_while` helped me keep my solution for [AoC 2018 Day 1 Part 2](https://adventofcode.com/2018/day/1) compact. I'm hoping to find more real-world use cases for it... but it's still one of my favorite data processing functions I've found.

## Juxt

While I admit that, at first glance, `juxt` is nothing special. Take an array of functions that operate on the same parameters, and then return a single function that takes that parameter and returns an array of each function run on those parameters? Why use that?

I've ported this function from Clojure into other work projects before. For instance, I had a *very* large collection of data (1MM+ entries!) and I couldn't afford to iterate over them multiple times. I used `juxt` to compose my analysis functions together so that I only had to iterate over the collection one time.

Similarly, since a keyword in Clojure can be treated as a function for retrieving a value out of a map with that key (`(:foo {:foo 5 :bar 3})` returns `5`), you can compose several keywords for accessing data out of a collection of maps and returning the results in kind of like a table format. I wrote about that as part of a previous post on dense Clojure code:

{% link https://dev.to/jdsteinhauser/a-verbose-explanation-of-compact-code-546g %}

## Frequencies

Because sometimes, you just need a histogram. `frequencies` provides that in one single function!

# Conclusion

Hopefully someone out there will find this useful, either as a cheat sheet or as a library. In the near-term, I will be investigating Ruby and Rust (in that order) to see what other handy functions I could foresee using across multiple languages. I'll also put Wormhole up as a package in your favorite package managers soon, and probably write about the things I do/don't like about each.

Happy coding, and I'd love to hear about other general purpose data manipulation functions you've found useful!