Powerful operators for effective JavaScript: "map" and "flat map" (not exclusive to arrays)
Tags: javascript, functional
Published: 2020-03-02 02:57:27

---
title: Powerful operators for effective JavaScript: "map" and "flat map" (not exclusive to arrays)
published: true
description: Rather than using loops for synchronous code, and writing some elaborate event processor, consider thinking of everything as containers, and apply `map`, and `flatMap`.
tags: javascript, functionalprogramming
---

You have an array. Given that array's elements, you want to create an entirely new array, with the data being different from the original. For-loops have been historically the construct used for such a task.

But in this article, I aim to introduce to you to two operators, that, when composed, can yield highly expressive code, and potentially improve your productivity. These operators are `map` and `flatMap`.

Along with the `map` and `flatMap` operators, I aim to have you think about where data originates from, and how they are stored, and how one can use `map` and `flatMap` to derive richer data.

I also aim to show you that `map` and `flatMap` can be used with just about any types that "hold" data.

By composing these operators, you will be able to work with clean and rich data, that are distinct from the source data, and allow you to quickly re-think how your application uses it.

## Synthesizing a new array

You have an array of objects, and each object represents a comment. Each comment object has a `date` field.

However, that field is a string, and—as the name would imply—represents the date.

```javascript
// Dummy data.
//
// Array of comments.

const comments = [
  {
    content: 'This is awesome',
    date: '2019-10-12'
  },
  {
    content: 'This is rad',
    date: '2019-11-05'
  },
  {
    content: 'I like your post!',
    date: '2020-01-12'
  },
  // ...
]
``` 

Given this array, you want to generate a whole new array of objects, with the `date` field converted to a JavaScript `Date` object.

In older versions of JavaScript, before the `map` method was added to arrays, for-loops were useful.

It will involve initializing an empty array, iterating through the previous array, and pushing the new object into the new array.

```javascript
// In a much, much older version of JavaScript, this is what people used to do.

const commentsWithDate = [];

for (let i = 0; i < comments.length; i++) {
  const currentComment = comments[i];

  commentsWithDate.push({

    ...currentComment,
    date: new Date(currentComment)

  });
}
```

Iterating through an array is a very common task. With a for-loop, it involves initializing a number to `0`, checking that it's less than the length of the array, and incrementing it. This becomes repetitive, and possibly error prone.

Thus, the `map` method was added to JavaScript (eventually, [iterables](https://javascript.info/iterable) became a thing. It didn't only become an idiom, but an important part of JavaScript. And eventually, for-of-loop was also introduced). Replacing the above with an invocation of `map` would look like so:

```javascript
// The following code achieves exactly the same outcome as the above for-loop
// example.
//
// The code below is using array.map for creating a new array.

const commentsWithDate = comments.map(comment => {
  
  // Rather than push to a new array, just return the new object, and it will
  // be appended into the new array for you.
  return {

    ...comment,
    date: new Date(comment)

  };

});
```

**Bear in mind that the concept of `map` is not exclusive to arrays.**

Any container types (even if the container type holds, by definition, only a single value) may have `map` implemented for it. More on this later.

### Exercise

1. Look for code that you've written, or code on GitHub that you find that synthesize new arrays. Are they pushing to arrays for synthesis, or are they using `map`? If they are pushing to arrays, try to see if you can refactor it to use `map`.
2. Try to imagine any container types, other than arrays (hint: JavaScript's `Map` and `Set` collections are such types). Try to implement some `map` function for them

## Joining things

Let's say you are re-implementing the ubiquitous instant messaging app, Slack.

Slack has a feature where you can view all (unread) messages, across all channels.

Let's re-implement that feature. But we'll keep it simple. We will only implement the ability to view _all_ messages (whether read or unread), across all channels, at a glance.

This is what the array object will look like:

```javascript
// Dummy data

const channels = [
  {
    tag: 'watercooler',
    messages: [
      {
        body: 'What\'s for lunch, today?',
        timestamp: '2020-03-01T01:42:17.836Z'
      },
      {
        body: 'I don'\t know. Let\'s have some tacos',
        timestamp: '2020-03-01T01:42:48.922Z'
      },
    ]
  },
  {
    tag: 'development',
    messages: [
      {
        body: 'Is anyone willing to get into a pair-programming session?',
        timestamp: '2020-03-01T01:43:09.339Z'
      },
      {
        body: 'Hit me up in five minutes. I may be able to help.',
        timestamp: '2020-03-01T01:44:00.590Z'
      },
    ]
  },
  {
    tag: 'product',
    messages: [
      {
        body: 'Does anyone have the analysis of last week\'s A/B test?',
        timestamp: '2020-03-01T02:04:41.258Z'
      },
      {
        body: 'It\'s in the Dropbox Paper document, titled "A/B test 2020-02',
        timestamp: '2020-03-01T02:04:49.269Z'
      },
    ]
  }
]
```

The `channels` variable is an array, that has objects, that each object has a field `messages`, which are the messages.

A solution would be to iterate through each channel, then iterate through each messages per channel.

```javascript
const messages = [];

for (const channel of channels) {
  for (const message of channel.messages) {
    messages.push(message);
  }
}
```

If you wanted to avoid pushing to an array, one can use `flatMap`.

The `flatMap` method joins all arrays returned by the mapping function.

```javascript
const messages = channels.flatMap(channel => {
  return channel.messages
});
```

### Formality of flatMap

Given some container type (such as an array), there exists a method called `flatMap`, which accepts a callback. That callback accepts a value of the type that the container type holds. The callback returns another container, whos values may not be the same as the original container type.

```javascript
someContainer.flatMap(theValue => {
  const somethingDifferent = transform(theValue);

  return createSomeContainer(theValue);
});
```

`flatMap` will unwrap each item in the container type, and invoke the callback with the value. The callback will then return a container, and `flatMap` will unwrap the value, and return an entirely new container.

### Exercise

1. From the above message retrieval example, modify `flatMap` code so that messages have a `date` field that is a `Date` object representation of `timestamp` (note: just calling `new Date(message.timestamp)` will suffice)
2. Look for code that you've written, or code on GitHub that you find that synthesize new arrays. Are they pushing to arrays for synthesis, or are they using `flatMap`? If they are pushing to arrays, try to see if you can refactor it to use `map`.
3. Try to imagine any container types, other than arrays (hint: JavaScript's `Map` and `Set` collections are such types). Try to implement some `flatMap` function for them

## Why not mutate the original array?

One of he most commonly touted benefits of using `map` and `flatMap` is that it avoids mutations. Many would say that mutations are major source of software faults. That's one reason.

Another reason is that although the source data has everything we need, certain aspects of our applications may require it in specific formats. And it's not just one or two aspects of our applications, but possibly dozens. Mutating the source can result in application code that is hard to manage.

Thus, you don't mutate, but synthesize. The original data remains untouched, but aspects of your code can still benefit from getting the data in the format that it expects.

## Promise's `then` method are like `map` and `flatMap`

The `then` method in promises act like both `map` and `flatMap`.

Let's say you issue a REST API call to get an article. You can invoke `then` to gain access to the retrieved article. But, you can derive an entirely new promise, by returning an entirely new object.

Thus, we are using `then` like `map`.

```javascript
getArticle().then(article => {
  return {
    ...article,
    date: newDate(article.date)
  };
}); // Returns a promise, with a comment that has a `Date` object attached.
```

Using `then` like `flatMap`, if you wanted to get comments from an article, you'd invoke it like so.

```javascript
// Here are using `then` like flatMap.
getArticle().then(article => {
  const commentsPromise = getComments(article.id);

  // Here, we are using `then` like `map`.
  return commentsPromise.then(comments => {
    return comments.map(comment => {
      ...comment,
      date: new Date(comment.date)
    });
  });
});
```

### Exercise

1. In the second promise example, refactor it so that the second `then` is no longer nested inside the callback
2. What if you wanted to not only return just the comments, but have the comments embedded in the `post` object. Would you be able to remove the nesting? If not, why?

## Functors (those that work with map) and monads (those that work with flatMap)

A container type that works with the `map` function is a functor. Both arrays, and promises are examples of functors.

A container type that works with the `flatMap` function is a monad. Both arrays, and promises are examples of monads.

Actually, you can turn just about any container type in a functor and/or a monad.

## Iterables as functors (map) and monads (flatMap)

Just as a primer, arrays are iterables. As iterables, you can splat them into arrays and function parameters, as well as iterate through them using `for-of`.

```javascript
// Some silly array example.
const arr = [ 1, 2, 3 ];

for (const el of arr) {
  // `el` should be an element of arr
}

// Splat into an array
const newArr = [ ...arr ];

// Splat into function parameter
someFunction(...newArr)
```

**But remember**: all arrays are iterables, but not all iterables are arrays.

And hence you don't enjoy the luxuries afforded to you by JavaScript arrays.

However, they are a container type that you can extract their values from.

Iterables don't natively have any `map` or `flatMap` function defined. Fortunately, we can define them ourselves.

We will use generator function for that. Generators return iterables.

With generators, we can use the `yield` keyword to simplify our lives.

```javascript
function * map(iterable, callback) {
  for (const value of iterable) {
    yield callback(value);
  }
}
```

Likewise, for `flatMap`.

```javascript
function * flatMap(iterable, callback) {
  for (const value of iterable) {
    for (const subIterable of callback(value)) {
      yield value;
    }

    // Actually, we could have just used `yield * callback(value)`, but those
    // who are not initiated with generators, this may be a bit much to digest.
  }
}
```

Then, perhaps we have posts stored in something other than an array, but is iterable, we can `map` each value to get the date.

```javascript
const commentsWithDate = map(comments, comment => {
  return {
    ...comment,
    date: new Date(comment.date)
  }
});

// Note: `commentsWithDate` is an iterable; not an array
```

Also with the example of getting messages from channels, we can do the following:

```javascript
const messages = flatMap(channels, channel => {
  return channel.messages;
});

// the above variable will now be a flat iterable of messages; not channels
```

And, if we are to take the above example, and have it so that the messages have a date field:

```javascript
const messages = flatMap(channels, channel => {
  return map(channel.messages, message => {
    return { ...message, date: new Date(message.timestamp) };
  });
});

// the above variable will now be a flat iterable of messages; not channels
```

### A helper library: IxJS

Above, I have introduced `map` and `flatMap` for iterables.

One problem with the above, however, is it requires that we pass in the iterable as a first parameter. Composing `map` and `flatMap` results in nesting, and renders it rather difficult to follow logically which operations are happening in what order.

IxJS introduces an iterator object that exposes a `pipe` method. This way, you are able to compose `map` and `flatMap`.

Here's what the above code would look like with IxJS

```javascript
const results = from(channels).pipe(
  flatMap(channel => channel.messages),
  map(message => ({ ...message, date: new Date(message.timestamp) }))
);
```

## Other examples: RxJS and observables

### Eventing primer

If you wanted to listen in on user mouse clicks, you can attach an event listener (in the form of a callback function) to the mouse click event.

Below is an example of listening to a click events on a button.

```javascript
button.addEventListener('click', event => {
  alert(`Button clicked at coordinate (${event.screenX}, ${event.screenY}).`);
});
```

The paradigm applied in the application programming interface (API) above is that the button itself is an event emitter. It's an event emitter because it exposes a method named `addEventListener`, and you attach an event listener, provided some event tag (in this case, the `'click'` event).

An alternative paradigm that browsers could have opted for instead, is that the button holds, as a property, an object that represents an event emitter (also referred to as an event stream).

So here's what the API could have looked like, had browsers instead opted for the paradigm that I am proposing:

```javascript
button.clickStream.subscribe(event => {
  alert(`Button clicked at coordinate (${event.screenX}, ${event.screenY}).`);
});
```

In the last example, `clickStream` is an object. And since it's an object that has the single responsibility of notifying click events, we can grab a hold of it, and apply all sorts of operators of our choosing.

Of course, in the context of this post, it's a perfect candidate for `map` and `flatMap`.

### Observables as functors (map) and monads (flatMap)

Earlier, I mentioned that you can think of promises as a container type. But if you are familiar with them, they are what are often returned after some asynchronous call (such as AJAX request, file read, etc.).

Nevertheless, it helps to think of them as container types; they asynchronously "hold" a value, that is exposed through callbacks. This is related to how `then` can act both like `map` and `flatMap`.

RxJS introduces a notion called "observables". Observables differ from promises in that promises represent a single instance of a value, where as observables represent a stream of values.

Like promises, we can treat observables like container types.

RxJS observables have a `pipe` method, that you can apply the `map` and `flatMap` functions to.

### Chat application notification example using RxJS observables

Throughout this post, I made repeated references to instant messaging (IM) applications. This is because IM apps are very event-driven.

The two events that we will concern ourselves with is

- when a contact logs in
- when a contact sends a direct message

For our IM app, we will have a server. We interface with it via a REST or GraphQL API (detail doesn't matter), as well as WebSocket for streams of data. It's through WebSocket that we will be subscribing to events.

Through WebSocket, our server supports subscribing to these two events, for now:

- users coming online
- which user sent us a message (note: when the user logs out, the subscription closes)

First, we want to listen in on events of a user logging in.

Below is a simple listener to our server for that very event.

```javascript
const socket = new WebSocket(`${endpoint}/log-ins`);

socket.on('message', (data) => {
  // Do whatever, here.
});
```

For the purposes of this article, I want to make use of RxJS, as much as possible.

With RxJS, we can convert an event emitter into an observable.

Let's convert the above socket message event into an observable.

```javascript
import { fromEvent } from 'rxjs';

const socketMessageStream = fromEvent(socket, 'message');
```

Next, we will filter for only valid JSON messages, and convert them to valid JavaScript objects.

We will peruse the RxJS `filter` operator.

The purpose of the `filter` operator is to generate a new stream, for all events that only test true, according to a callback (that callback has a fancy term, and it's called a predicate).

Let's create an RxJS operator that will filter only for valid JSON, and transform them into objects.

```javascript
/**
 * Operator for filtering out invalid JSON, and converting the messages to
 * objects.
 */
function onlyValidObjects(source) {

  return source.pipe(

    // Get all valid messages that can be converted to an object.
    filter(message => {
      try {
        JSON.parse(message.toString('utf8'));
      } catch (_) {
        return false;
      }
    }),

    // Convert those messages to 
    map(message => JSON.parse(message.toString('utf8')))

  );

}
```

Next create a stream of only valid events.

```javascript
import { filter, map } from 'rxjs/operators';

// All messages are valid objects.
const validEvents = socketMessageStream.pipe(onlyValidObjects);
```

Next, we filter exclusively for messages that are login events.

```javascript
import { filter } from 'rxjs/operators';

// loginStream will only have events that will exclusively hold log-in events.
const loginStream = socketMessageStream.pipe(
  filter(message => {
    return message.type === 'login';
  })
);
```

Finally, from the login stream, get a new stream of *all* new messages from all logged-in users.

For every login, this requires listening to new messages, from every user that logs in.

Listening to new messages from every new login can be their own event stream. This results in "nested" event listeners.

This is now a call for `flatMap`. The following demonstrates the use of `flatMap` to flatten out the messages stream.

```javascript
import { flatMap } from 'rxjs/operators';

const messageStream = loginStream.pipe(
  flatMap(user => {

    const instantMessages = new WebSocket(`${endpoint}/messages/${user.id}`);
    return fromEvent(instantMessage, 'message').pipe(onlyValidObjects);

    // Note: we don't have to worry about logging out, since `instantMessages`
    // closes the moment the user logs out.

  })
);
```

Now, with `messagesStream`, we are free to interpret the stream however we wish.

The first of which is to send a push notification to the browser.

```javascript
messageStream.subscribe(message => {
  notify(`${message.user.name} sent ${message.body}`);
});

// You can implement the `notify` function by following the "Example" section on
// MDN.
//
// https://developer.mozilla.org/en-US/docs/Web/API/notification
```

If we're using React, we should be able to add a counter.

Below is a React component that subscribes to the message stream, and for every message, increment the counter.

```javascript
import React, { useState, useEffect } from 'react';

/**
 * A counter that increments on every new message.
 */
function Counter() {

  const [ count, setCounter ] = useState(0);
  
  useEffect(() => {
    const subscription = messageStream
      .subscribe(() => { setCounter(count + 1); });

    return () => {
      subscription.unsubscribe();
    };
  }, []);

  return (
    <div>
      Notifications {count}
    </div>
  );

}
```

### Exercise

1. As is evidenced by `filter`, the `map` and `flatMap` operators aren't the only RxJS operators. Consider looking into using other operators from RxJS, and see what you can eventually build (hint: consider the `scan` operator)
2. Just like RxJS observables, `map` and `flatMap` aren't the only operators for arrays. Look at the [MDN documentation on arrays](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array), and explore all the other operators available. 

## Takeaways

- rather than relying on loops and actively mutating data, `map` and `flatMap` can be composed to synthesize new data
- these operators allow you to iterate quickly by limiting yout work mostly from synthesized clean but rich data
- `map` and `flatMap` can be applied to more than just arrays. As long as there exists some data holder, you can extract it, and apply operators to them, and derive new data
- streams are such examples of data holders (even if data is not yet available). RxJS re-imagines how events should be handled. Thus, as demonstrated above, `map` and `flatMap` were applied to

## In Closing

With `map` and `flatMap`, you are given two very powerful operators to work with data, without mutating them.

Hopefully, you will now be able to quickly iterate on new features for your JavaScript-related products (and even in other environments other than JavaScript).