Using array map, filter and reduce in MongoDB aggregation pipeline
Tags: mongodb, javascript
Published: 2021-07-31 20:46:34

If you work with javascript, the chances of you are using array methods like **map**, **filter** and **reducer** today are really great. 

All simplicity offered by javascript [higher-order functions](https://blog.bitsrc.io/understanding-higher-order-functions-in-javascript-75461803bad) makes our code more readable and concise, mainly when we work with array data transformations.

Let's remember these methods:

```js
const numbers = [2, 8, 15];

const greaterThanFive = (num) => num > 5;
const multiplyBy2 = (num) => num * 2;
const sum = (acc, num) => acc + num;

const filtered = numbers.filter(greaterThanFive);
const mapped = numbers.map(multiplyBy2);
const reduced = numbers.reduce(sum);

console.log(filtered); // [8, 15]
console.log(mapped); // [4, 16, 30]
console.log(reduced); // 25
```

That's really amazing! 

However, in databases scenario, querying data with this simplicity is usually unusual, unless that database is **MongoDB**. 

Because MongoDB is a [NoSQL](https://www.mongodb.com/nosql-explained) database with JSON based model, some javascript array methods have similar [expression operators](https://docs.mongodb.com/manual/reference/operator/aggregation/#array-expression-operators)
 in MongoDB Aggregation Pipeline. 

About its JSON nature, official website cites:

> MongoDB’s document data model naturally supports [JSON](https://www.mongodb.com/json-and-bson) and its expressive query language is simple for developers to learn and use.

And that makes all difference folks...

Let's get `numbers` array data used in the javascript example to create a new document in a generic collection. For improve the understanding, I will use MongoDB Playground to test our queries:

```json
[
  {
    "numbers": [
      2,
      8,
      15
    ]
  },
]
```
[Mongo Playground](https://mongoplayground.net/p/kOCZ5hJSqOf)

Good! Our collection is ready to receive queries now :)

## $filter

Starting, let's use [$filter](https://docs.mongodb.com/manual/reference/operator/aggregation/filter/) aggregation pipeline operator.

**Query**
```js
db.collection.aggregate([
  {
    $project: {
      _id: 0,
      filtered: {
        $filter: {
          input: "$numbers",
          as: "num",
          cond: {
            $gt: [
              "$$num",
              5
            ]
          }
        }
      }
    }
  }
])
```

- Start using `aggregate` method to submit the query. That method enables aggregation framework;
- Pipeline starts using `$project` aggregation pipeline stage. The specified fields inside it can be existing fields from the input documents or newly computed fields. In our case, `filtered` field will be created and added to response;
- The computed value for `filtered` field will be given by 
`$filter` aggregation pipeline operator;
- Inside filter operator, set *input* to `$numbers`. That's our array to be iterated;
- Set *as* to `num` to get each array value to test in filter condition. You could use any name here, just like you did in javascript filter method;
- Then, set the filter condition in `cond` using `$gt` expression to return a boolean if current array value `$$num` is greater than 5;

**Response**
```json
[
  {
    "filtered": [
      8,
      15
    ]
  }
]
```
[Mongo Playground](https://mongoplayground.net/p/KNtosi_ymwJ) 

## $map

[$map](https://docs.mongodb.com/manual/reference/operator/aggregation/map/) operator is pretty similar to `$filter`, however while `$filter` needs a condition, `$map` you must set `in` to output a new array value according to some rule.

**Query**
```js
db.collection.aggregate([
  {
    $project: {
      _id: 0,
      mapped: {
        $map: {
          input: "$numbers",
          as: "num",
          in: {
            $multiply: [
              "$$num",
              2
            ]
          }
        }
      }
    }
  }
])
```
In case, using `$multiply` expression to return all array values multiplied by 2.

**Response**
```json
[
  {
    "mapped": [
      4,
      16,
      30
    ]
  }
]
```
[Mongo Playground](https://mongoplayground.net/p/EuZAMEyeVz_) 

## $reduce

[$reduce](https://docs.mongodb.com/manual/reference/operator/aggregation/reduce/) operator applies an expression to each element in an array and combines them into a single value.

**Query**
```js
db.collection.aggregate([
  {
    $project: {
      _id: 0,
      reduced: {
        $reduce: {
          input: "$numbers",
          initialValue: 0,
          in: {
            $sum: [
              "$$value",
              "$$this"
            ]
          }
        }
      }
    }
  }
])
```
- Again, set `$numbers` array as `input` to iterate;
- The initial cumulative value set before `in` is applied to the first element of the input array, `initialValue` is set to 0;
- Finally, `in` expression give us two special variables: `$$value` is the variable that represents the cumulative value of the expression (*`acc` in javascript example*) and `$$this` is the variable that refers to the element being processed (*`num` in javascript example*). In case, using `$sum` expression to return the new accumulated value.

**Response**
```json
[
  {
    "reduced": 25
  }
]
```

[Mongo Playground](https://mongoplayground.net/p/AmXgwpR3lKp) 

## All in one

In previous examples, we worked with each operator in a separated query, however we could do a single query requesting all operators at once.

**Query**
```js
db.collection.aggregate([
  {
    $project: {
      _id: 0,
      filtered: {
        $filter: {
          input: "$numbers",
          as: "num",
          cond: {
            $gte: [
              "$$num",
              5
            ]
          },
          
        }
      },
      mapped: {
        $map: {
          input: "$numbers",
          as: "num",
          in: {
            $multiply: [
              "$$num",
              2
            ]
          }
        }
      },
      reduced: {
        $reduce: {
          input: "$numbers",
          initialValue: 0,
          in: {
            $sum: [
              "$$value",
              "$$this"
            ]
          }
        }
      }
    }
  }
])
```
**Response**
```json
[
  {
    "filtered": [
      8,
      15
    ],
    "mapped": [
      4,
      16,
      30
    ],
    "reduced": 25
  }
]
```
[Mongo Playground](https://mongoplayground.net/p/VbBLVHR1Yzh)

Going further, if you add more documents to collection, this same query computes data for each of them. Let's query a collection with 3 documents now:  

**Collection**
```json
[
  {
    "numbers": [
      2,
      8,
      15
    ]
  },
  {
    "numbers": [
      4,
      8,
      9,
      13
    ]
  },
  {
    "numbers": [
      1,
      3,
      7
    ]
  }
]
```
**Response**
```json
[
  {
    "filtered": [
      8,
      15
    ],
    "mapped": [
      4,
      16,
      30
    ],
    "reduced": 25
  },
  {
    "filtered": [
      8,
      9,
      13
    ],
    "mapped": [
      8,
      16,
      18,
      26
    ],
    "reduced": 34
  },
  {
    "filtered": [
      7
    ],
    "mapped": [
      2,
      6,
      14
    ],
    "reduced": 11
  }
]
```
[Mongo Playground](https://mongoplayground.net/p/__W2NufcN7j)

## Conclusion

MongoDB for javascript developers is intuitive by nature! Aggregation framework does the hard work directly in the database server using many of features already known by us and data can be delivered ready-to-use, which normally decreases the workload for the application server.

See also the complete [Array Expression Operators](https://docs.mongodb.com/manual/reference/operator/aggregation/#array-expression-operators) list in MongoDB official website.