Removing duplicates with Map in JavaScript
Tags: javascript, webdev, react
Published: 2019-11-18 18:54:57

_The article was originally posted on [my personal blog](https://claritydev.net/blog/removing-duplicates-with-map-in-javascript)._


It is quite common to use `Set` to remove duplicate items from an array. This can be achieved by wrapping an existing array into `Set` constructor and then transforming it back into array:
```javascript
const arr = [1, 2, 'a', 'b', 2, 'a', 3, 4];
const uniqueArr = [...new Set(arr)];

console.log(uniqueArr); // [1, 2, "a", "b", 3, 4]
```
This works great for arrays of primitive values, however when applying the same approach to array of arrays or objects, the result is quite disappointing: 
```javascript
const arr = [[1, 2], {'a': 'b'}, {'a':2}, {'a':'b'}, [3, 4], [1, 2]];
const uniqueArr = [...new Set(arr)];

console.log(uniqueArr); // [[1, 2], {'a': 'b'}, {'a':2}, {'a':'b'}, [3, 4], [1, 2]]
```
This is because `Set` compares non-primitive values by reference and not by value, and in our case all values in array have different reference. 

A bit less-known fact is that `Map` data structure maintains key uniqueness, meaning that there can be no more than one key-value pair with the same key. While knowing this wont help us magically transform any array into array of unique values, there are certain use cases which can benefit from Map's key uniqueness. 

Let's consider a sample React app, which displays a list of books and a dropdown that allows filtering books by their authors. 
```javascript
const App = () => {
  const books = [
    {
      id: 1,
      name: "In Search of Lost Time ",
      author: { name: "Marcel Proust", id: 1 }
    },
    { id: 2, name: "Ulysses", author: { name: "James Joyce", id: 2 } },
    {
      id: 3,
      name: "Don Quixote",
      author: { name: "Miguel de Cervantes", id: 3 }
    },
    { id: 4, name: "Hamlet", author: { name: "William Shakespeare", id: 4 } },
    {
      id: 5,
      name: "Romeo and Juliet",
      author: { name: "William Shakespeare", id: 4 }
    },
    { id: 6, name: "Dubliners", author: { name: "James Joyce", id: 2 } }
  ];
  const [selectedAuthorId, setSelectedAuthorId] = useState(null);

  const filteredBooks = () => {
    if (!selectedAuthorId) {
      return books;
    }
    return books.filter(book => String(book.author.id) === selectedAuthorId);
  };

  return (
    <div className="books">
      <select
        className="books__select"
        onChange={({ target }) => setSelectedAuthorId(target.value)}
      >
        {/*Show author options*/}
      </select>
      <ul className="books__list">
        {filteredBooks().map(book => (
          <li className="books__item">
            {book.name} by {book.author.name}
          </li>
        ))}
      </ul>
    </div>
  );
};
```
For simplicity `books` array is hardcoded here, although in a real-world app the data will be probably fetched from an API. 

The app is almost complete, we just need to render the dropdown of authors to filter by. A good way to approach it would be to collect the `id` and `name` of each author from our list of books into a separate array and render it as options inside the `select`. However, there's one condition - this list should contain only unique authors, otherwise authors of more than one book will appear in the dropdown multiple times, which is something we don't want to happen. We need both `id` for option's `value` and `name` to display the option's label and since the author's data is contained inside an object we can't just apply the `Set` trick to get unique values only. One option would be to first get all the `id`s for the authors into an array, then apply `Set` to it in order to get unique ones and after that iterate over the authors array once more to collect their names based on the `id`s. That sounds like a lot of work, and luckily there's an easier solution.

Considering that we basically need an array of `id` - `name` pairs, we can extract those from the `books` list and transform them into a `Map`, which would automatically take care of preserving only the pairs with unique keys. 
```javascript
  const authorOptions = new Map([
    ...books.map(book => [book.author.id, book.author.name])
  ]);
```
That's it! Now we have a Map of unique key-value pairs, which we can feed directly into our select component. 

It's worth keeping in mind that when `Map` preserves key uniqueness, the last inserted item with the existing key stays in the Map, while previous duplicates are discarded. ```javascript
const map1 = new Map([[1,3], [2,3]]);

const map2 = new Map([[1,2]]);

var merged = new Map([...map1, ...map2]);

console.log(merged.get(1)); // 2
console.log(merged.get(2)); // 3
```

Luckily, in our example app all the author `id` - `name` pairs are unique so we don't need to worry about accidentally overriding any data. 

Now we can combine everything into the final version of the component.
```javascript
const App = () => {
  const books = [
    {
      id: 1,
      name: "In Search of Lost Time ",
      author: { name: "Marcel Proust", id: 1 }
    },
    { id: 2, name: "Ulysses", author: { name: "James Joyce", id: 2 } },
    {
      id: 3,
      name: "Don Quixote",
      author: { name: "Miguel de Cervantes", id: 3 }
    },
    { id: 4, name: "Hamlet", author: { name: "William Shakespeare", id: 4 } },
    {
      id: 5,
      name: "Romeo and Juliet",
      author: { name: "William Shakespeare", id: 4 }
    },
    { id: 6, name: "Dubliners", author: { name: "James Joyce", id: 2 } }
  ];
  const [selectedAuthorId, setSelectedAuthorId] = useState(null);

  const authorOptions = new Map([
    ...books.map(book => [book.author.id, book.author.name])
  ]);

  const filteredBooks = () => {
    if (!selectedAuthorId) {
      return books;
    }
    return books.filter(book => String(book.author.id) === selectedAuthorId);
  };

  return (
    <div className="books">
      <select
        className="books__select"
        onChange={({ target }) => setSelectedAuthorId(target.value)}
      >
         <option value=''>--Select author--</option>
        {[...authorOptions].map(([id, name]) => (
          <option value={id}>{name}</option>
        ))}
      </select>
      <ul className="books__list">
        {filteredBooks().map(book => (
          <li className="books__item">
            {book.name} by {book.author.name}
          </li>
        ))}
      </ul>
    </div>
  );
};

```