JavaScript: How to Remove Duplicate Values from Arrays
Tags: javascript, webdev, tutorial
Published: 2020-07-03 18:15:06

*Originally posted on [Will's blog](https://www.willharris.dev/garden/remove-array-duplicates)*

---

In a [previous post](https://bikesandbytes.net/check-for-array-duplicates) we saw how to determine whether a JavaScript array contains duplicate values. Today, I want to show a few different methods I've found for removing duplicate values from an array.

## Using the `Array.prototype.filter()` & `Array.prototype.indexOf()` methods

```js
let originalArray = [1, 2, 3, 4, 1, 2, 3, 4]

let uniqueArray = originalArray.filter((item, index, array) => {
  return array.indexOf(item) === index
})

// uniqueArray === [1, 2, 3, 4]
```

The basic strategy here is to iterate through `originalArray` and check to see if the index of the item we are currently examining is the same as the index of the item in the `originalArray`.

Because `indexOf` returns the first index that it finds for a given value, if it isn't a duplicate value then the index for that item must be the same!

Note that this method is not the most efficient: it executes in quadratic time. So if the arrays you're checking are very large in size, you may want to use a different method.

Another thing worth nothing is that we can use the same method to return only the duplicate values by inverting the comparison:

```js
let duplicateArray = originalArray.filter((item, index, array) => {
  return array.indexOf(item) !== index
})
```

## Using `Array.prototype.reduce()` & `Array.prototype.includes()`

```js
let originalArray = [1, 2, 3, 4, 1, 2, 3, 4]

let uniqueArray = originalArray.reduce((unique, item) => {
  unique.includes(item) ? unique : [...unique, item]
}, [])

// uniqueArray === [1, 2, 3, 4]
```

Here the strategy is to keep a running list of the unique items in our reducer function's 'accumulator' (`unique`). For each item in `originalArray` we check to see if the accumulator includes the item under consideration.

- If it does contain the item, return the accumulator without making any changes, in effect 'skipping over' that item.
- If it does not contain the item, spread the values in the accumulator into a new array, and add the item under consideration.

`Array.prototype.includes` returns a boolean value -- `true` if the value is found in the array, `false` if not. This boolean value drives our conditional, determining what to do with each value.

I find this approach less intuitive and harder to read, but it works.

Also note that the empty array that is passed in after the reducer function is the starting value for the accumulator, so the first pass through the `reduce`, `unique` is an empty array.

## ⚡ Using the ES6 `Set` object ⚡

```js
let originalArray = [1, 2, 3, 4, 1, 2, 3, 4]

let uniqueArray = array => [...new Set(array)]

// or

let uniqueArray = Array.from(new Set(originalArray))

// uniqueArray = [1, 2, 3, 4]
```
This approach harnesses the power of the `Set` object, introduced in ES6.

Sets are guaranteed to preserve the order of the inserted items, and to only contain unique values. Therefore it is by definition impossible for a set to contain duplicates!

Here we call the `Set` object's constructor, passing it the array we'd like to construct a `Set` from. Then, once we've trimmed out all the duplicates and stored the remaining values in our `Set`, we convert back to an array and return the result.

I've seen some discussion of this approach being a bit less performant if the array under consideration is very large and contains many duplicate values. However, the same discussion found that this approach is very efficient in a scenario where the data has very few duplicates.

Personally I think the conciseness of this last approach is enough of a benefit to warrant using the `Set` object approach, unless there's a compelling performance reason not to.