Back to the Basics: Insertion Sort
Tags: Algorithms, Insertion Sort, Bubble Sort

“Bubble Sort is to sorting algorithms as Singleton Pattern is to design patterns.”

Developer A: “What’s your favorite Design Pattern?”

Developer B: “Singleton — it’ can be applied to so many things.”

Developer A: “You don’t really know any other Design Patterns do you?”

Developer B: “Nope.”

In this section, we will look at the insertion sorting algorithm; another in a line of “NAIMC” algorithms (Not Appropriate In Most Cases). Both Bubble Sort and Insertion Sort have a worse case, quadratic time complexity of O(n²); in other words, the number of operations it performs grows in proportion to the square of its input.

What I mean by this is:

For an array of 10 elements there could be 10² or 100 operations to facilitate a complete sort.

For an array of 100 elements there could be 100² or 10000 operations.

For an array of 1000 elements there could be 1000² or 1,000,000 operations.

Now the million dollar question, “why learn these algorithms?” The answer is to develop better heuristics. Each of these algorithms force us to look at solving the sort problem in a different way.

Once we understand these first few sorting algorithms, we can build a foundation for more efficient and scalable algorithms

Insertion Sort Characteristics

Before we begin let’s draw some similarities and distinctions between bubble sort and insertion sort algorithms, and revisit some important characteristics of a sorting algorithm.

Both sorting algorithms are in place algorithms. In place algorithms are algorithms which transforms their input with out leveraging an auxiliary (or outside) data structure; and the input is usually overwritten by the algorithm itself. In other words if we have an array that we would like to sort, neither of these algorithms would necessitate creating another array. Instead we would use the array itself to bubble up, or insert values into our array indices. This is an important point when considering memory allocation.

Both sorting algorithms are stable algorithms. A sort is considered stable if when there are two objects with equal keys being sorted, the sorting algorithm preserves the order output as it was with the originating input. A sort is considered unstable if when two objects with equal keys are being sorted and the algorithm does not preserve order.

To drive this point home, lets take the following array:

If we were to run it through a stable sort algorithm, we would have the following:

Notice the bold 15 is still before the ‘green background’ 15. If we were to run it through an unstable sort we could end up with the following:

Notice that the ‘green background’ 15 is ordered before the bold 15.

In this example we are using integers and this is most likely not a problem. A 15 is a 15 who cares, however; consider a situation where we are trying to sort objects; we may want to preserve the order.

Insertion Sort Theory

Let’s now look at that algorithm’s theory.

Lets take a given array:

We will start at the first position of the array, and we will start looking at the array in partitions. The left side is the sorted partition and the right being unsorted. This is to say we will grow the sorted partition from left to right.

When starting our first iteration, we start at the first position of the array at position [0], by default our first element is considered sorted. and the elements in position [1], and onward are considered unsorted.

In our first iteration, we will compare the item in position [0] which is our sorted partition, against item in position [1]; and we will then insert the value of the element in position [1] to position [0], if when comparing the value in position [1] is less than or equal to the value of position [0].

Looking at the diagram above, notice we denote the left side in green which is ‘Sorted’; the sorted partition is only [0], so 88 is considered ‘sorted’. We look at position [1] (denoted in faded blue) and what we want to do is compare it against all the sorted items in our sorted partition, which again is only 88.

Lets take a look at the next iteration:

Now our sorting partition has two elements, elements at potion [0] and position [1] (denoted with a green background), with values of 18 and 88 respectively. We will compare them against element [2] (denoted with faded blue text). We will take item in element [2] of the array, and start ‘walking up’ the sorted array, comparing element [2] against items in the sorted array looking to insert the value in proper position.

We take the current element [2], value being 59, and while its less than the values in the sorted array we swap elements. We compare 59 and 88, so element [2] becomes 88, and insert 59 into element [1].

Let’ s move to Iteration 3.

Again, at the start of iteration 3, we have a sorted partition in our array now with 3 elements at position [0][1] & [2] with values of {18, 59, 88}.

And the element at position[3] will be the next inline for our unpartitioned element, or key, to be sorted. In this iteration the value in position [3] is 1, and while it’s less than the values in the sorted array we swap elements. So when evaluating 18, 59, 88, element [3] becomes 88, and insert 1 into element [0].

To continue on:

Implementation Detail

Let’s take a look at the the code. To drive this point home, I will write this using JavaScript but will do so avoiding high order functions in an attempt to explain the algorithm following imperative language constructs.

Lets start by creating a loop over the array:

const arr = [ 88, 18, 59, 1, 8];

for (let index = 1; index < arr.length; ++index) { let currentValue = arr[index];

let previousPosition = index — 1;

}

Notice that we start the array at position [1] , if you remember, this is because we start with the partition of sorted and unsorted. Within the loop we do two things:

Create a variable that stores the current value. Create a variable that stores previous position of the iteration.

So now we need to create the mechanism that will compare the current value against the sorted partition and insert the current value into the correct position.

while (previousPosition >= 0 && arr[previousPosition] > currentValue) { arr[previousPosition + 1] = arr[previousPosition];

previousPosition = previousPosition — 1;

} arr[previousPosition + 1] = currentValue; } return arr;

The above portion is really the ‘substance’ of this algorithm. So let’s look at it in detail. At this point we are already within a for loop. The for loop represents our unsorted array which is why we start at position [1] not zero. Remember, again, position [0] is our sorted partition.

Lets focus for a second on this while loop we can think of this as, ‘now iterating over our sorted partition’ we sort while two things are to be true.

We haven’t past the beginning position (index) of our array. The sorted element is smaller than the current value.

If both those criteria are met we want to move the value in the sorted array down one to make room for our insert.

Once we are either at the beginning of the array or the sorted element is larger than the current unsorted element; we place the current element in the current position we are in.

Putting it all together:

Criticism of the code above

Before we cut and paste the code above into our production applications, I feel like it is extremely important to talk about ways to make the code better. After all, anytime you step away from the keyboard to analyze and iterate over your code, you should always ask yourself how can we make it better?

First and foremost, keep in mind that the code above is using javascript. Typically in our javascript applications, we do NOT like to mutate our arrays. For those who have worked with Redux may understand the dilemma. Let me explain:

If data is immutable, then checking whether the state has changed is trivial, If the data is mutable, change detection is much harder. If you return the same object that was passed in as the “last known state”, redux will assume your state hasn’t changed and will not notify your application.

Moreover, we end up with odd bugs. Suppose the possibility of code elsewhere in the application which still has a reference to the same array, as it has been mutated but not replaced. This can result in very hard to identify / reproduce bugs.

In addition, the exercise is to explain the algorithm not write it the most elegant fashion. Anytime we write javascript, we should consider composition and attempt to utilize higher order functions.

Summary

Insertion in a nutshell — it is an in place, quadratic algorithm with a time and space complexity of:

Time Complexity: O(n*2) Auxiliary Space: O(1)

You should use this algorithm, [mostly] never; however, now when you are at a cocktail party and are ‘working the room’ you can hold a fascinating conversation about this sorting algorithm. But seriously, this starts to lay the foundation for the next few algorithms that are in the Back to the Basic series that are useful. Can you say Divide-and-Conquer Algorithms 😄

If you enjoyed this please hit the follow button. If you have any questions, or if I got anything incorrect, please leave a comment below!