Insertion-Sort — Everything you need to know!
By: Leonardo Sampaio De Sousa
Tags: Algorithms, Computer Science, Sorting Algorithms, Java, Programming

Hello reader! If you are here, you’re probably a computer science student or a computer science enthusiastic (or not…). Either way, this article was made for you.

The study of sorting algorithms is one of the most fundamental areas of computer science. There are many sorting algorithms out there and they all perform in different ways. So, I decided to create a series where I’ll cover the basics of the most famous sorting algorithms.

The first one of this series will be Insertion-sort, a pretty straightforward and easy to understand algorithm. Shall we begin?

Insertion-sort

Intuition

Let A be a array of numbers. Let n be the number of elements of A. So, we will iterate over A starting from the second index until n. Let j be the current index of the iteration.

The main idea of the algorithm is to ensure that A[1]…A[ j - 1] is always sorted. So, if necessary, each element A[ j] will be inserted in the correct position of the sequence A[1]…A[j -1] by shifting the elements of this sequence to the right until the element A[ j] is in the right spot. This ensures that in the next step of the iteration, the sequence A[1]…A[j -1] remains sorted. Note that, in the first iteration, the sequence A[1]…A[ j - 1] = A[1].

Consequently, in the last iteration, j = n and the sequence A[1]…A[n-1] is sorted. So, after A[n] be placed in the correct position of the sequence, the array A is fully sorted.

Pseudocode

Here’s the algorithm in pseudocode:

Code image created in carbon.now.sh

Note that, in our pseudocode, array indexing start’s from 1. So, the first element of a given array A is A[1].

Line by line explanation

In the first line of the code, we create a loop starting from the second index until the last index.

In the second line, we save the value of A[j] in a variable called key. This is necessary because we need this value through the whole iteration and it can be updated in line 5.

In the third line, we set the value of i to j - 1. (Remember that we want to shift each element of the sequence A[1]…A[j-1] to the right until we find a place to put A[j]. So, we will start with A[j - 1].)

Code image created in carbon.now.sh

In the fourth line, we have a while loop that goes all the way to line 6. If both conditions are satisfied, the A[j] element is in the wrong position and we need to shift the A[ j - 1] element to the right. The fifth line is responsible for shifting the elements of the sequence one position to the right and after that, line 6 decrements i to change the current element of the sequence A[1]…A[j - 1].

If we got here in the seventh line, it means that one of these conditions in the while loop happened: i = 0 or A[i] ≤ key.

If i = 0, all elements of the sequence A[1]…A[ j - 1] are greater then A[ j] and were shifted to the right. The line 7 will put the element A[j] in the first position of the sequence.

if A[i] ≤ key, all elements of the sequence A[i + 1]…A[j - 1] are greater then A[j] and were shifted to the right. The line 7 will put the element A[j] right after A[i], keeping the sequence A[1]..A[j -1] sorted in the next iteration.

Complexity

Now, let’s calculate the time complexity of Insertion-sort.

Well, we know that each line of the code will spend some time to execute. Lets call this time as ci, where i is the number of the line. So c1 is the execution time of the line 1, c2 is the execution time for the line 2, etc…

To calculate the time complexity, we will calculate how many times each line i is executed and multiply it by ci. After that, we sum the values for all lines.

Let tj be the number of times the while loop is executed for a given j.

Code image created in carbon.now.sh

So, for a given n, line 1 is executed n times.

Lines 2, 3, 7 are executed (n - 1) times.

Line 4 is executed sum of tj, j = 2,3,..,n times.

Lines 5 and 6 are executed sum of (tj - 1), j = 2,3,..,n times.

After summing all these values, we get this formula below:

Equation 1

Best case

The best case is when the input array is already sorted. In this situation, the while condition is always false and is executed only one time for each j, giving a total number of times equals to n - 1. Lines 5 and 6 are never executed.

So, we replace the number of execution times of line 4 to n - 1 and of lines 5, 6 to 0 into the formula 1.

And now, we can rewrite this equation as an + b, where a and b are constants.

So, for the best case, Insertion-sort runs in Θ(n) (Linear time) for a given n.

Worst case

Let A be the input array. The worst case is when A is inversely ordered. In this situation, every element of the sequence A[1]…A[j-1] is going to be greater then the A[j] element. So, the while loop will run exactly j times, tj = j for each j.

When, replacing tj for j in equation 1, we will have to solve this summations. Just remember that this is a sum of a arithmetic progression.

And now we replace those values in equation 1 and rewrite the equation in the form of an² + bn + c where a,b,c are constants.

So, for the worst case, Insertion-sort runs in Θ(n²)

Average case

Let A be the input array. On average, half of the elements of the sequence A[1]…A[j-1] are going to be smaller than A[j] and half the elements are greater. Therefore, on average, we will check only the half of the sequence A[1]…A[j-1]. So, tj is about j/2.

After replacing tj to j/2 in equation 1, we will have to solve this summations

And now we put those values in equation 1 and rewrite the equation in the form of an² + bn + c where a,b,c are constants.

So, for the average case, Insertion-sort runs in Θ(n²)

Implementation

Here’s the implementation of Insertion-sort in Java.

There are few differences from the pseudocode version.

The first one, is that, in this version, we don’t need to pass n as an input parameter. That’s because, in java, we can access the number of elements of an array by accessing the attribute length. So, for a array A, n = A.length.

The second one, is that, in Java, array indexing start’s from 0. So, we need to change the first part of the condition in the while loop to i ≥ 0, instead of the pseudocode version i > 0.

Here’s the link of the github repository in case you want to test it out!

Link: https://github.com/leosampsousa/Insertion-sort

Conclusion

I hope you have enjoyed this article and learned more about Insertion-sort! We will see in the next articles of this series, that there are other sorting algorithms that perform much better!

This was the first article i wrote here, so feel free to give me some insights or to correct any mistake i could have made above.

I’ll see you in the next algorithm!