Algorithms Revisited Part 4: Randomized Algorithms
By: Can Bayar
Tags: Algorithms, Programming, Randomization, Computer Science, Software Development

Okay, this one should not take long. The entire purpose of this post is to introduce the idea of utilizing randomness in order to improve the performance.

Initially, I was planning to write on a different subject but half-way through I realized a short post on randomized algorithms might be beneficial for the subject. So, here goes: a randomized algorithm is simply an algorithm that uses some sort of randomness at some point during your algorithm to improve the performance by uniformly distributing the inputs.

This subject is heavily based on probabilistic analysis, hence requires a strong mathematical background. However, I will be skipping the calculations in this post and just going to show you how randomized algorithms can be used to improve performance.

Randomized Quick-Sort

The quicksort algorithm is based on the partitioning procedure. Partitioning procedure works like this: You pick an element from the array to be the pivot. Your array will be divided into two subarrays around the pivot item. Elements larger than the pivot will end up in the right subarray and the ones that are smaller will end up in the left subarray. In the example, 4 is taken as the pivot.

If you keep applying this partitioning procedure recursively to the left and right subarrays until you reach subarrays of size 1, you will obtain a sorted array. So here is the quicksort algorithm:

void quicksort(vector<int> &numbers, int left, int right) {



if (left < right) {

int pivotIndex = partition(numbers, left, right);

quicksort(numbers, left, pivotIndex - 1);

quicksort(numbers, pivotIndex + 1, right);

}

}

The average runtime of the quicksort algorithm is O(n log n). I will not go further into the algorithm, it is not related and you can just Google it. What matters for us here is the analysis of the partitioning procedure. To be more specific, how are we going to select the pivot element.

You can really select any element on the array to be the pivot but you would probably prefer a deterministic approach and either select the first or the last element. Let’s assume we are always choosing the first element. In the best case, the partition procedure will evenly divide the array into two subarrays of the same size. What does this tell us about the worst-case?

The worst-case occurs when the array is already sorted in this scenario. At each step, the left subarray will be empty while the right subarray contains all the elements except the pivot. So, at each step, we will have to partition arrays of size n, n-1, n-2, n-3, …, which we add up to O(n²) time-complexity.

How can we overcome this problem? By using a randomized pivot selection instead of a deterministic approach. So, instead of using the first element, we will randomly select an element to be the pivot. If the algorithm selects the pivots uniformly at random, the algorithm shall produce reasonably balanced subarrays regardless of the input arrays.

Obviously, even though you randomly select the pivots, your subarrays may not be well-balanced. However, the possibility of this to occur will be very small compared to selecting the first item. In fact, there is a very little deviation from the average case while using random pivots and you can check the proven probabilistic analysis of randomized quicksort from here if you are interested. You can even test it yourself by generating random arrays.