The Gist of Big-O Notation Explained via Bubble Sort
By: Andre Unsal
Tags: Algorithms, Software Development, Software Engineering, Data Structures, Big O

Regardless of where you are in your software career, or what role you fill, you’ve surely heard of the idea of Big-O notation and its relation to algorithmic performance. Many of the analytics behind Big-O can appear quite daunting, and the high-level idea which applies can be lost. Having a solid grasp on Big-O can be a game-changer if you need to answer technical interview questions, grow a keener eye for the implementations in your codebase, or design new products while cognizant of performance characteristics. While you may never find yourself at a whiteboard showing off your analyses, you will surely find yourself asking “do I really need to nest that many loops together?” the next time you churn out a snippet if you come to understand the asymptotic analysis behind Big-O.

The Gist of Big-O

Big-O notation is simply a framework for performing asymptotic analysis. It comes with a simple vocabulary and serves as a sweet-spot for high-level reasoning about algorithms. It is also coarse enough to suppress any dependencies related to the architecture, language, compiler, and other system elements. Additionally, it is sharp enough to make useful comparisons between different algorithms on very large inputs. This enables it to be a useful tool for quickly making decisions and conclusions about a whole universe of algorithms.

Motivating Example (Bubble Sort):

We’re going to see how we can apply asymptotic analysis to Bubble Sort —a simple (but also inefficient) sorting algorithm. The way it works is by simply reading an array of integers in steps, and comparing the current value to the next value to determine which is greater (or less) than the other. The final output yields a sorted list in either ascending or descending order — depending on how you implement it.

Let’s say we’ve got an array of integers [1, 4, 2, 5, 8, 6, 3, 9, 7] . This array of integers contains n = 9 elements (note that n is simply the number of elements in this array — and is typically the defining variable which denotes the count of elements in an input for virtually all problems in algorithm analysis).

To sort our array you would implement the following algorithm (we won’t go into the details of implementation since the scope of this article is the asymptotic analysis):

isSwapped = true

while isSwapped

isSwapped = false

for j from 0 to N - 1

if a[j] > a[j + 1]

swap(a[j], a[j+1])

swapped = true

This algorithm requires n iterations to perform the sort. In each iteration we perform a comparison, and additionally a swap if required. Given an array of size n (or 9 in our example), we need to perform (n — 1) = 8 comparisons on the first iteration. On the second iteration we will need to perform (n — 2) = 7 comparisons. The third iteration will require (n — 3) = 6 , and so on… until you exhaust all of the inputs. This yields a formula as follows:

If you are wondering why we have simplified the time complexity of the bubble sort to O(n^2) it is simply because the constant factors and lower order terms have little impact on the conclusions we can draw about its running time performance as the input size of the array increases. Note in the chart below, bubble sort aligns with the quadratic running time in the “Horrible” category. As we increase the number of elements in our input array, Bubble Sort will simply perform worse and worse as n grows in size. The motivation to take from this example, is that we should constantly question if we can achieve better performance as input sizes grow.

Formal Definition

Big-O defines the running-time performance of any algorithm in general terms with a concern for increasing input size (i.e. can the algorithm still perform favorably if we go from 9 integers up to 1,000,000 integers?).

Put quite plainly, Big-O is concerned with functions that are defined on all positive integers n = 1, 2, 3, 4, 5, .... (we won’t ever have an input with a negative count of values … ).

Additionally, the running time of an algorithm will always aid in denoting the worst case running time of an algorithm.

The definition provided relates well to the chart below (which signifies that no matter what constant factors are applied, we can always apply a bound above the running time of our function). Therefore, we can always ignore constant factors and lower-order terms (in the case of algorithms with polynomial complexity) to infer a succinct running time complexity. The takeaway should always be that the running time of an algorithm increases according to Big-O as input sizes grow large. The magnitude or severity of this increase comes down to the unique algorithm in question.

T(n) upper bounded by f(n) with arbitrary constant c

When To Care About Constant Factors and Lower Order Terms

Generally speaking, writing loops and snippets which operate on collections or data structures in your codebase won’t require profound analysis. You can get away with the general Big-O running time characteristics for your use case.

However, its entirely possible that you’re in a startup, or in a highly specialized environment where your domain knowledge can come in handy. In very specific scenarios, where your product or service relies on an algorithm you or your team have implemented, it can behoove you to uncover your constants and lower order terms so that you can further optimize your algorithm — especially if its not feasible for you to vastly modify your design to conform to a totally different implementation.

Typically, constant factors and lower order terms are generated as a result of basic setting operations, and other primitive operations (like initializing a variable). The cost with such operations is very low, hence its ignored by Big-O mostly. But, you can take the reigns and utilize Big-O while retaining these factors and terms, and develop ways to reduce those costs to maximally optimize your algorithm (perhaps you can lower the number of variables, and reduce the times you reference addresses in your memory). The little gains made in this area could allow your product to hit the market stronger than your competition — even if marginally.

What To Take Away

Consider what input you have, and what you need to output

Come up with an algorithm, by brute force, or via domain knowledge (use best practices when possible)

Analyze your implementation, uncover how many times you are operating on your input (is it quadratic, cubic, linear, etc.?)

Ask if your implementation can be better — can you reduce your operations on the input from, say, quadratic to cubic, by using one less loop?

Consult a Big-O chart when performing analysis, decide if your implementation is critical to your system

chart when performing analysis, decide if your implementation is critical to your system Look up a Big-O chart and uncover the general running times of an array of algorithm types and primitive operations — you don’t need to whiteboard everything on the job

What Else To Learn