What’s the Deal With Big O Notation?
By: Razvan Soare
Tags: Programming Tips, Big O, Big O Notation, Software Development, Programming Interviews

If you are a programmer, even if you are a beginner, chances are you’ve heard the term Big O being mentioned. So what exactly is Big O and how important is to know about it?

Big O

According to Wikipedia, the general definition for Big O notation is:

“…a mathematical notation that describes the limiting behaviour of a function when the argument tends towards a particular value or infinity.”

Specific to programming:

“… Big O notation is used to classify algorithms according to how their run time or space requirements grow as the input size grows.”

In other words, Big O is concerned with how well our programs scale as the input grows. It can be referring to time or space complexity. Time complexity means how is the execution time of our code impacted by increasing the input. Space complexity means how much additional memory does our program need in order to run by increasing the input.

For most programmers, memory is no longer a major concern as our devices have GBs of RAM. On the other hand, how fast a program runs is still very important even with our crazy fast CPUs. That being said, in this article, we’ll focus more on time complexity though memory shouldn’t be ignored.

Common complexities

Let’s give a few examples of the most frequently encountered complexities. We’ll begin with constant complexity which, in Big O notation, is written as O(1). An O(1) algorithm takes the same amount of execution time no matter how large is the input. To give a simple imaginary example, let’s suppose that a print method that just prints a number on the screen takes 1 second to run. Writing the program:

print(23);

will take 1 second.

Writing the program:

print(100000)

will also take a second.

Accessing an element of an array is known to be an O(1) operation. In our imaginary awfully slow computer, let’s say that accessing an array element takes 1 second. That means that if you either print(array[0]) or print(array[1000]) it will still take only roughly two seconds. So in this O(1) “alogrithm” the size of the array simply doesn’t matter.

Linear complexity, written as O(n), means that an algorithm’s execution time increases linearly with the size of its input. As an example, we’ll use the same computer that takes one second to access an array element and one second to print something on the screen. So let’s consider printing every item in an array with n elements:

for(int i = 0; i < n; i++) {

print(array[i]);

}

So how long will this algorithm take? The answer is 2 * n seconds. So if we have an array of 10 elements, our program will take 20 seconds to print all the elements of the array. If, on the other hand, the array has 1800 elements, our code will take 1 hour to complete.

Note that Big O notation is only concerned with how well our code scales with respect to input growth, not how long it takes in absolute time. If in that for loop we also print 5 other things, in order to increase the execution time of one loop to 7 seconds, the complexity will still be O(n). So O(n) doesn’t care if our code takes 1ms to execute or 2 days. All it cares about is that if we give our algorithm an input that is 5 times larger than the previous input, it will take 5 times longer to execute.

Quadratic complexity, written as O(n²), means that the execution time will increase quadratically with respect to the input. A quick example:

for(int i = 0; i < n; i++){

for(int j = 0; j < n; j++) {

print(i + j);

}

}

For n = 3 this code will take 9 seconds to execute, For n = 30 it will take 900 seconds (or 15 minutes). It’s easy to see that O(n²) complexity is far worse that O(n). If we add another nested loop in the second one we will get O(n³) complexity which by now is pretty self explanatory. One way to spot this kind of complexity is by noticing the amount of nested loops. We should be on the lookout for unnecessary nested loops as they lead to increased complexity.

Most modern programming languages have cool features like forEach, filter, map and others that make working with arrays much more enjoyable. However, we should be careful when using them as they are basically hidden loops. If we don’t pay attention to what we’re doing, we may end up with very unscalable code. That doesn’t mean we shouldn’t use these features, it only means that we should be aware that they’re still loops, not magic.

Logarithmic complexity or O(log(n)) is much better than O(n). An example of an algorithm of logarithmic complexity is binary search. I’ll give a more complex example of this in the section in which we’ll write a division program for our fictional “a print takes 1 second” computer.

Linearithmic complexity or O(n log(n)) is better than O(n²) but worse than O(n). This is the complexity that we strive for if our current algorithm is O(n²). An example of an O(n log(n)) algorithm is merge sort.

Exponential complexity or O(c^n), where c is a constant, is among the worse complexities your algorithm can have. Many brute force algorithms are in this category. An example is trying to break a numeric password of length n which has O(10^n) complexity.

Factorial complexity or O(n!) is the worse common complexity and an example would be finding all permutations of a subset.

Division on our fictional computer

To expand on logarithimc (O(log n)) complexity, we’ll go on with a fictional assignment for our fictional computer.

Besides being able to print a number in an astonnishing 1 second, our computer is unable to compute divisions. It can only do integer additions and subtractions. It can do either an addition or a subtraction in “only” 1 second. In the following examples we’ll ignore all other execution costs and just focus on the addition times.

How our fictional computer might look like

A customer comes to us with the request for a program that computes integer divisions. So 3/2=1, 17/3=5 and so on.

Let’s think how to achieve this goal using only additions and subtractions. A pretty straight-forward way is to add the divisor until we reach or overshoot the dividend, and count the number of additions. We build the first version of the program:

int divide(int a, int b) {

int result = 0;

int acc = 0;

while(acc + b <= a) {

result += 1;

acc += b;

} return result;

}

We give the program to our customer and he starts testing it. He inputs divide(17,3) and receives 5, the correct result, in 10 seconds (there are 2 additions for each loop iteration and in this situation it loops 5 times). Not that bad for our extreemly slow computer. He then inputs divide(120, 2) and receives the correct result, 60 in 2 minutes. That’s an extreemly long time by any standards but considering our absurd 1 second per addition computer, it is acceptable. After that, he inserts divide(32768, 2). After 1 hour of waiting he still doesn’t get a result. It turns out that this division would take our computer roughly 9 hours to compute (the loop would run 16384 times). Our client is not satisfied with the result and requests a faster result time.

So is there any way we can make our program significantly faster? It turns out that there is a way. At the moment, our code has O(n) complexity. As you already know, that means that the execution time increases linearly with the size of the input. In our case it increases linearly with the size of the result of our division. So the relationship between the two inputs a and b impacts the program complexity. In order to improve it, we should strive to get our code to run closer to O(log(n)) time.

One approach is to increase the acc variable from our previous code by more than b each time. The code would look something like:

int divide(int a, int b) {

int result = 0;

int acc = 0;

int resultIncrement = 1;

int accIncrement = b;



while(acc < a) {

if(acc + accIncrement <= a) {

result += resultIncrement;

acc += accIncrement;

resultIncrement += resultIncrement;

accIncrement += accIncrement;

} else {

int newA = a — acc;

result += divide(newA, b);

}

} return result;

}

By doing this we double both the resultIncrement and the accIncrement each loop. The else branch occurs when further incrementing the acc yields a result larger than the dividend. In this situation we create a new dividend that is the difference between the acc so far and the dividend and call divide again. It turns out that calling divide(32768, 2) takes around 1 minute instead of 9 hours.

This baby might be faster than our fictional computer

Let’s see a step by step computation. The additions that happen inside the if branch in our loop are:

acc = 2+4+8+16+32+128+256+512+1024+2048+4096+8192+16384=32766

result = 1+2+4+8+16+32+64+128+256+512+1024+2048+4096+8192=16382

If we make another addition, acc would be greater than 32768 so we reach the else branch where we have a single addition yielding an acc of 2 and a result of 1. Adding 1 to the result yields a result of 16383 which is the correct answer for 32766/2.

If we compute the base 2 log of 16383 we arrive at the result of ~14 which is the number of additions that we need to perform on the result variable in order to get the correct answer. So our algorithm is O(log (n)) instead of the previous logarithm which was O(n).

To insist on the point that Big O is not interested in absolute execution time but rather on scalability, we can see that a loop in the first implementation contained 2 additions while in the second implementation it contains 4 additions(if we don’t account for the else branch). So one loop takes 4 seconds instead of 2. If we divide(4, 2) our O(log(n)) approach is actually slower than our O(n) version.

Note that usually the size of an input number (like a dividend/divisor) doesn’t influence the algorithm complexity, but in our case that input number (more exactly the relationship between our 2 input numbers) influences the number of loop iterations. Most of the times the input would be a collection, like an array, and the size of that collection would influence the algorithm complexity.

Other “Big” notations

Big O is not the single notation in the family. Big O represents the worst case complexity. There are also other notations like Big Θ which is the average complexity and Big Ω which is the best case complexity.

In practice when using data structures we can orient ourselves by the Big Θ complexity of the operations that we require rather than Big O. In most cases the average complexity is what works best for us. If on the other hand we have some real-time critical software we shouldn’t hope for the best and instead program assuming the worst. You can find a quick cheat sheet for various data structure complexities here.

Real life example

A recent example of when knowing about Big O came in handy, is when I was tasked with fixing an iOS app UITableView scrolling issue. The frame rate kept dropping when scrolling the table.

After looking into it, I saw that when a cell needed to be drawn, a filter operation was triggered that looked for a struct with a certain id. These structs that had fields like {id, name, description}, were stored in an array with over 2000 elements. Every time you scrolled tens of cells started looping through this array in search for a struct with a certain id. This caused the frame rate to drop.

Searching an array is an O(n) operation. In this case n is greater than 2000. Even though this was happening tens of times (once for each cell that needed to be drawn), the complexity remains O(n) as the number of visible cells is small compared to 2000.

A reasonable target would be to reduce the complexity to O(log(n)). It turns out, that in this situation, we can do better. Before displaying the table view, we can store all the array elements in a dictionary where the keys are the struct ids. Instead of filtering through the array, we can just access the dictionary. A dictionary has typically O(1) access time (It is actually Θ(1) and O(n), but in practice in Swift we can pretty much suppose it is O(1). For more details you can search how hash tables work).

After making this change the scrolling went smoothly. The trade-off is that using a dictionary increases the amount of memory that the app uses. In this specific situation, current devices have no trouble with this.

If memory was an issue, another solution would have been to filter the arrays on a different thread. This, however, would mean that the user had to see dummy data or loading animations for each cell while quickly scrolling. Instantly seeing the data is a way better user experience (considering that memory was a non-issue in this case).

Conclusion

In real computers additions don’t take full seconds like in our exaggerated examples. Still, keeping the user waiting behind a loading screen is not a desired behaviour. Knowing a little bit about Big O can help us understand how code scales and write better software.