Introduction to Big O Notation. Getting started with Big O Notation... | by Andrew Jamieson | Towards Data Science

Big O Notation is a way to measure an algorithm’s efficiency. It measures the time it takes to run your function as the input grows. Or in other words, how well does the function scale.
There are two parts to measuring efficiency — time complexity and space complexity. Time complexity is a measure of how long the function takes to run in terms of its computational steps. Space complexity has to do with the amount of memory used by the function. This blog will illustrate time complexity with two search algorithms.
Big O is sometimes referred to as the algorithm’s upper bound, meaning that it deals with the worst-case scenario. The best-case scenario doesn’t really tell us anything — it will be finding our item in the first pass. We use worst-case to remove uncertainty — the algorithm will never perform worse than we expect.
Let’s look at a simple example.
We will search for the number eight out of a range from 1–8.
Our first strategy is to start with the first number of the array and move up by one until we find our target number. Our algorithm steps would look something like this.
Start at the beginningCompare the current value to our targetMove to the next valueReach the end of the list
Start at the beginning
Compare the current value to our target
Move to the next value
Reach the end of the list
In round 1, we choose number one; that’s not correct, so we move to round 2 and eliminate number one as an option. Step 2 is also not correct, and we continue all the way until we select number eight.
In our worst-case scenario, this is not very efficient. We have to check every single number in the list until we get to our answer. This method is called Linear Search. The Big O notation for Linear Search is O(N). The complexity is directly related to the size of the inputs — the algorithm takes an additional step for each additional data element.
def linear_search(arr, x):  #input array and target      for i in range(len(arr)):           if arr[i] == x:             return i       return -1   # return -1 if target is not in the array
Let’s try a different strategy. The critical part of this strategy is that the list must be in order.
This time we start in the middle of the list. We check to see whether we have chosen the target number. If not, we check whether the number is smaller or larger than our choice. In either case, we eliminate half of the list that doesn’t include our target. Then we select a number in the middle of the remaining values.
Find the middle of the list — the midpointCompare the midpoint to the targetIf our value and the target match we stop — we’ve found our targetIf our value is smaller than the target, we make a new list ranging from the midpoint plus one to the largest value.If our value is larger than the target, we make a new list ranging from the smallest value to the midpoint minus one.Repeat until we find the target or reach that last element, and it does not match the target.
Find the middle of the list — the midpoint
Compare the midpoint to the target
If our value and the target match we stop — we’ve found our target
If our value is smaller than the target, we make a new list ranging from the midpoint plus one to the largest value.
If our value is larger than the target, we make a new list ranging from the smallest value to the midpoint minus one.
Repeat until we find the target or reach that last element, and it does not match the target.
The midpoint for our first choice is four. The target is greater than four, so we exclude values four and below and create a new list from five to eight. We pick a midpoint value of six and choose again. We continue until only number eight remains.
This strategy is called Binary Search. It is efficient because it eliminates half of the list in each pass. If we were to double our search and look for number 16 in a range of 1–16, it would take us only one more round. Amazingly, if we were to pick a number between one and a million, the worst case would be 20 guesses.
When we get to large numbers like this, we can clearly see how much more efficient this method is than using linear search: 20 guesses vs. 1,000,000 guesses.
We can model this mathematically. The number eight is equivalent to 2 x 2 x 2. We can also express this equation using exponents: two to the power of three, or 23.
The inverse of an exponent is a logarithm. Log to the base 2 of 8 means how many times do you have multiply two by itself to get eight. As illustrated above 2 x 2 x 2 =8, so Log2 8 = 3.
In general, the worst-case scenario of a Binary Search is Log of n + 1. The Big O notation for Binary Search is O(log N). In contrast to O(N) which takes an additional step for each data element, O(log N) means that the algorithm takes an additional step each time the data doubles.
def binary_search3(arr, target):    left = 0    right = len(arr) - 1while left <= right:        mid_point = (left + right) // 2             if target < arr[mid_point]:            right = mid_point - 1        elif target > arr[mid_point]:            left = mid_point + 1        else:            return mid_pointreturn -1
Note: Even though Binary Search is far more efficient than Linear Search, you wouldn’t always choose it in every example. The reason for this is the very first qualification. Binary Search requires that the list is in order. Sorting a list introduces its own complexity — so in some cases, it may be more efficient to use Linear Search rather than sorting the list first.
When we write Big O notation, we look for the fastest-growing term as the input gets larger and larger. We can simplify the equation by dropping constants and any non-dominant terms. For example, O(2N) becomes O(N), and O(N2 + N + 1000) becomes O(N2).
Binary Search is O(log N) which is less complex than Linear Search. There are many more complex algorithms. A common example of a quadratic algorithm or O(N2) is a nested for loop. In a nested loop, we iterate through the entire data in an outer loop. Then for each element, we iterate through the data in an inner loop. This is N x N times of N2.
Here’s how these Big O runtimes look on a chart. The examples in this blog are two of the least complex notations.
There are often many choices of writing code to get to a solution. Understanding Big O notation is important in writing algorithms. It helps you determine when your algorithm is getting faster or slower. You can also compare different methods and choose the most efficient.
Treehouse via freeCodeCamp has a great YouTube course to take you from beginner to understanding Big O notation. Warning it is over five hours long. (Here’s a blog with some tips to watch YouTube efficiently)