Space and Time Complexity in Computer Algorithms | by Areeba Merriam | Towards Data Science

In this article, I will discuss computational complexity which was developed by Juris Hartmanis and Richard E. Stearns to analyze the difficulty of an algorithm.
We all know, human nature aspires to seek an efficient way to assemble their daily tasks. The predominant thought process behind innovation and technology is to make life easier for people by providing ways to solve problems they may encounter. The same thing happens within the world of computer science and digital products. We write algorithms that are efficient and take up less memory to perform better.
Time complexity is the time taken by the algorithm to execute each set of instructions. It is always better to select the most efficient algorithm when a simple problem can solve with different methods.
Space complexity is usually referred to as the amount of memory consumed by the algorithm. It is composed of two different spaces; Auxiliary space and Input space.
The factor of time is usually more important than that of space.
Note: — In computer programming, you are allowed to use 256MB for a particular problem. If you create an array having a size of more than 108, you will get the error. Also, you can’t make an array having size 106 because the maximum space allotted to a function is 4MB. We must define a global to solve this problem.
Though they seem trivial, these factors are crucial in determining how a computer program is developed, designed and how it adds value to the life of its user. Remember, time is money.
A good algorithm is one that takes less time in execution and saves space during the process. Ideally, we have to find a middle ground between space and time, but we can settle for the average. Let’s look at a simple algorithm of finding out the sum of two numbers.
Step #01: Start.Step #02: Create two variables (a & b).Step #03: Store integer values in ‘a’ and ‘b.’ -> InputStep #04: Create a variable named ‘Sum.’Step #05: Store the sum of ‘a’ and ‘b’ in a variable named ‘Sum’ -> OutputStep #06: End.
It is as simple as it looks.
Note: — If you are a gamer, you would know the average game size (in hard disk) is increasing day by day, along with improvements in loading times. Similarly, in websites, loading time reduces significantly, and the storage space of its servers is increasing day by day. So, as we discussed earlier, in terms of space & time, time plays a crucial role in the development phases of any software.
According to the research conducted by Neil Patel,
“47 percent of consumers expect a website to load in no more than two seconds.”
You can find many sophisticated algorithms over the internet. It’s the possible reason why almost all big companies invest a lot in research and writing these sophisticated sets of instructions.
Following are the factors that play a significant role in the long term usage of an algorithm:
Efficiency — We’ve already talked about how much efficiency matters in creating a good algorithm. It is the efficiency that reduces the computational time and generating quick output.Finiteness — The algorithm must terminate after completing a specified number of steps. Otherwise, it’ll use more memory space, and it’s considered a bad practice. Stack overflow and out-of-bounds conditions may occur if it goes on for infinite loops or recursion.Correctness — A good algorithm should produce a correct result irrespective of the size of the input provided.
Efficiency — We’ve already talked about how much efficiency matters in creating a good algorithm. It is the efficiency that reduces the computational time and generating quick output.
Finiteness — The algorithm must terminate after completing a specified number of steps. Otherwise, it’ll use more memory space, and it’s considered a bad practice. Stack overflow and out-of-bounds conditions may occur if it goes on for infinite loops or recursion.
Correctness — A good algorithm should produce a correct result irrespective of the size of the input provided.
Time complexity is profoundly related to the input size. As the size of the input increases, the run time (which is — time taken by the algorithm to run) also increases.
Example: Consider a sorting algorithm.
Let suppose we have a set of numbers named A = {10, 5, 21, 6, 9},
There are many algorithms available to sort the given numbers. But not all of them are efficient. To find out the most effective one, we have to run a computational analysis on every algorithm.
Leonardo Galler and Matteo Kimura did one of the finest research on LAMFO about “Sorting Algorithm.”
Here are some of the important observations from the graph:-
This test showed five of the most used sorting algorithms: Quicksort, Insertion sort, Bubble sort, Shell sort, Heapsort.
The programming language used to perform the task is Python and the size of the input ranges from 2500 to 50000.
The results were: “Shell sort and Heap Sort algorithms performed well despite the length of the lists, on the other side we found that Insertion sort and Bubble sort algorithms were far the worse, increasing computing time significantly. See the results in the chart above.”
Before running an analysis on any algorithm, we must have to check if it’s stable or not. Understanding our data is the most important part of running a successful analysis.
If you are not from a Computer Science background, you may find this concept a little more complex than usual. No worries! I got you covered.
So, what is Asymptotic notation?
In simple words, it tells us how good an algorithm is when compared with another algorithm.
We cannot directly compare two algorithms side by side. It heavily depends on the tools and the hardware we use for comparisons, such as the Operating System, CPU model, processor generation, etc. Even if we calculate time and space for two algorithms running on the same system, their time and space complexity may be affected by the subtle changes in the system environment.
Therefore, we use Asymptotic analysis to compare space and time complexity. It analyzes two algorithms based on changes in their performance concerning the increment or decrement in the input size.
Primarily there are three types of Asymptotic notations:
Big-Oh (O) notation.Big Omega (Ω) notation.Big Theta (Θ) notation — widely used.
Big-Oh (O) notation.
Big Omega (Ω) notation.
Big Theta (Θ) notation — widely used.
The big-O notation was introduced in 1894 by Paul Bachmann. He thoughtlessly introduced this notation in his discussion on approximating a function.
From the definition: O(g(n)) = {f(n): there exist positive constants c and n0 such that 0 <= f(n) <= c*g(n) for all n >=n0}
Here ’n’ gives the upper bound value. If a function is O(n), then it’s O(n2), O(n3), as well.
It is the most commonly used notation for the Asymptotic analysis. It defines the upper bound of a function i.e., the maximum time taken by an algorithm or the worst-case time complexity of an algorithm. In other words, it gives the maximum output value (big-O) for a respective input.
From the definition: The function f (n) is Ω(g(n)) if there exist positive numbers c and N, such that f (n) ≥ cg(n) for all n ≥ N.
It defines the lower bound of a function — i.e., the minimum time taken by an algorithm. It gives the minimum output value (big-Ω) for a respective input.
From the definition: f(n) is Θ(g(n)) if there exist positive numbers c1, c2, and N such that c1g(n) ≤ f (n) ≤ c2g(n) for all n ≥ N
It defines the lower bound and upper bound of a function i.e., it exists as at the both, most and at least boundaries for a given input value.
Note: — Big-O defines in terms of ≤ and Ω in terms of ≥; = is included in both inequalities. It suggests a way of restricting the sets of possible lower and upper bounds. This restriction accomplishes by the Big Theta (Θ) notation.
Best Case: It defines as the condition that allows an algorithm to complete the execution of statements in the minimum amount of time. In this case, the execution time acts as a lower bound on the algorithm’s time complexity.Average Case: In the average case, we get the sum of running times on every possible input combination and then take the average. In this case, the execution time acts as both lower bound and upper bound on the algorithm’s time complexity.Worst Case: It defines as the condition that allows an algorithm to complete the execution of statements in the maximum amount of time. In this case, the execution time acts as an upper bound on the algorithm’s time complexity.
Best Case: It defines as the condition that allows an algorithm to complete the execution of statements in the minimum amount of time. In this case, the execution time acts as a lower bound on the algorithm’s time complexity.
Average Case: In the average case, we get the sum of running times on every possible input combination and then take the average. In this case, the execution time acts as both lower bound and upper bound on the algorithm’s time complexity.
Worst Case: It defines as the condition that allows an algorithm to complete the execution of statements in the maximum amount of time. In this case, the execution time acts as an upper bound on the algorithm’s time complexity.
When I started programming, the concept of space and time complexity was always a stepping stone for me. So today, I thought to bring about a simplified discussion on how these two factors significantly affect the algorithm.
Thank you for reading this story and see you again. Feel free to leave a comment if you have any thoughts, feedback, or suggestions about it!
If you like my work and wanna support me then please sign up to become a medium member using this link or else, you can buy me a coffee ☕️.
Space complexity of an algorithm — Studytonight
Why is Time Complexity Essential and What is Time Complexity? — Greatlearning
Time and Space Complexity — HackerEarth
Complete guide to understanding time and space complexity of algorithms — Learn Code Stream
How to Increase Page Speed — Neil Patel
Sorting Algorithms — LAMFO