3 techniques to make your Python code faster | by Dhanesh Budhrani | Towards Data Science

In this post I’ll be sharing 3 Python efficiency techniques that you may use in your daily scripting, and how to measure the performance improvement between 2 solutions. Let’s get started!
Performance may refer to many different factors in a solution (e.g. execution time, CPU usage, memory usage, etc.). In this post, though, we’ll be focusing on the execution time.
The improvement in the execution time of a new solution can be computed as simply as making a division. That is, we’ll be dividing the execution time of the old (or non-optimized) solution by the new (or optimized) solution: Told / Tnew. This metric is commonly referred to as speedup. For instance, if we got a speedup factor of 2, our improved solution would be taking half as much time as the original solution.
In order to compare the performance of our functions, we’ll create a function that receives both of them, calculates their execution time and computes the obtained speedup:
import timedef compute_speedup(slow_func, opt_func, func_name, tp=None):  x = range(int(1e5))  if tp: x = list(map(tp, x))  slow_start = time.time()  slow_func(x)  slow_end = time.time()  slow_time = slow_end - slow_start  opt_start = time.time()  opt_func(x)  opt_end = time.time()  opt_time = opt_end - opt_start  speedup = slow_time/opt_time  print('{} speedup: {}'.format(func_name, speedup))
In order to obtain significant results, we’ll be using a relatively large array (100.000 elements), and pass it to both functions as an argument. We’ll then compute the execution time using the time module and finally deliver the obtained speedup.
Note that we are also passing an optional argument that would allow us to change the type of our list elements.
A common situation you may find is having to form a string with multiple subparts of it. Python has a convenient + operator that allows us to comfortably concatenate strings in the following way:
def slow_join(x):  s = ''  for n in x:    s += n
Even though it seems a clean approach to us, Python’s strings were created to be immutable, and therefore cannot be modified. That means that every time we use the + operator, Python is actually creating a new string based on both substrings and returning the new string. Consider that, in our case, this operation would be executed 100.000 times.
This approach clearly has a cost, and we may find a cheaper solution using join(), such as in the following example:
def opt_join(x):  s = ''.join(x)
This solution takes the array of substrings and joins them with the empty string separator. Let’s check our performance improvement:
compute_speedup(slow_join, opt_join, 'join', tp=str)
I’m getting a speedup factor of 7,25! Not too bad I’d say, considering the small amount of work needed to implement this technique.
When we need to operate on every element in a list we may normally do it in the following way: we apply generator comprehension and work on the current element. Then, we may iterate over it when necessary:
def slow_map(x):  l = (str(n) for n in x)  for n in l:    pass
However, in many situations you may prefer using Python’s built-in map function, which applies the same operation to every element in an iterable and yields the results. It may be implemented as simply as follows:
def opt_map(x):  l = map(str, x)  for n in l:    pass
It’s time to check how much we’ve improved the execution time! Run our compute_speedup function as follows:
compute_speedup(slow_map, opt_map, 'map')
I’m obtaining a speedup of 1,55. It may be argued that comprehensions are more readable, but I’d say that it takes very little to get used to map’s syntax as well, at least in simple scenarios (e.g. when no condition over the iterable is required).
Whenever you find yourself repeatedly using the same function over an element inside a loop block, such as:
y = []for n in x:  y.append(n)  y.append(n**2)  y.append(n**3)
... or simply using such function once inside the loop block, but over a large list, such as in the following case:
def slow_loop(x):  y = []  for n in x:    y.append(n)
... you may take advantage of another optimization technique.
You may save the cost of reevaluating a function if you previously keep it as a variable and reuse it in the loop block. The following snippet shows this behaviour:
def opt_loop(x):  y = []  append = y.append  for n in x:    append(n)
Note that if you need to append the current element to different lists, you’ll have to create a new variable for each list’s append function.
Let’s check the speedup using compute_speedup:
compute_speedup(slow_loop, opt_loop, 'loop')
In this case, I’m obtaining a speedup of 2,07! Again, we didn’t need to make any major changes to obtain such improvement.
—
Want more efficiency tips? Check these articles out!
3 simple Python efficiency tips
Finding performance bottlenecks in Python