How to Benchmark functions in Python
Tags: python, programming, 100daysofcode, codenewbie
Published: 2021-03-23 13:40:19

<!-- wp:separator -->
<hr class="wp-block-separator"/>
<!-- /wp:separator -->

<!-- wp:heading {"level":4} -->
<h4>In this article, we will discuss 4 approaches to benchmark functions in&nbsp;Python</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>The first 3 methods will help us measure the execution time of a function while the last method will help us measure the memory usage.</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3>Table of&nbsp;Contents</h3>
<!-- /wp:heading -->

<!-- wp:list -->
<ul><li>Using the time library</li><li>Using timeit</li><li>Using line_profiler</li><li>Using memory_profiler</li><li>Conclusion</li></ul>
<!-- /wp:list -->

<!-- wp:heading {"level":3} -->
<h3>Using the time&nbsp;library</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>This is one of the simplest ways to calculate the execution time of a function. We essentially get the time in seconds right before and after we invoke the function. The difference between the two can give us an estimate of the time of the function. I say estimate since this measures the time for a single run. A better benchmark would be running it multiple times and calculating the average time taken.</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">import time
def func():
   lst = [i for i in range(100000)]
   start = time.perf_counter()
func()
print(f"Completed Execution in {time.perf_counter() - start} seconds")</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>Below is the output</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">Completed Execution in 0.007916 seconds</pre>
<!-- /wp:preformatted -->

<!-- wp:heading {"level":3} -->
<h3>Using timeit</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p><a href="https://docs.python.org/3/library/timeit.html" rel="noreferrer noopener" target="_blank">Timeit</a> is a built-in method in Python. It lets us specify the number of times we want to run the function, this helps us to calculate the average time the function takes. This is a better measure than the execution time of a single run. However, for a complex function that takes a long time, this method may not be ideal.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>The general syntax is below</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">timeit.Timer(funcName).timeit(number=number)</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p><strong>number </strong>is the number of times we want the function to be executed. It returns the sum of all individual runtimes. To get the average, we can just divide the sum by the number of runs.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Below is how we would measure the average execution time of our function <strong>func()</strong></p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">import timeit
num_runs = 10
duration = timeit.Timer(func).timeit(number = num_runs)
avg_duration = duration/num_runs
print(f'On average it took {avg_duration} seconds')</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>The output is below</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">On average it took 0.004649160000000001 seconds</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>We can also use the <strong>repeat()</strong> method to run the experiment (running the function n times) multiple times. In simpler terms, if we consider an experiment as running our function <strong>func()</strong> 10 times, we can do this experiment 3 times and get the execution time of each experiment.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>The general syntax is&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">timeit.Timer(funcName).repeat(repeat=num_repetions,number=num_runs)</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>Below is using it to benchmark our function <strong>func()</strong></p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">num_runs = 10<br>num_repetions = 3<br>ex_time = timeit.Timer(func).repeat(<br>                     repeat=num_repetions,<br>                     number=num_runs)<br>print(f'It took {ex_time}')</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>This will return the following output</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">It took [0.0494772, 0.04936369999999998, 0.048738000000000004]</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>Similar to the <strong>timeit()</strong> method, it returns the total time for the 10 runs. We can use the <strong>max()</strong> to get the worst time, <strong>min()</strong> to get the best time, and <strong>sum(lst)/len(lst) </strong>to get the average execution time.</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3>Using line_profiler</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>line_profiles is a really cool library that gives a line-by-line analysis of a function. Before using the library, we will need to install it</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">conda install -c anaconda line_profiler</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>We will need to add a decorator before our function</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">@profile<br>def func():<br>    lst = []<br>    for i in range(100000):<br>       lst.append(i)</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>Since line-profiler provides a line-by-line analysis, it doesn’t make much sense to use it with a list comprehension.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Typ the following commands in the Anaconda Prompt</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">kernprof -l main.py<br>python -m line_profiler main.py.lprof</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>The first command runs our python script and stores the logs in a file. The second command displays the logs in the form of an easy-to-understand table.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Below is the output of the second command</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://cdn-images-1.medium.com/max/1200/1*0c_xRQBcV3ZDqg0dkpfmWg.jpeg" alt=""/><figcaption>Screenshot of line_profiler</figcaption></figure>
<!-- /wp:image -->

<!-- wp:list -->
<ul><li><strong>Hit- </strong>The number of times that line was executed.</li><li><strong>Time-</strong> Total time taken by that line</li><li><strong>Per Hit-</strong> Average time taken by the line</li><li><strong>% Time- </strong>Fraction of total execution time. As we can see from the above picture, the append function takes around 57% of the execution time.&nbsp;</li></ul>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>If you want to change the number of times the function is executed for line_profiler, use the following code</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">prof = profile(func)<br>for i in range(10):<br>   prof()</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>This will run the function 10 times.</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3>Using Memory_profiler</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>It is pretty similar to line_profiler, the difference being the logs generated tell us about the memory usage instead of time taken.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>First, we will need to install the library</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">conda install -c anaconda memory_profiler</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>We will have to add the same decorator as before</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">@profile<br>def func():<br>    lst = []<br>    for i in range(100000):<br>       lst.append(i)</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>Then type the following command in the Anaconda Prompt</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">python -m memory_profiler main.py</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>Below is the output</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://cdn-images-1.medium.com/max/1200/1*Mkkhkc-5BmG8RZshD-apGA.jpeg" alt=""/><figcaption>Screenshot of memory_profiler</figcaption></figure>
<!-- /wp:image -->

<!-- wp:list -->
<ul><li><strong>Mem usage</strong>- The total memory usage at the line</li><li><strong>Increment</strong>- memory usage by each execution of that line</li><li><strong>Occurrences</strong>- number of times the line was executed</li></ul>
<!-- /wp:list -->

<!-- wp:heading {"level":3} -->
<h3>Conclusion</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>If you want a quick time performance test of a piece of code or a function, you should try measuring the execution time using the <strong>time </strong>library. However, if you want a better estimate, consider using the <strong>timeit </strong>library.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>If you want a more detailed line-by-line analysis, consider using line-profiler and memory-profiler.&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:separator -->
<hr class="wp-block-separator"/>
<!-- /wp:separator -->

<!-- wp:quote -->
<blockquote class="wp-block-quote"><p>Connect with me on <a href="https://www.linkedin.com/in/rahulbanerjee2699/" rel="noreferrer noopener" target="_blank">LinkedIn</a></p></blockquote>
<!-- /wp:quote -->

<!-- wp:quote -->
<blockquote class="wp-block-quote"><p>I recently started a modified version of the #100daysofcode challenge. I aim to write content related to Python, Data Science, or Programming every day. Follow my progress on <a href="https://twitter.com/rahulbanerjee99" rel="noreferrer noopener" target="_blank">Twitter</a>, <a href="https://medium.com/daily-programming-tips" rel="noreferrer noopener" target="_blank">Medium</a>, <a href="https://dev.to/rahulbanerjee99" rel="noreferrer noopener" target="_blank">Dev.to</a>, <a href="https://realpythonproject.hashnode.dev/series/daily-programming-content/" rel="noreferrer noopener" target="_blank">Hashnode</a>, or <a href="https://www.realpythonproject.com/category/daily-programming-tips/" rel="noreferrer noopener" target="_blank">my WordPress Blog</a></p></blockquote>
<!-- /wp:quote -->