Multi-Threading in Python
Tags: multithread, python, multithreading
Published: 2019-07-07 15:04:28

### Multi-Threading in Python using the ‘Activity’ Application


Threads in the computer world are described as simple executors (which execute the given instructions). These are lightweight and used to perform tasks asynchronously.

Eg: Different tabs in browser act as different threads

In general, At the OS level, threads have their own resources but at an abstract level, they use the same code, data and files.

![](https://cdn-images-1.medium.com/max/800/0*1C4e7YzuhSZQcwUf.)
<span class="figcaption_hack">Source: [https://www.studytonight.com](https://www.studytonight.com/)</span>

**Multithreading in activity application:**

[Activity](https://github.com/sayanarijit/activity) is an application developed by my good friend [Arijit](https://github.com/sayanarijit). It performs bulk server command execution and validations over ssh asynchronously.

It was developed in python 3.6. The core concept of this module is that, it uses multi-threading to perform command execution in parallel on all the given servers because of which it consumes less time and performs tasks in an organised and efficient way.

To understand multithreading, Let's dive into some code which is part of
activity app, which performs ping check on all the hosts asynchronously.

{% gist https://gist.github.com/suryasr007/ce04992426493e7f260a9b3bdcf6384c %}

The above-defined function is a callable function, which performs the required activity.

{% gist https://gist.github.com/suryasr007/b7331fa7267c73b6e179b9d8d94a6ba4 %}

To understand this, we should know a brief background of how parallel execution works in python.

#### concurrent.futures: Launching parallel tasks

The [concurrent.futures](https://docs.python.org/3/library/concurrent.futures.html#module-concurrent.futures) module provides a high-level interface for asynchronously executing callables which is an In-built module.

In our program, `_ping_single(args):` is a executing callable function

The asynchronous execution can be performed using threads and processes. Sub-function for threads in [concurrent.futures](https://docs.python.org/3/library/concurrent.futures.html#module-concurrent.futures) is **ThreadPoolExecutor** and processes is **ProcessPoolExecutor**.

To import required module

`from concurrent.futures import ThreadPoolExecutor, as_completed`

Once imported,

`with ThreadPoolExecutor(max_workers=PARALLEL_LIMIT) as pool:`

The above line is used to initialize the threads. It is initialized with `with` so that when execution of a threads complete, the resources of the thread are freed and then shutdown the Executor.


The execution starts once the callable function is submitted along with
arguments as shown above. In the above line hosts, as well as required arguments are submitted along with callable function.

We can use `pool.map` to submit the tasks to threads, The difference would be seen in while capturing the output.
```python
    for f in as_completed(futures):
        output.append(f.result())
```
Once the execution of the thread is started, we can capture the output using `as_completed` or the `wait` function.

The `as_completed()` function takes an iterable of Future objects and starts yielding values as soon as the futures (threads) start resolving. The main difference between the aforementioned map method with as_completed is that map returns the results in the order in which we pass the iterables. That is the first result from the map method is the result of the first item. On the other hand, the first result from the as_completed function is from whichever future completed first.

The `wait()` function would return a named tuple which contains two set — one set contains the futures which completed (either got result or exception) and the other set containing the ones which didn’t complete.

*Note:* If the threads are not configured properly, there will be performance issues like deadlock, memory leaks.

**Conclusion:**

Multithreading if configured properly, is an efficient and organized way to reduce the time taken for execution as well as improve the performance of the program.

For more explanation on multithreading please refer the below article:
[https://medium.com/@bfortuner/python-multithreading-vs-multiprocessing-73072ce5600b](https://medium.com/@bfortuner/python-multithreading-vs-multiprocessing-73072ce5600b)

Source:

* [Python](https://medium.com/tag/python?source=post)
* [Multithreading](https://medium.com/tag/multithreading?source=post)
* [Activity](https://medium.com/tag/activity?source=post)