Parallel monitoring of CPU usage with Python's psutil
Tags: python
Published: 2022-08-09 07:41:15

I wrote a script that periodically monitors CPU utilization while running some process in Python.

Multithreading allows periodic execution without stopping the main process. It may be useful when you want to report time series data of heavy computer load. The full code is at the bottom.

## Library Installation
The library [psutil](https://pypi.org/project/psutil/) can be used to get the CPU utilization in Python. You can install it via pip.
```
pip install psutil
```
The basic usage is as follows.
```python
# percpu=True returns a list of utilization (%) per core
>>> psutil.cpu_percent(interval=1, percpu=True)
[7.0, 1.0, 8.9, 0.0, 5.0, 1.0, 4.0, 0.0]

# percpu=False returns an average utilization (%)
>>> psutil.cpu_percent(interval=1, percpu=False)
8.9

# Smaller interval(sec) shortens the measurement time, but also increases the error.
>>> psutil.cpu_percent(interval=0.5, percpu=True) 
[6.0, 0.0, 7.8, 1.9, 4.0, 0.0, 5.8, 0.0]

```
## Parallel processing
This time, we will implement parallel processing using the native library [threading](https://docs.python.org/3/library/threading.html). 
Thread instances can be created and executed as follows.
```python
m = threading.Thread(target=monitor_cpu,args=((initial_time,)))
m.start()
```
As arguments for `threading.Thread()`, `target` is a method to execute and `args` is the argument for the method.

`monitor_cpu` is the method for monitoring.

## Monitoring method
Manage monitoring threads using `threading.Event`.
`threading.Event` is used to make a thread wait until an event occurs, and then resume the waiting thread when an event is raised by another thread.

We use 2 events.
- `wait(timeout)` : make the current thread wait until an event occurs or `timeout` seconds elapse. 
- `set()` : fires an event.

```python
def monitor_cpu(initial_time):
    print("START monitor_cpu")
    while not event.wait(1):  # Wait for 1 second, if no event occurs, execute inside the loop and wait again
        elapsed_time = time.time() - initial_time
        cpu_percent = psutil.cpu_percent(percpu=True)
        cpu_percent = '\t'.join(["{:10.4f}".format(v) for v in cpu_percent])
        print("time:", int(elapsed_time), cpu_percent)
    print("END monitor_cpu")  # When an event occurs, exit the loop and terminate execution


if __name__=="__main__":
    event = threading.Event()
    initial_time = time.time()
    m = threading.Thread(target=monitor_cpu,args=((initial_time,)))
    m.start()

    # Main process

    event.set()  # Fires an event
```

Inside `monitor_cpu`, an infinite loop runs with an execution interval of 1 second. When an event occurs, it breaks the loop and terminates execution. 

```python
def monitor_cpu(initial_time):
    print("START monitor_cpu")
    while flag:
        time.sleep(1)
        elapsed_time = time.time() - initial_time
        cpu_percent = psutil.cpu_percent(percpu=True)
        cpu_percent = '\t'.join(["{:10.4f}".format(v) for v in cpu_percent])
        print("time:", int(elapsed_time), cpu_percent)
    print("END monitor_cpu")

if __name__=="__main__":
    event = threading.Event()
    initial_time = time.time()
    flag = True
    m = threading.Thread(target=monitor_cpu,args=((initial_time,)))
    m.start()
    tmp = 0
    for i in range(100000000):
        tmp = i+i
    flag = False
```
## Entire code
```python
import threading
import time
import psutil

def monitor_cpu(initial_time):
    print("START monitor_cpu")
    while not event.wait(1):
        elapsed_time = time.time() - initial_time
        cpu_percent = psutil.cpu_percent(percpu=True)
        cpu_percent = '\t'.join(["{:10.4f}".format(v) for v in cpu_percent])
        print("time:", int(elapsed_time), cpu_percent)
    print("END monitor_cpu")

if __name__=="__main__":
    event = threading.Event()
    initial_time = time.time()
    m = threading.Thread(target=monitor_cpu,args=((initial_time,)))
    m.start()
    tmp = 0
    for i in range(100000000):
        tmp = i+i
    event.set()

```
## Results
From left to right: time (seconds), CPU1 utilization (%), CPU2 utilization ... CPU8 utilization.
```
START monitor_cpu
time: 1  52.0     1.0    29.7     1.0    28.7     2.0    30.4     0.0
time: 2  45.0     3.0    36.3     3.0    32.4     1.0    29.4     1.0
time: 3  43.6     2.0    31.0     1.0    33.7     0.0    27.7     2.0
time: 4  45.5     2.0    25.2     1.0    22.8     0.0    19.8     0.0
time: 5  41.6     1.0    26.0     1.0    26.7     1.0    20.8     1.0
time: 6  46.1     5.0    34.7     3.0    38.0     3.0    31.7     4.0
time: 7  63.0    10.0    52.5    10.0    55.4    10.9    55.0    10.0
time: 8  51.5     6.9    36.0     4.9    41.2     5.0    39.6     4.9
time: 9  55.0     0.0    20.6     1.0    26.0     0.0    14.9     0.0
time: 10  49.5    2.9    22.8     1.0    25.2     1.0    23.5     1.0
time: 11  43.6    1.0    32.7     0.0    28.3     0.0    23.8     1.0
time: 12  47.5    2.9    22.8     1.0    20.6     2.0    25.0     2.0
END monitor_cpu
```