<figure><img src="image_1.png" /></figure>

HAL
open science

# ResourceMonitor: Coarse-grain resource monitoring tool for HPC applications

Esragul Korkmaz

► To cite this version:

Esragul Korkmaz. ResourceMonitor: Coarse-grain resource monitoring tool for HPC applications. 2025. hal-05368647

HAL Id: hal-05368647

https://hal.science/hal-05368647v1

Submitted on 17 Nov 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

AIRBUS

<figure><img src="image_4.png" /></figure>

CERFACS

<figure><img src="image_5.png" /></figure>

concace

# ResourceMonitor<sup>ab</sup>: Coarse-grain resource monitoring tool for HPC applications

November 18, 2025

Esragul Korkmaz

Inria Bordeaux Sud-Ouest, France
Team-project CONCACE

<footnote>ahttps://gitlab.inria.fr/composyx/tools/ResourceMonitor/</footnote>

<footnote>bOpen source: CeCILL-C FREE SOFTWARE LICENSE AGREEMENT</footnote>
* HPC applications run across many cores/nodes

* Observing resource (Memory/CPU/Disk/Threads) behavior is hard

* Motivation: Quick coarse-grained resource usage info over time to detect any unexpected behavior

* Big picture: No complex details

* Easy-to-use: Only two python scripts

* traces.py: Generate the log files

* plot_traces.py: Visualize the log files

* Two usage modes:

* Basic usage: No instrumentation required

* Advanced usage: Easy instrumentation is required

* For coloring the time intervals according to the function being executed
Basic Usage: Generating log files

Run the monitored application

python3 traces.py ./my_program --arg1 --arg2
Run the monitored application

python3 traces.py ./my_program --arg1 --arg2

This produces four log files (one per metric and per process rank)

rss-{rank}.log (Memory usage)

top-{rank}.log (CPU usage)

hdd-{rank}.log (Disk usage)

linux_threads-{rank}.log (Total & active threads)
Run the monitored application

python3 traces.py ./my_program --arg1 --arg2

This produces four log files (one per metric and per process rank)

rss-{rank}.log (Memory usage)

top-{rank}.log (CPU usage)

hdd-{rank}.log (Disk usage)

linux_threads-{rank}.log (Total & active threads)

Log file structure

Each log file contains time-series data:

• `rss`, `top`, `hdd`: `<time> <value>`

• `linux_threads`: `<time> <total_threads> <active_threads>`
## Usage Example

```
python3 plot_traces.py
--taskNum=2
--threadNum=36
--inputDir=./example_logs
--outputDir=./example_logs
--rssUnit=MB
```
## Usage Example

```
python3 plot_traces.py
--taskNum=2
--threadNum=36
--inputDir=./example_logs
--outputDir=./example_logs
--rssUnit=MB
```

## Generated Figures

* all processes data.pdf — raw traces for all processes

* each process data.pdf — raw traces of each process

* aggregated_data.pdf — aggregated max/mean/min usage across processes
<figure><img src="image_6.png" /><figcaption>all processes data.pdf</figcaption></figure>
each_process_data.pdf

Per-Process Resource Usage

<figure><img src="image_7.png" /></figure>
<figure><img src="image_8.png" /><figcaption>aggregated_data.pdf</figcaption></figure>
* Colors the time intervals according to the function being executed

* An extra timeline file of routines (e.g. timeline.log) is necessary<fnref n="1" />

* Add a copy-paste C++ instrumentation block (see timeline.hpp)

* Add `enter_context` and `exit_context` calls before and after each desired function, respectively (see example.cpp)

<figure><img src="image_9.png" /></figure>

<footnote n="1">¹https://gitlab.inria.fr/composyx/tools/ResourceMonitor/-/tree/main/example_context_usage</footnote>
traces.py is used as it is:

python3 traces.py ./my_program --arg1 --arg2

plot_traces.py requires two extra arguments (in red):

python3 plot_traces.py

--taskNum=2

--threadNum=36

--inputDir=./example_logs

--outputDir=./example_logs

-- timelineFile= timeline.log

--routines="MPF_multifact_MUMPS_HMAT,MatFactor_HMAT()"

--rssUnit=MB
all processes data.pdf

<figure><img src="image_10.png" /></figure>
<figure><img src="image_11.png" /><figcaption>each_process_data.pdf</figcaption></figure>
<figure><img src="image_12.png" /><figcaption>aggregated_data.pdf</figcaption></figure>
* **ResourceMonitor**<sup>23</sup> provides a clear, high-level view of memory, CPU, disk and thread usage

* **Works immediately:** no code changes needed

* **Optional timeline instrumentation:** to color function phases in plots

* Designed for users who want to **see the big picture quickly**, without the complexity of full profilers

<footnote n="2">²https://gitlab.inria.fr/composyx/tools/ResourceMonitor/</footnote>

<footnote n="3">³Open source: CeCILL-C FREE SOFTWARE LICENSE AGREEMENT</footnote>
# Questions?

Any contributions, additions, corrections are welcome :)