<figure><img src="image_1.png" /></figure>

HAL
open science

Internship report - CRPVIS, Visualization for
application optimization, CORHPEX (COmpiler
Runtime and Hardware Parameter EXplorer)
visualization tool

Raghid Osseiran

<figure><img src="image_2.png" /></figure>

To cite this version:

Raghid Osseiran. Internship report - CRPVIS, Visualization for application optimization, CORHPEX (COmpiler Runtime and Hardware Parameter EXplorer) visualization tool. Université de Bordeaux; Inria. 2025. hal-05224515

HAL Id: hal-05224515

https://inria.hal.science/hal-05224515v1

Submitted on 26 Aug 2025

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Internship report - CRPVIS, Visualization for application optimization, CORHPEX (COmpiler Runtime and Hardware Parameter EXplorer) visualization tool

Raghid Osseiran

June 2025

* **Institution:** Université de Bordeaux

* **Laboratory:** Centre Inria l'Université de Bordeaux, Talence

* **Team / Domain:** STORM / HPC

* **Supervisors:** Olivier Aumage, Lana Huong Scravaglieri

* **Duration:** May 14 2025 – July 11 2025 (2 month)
Contents

<table><tr><td>1</td><td>Introduction</td><td>3</td></tr><tr><td>1.1</td><td>Plan</td><td>3</td></tr><tr><td>2</td><td>Context</td><td>5</td></tr><tr><td>2.1</td><td>Simulations of Complex Phenomena</td><td>5</td></tr><tr><td>2.1.1</td><td>What is a Simulation</td><td>5</td></tr><tr><td>2.1.2</td><td>When Simulation is Not Appropriate</td><td>5</td></tr><tr><td>2.2</td><td>Impact of Compiler, Runtime, and Hardware Parameters on HPC Performance</td><td>6</td></tr><tr><td>2.3</td><td>Importance of Visualization in HPC and Research</td><td>7</td></tr><tr><td>3</td><td>Visualizations</td><td>8</td></tr><tr><td>3.1</td><td>Usage and layout</td><td>8</td></tr><tr><td>3.2</td><td>Heatmaps</td><td>10</td></tr><tr><td>3.3</td><td>Scatter Plots</td><td>16</td></tr><tr><td>3.4</td><td>Nested Plots</td><td>17</td></tr><tr><td>3.5</td><td>Correlation Matrices</td><td>20</td></tr><tr><td>3.6</td><td>UpSet Plots</td><td>23</td></tr><tr><td>3.7</td><td>More Plots</td><td>25</td></tr><tr><td>4</td><td>Conclusion</td><td>27</td></tr><tr><td>4.1</td><td>Approach and achievements</td><td>28</td></tr><tr><td>4.2</td><td>Benefits and limitations</td><td>28</td></tr><tr><td>4.3</td><td>Future Work</td><td>28</td></tr><tr><td>4.4</td><td>Concluding remarks</td><td>29</td></tr></table>
# 1 Introduction

Simulating natural phenomena with a computer typically requires a discretization<fnref n="1" /> step, using a sufficiently fine granularity<fnref n="2" /> to model them accurately. As a result, these simulations are extremely costly and demand significant computational resources. To meet this need, high-performance computing (HPC) has become essential. HPC systems consist of powerful clusters of machines capable of handling data-intensive and computationally demanding tasks efficiently. In particular, HPC is widely used to support complex scientific simulations in fields such as physics, biology, chemistry, and healthcare, where traditional analytical methods fall short.

These HPC applications are increasingly complex in the sense where there exists a multitude of tunable parameters that are to be set in order to achieve optimal performance. The way these parameters are set is crucial, since they directly affect an application's performance, therefore it is very important to find the best combination or configuration possible for these parameters in each possible context, no matter the size or the complexity of the application. The main challenges of these systems are finding the appropriate configuration per application on any given system and understanding how the configurations affect applications' metrics on a system.

To address this problem and provide solutions towards finding the most appropriate configuration per application on any given system, the CORHPEX [2] (Compiler, Runtime, and Hardware Parameters Design space Exploration) framework will be essential. This framework tackles this problem using what is called design space exploration, design space exploration involves selectively testing different combinations of settings or parameters to identify the best configuration for running applications effectively. It is a framework that proposes a unified methodology to perform the exploration, it enables choosing the exploration strategy, the design space to study, the applications to execute and the metrics to collect independently while involving little coding overhead. However, a significant challenge remains: effectively interpreting the vast amounts of data and metrics produced by these explorations. Raw numerical outputs alone provide little meaningful insight to programmers or end users, making it difficult to extract actionable conclusions or understand the underlying performance behaviors.

Visualization then plays a crucial role in Design Space Exploration as we could turn complex datasets into intuitive representations, enabling users to quickly interpret patterns, relationships, and performance trends. It facilitates the identification of optimal parameter settings by highlighting promising regions of the design space and making it easier to detect subtle interactions among parameters, working hand in hand with the framework used, in order to achieve optimality but also understand how we got there. Ultimately, visualization accelerates the decision-making process, enhances user understanding of system behaviors, and helps researchers gain deeper insights into the performance impacts of different configurations, which might otherwise remain hidden in raw numerical data. As such, visualization will be the main focus of this report, reflecting the core aspect of my work during the internship.

## 1.1 Plan

The rest of this report is organized as follows:

* **Section 2: Context.** We first review the role of simulation in scientific computing (2.1), then motivate why and how tunable parameters in the compiler, runtime, and hardware layers critically affect performance, energy, and reliability on HPC systems (2.2). We conclude the context with a brief primer on visualization as a tool for making sense of high-dimensional performance data (2.3).

<footnote n="1">¹the process of converting continuous data into a set of discrete intervals or categories</footnote>

<footnote n="2">²refers to the degree of detail or precision in the division or representation of data, processes, or models.</footnote>
* **Section 3: Visualizations.** This is the core of the report. We describe the overall architecture and command-line interface of the CRPVIS tool (3.1), then present each plot type in turn (P-I-R = Proposition, Implementation, Results):

- Heatmaps (3.2): P-I-R.

- Scatter plots (3.3): P-I-R.

- Nested plots (3.4): P-I-R.

- Correlation matrices (3.5): P-I-R.

- UpSet plots (3.6): P-I-R.

- line plots (3.7): P-I-R.

- and gain plots (3.8): each with a brief overview of purpose, design, and representative output.

* **Section 4: Conclusion.** We summarize key insights gleaned from visual exploration, discuss limitations, and outline avenues for future enhancement of both CRPVIS and the underlying CORHPEX framework.
# 2 Context

## 2.1 Simulations of Complex Phenomena

### 2.1.1 What is a Simulation

Many real-world systems (manufacturing lines, financial markets, climate patterns, high-performance workloads, etc.) evolve dynamically and interact non-linearly with their environment. Their behavior depends on numerous intertwined parameters, making closed-form analytical modeling either intractable<fnref n="3" /> or too coarse<fnref n="4" /> to be useful. To study such systems, engineers or scientists build an abstract model that mimics the essential casual mechanisms and execute this model on a computer. Running the model—i.e., performing a simulation, yields the system’s trajectory through time under chosen initial conditions and control parameters.

Because the model lives in silico<fnref n="5" /> we can repeatedly ask “what-if?” questions, compress or expand simulated time, replicate experiments with different random seeds, and record internal states that would remain hidden in a physical prototype. When the underlying dynamics are simple the simulation can be cross-checked against a closed-form analytical solution, providing validation. In more complex cases, simulation becomes the only practical solution, revealing the dynamic response of a system to controlled perturbations and quantifying performance or risk before costly real-world implementation.

### 2.1.2 When Simulation is Not Appropriate

Although versatile, simulation is not a universal remedy. If a problem admits a common-sense or closed-form solution, or if an inexpensive physical experiment is feasible, resorting to a computer model wastes time and resources. A classic illustration is the newsboy problem. Mike sells Sunday papers that cost $0.64 and retail at $2.41; daily demand is uniformly distributed between 18 and 38 copies. A short analytical derivation identifies the optimal order quantity:

* Shortage cost $C_s = 2.41 - 0.64 = 1.77$ (lost margin),

* Cost of over ordering $C_o = 0.64$,

* Critical fractile $F^* = C_s/(C_s + C_o) \approx 0.73$,

* Optimal order quantity $Q^* = 18 + F^*(38 - 18) \approx 33$ copies.

A one-line formula outperforms a full-scale simulation that would require sampling demand, running many replications, and analysing the resulting statistics. Simulation also becomes ill-advised when expected savings do not cover modelling costs, when hardware, software, or expert availability is insufficient, when data scarcity prevents credible calibration, or when human behaviour in open social systems defies tractable abstraction.

While the strengths and limitations of simulation frame its value as an investigative tool, the insights it produces ultimately hinge<fnref n="6" /> on the parameters we choose to vary and the hardware-software stack on which those variations run. Each compiler flag, runtime option, or architectural feature can amplify or mask the behaviours a simulation aims to reveal. Consequently, understanding how individual parameters, and especially their interactions shape performance, energy, and reliability on modern HPC systems is the next critical step.

<footnote n="3">³intractable: Hard to control or deal with.</footnote>

<footnote n="4">⁴coarse: Lacking refinement.</footnote>

<footnote n="5">⁵in silico: conducted or produced by means of computer modelling or computer simulation.</footnote>

<footnote n="6">⁶hinge: a type of joint that attaches two things together while allowing for limited movement.</footnote>
## 2.2 Impact of Compiler, Runtime, and Hardware Parameters on HPC Performance

HPC application performance is determined not only by algorithms, but also by a multitude of tunable parameters across the compilation, runtime, and hardware layers. Modern systems expose numerous knobs – from compiler optimization flags and precision settings, to runtime environment variables (thread counts, scheduling policies, NUMA memory placement), to hardware features (clock frequencies, Simultaneous Multi-Threading (SMT) toggles, cache prefetchers, power limits, etc.). Each of these parameters can significantly influence execution time, energy consumption, and even system reliability. Indeed, HPC systems have become increasingly complex with many tunable parameters impacting applications’ metrics (such as performance and energy). The challenge is that there is no one-size-fits-all setting: the optimal configuration is highly application and system dependent. Understanding how each parameter (and their combinations) affects performance metrics is a non-trivial task for HPC practitioners.

To appreciate the impact of these parameters, consider concrete examples. Thread and core allocation is a critical runtime choice: using too few threads may underutilize available CPU cores (leading to longer runtimes), whereas using too many threads can cause contention (e.g. threads fighting over shared resources) and even slow the program down. The way threads are pinned to cores and NUMA nodes matters as well. If an application’s threads and memory allocations are spread across different NUMA domains without care, remote memory accesses incur additional latency. Enabling NUMA-aware policies (ensuring a thread allocates memory on its local node) often improves data locality and thus performance. In fact, prior experiments have found that NUMA binding optimizations can noticeably boost performance and save energy, the CORHPEX [2] study even reported that tuning NUMA and cache prefetch settings yielded up to 1.85× speedup and 5.9× reduction in energy compared to a baseline run. This dramatic difference highlights how hardware-level parameters (memory placement, prefetchers) can be as influential as algorithmic improvements in HPC. The hardware prefetchers themselves are another example: modern CPUs have multiple prefetching mechanisms that speculate on memory access patterns. Turning certain prefetchers off or on changes memory traffic. For memory-bound codes, an effective prefetch strategy can hide latency and improve throughput; however, an aggressive prefetcher that fetches useless data might waste bandwidth and power. Thus, finding the best prefetcher configuration can directly impact both runtime and energy efficiency.

SMT (Simultaneous Multi-Threading), or hyper-threading, is a hardware parameter that often requires careful tuning in HPC. Enabling SMT allows two (or more) threads to share one physical core, which can improve resource utilization if one thread leaves functional units idle. For certain workloads, especially those that are I/O-bound or latency-sensitive, SMT can increase throughput. However, for CPU-bound numerical kernels, two threads on the same core might contend for the same caches and execution units, sometimes yielding less than a 2× speedup (or even slowdowns if the threads interfere). HPC practitioners often experiment with SMT on vs. off to see which yields better performance per watt for their specific application. This again ties into energy: SMT on means higher total power draw (more active threads) but potentially shorter execution, whereas SMT off might run cooler but longer. The trade-off is workload-dependent, and thus an important part of the tuning process.

At the compiler level, numerous parameters (flags) influence how code is optimized. Selecting a higher optimization level (e.g. `-O3` or `-Ofast`) can enable advanced optimizations like vectorization, loop unrolling, and inlining, often improving raw performance. These optimizations can reduce execution time by better exploiting the CPU's capabilities (e.g., using SIMD vector instructions). However, they may also increase instantaneous power usage, for example, AVX-512 instructions deliver higher performance but at a cost of greater power draw and potential thermal throttling. Moreover, certain compiler flags can affect numerical reliability: enabling aggressive floating-point optimizations might slightly change arithmetic associativity and lead to small differences in results, which is a reliability consideration if bit-perfect reproducibility is required. Thus, HPC users must balance raw speed with numerical stability. Other compiler parameters, such as linking against different math libraries or toggling precision (like using single vs. double precision),
also play a role. All these choices illustrate that compiler parameters are part of the performance-energy-reliability equation alongside runtime and hardware settings.

Crucially, these parameters do not exist in isolation, they interact in complex ways. An optimal setting for one aspect might undermine another. For example, pushing the CPU frequency to the maximum (to minimize runtime) also increases power consumption exponentially, which can lead not only to inefficiency energy-wise but also thermal stress. Higher temperatures and power draw can reduce hardware reliability or even cause frequency throttling. It has been observed that reducing frequency (or using dynamic voltage-frequency scaling) can significantly lower power and heat, thereby improving system reliability and longevity. Thus, HPC tuning often involves trade-offs: a configuration that achieves the fastest time may consume more energy or run the hardware harder, whereas a configuration optimized for energy (or reliability) might sacrifice some speed. The multi-objective nature of HPC performance (balancing speed, energy, correctness, reliability) means that the “best” configuration depends on what metric one prioritizes.

Given the myriad of parameters and their interdependencies, the space of possible configurations is enormous. Even for a single application on a fixed machine, one might have dozens of tunable settings, exploring all combinations is combinatorially explosive (a “prohibitively large search space for optimal parameters”). Exhaustive trial-and-error is usually infeasible in practice, especially since each trial may require a full application run. Researchers have therefore begun to employ systematic Design Space Exploration (DSE) methods to navigate this high-dimensional space. For instance, Scravaglieri et al. [2] developed the CORHPEX framework to automate the exploration of compiler, runtime, and hardware parameters. In their study, hundreds of configurations were evaluated for benchmarks and real applications on different systems. The results vividly demonstrated how tuning can yield extreme variations in outcomes. As noted above, certain combinations yielded nearly 6× better energy efficiency than the default settings with substantial performance gains. However, while automated DSE can generate this data, it leaves us with a new challenge: making sense of a deluge of performance results across a high-dimensional parameter space. A single experiment can produce thousands of data points (each a configuration with associated metrics), and interpreting this without guidance is extremely difficult. This is where visualization becomes essential, bridging the gap between raw data and human insight. In the next section, we discuss the importance of visualization in understanding such complex performance data, setting the stage for tools and methodologies (like the ones developed in this internship) to tackle the interpretation challenge.

## 2.3 Importance of Visualization in HPC and Research

Visualization plays a pivotal role in high-performance computing and in scientific research at large. In fields where data is abundant and multi-dimensional, human intuition often fails without visual aids. This is especially true in HPC performance analysis: the logs and results of large-scale experiments contain a wealth of information that is notoriously difficult to manage and make sense of, due to their complexity and volume. By converting data into a visual form, we leverage the human visual system's pattern-recognition capabilities to discern trends, outliers, and correlations that would remain hidden in raw numerical data. In fact, performance visualization of HPC codes, though a non-trivial endeavor, can be one of the most useful tools in analysis, saving considerable time and effort in the optimization process.

In the context of HPC design space exploration (such as the multi-parameter studies enabled by CORH-PEX), visualization is indispensable for interpreting the high-dimensional datasets produced. A tool like CORHPEX might run an application under hundreds of different parameter configurations, collecting metrics like execution time, energy usage, cache misses, etc., for each run. The result is a large table of results with many dimensions (one dimension per parameter varied, plus multiple output metrics). Understanding this high-dimensional data is challenging – one cannot simply look at a spreadsheet and immediately see which parameters matter the most or how they interact. This is where visualization techniques prove their value: by projecting the data into thoughtfully designed visual representations, we make the patterns salient. For example, a simple scatter plot can be used to compare two metrics, say runtime vs energy, across all tested configurations, this might reveal a Pareto frontier of “optimal” configurations that offer the best
trade-off between speed and energy. A heatmap or surface plot can depict how performance varies with two parameters (e.g. thread count on one axis, CPU frequency on the other), quickly showing ridges of high performance or valleys of inefficiency.

Good visualization supports not only trend-finding but also debugging and optimization in HPC. Often, when plotting performance data, certain configurations will stand out as outliers, for instance, one experiment run might be dramatically slower than others. Visualizing the results might immediately highlight that outlier point, prompting an investigation: perhaps it corresponds to a misconfigured run, a performance bug, or an out-of-memory situation. Without the visual cue, such an anomaly could be overlooked in a sea of numbers. Likewise, visualization can reveal subtle performance phenomena. Imagine an interactive timeline or sequence of plots showing how performance changes as we vary one parameter; if performance suddenly drops beyond a certain threshold, the visual cue might suggest a saturation point or a shift in the bottleneck (e.g. moving from CPU-bound to memory-bound regime). This insight can guide developers to focus their optimization efforts.

Visualization is also a powerful communication tool. In research, complex ideas often need to be conveyed to others, whether in papers or presentations, and a clear visual can succinctly illustrate a point that would otherwise require lengthy explanation. For HPC performance studies, graphs and charts are the lingua franca<fnref n="7" /> to report findings: speedup curves, efficiency plots, roofline diagrams, etc., all help distill complex performance characteristics into understandable visuals. Within the scope of this internship, harnessing visualization was crucial: we needed to take the massive output of parameter exploration and turn it into digestible knowledge. Effective visual analytics enable what we might call “data-driven reasoning”, seeing the evidence of how the system behaves under different settings, and using that evidence to drive decisions about where to optimize or how to configure future runs.

By intelligently combining multiple views of the data, a good visualization framework lets researchers derive insights that would be nearly impossible to obtain otherwise. This philosophy underpins the development of our own visualization toolkit in this project, which we turn to next.

# 3 Visualizations

## 3.1 Usage and layout

The CORHPEX visualization tool (CRPVIS) I have implemented is mainly broken down into four main components 1. Each box representing a component, and the arrows representing dependencies, with each component depending on the one(or ones) it points to.

<footnote n="7">⁷a language that is adopted as a common language between speakers whose native languages are different.</footnote>
<figure><img src="image_4.png" /><figcaption>Figure 1: CRPVIS Components</figcaption></figure>

The visualisation module is invoked from the command line and reads the SQLite database produced by
CORHPEX. A typical call is:

python crpvis.py <database> -p <plot> -m <metric1> <metric2> ...

- `-p/--plot` selects the plot type,

- `-m/--metrics` lists one or more metrics to visualise, and

- Additional flags (color-map, interactive HTML output, Highlighting, Normalization, etc.) are documented in `crpvis.py --help` and on the project wiki [1].

Furthermore, before jumping into plot-specific implementations, I need to explain how certain necessities to understand how the code works.

Based on the plot option chosen by the user, the corresponding visualization function is called. I chose to implement this using an array that stores pointers to functions:

crpvis.py - line 56.

self.__chart__visu_funcsn_ = [
    self.visualize_heat,          # 0: Standard heatmaps
    self.visualize_heat_fused,    # 1: Fused two-metric heatmaps
    self.visualize_scatter,       # 2: Scatter plots between two metrics
    self.visualize_nested,       # 3: 4*4 grid plots by prefetcher
    self.visualize_correlation,   # 4: Correlation matrices
    self.visualize_upset,        # 5: UpSet plots for set intersections
```python
self.visualize_lineplot, # 6: Line plots across regions
self.visualize gainplot  # 7: Shows gains for each dimension combination
]
```

Because each function serves as the visualization-setup method for a given plot type, I store them in a specific order: the list index doubles as the plot-type ID, which is assigned by the switcher and make_entry methods in crpvis_data.py.

```python
crpvis_data.py - line 85 and 194.

# ===================================================================
# Chart Type Registration System
# ===================================================================
# Maps chart names to entry dictionaries containing ID and output
# directory

self swtcher = {
    "heat": self.make_entry("heats"),
    "heat_fused": self.make_entry("heats_fused"),
    "scatter": self.make_entry("scatterplots"),
    //.....
}
```

```python
def make_entry(self, dir_):
    """
    Create chart type entry with unique ID and output directory.

    Part of the chart type registration system that maps chart names
    to unique IDs and their corresponding output directories.

    Argss:
        dir_ (str): Output directory name for this chart type

    Returns:
        dict: Entry with unique ID and directory name
    """
    ret = {"id": self.entry_id, "dir": dir_}
    self.entry_id += 1
    return ret
```

As we can see, the entry not only associates an ID with the chosen plot option but also a sub-directory name, so the generated plots end up in their own folder for a cleaner, more streamlined project structure. All directory logic is handled in dir_manager.py; I will not discuss it further here.

## 3.2 Heatmaps

### 3.2.1 Proposition

Heatmaps provide an immediate visual answer to a central question of design-space exploration: which parameter combinations systematically outperform the baseline? Representing the parameter space as a colored 2-D matrix lets us see performance frontiers at a glance, even when the dataset spans thousands of configurations. For each metric we therefore construct a heatmap, raw or baseline-normalized, using a fixed
color scale that highlights “good” (red) versus “bad” (blue) regions. Normalization is defined as:

$$Norm\_Val = Baseline\_value/Current\_value$$

Values greater than 1 indicate better performance than the baseline, values less than 1 indicate worse performance than the baseline, and values equal to 1 show identical performance.

<table><caption>Table 1: Normalised value indication.</caption><thead><tr><td>Norm\_Val</td><td>Performance</td></tr></thead><tbody><tr><td>&gt; 1</td><td>Better</td></tr><tr><td>&lt; 1</td><td>Worse</td></tr><tr><td>= 1</td><td>Same</td></tr></tbody></table>

### 3.2.2 Implementation

With the plot option being heat, the visualize_heat function is then called. Like the other visualization functions, it is tasked with setting up and invoking the routine that actually draws the plot.

```
crpvis.py - line 101.

def visualize_heat(self):
    """
    Generate heatmap visualizations for each metric.

    Creates either interactive 3D heatmaps (Plotly) or static 2D heatmaps ( matplotlib )
    showing performance metrics across regions (rows) and configurations (columns).
    """
    for metric in self.dtm.metrics:
        if self.dtm.interactive: # Ignore this case for the moment
            self.draw_interactive_heatmap(metric)
        else:
            self.draw_heatmap(metric)
```

All that is left to do now is to setup the pandas dataframe. Before doing so, I must explain how our experimental data are stored. For each region I chose to store a unique dataframe, that contains in the rows (so the index of the dataframe) the configuration names, and in the columns, the parsed hardware parameters (so for example nb_threads, nb_numa_nodes, etc.) + the requested performance metrics (example time, energy). The dataframe would look like this:

<table><caption>Table 2: Example region dataframe layout</caption><thead><tr><td>X</td><td>nb_threads</td><td>....</td><td>time</td><td>energy</td></tr></thead><tbody><tr><td>dtmaploc_th32...</td><td>32</td><td>...</td><td>2.3453</td><td>0.432</td></tr></tbody></table>

Here is the implementation of the method that builds and returns this DFame:

```
crpvis_data.py - line 367.

def build_region_dataframe(self, region):
    """
    region (str): Region name to process
    """
```
Returns:
pd.DataFrame: Region data with configurations as rows, parameters+metrics as columns
Returns empty Frame if no valid configurations found
"
# Get raw metric data for this region from CORHPEX database
appMetrics = corhpex. get_appMetrics (region)
parsed_cfg_info = {}

for config in appMetrics.keys():
# Use global cache to avoid reparsing configuration names
if config not in self.global_parsed_cfg_info:
self.global_parsed_cfg_info[config] = self.parse_config_name (config)

# Start with parsed hardware parameters
row = self.global_parsed_cfg_info[config].copy()
row["config_name"] = config

# Add performance metrics with optional normalization
metric_found = False
for metric in self.metrics:
metric_value = appMetrics[config][metric][0]

# Skip configurations with zero values (assumed invalid/incomplete)
if metric_value != 0:
if not self normalized:
# Use raw metric value
row[metric] = metric_value
else:
# Normalize relative to baseline: baseline_value / current_value
# Values > 1 indicate better performance than baseline
baseline_value = appMetrics[ self.default_conf ][metric][0]
row[metric] = baseline_value / metric_value
metric_found = True

# Only include configurations with at least one valid metric
if metric_found:
parsed_cfg_info[config] = row

# Handle case where no valid configurations found
if not parsed_cfg_info:
logger. error ("No valid configuration data found for region: {region}")
return pd. Frame()

# Convert to Frame with configuration names as index
return pd. Frame( list( parsed_cfg_info. values()).set_index ("config_name") )

Once that is done, I use another method that I implemented to combine all regional DataFrames into a single aggregated Frame, which is precisely what we need for heatmap plotting. In a heatmap, we care only about the values, which, as we have seen, are usually at the end of the Frame. We can therefore ignore the columns that hold hardware parameters. In the function that draws the heatmap, I then pivot the Frame so that the rows become the regions and the columns become the different configurations, and inside are the values, here is a snippet:

crpvis.py - line 208.
crpvis_data.py - line 388.
```python
def draw_heatmap(self, metric):
    """
    Create static 2D heatmap using matplotlib/seaborn.

    **Args:**
        metric (str): Performance metric to visualize

    **Features:**
        - Supports both normalized (relative to baseline) and raw value visualization
        - Highlights baseline configuration in green when normalization is enabled
        - Uses logarithmic normalization for raw values to handle wide value ranges
        - Automatic clipping of normalized values at 2.0 to prevent extreme outliers
    """
    df = self.dtm.buildCombined_dataframe()
    heatmap_df = self.dtm.pivot DF(df, metric)
    plt. figure(figure size=(50, 30))
    //........
```

```python
def pivot_df(self, df, metric):
    """
    Create a pivot table for heatmap visualization.

    **Transforms the combined Data Frame into a 2D matrix suitable for heatmaps:**
        - Rows: Region names
        - Columns: Configuration names
        - Values: Metric values

    **Args:**
        df (pd. Frame): Combined Frame from build combined data frame()
        metric (str): Performance metric to pivot on

    **Returns:**
        pd. Frame: Pivot table ready for heatmap visualization
    """
    return df. pivot_table(index='region', columns='config_name', values=metric)
```

### 3.2.3 Experiences and Results

Here are the resulting heatmaps, generated using data acquired by running experiments and simulations using the CORHPEX framework:
<figure><img src="image_5.png" /><figcaption>Figure 2: Normalized Heatmap energy</figcaption></figure>

Figure 2 was generated with an eight-level, quantized color scale to emphasize zones of interest and improve contrast. We clipped all values to the interval [0, 2], since any result twice as good as the baseline already marks a significant win, further gains (4×, 6×, etc.) add little extra insight. Users may also select a standard continuous color map; a snippet of that version appears below.
<figure><img src="image_6.png" /><figcaption>Figure 3: Snippet continuous color scale Heatmap</figcaption></figure>

In these maps, red cells mark configurations that outperform the baseline, while blue cells underperform. For instance, if one entire row is red, this means every configuration in that region yields lower energy than the baseline. Likewise, certain columns remain red across all rows, indicating those specific configurations deliver consistent improvements across every region. The user also has the option to generate fused heatmaps which combine the values of the first two metrics specified when executing the script. This is done by associating to color (red or blue, the hue) to one of the metrics, and the saturation (the brightness) to the other, so for example values that are red, and are very dark, perform better than the baseline in both energy and time, while values that are blue and bright perform worst than the baseline for both metrics. Here is an example for a fused heatmap generated using CRPVIS:

<figure><img src="image_7.png" /><figcaption>Figure 4: Fused heatmap snippet</figcaption></figure>
## 3.3 Scatter Plots

### 3.3.1 Proposition

Scatter plots are a standard way to explore relationships between two (sometimes three) variables. In our implementation, we draw one plot per region, placing execution time on the x-axis and energy on the y-axis—one point for each configuration. We also offer an optional highlighting mode so that readers can instantly see which points correspond to which configurations.

### 3.3.2 Implementation

I start by filtering for each region, the configurations based on the current prefetch profile, which for the moment is set by default to 5.

```python
for region, region DF in self.dtm.region_dfs.items():
    # Filter configurations based on current prefetch profile
    region DF = region DF[region DF.index.str.contains(self.dtm.prefetch_profile)]
    configs = [cfg for cfg in region DF.index]
    m1 vals = region DF[metric1].to numpy()
    m2 vals = region DF[metric2].to numpy()
```

Then all that is simply left to do is sort our data, and call the scatter method provided by matplotlib to generate the scatter plot. If the user enabled highlighting using the -hl flag, we identify the three worst, middle, and best performing configurations, highlighting each one with different colors and markers, adding also configuration name annotations, and correctly calculating the position of each annotation.

### 3.3.3 Experiences and Results

Here is a scatter plot generated for one of the regions:

<figure><img src="image_8.png" /><figcaption>Figure 5: Scatter plot (no highlighting)</figcaption></figure>

Figure 5 shows the raw scatter of time (x-axis) versus energy (y-axis) for all configurations in the region `jpdc_cere_3D_computetempOMP_150_large`. As expected, there is a loose positive correlation: configurations that run faster tend also to consume less energy. However, the cloud of purple points is broad, without further cues, it's hard to pick out the truly standout settings.
<figure><img src="image_9.png" /><figcaption>Figure 6: Scatter plot (with highlighting)</figcaption></figure>

Adding highlighting (Figure 6) immediately reveals the structure. The three lowest energy/time points (in green) cluster in the bottom left corner, all sharing 16 threads with one NUMA (Non-uniform memory access) node, prefetcher combination 5 (0101), and the "thmap_compact" compact thread mapping. The three worst performers (in orange) sit in the top right, where higher thread counts (32, 64) and scatter mappings drive both time and energy up by almost 50 %. The middle group (yellow) occupies an intermediate strip around (1.2×10⁹, 100), showing how intermediate thread counts or prefetcher settings trade off performance and efficiency.

These plots confirm two key insights:

• 1. A single configuration (16 threads + thmap_compact) dominates both metrics.

• 2. Increasing thread counts beyond 16 on this region yields diminishing returns and higher energy.

With highlighting enabled, users can instantly link back to configuration names (and thus their hardware parameters), turning what was once a diffuse cloud into a clear performance frontier.

## 3.4 Nested Plots

### 3.4.1 Proposition

Having four independent prefetcher flags each having two possible states, 0 or 1, there are $2^4 = 16$ possible prefetcher masks. To compare them all at once, we lay out 16 small scatter-plot subpanels in a 4×4 grid—one subplot per mask. Each subplot shows execution time (x-axis) vs. energy (y-axis) for that mask, so readers can instantly see how different prefetcher combinations shift the time/energy trade-off. I call this arrangement a “nested plot.”

### 3.4.2 Implementation

To build a nested plot, we first leverage Matplotlib's `subplots` to allocate a 4×4 grid of axes—one cell per prefetcher mask (0–15). For each region:
1. Create a 4×4 grid of subplots:

```python
fig, axes = plt.subplots(
    nrows=4, ncols=4, figsize=(16,12),
    sharex=True, sharey=True
)
fig.suptitle(f"{region}", fontsize=16)
```

2. Group by prefetch mask:

```python
# df_region is a DataFrame with a 'prefetch' column (0 15)
grouped = df_region.groupby("prefetch")
```

3. Populate each cell (or gray it out):

```python
for mask in range(16):
    ax = axes[mask//4, mask%4]
    if mask not in grouped.groups:
        ax.setFacecolor("#f0f0f0")
        ax.set_title(f"prefetch={mask}\n(no data)")
        continue
    subdf = grouped.get_group mask)
    ax(scatter(subdf[metric1], subdf[metric2], s=20, alpha=0.6)
    ax.set_title(f"prefetch={mask}")
```

4. Overlay best/mid/worst highlights (when enabled):

```python
if highlight:
    vals = subdf[metric2].values
    idx = np.argsort vals)
    best, mid, worst = idx[:3], idx[n//2-1:n//2+2], idx[-3:]
    ax.scatter(..., edgecolor="green", marker="o", label="best-3")
    ax.scatter(..., edgecolor="orange", marker="s", label="mid-3")
    ax.scatter(..., edgecolor="red", marker="^", label="worst-3")
```

5. Share axes, add legend, and save:

```python
fig.legend(loc="upper right", fontsize=10)
plt.tight_layout rect=[0,0,1,0.96])
self.drm.save_plot(plt, f"{region}_nested_scatter.png")
```

That's all—16 small scatter panels neatly arranged, each instantly revealing how prefetch flags reshape the time/energy trade-off.
3.4.3 Experiences and Results

<figure><img src="image_10.png" /><figcaption>Figure 7: Nested plot (no highlighting)</figcaption></figure>

Results are very similar to those of the scatter plots and can allow us to get similar insight, but one
major thing nested plots adds, is the option to compare the performances across all different prefetcher
combinations. Figure 8 shows what the nested plot would look like with highlighting on. I decided to remove
the annotations showing the configuration names unlike the scatter plots, since it would over-encumber the
plot.
<figure><img src="image_11.png" /><figcaption>Figure 8: Nested plot (with highlighting)</figcaption></figure>

The main reason I decided to continue on with these nested plots, was to be able to identify the best prefetcher profile for a certain goal, to also give the user default profiles to choose from before the script is to be executed, since prefetcher settings could only be changed from a sysadmin, and not the user himself.

## 3.5 Correlation Matrices

### 3.5.1 Proposition

Correlation matrices are used to show relationships between variables. I decided that it was essential to implement this plot as it provides a much clearer view of which parameters correlate strongly (or weakly). To do that, I used the Spearman correlation coefficient. Spearman's $\rho$ ranges from -1 to 1 and measures the strength and direction of a monotonic relationship between two ranked variables. Let's look at an example:

<table><caption>Table 3: Spearman p example</caption><tr><td>nb_th</td><td>1</td><td>0.78</td></tr><tr><td>energy</td><td>0.78</td><td>1</td></tr><tr><td>X</td><td>nb_th</td><td>energy</td></tr></table>

A value of 0.78 indicates a fairly strong positive monotonic relationship between the number of threads (nb_th) and energy consumption, in other words, as the number of threads increases, energy consumption tends to increase as well. Conversely, a negative coefficient indicates an inverse relationship: as one variable rises, the other tends to fall.
### 3.5.2 Implementation

The method to calculate the correlation for a dataframe is already provided to us by the pandas library, all we have to do then is to call it selecting the corresponding methode we wish to use, which in my case is spearman:

```python
corr_full = region DF[numeric_COLS].corr(method="spearman")
```

The all that is left to do is to plot using matplotlib for the legends, and seaborn's heatmap method for the plot itself:

```python
fig, ax = plt.subplots(figsize=(max(10, 0.5*len(numeric_COLS)), max(5, 0.5*len(numeric_COLS))

hm = sns. heatmap(
    corr_full,
    mask=mask,
    annot=True,
    //.....
)

ax.set_title(
    f"Correlation for region '{region}'\n"
)

cbar_ ax = ax. collections [0]. colorbar
//.....

self. dtm. drm.save_ plot ( plt, f"{region}_corr_num_ metrics.png")
```
### 3.5.3 Experiences and Results

Here is what a spearman correlation generated by CRPVIS looks like:

<figure><img src="image_12.png" /><figcaption>Figure 9: Spearman correlation matrix</figcaption></figure>

I have chosen to display only the lower triangle of each square Spearman correlation matrix to eliminate redundancy and keep the visualization clean and easy to read. As stated above, Spearman's ρ ranges from -1 (strong negative monotonic relationship) to +1 (strong positive monotonic relationship). In our example for the region `jpcd__cere__3D_computTempOMP_150_large`, we immediately see for example that the number of NUMA nodes correlates positively (ρ ≈ 0.71) with energy consumption, meaning more NUMA nodes tend to drive up energy use.

By summarizing correlations in this way, you can:

* Identify pairs of parameters to tune jointly. For instance, since threads (ρ ≈ 0.68) and `numa_node` both show strong positive correlations with energy, you might explore configurations that moderate thread counts on fewer NUMA nodes to balance performance and power.

* Perform feature selection. Variables that correlate very highly with one another (e.g., if two hardware settings always move in lockstep) may be redundant; dropping one can simplify modeling without losing much predictive power.

* Cluster-related settings. You can apply hierarchical clustering to the correlation matrix to discover groups of parameters that behave similarly, guiding both visualization and automated tuning strategies.

Finally, we can aggregate data across all regions, averaging each pairwise ρ, to generate a single “global” correlation matrix. This consolidated view helps you spot hardware parameters that consistently interact (positively or negatively) across the entire application suite, and prioritize those relationships when designing cross-region optimizations.
<figure><img src="image_13.png" /><figcaption>Figure 10: Average Spearman correlation across all regions</figcaption></figure>

## 3.6 UpSet Plots

### 3.6.1 Proposition

UpSet plots let us explore how different prefetcher combinations contribute to improvements over the default hardware configuration. By treating each of the four prefetcher bits as a Boolean “set”, we can visualize:

* Which combinations of bits yield the largest number of “better-than-default” configurations,

* How these sets overlap (e.g. how many configurations require both bit 0 and bit 2 to beat the default),

* And which individual bits provide unique gains by themselves.

This is invaluable for guiding targeted tuning: rather than testing every configuration in isolation, you immediately see which features work best together, and which features on their own are most effective.

### 3.6.2 Implementation

The core routine `draw_upset_plot()` iterates over each metric and region, filters out the configurations that outperform the default, extracts the four bits of the `prefetch` word as Boolean columns, and feeds them into `UpSet` (from the `upsetplot` library). I also offer an “extract_info” mode that dumps detailed intersection counts into JSON, so you can post-process the data programmatically or generate custom tables. This option to generate json files containing plot specific data with the plot is to be set using the `-ext 9` flag while running the script.
### 3.6.3 Experiences and Results

<figure><img src="image_14.png" /><figcaption>Figure 11: Upset plot (with highlighting)</figcaption></figure>

In this Upset plot, we are showing the number of configuration that have an execution time below that of the default configuration for each possible prefetcher combination. Intersection-size is the number of configurations that satisfy our current testing need (is this case time under a certain threshold) under that given prefetcher combination, and the horizontal boxes show for each unique prefetcher, the sum of all configurations that verified the testing need having that unique prefetcher active.

To facilitate downstream analysis, we also export these numbers in JSON form. A trimmed example of the JSON structure is:
<figure><img src="image_15.png" /><figcaption>Figure 12: UpSet plot JSON data</figcaption></figure>

With this JSON in hand, you can programmatically rank feature combinations across regions or metrics, feed the counts into other dashboards, or even train a simple model to predict which prefetcher settings will yield the best speedups in unseen regions.

## 3.7 More Plots

Many more plot types are available with the CRPVIS tool like lineplots and gainplots but they aren't fully done yet, since I believe they could be improved. Here are what they currently look like.

**Line plots:** The idea is to show the values of a given metric, over all regions using points, where each point is a different configuration. Since the values for metrics like time tend to be large, I decided it would be best to use a logscale (Unlike commonly used linear functions that show an increase or decrease along equivalent or equally spaced out increments, log scales are exponential, which means increasing quickly by large numbers, changing by factors rather than linearly) for better clarity and understanding in the variety and jump between values.
<figure><img src="image_16.png" /><figcaption>Figure 13: Line plot time</figcaption></figure>

Gain plots: Understanding how certain hardware parameters behave when combined together vs. on their own is crucial, and yet to be done in any of the previous plots. This is where gain plots come in, as it allows us to visualize if there is something to gain from conjointly changing some parameters together or on their own.
<figure><img src="image_17.png" /><figcaption>Figure 14: Gain plot energy subspace analysis</figcaption></figure>

n represents the number of configurations that were used to calculate the average gain for that given hardware parameter. To calculate the gains, I take the exact same hardware parameter settings as the default configuration, and change only the one (or ones) i am currently looking to calculate the gain for, for example smt (simultaneous multithreading) only has two possible states 0 (off) or 1 (on), and the default configuration has smt set to 0, so to calculate the gains we copy all other settings but set smt to 1, thus also explaining why only 1 configuration contributed to calculating the gain for smt.

# 4 Conclusion

The central challenge addressed in this internship was the interpretation of large volumes of performance data produced by comprehensive HPC parameter explorations. When faced with hundreds or thousands of experiment runs (each corresponding to a unique combination of compiler, runtime, and hardware parameters), the amount of data generated is overwhelming. Important trends and insights are easily obscured by the sheer scale and dimensionality of the dataset. Manually poring over tables of results to find optimal configurations or to understand performance quirks is impractical and error-prone. In other words, we had a classic “big data” problem in the HPC performance domain: how to extract meaning and actionable knowledge from an unwieldy collection of results.
## 4.1 Approach and achievements

To tackle this, the internship focused on developing CRPVIS, CORHPEX Visualization tool, as a direct response to the interpretation challenge. The CRPVIS tool [1] was designed to ingest the complex output of CORHPEX, and present it through a range of intuitive visualization plots. Throughout the project, I implemented and refined multiple visualization techniques specifically tailored to HPC performance data. For example, CRPVIS provides scatter plot views for two-metric analysis (allowing users to plot any metric against any other, such as execution time vs energy, or speedup vs number of threads), multi-dimensional parallel coordinates plots to observe how each configuration (line) performs across all metrics, and interactive filtering to focus on subsets of interest (e.g. highlighting configurations where reliability metrics pass a threshold, or isolating runs on a particular NUMA node configuration). By building CRPVIS, I essentially created a bridge between the raw data and the user's understanding, the tool turns databases into visual stories about performance.

Applying CRPVIS to our case studies yielded immediate benefits. Large spreadsheets of performance results were transformed into interactive dashboards where patterns leapt out. We could quickly identify which parameters had the most impact: for example, a parallel coordinates plot helped expose that configurations which disabled a certain hardware prefetcher had consistently lower energy consumption without much performance loss, suggesting a potential energy-saving tweak. These insights would have been much harder to glean without visual representation.

## 4.2 Benefits and limitations

The work done in this project demonstrates several clear benefits. First, it accelerates the analysis cycle, what used to take hours of manual analysis can now be done in minutes by interactively visualizing the results and immediately spotting the key information. Second, it improves the depth of understanding, by visualizing multi-dimensional relationships, we can observe not just single-factor effects but also interactions. This holistic view can lead to more robust performance tuning decisions. Third, CRPVIS serves as a communication tool, the plots and charts produced can be used to communicate findings to colleagues or in reports, thus disseminating the insights gained. For instance, instead of saying "we found that parameter X affects performance", we can show a graph that vividly demonstrates that effect. However, it is important to acknowledge the limitations of our approach as well. One limitation is scalability: as the number of parameters and data points grows even further, certain visualizations can become cluttered or less responsive. While CRPVIS handles the current dataset well, extremely large design spaces might require additional strategies (such as automatic clustering of similar configurations, or server-side pre-processing) to remain effective. Another limitation is that visualization is inherently an aid to human analysis, not a fully automatic solution. Our tool helps highlight patterns, but it still relies on the user to draw conclusions and make decisions. In cases where the data is ambiguous or noisy, a human analyst might still struggle to interpret the visuals correctly. In other words, CRPVIS does not replace expert knowledge, it complements it. Lastly, our focus was primarily on performance and energy metrics; reliability metrics (like error rates or variability between runs) were less central in our test cases. If those become important (for example, if exploring undervolting techniques that might induce errors), the visualization might need enhancement to properly represent reliability-related outcomes (perhaps via specialized plots for error distribution, etc.).

## 4.3 Future Work

There are several avenues to extend and improve upon this work. One obvious direction is to enhance scalability and automation. This could involve integrating machine learning techniques with visualization, for instance, using clustering or dimensionality reduction to guide the user to interesting regions of the parameter space automatically (essentially highlighting "these configurations are similar to each other" or "these few runs are very atypical"). Another direction is to broaden the metrics and data sources that
CRPVIS can handle. Additionally, supporting real-time visualization could be transformative: instead of visualizing only after all experiments finish, the tool could update plots on the fly as new data comes in, allowing experimenters to steer the exploration dynamically (for example, aborting unpromising runs early or adjusting the search based on interim results). From a usability standpoint, future work could focus on adding more interactive storytelling features, enabling users to annotate plots, save “views” or states of the analysis, and compare different experiments side by side. This would further aid in using the tool for collaborative performance tuning and reporting results. Finally, while CRPVIS was developed with CORHPEX in mind, the underlying concepts are applicable broadly. I could generalize the tool to support data from other HPC performance studies or even integrate it with runtime tuning frameworks so that it becomes part of an end-to-end autotuning system (where a user could visualize and then directly select a configuration to deploy). Such integrations would push the state-of-the-art in how performance engineering is conducted, making it more interactive and data-driven.

## 4.4 Concluding remarks

In summary, this internship confronted the challenge of making sense of HPC performance data at scale and delivered a practical solution in the form of a visualization-driven analysis tool. The combination of systematic exploration (via CORHPEX) and insightful visualization (via CRPVIS) proved to be a powerful approach for performance engineering. We reassert the core finding that understanding the performance-energy-reliability trade-offs in modern HPC systems requires not just collecting data, but presenting it in the right way. By enabling users to see the impact of compiler optimizations, runtime settings, and hardware features on their applications, we empower them to make better informed decisions. The work completed lays a foundation for future improvements and I anticipate that such visualization-guided performance tuning will become increasingly important as HPC systems and applications continue to grow in complexity. Ultimately, the techniques and tools developed here contribute to the larger goal of harnessing the full potential of HPC machines, not by brute-force experimentation alone, but by augmenting human insight with effective visual analytics.
References

[1] Corhpex Visu Page. URL: https://gitlab.inria.fr/storm-ifpen/corhpex-visu.

[2] Olivier Aumage Thomas Guignon Lana Scravaglieri, Ani Anciaux-Sedrakian and Mihail Popov. Compiler, runtime, and hardware parameters design space exploration. *IPDPS 2025 - 39th IEEE International Parallel and Distributed Processing Symposium*, 2025. URL: https://inria.hal.science/hal-04969854v1.