# Improving Productivity of Threaded Scientific Applications with Quo Vadis

Edgar A Leon, Samuel Gutiérrez, Guillaume Mercier

► To cite this version:

Edgar A Leon, Samuel Gutiérrez, Guillaume Mercier. Improving Productivity of Threaded Scientific Applications with Quo Vadis. PASC 2025 - Platform for Advanced Scientific Computing, Jun 2025, Windisch, Switzerland. hal-05176401

HAL Id: hal-05176401

https://inria.hal.science/hal-05176401v1

Submitted on 22 Jul 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
**Scientific Domain**

Computational Methods and Applied Mathematics: 1

**Title (Maximum 25 words)**

Title (Maximum 25 words): Improving Productivity of Threaded Scientific Applications with Quo Vadis

**Author Information**

**Author 1:**

Gender: male

Name: Dr. Edgar A. Leon

Email: leon@llnl.gov

Affiliation: Lawrence Livermore National Laboratory

2nd Affiliation:

Country of the Primary Affiliation: United States of America

Will this person present the poster at the conference? Yes

**Author 2:**

Gender: male

Name: Dr. Samuel K. Gutierrez

Email: samuel@lanl.gov

Affiliation: Los Alamos National Laboratory

2nd Affiliation:

Country of the Primary Affiliation: United States of America

Will this person present the poster at the conference? Yes

**Author 3:**

Gender: male

Name: Dr. Guillaume Mercier

Email: guillaume.mercier@bordeaux-inp.fr

Affiliation: Bordeaux-INP

2nd Affiliation: Inria, CNRS, LaBRI UMR 5800

Country of the Primary Affiliation: France

Will this person present the poster at the conference? Yes

**Long Abstract**

Long Abstract (Maximum 800 words):

Scientific discovery is increasingly enabled by heterogeneous computing hardware that includes multiple processor types (latency-and throughput-optimized cores), deep memory hierarchies (high-bandwidth and high-capacity memories), and heterogeneous memories (volatile and non-volatile). To effectively utilize this complex hardware, computational scientists must compose their applications using a combination of programming models, middleware, and runtime systems.

Since these systems are often designed in isolation from each other, their concurrent execution results in resource contention and interference, which limits application performance, scalability, and robustness. This problem adds to the already complex interactions between multiple physics libraries and emerging machine learning components in scientific applications. Consequently, real-world applications face salient challenges on heterogeneous machines.
To better understand these challenges, consider applications in the area of Climate Sciences. These applications are comprised of multiple kernels corresponding to different models such as Atmospheric, Ocean, and Marine Bio-geochemistry physics. While these models interact with each other, they tend to be developed independently and may use different programming abstractions. The different models are coupled into a single multi-physics application.

For instance, a multi-kernel simulation using the OASIS code coupling software from the CERFACS basic and applied research center may consist of three different kernels or phases. The first phase consists of MPI code whose performance is best using one MPI process per core; the second phase consists of OpenMP code (single process) whose performance is best using one OpenMP thread per core; and the third phase uses hybrid MPI+OpenMP code whose performance is best using one MPI process per NUMA domain and launching one thread per core within each process. A static parallel configuration across phases would severely impact the code's performance.

To achieve optimal performance, each phase must be mapped differently to the hardware. However, it is difficult for application developers to code the transitions from one phase to the next: (1) The topology of the machine needs to be discovered; (2) MPI processes need to be enabled or disabled selectively on a per phase basis; and (3) Processes and threads need to be mapped and pinned to different resources based on the current phase. For these (and other) reasons, tuning multi-physics applications is a difficult, complex, and non-portable process.

In this work, we present the thread interface of Quo Vadis (QV), a runtime system that helps hybrid applications make efficient use of heterogeneous hardware, eases programmability in the presence of multiple programming abstractions, and enables portability across systems. The runtime system abstracts out low-level details of the hardware and presents an architecture-independent interface applications can use to leverage local resources automatically and without user intervention. This interface enables portability across computing platforms.

At PASC'24, we introduced the split operation for processes that partitions hardware resources into an arbitrary number of pieces and assigns processes to these pieces, enabling concurrency and avoiding resource interference. This operation does not involve machine-specific information and is portable across computer architectures.

We now introduce similar operations for both OpenMP and POSIX threads. This enables hybrid applications (with processes and threads) to fully take advantage of managing resources at a level of abstraction that computational scientists can employ rapidly. This is particularly useful when the mapping of one application's phase to the machine needs to change on the next phase. The QV thread interface has similar semantics (and syntax) to the process interface, allowing scientists to leverage a single-semantics model for partitioning and assignment of resources to workers, whether they are processes or threads.

The goal of this poster is three-fold: (1) Introduce the Quo Vadis thread interface to the computational science community; (2) Highlight a single-semantics model for both processes and threads on hybrid-application scenarios; and (3) Exchange ideas and expertise with computational scientists so that QV interfaces serve the needs of applications more effectively. The poster will include skeleton-application examples demonstrating how the challenges described above can be met in a portable way across systems and with a small effort from application writers.

When combined with application-specific heuristics, QV enacts tailored execution policies by using dynamic hardware affinities that are exposed through a straightforward stack semantics. Where before a new phase is entered, we can push a particular policy, execute in that new environment, and then revert the policy by popping it off the affinity stack. With these semantics, a user can
stack an arbitrary number of binding policies that map to the potentially stacked composition of coupled components in a QV-enabled application.

We believe computational scientists should focus on their domain science rather than computer science, which they now deal with due to the increasing complexity of the hardware. To this end, we extended QV to provide a thread interface for multi-physics applications to enable productivity in the face of different programming abstractions and heterogeneous hardware. Quo Vadis is open source and available on GitHub.

## Short Abstract

Short Abstract (Maximum 200 words):

Scientific discovery is increasingly enabled by heterogeneous hardware that includes multiple processor types, deep memory hierarchies, and heterogeneous memories. To effectively utilize this hardware, computational scientists must compose their applications using a combination of programming models, middleware, and runtime systems. Since these systems are often designed in isolation from each other, their concurrent execution results in resource contention and interference, which limits application performance and scalability. Consequently, real-world applications face numerous challenges on heterogeneous machines.

This poster presents the thread interface of Quo Vadis, a runtime system that helps hybrid applications make efficient use of heterogeneous hardware, eases programmability in the presence of multiple programming abstractions, and enables portability across systems. Applications using OpenMP or POSIX threads can now benefit from Quo Vadis' high-level abstractions to map and remap full physics packages to the machine dynamically with a handful of functions. Furthermore, the thread interface has similar semantics to the process interface, allowing scientists to leverage a single-semantics model for partitioning and assignment of resources to workers, whether they are processes or threads. With both process and thread interfaces, Quo Vadis broadens its applicability to improve the productivity of computational scientists across programming abstractions and heterogeneous hardware.

## Acknowledgement

Acknowledgement: yes