# Reproducibility Report for SC25 Paper Story of Two GPUs: Characterizing the Resilience of Hopper H100 and Ampere A100 GPUs

Philippe Swartvagher

► To cite this version:

Philippe Swartvagher. Reproducibility Report for SC25 Paper Story of Two GPUs: Characterizing the Resilience of Hopper H100 and Ampere A100 GPUs. Inria, LaBRI, Bordeaux INP. 2025, pp.2278-2279. hal-05363360

HAL Id: hal-05363360

https://inria.hal.science/hal-05363360v1

Submitted on 13 Nov 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Reproducibility Report for SC25 Paper
## Story of Two GPUs: Characterizing the Resilience of Hopper H100 and Ampere A100 GPUs

Philippe Swartvagher

Bordeaux INP, Inria

Talence, France

philippe.swartvagher@inria.fr

## Abstract

This reproducibility report provides details about the artifact evaluation done with regards to the Artifact Description and Evaluation appendix of SC25 paper Story of Two GPUs: Characterizing the Resilience of Hopper H100 and Ampere A100 GPUs by Cui et al. The work was done as part of the Reproducibility Initiative of SC25. The author is a member of the SC25 Reproducibility Committee.

## Keywords

Artifact Evaluation, Reproducibility

## ACM Reference Format:

Philippe Swartvagher. 2025. Reproducibility Report for SC25 Paper Story of Two GPUs: Characterizing the Resilience of Hopper H100 and Ampere A100 GPUs. In *The International Conference for High Performance Computing, Networking, Storage and Analysis (SC '25), November 16-21, 2025, St Louis, MO, USA*. ACM, New York, NY, USA, 2 pages. https://doi.org/10.1145/3712285.3769462

## 1 Overview of Reproduction of Artifacts

The following table provides an overview of each computational artifact's reproducibility status. Artifact IDs correspond to those in the AD/AE Appendices.

<table><thead><tr><th>Artifact ID</th><th>Available</th><th>Functional</th><th>Reproduced</th></tr></thead><tbody><tr><td>A<sub>1</sub></td><td>•</td><td>○</td><td>○</td></tr><tr><td>A<sub>2</sub></td><td>•</td><td>○</td><td>○</td></tr><tr><td>A<sub>3</sub></td><td>•</td><td>○</td><td>○</td></tr><tr><td>A<sub>4</sub></td><td>•</td><td>•</td><td>○</td></tr><tr><td>README</td><td>•</td><td>•</td><td>•</td></tr><tr><td>Badge awarded</td><td>yes</td><td>no</td><td>no</td></tr></tbody></table>

## 2 Reproduction of Computational Artifacts

### 2.1 Timeline

The artifact evaluation was conducted from Septembre 2, 2025, to September 3, 2025.

### 2.2 Computational Environment and Resources

The experiments conducted for artifact evaluation were performed on a laptop equipped with a Intel i7-1370P processor and 64 GB of RAM.

### 2.3 Details on Artifact Reproduction

The main problem for the reproducibility of this article is that the original dataset analyzed by the authors and presented in the paper is not available.

In the artifact, authors provide a subset of preprocessed dataset as Pickle files, which naturally gives different results than in the paper and prevents to test the artifact supposed to do this preprocessing.

The rest of this section explains more in details the encountered shortcomings.

*Introduction of Artifact Evaluation.* The scripts `statisticsplications_analysis.sh`, `application_impact_analysis.sh`, `dcgm_analysis.sh` and `emulation.sh` are not available in the provided artifact. The README does not contain instructions to access a pre-setup VM.

*Artifact A<sub>1</sub>.* There are several problems in the artifact setup: one = is missing to precise the version of Matplotlib: there is no version 7.0.8 for the Jupyter package; and the requested versions of Pandas, Matplotlib and NumPy are incompatible. Removing the version constraints of these packages allows to install the set of packages.

In most of the provided commands, the extension `.py` is missing in the script name executed by Python.

Raw log files are not provided, so the execution cannot be performed and the Pickle files needed for the following experiments are not generated nor provided in the artifact.

*Artifact A<sub>2</sub>.* For `JobDetailCollect.sh`, having access to the `sacct` command does not seem enough: on two clusters with Slurm, as a regular user, the `sacct` command returns nothing. Moreover, one needs to set several variables in the script, which is not precised in the instructions. The expected format for the variable content could also be explained (which format should have the date?).

*Artifact A<sub>3</sub>.* Why different versions of the Python packages than versions used for artifacts A<sub>1</sub> and A<sub>2</sub> are required?

Since DCGM data are not provided with the artifact, the script fails.

*Following instructions in the README.* The README in the artifact contains instructions to execute several scripts on a subset of the preprocessed dataset. All instructions given in the README

<footnote>Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).</footnote>

<footnote>SC '25, St Louis, MO, USA</footnote>

<footnote>© 2025 Copyright held by the owner/author(s).</footnote>

<footnote>ACM ISBN 979-8-4007-1466-5/2025/11</footnote>

<footnote>https://doi.org/10.1145/3712285.3769462</footnote>
work as expected and the results are similar to the provided ex-
pected outputs. However, the command
compute GPU XID error stats given to
SystemLogAnalysisPipeline.py is never mentioned in the Arti-
fact Evaluation instructions (while this script is mentioned several
times, but with different commands).

*Conclusion*. Since several artifact could not be executed and the instructions in the README differ from what is in the Artifact Evaluation instructions, only the badge *Available* is granted instead of the requested *Reproduced*.

**Disclaimer:** This Reproducibility Report was crafted by volunteers with the goal of enhancing reproducibility in our research domain. The time period allocated for the reproducibility analysis was constrained by paper notification deadlines and camera-ready submission dates. Furthermore, the compute hours in the shared infrastructure (e.g., Chameleon Cloud) available to the authors of this report were limited and restricted the scope and quantity of experiments in the review phase. Consequently, the inability to reproduce certain artifacts within this evaluation should not be interpreted as definitive evidence of their irreproducibility. Limitations in the time allocated to this review and the compute resources available to the reviewers may have prevented a positive outcome. Furthermore, reviewers assess the reproducibility of the artifacts provided by the authors; however, they are not accountable for verifying that the artifacts support the main claims of the paper.