Reproducible HPC Software Installation on Cray Systems with EasyBuild
Channel: FOSDEM
Date: 2018-03-06

by Guilherme Peretti-Pezzi

At: FOSDEM 2017

EasyBuild is a framework for building and installing (scientific) software onHPC clusters, implemented in Python and available under the GPLv2 FOSSlicense. It has served HPC support teams well on standard Linux/x86 systemssince 2012.

More recently, interest has grown to also employ EasyBuild on Cray systems,where the Cray Programming Environment (PE) not only provides common tools(compilers, ...) and libraries (MPI, BLAS/LAPACK, FFTW, ...), but also tunedinstallations of 3rd party libraries like HDF5, netCDF and PETSc. The Cray PEconsists of an extensive stack of environment modules, providing a familiaryet complex working environment of users.

To leverage the Cray PE a couple of enhancements had to be made to EasyBuild,including support for using 'external' modules, and defining Cray-specificcompiler toolchains. In addition, the way in which EasyBuild deals withenvironment modules had to be carefully revisited, since standard operationslike 'module purge' are not supported by the Cray PE.

This work has led to the use of EasyBuild in production on the largest Craysupercomputer in Europe, the Piz Daint system at the Swiss NationalSupercomputing Centre.

In this talk, we will outline how Cray systems are different from standard HPCsystems, motivate the need for a tool like EasyBuild despite the presence ofCray PE, and show how EasyBuild was tweaked to support reproducible softwareinstallations on Cray systems.

We believe this is interesting success story of how FOSS has significantlyenhanced the software environment on a multi-million dollar production system.


Room: H.2213
Scheduled start: 2017-02-04 11:00:00