DOI: 10.1594/pangaea.849428
title: Towards convection-resolving, global atmospheric simulations with the Model for Prediction Across Scales (MPAS) v3.1: an extreme scaling experiment, links to sourcecode of model and grides

description: The Model for Prediction Across Scales (MPAS) is a novel set of Earth system simulation components and consists of an atmospheric model, an ocean model and a land-ice model. Its distinct features are the use of unstructured Voronoi meshes and C-grid discretisation to address shortcomings of global models on regular grids and the use of limited area models nested in a forcing data set, with respect to parallel scalability, numerical accuracy and physical consistency. This concept allows one to include the feedback of regional land use information on weather and climate at local and global scales in a consistent way, which is impossible to achieve with traditional limited area modelling approaches. Here, we present an in-depth evaluation of MPAS with regards to technical aspects of performing model runs and scalability for three medium-size meshes on four different high-performance computing (HPC) sites with different architectures and compilers. We uncover model limitations and identify new aspects for the model optimisation that are introduced by the use of unstructured Voronoi meshes. We further demonstrate the model performance of MPAS in terms of its capability to reproduce the dynamics of the West African monsoon (WAM) and its associated precipitation in a pilot study. Constrained by available computational resources, we compare 11-month runs for two meshes with observations and a reference simulation from the Weather Research and Forecasting (WRF) model. We show that MPAS can reproduce the atmospheric dynamics on global and local scales in this experiment, but identify a precipitation excess for the West African region. Finally, we conduct extreme scaling tests on a global 3 km mesh with more than 65 million horizontal grid cells on up to half a million cores. We discuss necessary modifications of the model code to improve its parallel performance in general and specific to the HPC environment. We confirm good scaling (70 % parallel efficiency or better) of the MPAS model and provide numbers on the computational requirements for experiments with the 3 km mesh. In doing so, we show that global, convection-resolving atmospheric simulations with MPAS are within reach of current and next generations of high-end computing facilities.
description: The data archive is comprised of three medium-size test cases (mpas_120km.tar.bz2, mpas_100-25km.tar.bz2, mpas_60-12km.tar.bz2) and the model code MPAS v3.1, including minor modifications for a detailed assessment of the model performance (MPAS-Release-3.1_DOM.tar.gz). See also the "new version" link for a new version of the MPAS tool).The installation of the model code requires the following libraries to be installed on the system, using the same compiler as for MPAS itself: PnetCDF3 (Argonne Labs Parallel NetCDF3, current version 1.6.1), PHDF5 (Parallel HDF5, current version 1.8.14), NetCDF4 (current version 4.3.2), NetCDF-Fortran (current version 4.4.2) and PIO (Parallel I/O, https://github.com/PARALLELIO/ParallelIO, current version 1.9.19). The compilation of MPAS itself is relatively straightforward using the make utility.The three .tar.bz2 files contain the so-called run directories for the three test cases. After extracting the .tar.bz2 files and linking the model executable (atmosphere_model) from the MPAS installation in this directory, they can be run straight away without further modifications. The model output is written in standard NetCDF3 format on an irregular grid. In order to visualise this data, the output must be regridded to a regular lat-lon grid.
description: Supplement to: Heinzeller, Dominikus; Duda, Michael G; Kunstmann, Harald (2016): Towards convection-resolving, global atmospheric simulations with the Model for Prediction Across Scales (MPAS) v3.1: an extreme scaling experiment. Geoscientific Model Development, 9(1), 77-110