# Compression Challenges in Large Scale Partial Differential Equation Solvers
## 5 Mass Storage
### Checkpoint/Restart  
In exa-scale HPC systems, node failure will be a common event. Checkpoint/restart is thus mandatory, but snapshotting for fault tolerance is increasingly expensive due to checkpoint sizes. Application-based checkpointing aims at reducing the overhead by optimizing snapshot times, i.e., when to write a checkpoint, and what to write, i.e., store only information that cannot be re-computed in a reasonable amount of time. Moreover, information should only be stored with required accuracy, which might be significantly smaller than double precision values. Lossy compression for fault-tolerant iterative methods to solve large-scale linear systems is discussed by [PERSON] et al. (2018). They derive a model for the computational overhead of checkpointing both with and without lossy compression, and analyze the impact of lossy checkpointing. Numerical experiments demonstrate that their lossy checkpointing method can significantly reduce the fault tolerance overhead for the Jacobi, GMRES, and CG methods. [PERSON] et al. (2019) investigate using lossy compression to reduce checkpoint sizes for time stepping codes for PDE simulations. For choosing the compression tolerance they aim at an error less than the simulation's discretization error, which is estimated a priori using information about the mesh width of the space discretization and order of the numerical methods. Compression is performed using SZ ([PERSON] et al., 2018; [PERSON] et al., 2018). Numerical experiments for two model problems (1D-heat and 1D-advection equations) and two HPC applications (2D Euler equations with PlacComCM, a multiphysics plasma combustion code, \begin{table}
\begin{tabular}{l c c} \hline \hline  & **Double** & **Transform Coding** \\ \hline uniform & 1 & 54 \\ adaptive & 62 & 744 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Compression factors for adaptive mesh refinement and transform coding of the peak function shown in Figure 10. Adaptive and uniform mesh refinement yield the same interpolation error; the same error tolerance for lossy compression was used in the two cases. Figure 10: _Left_: highly local peak function. _Right_: adaptively refined mesh with 4237 vertices for minimal finite element interpolation error. A uniform grid with the same local resolution has 263,169 vertices. and 3D Navier-Stokes flow with the code Nek5000) demonstrate that restart from lossy compressed checkpoints does not significantly impact the simulation, but reduces checkpoint time. Application-specific fault-tolerance including computing optimum checkpoint intervals [91; 92] or multilevel checkpointing techniques [93] has been a research topic for many years. In order to illustrate the influence of lossy compression, we derive a simple model similar to [92], relating probability of failure and checkpoint times to the overall runtime of the application. We assume equidistant checkpoints and aim at determining the optimum number of checkpoints \(n\). For this we consider a parallel-in-time simulation application (see Section 4.1) and use the notation summarized in Table 2. The overall runtime of the simulation consists of the actual computation time \(T_{C}\), the time it takes to write a checkpoint \(T_{\text{CP}}\), and the restart time \(T_{\text{RS}}\) for \(N_{\text{RS}}\) failures/restarts, \(T=T_{C}+nT_{\text{CP}}+T_{\text{RS}}N_{\text{RS}}\). Note that here \(T_{C}\) depends implicitly on the number of cores used. The time for restart \(T_{\text{RS}}\) consists of the average required re-computation from the last written checkpoint to the time of failure, here for simplicity assumed as \(\frac{1}{2}\frac{T}{n}\) (see also [91]) and time to recover data structures \(T_{\text{R}}\). For \(N\) compute cores, and probability of failure per unit time and core \(p_{\text{RS}}\), we get the estimated number of restarts \(N_{\text{RS}}=p_{\text{RS}}TN\).