Reducing the HPC-datastorage footprint with MAFISCMultidimensional Adaptive Filtering Improved Scientific data Compression
Large HPC installations today also include large data storage installations. Data compression can significantly reduce the amount of data, and it was one of our goals to find out, how much compression can do for climate data. The price of compression is, of course, the need for additional computational resources, so our second goal was to relate the savings of compression to the costs it necessitates.
data compression netcdf hdf5