climpred: weather and climate forecast verification in python
Authors: Spring Aaron 
Published: March 2022

&lt;p&gt;Predicting subseasonal to seasonal weather and climate yields numerous benefits for economic and environmental decision-making.&lt;br&gt;Forecasters verify the forecast quality of models by initializing large sets of retrospective forecasts to predict past variations and phenomena in hindcast studies.&lt;/p&gt;&lt;p&gt;Quantifying prediction skill for multi-dimensional geospatial model output is computationally expensive and a difficult coding challenge. The large datasets require parallel and out-of-memory computing to be analyzed efficiently. Further, aligning the many forecast initializations with differing observational products is a straight-forward, but exhausting and error-prone exercise for researchers.&lt;/p&gt;&lt;p&gt;To simplify and standardize forecast verification across scales from hourly weather to decadal climate forecasts, we built &lt;em&gt;climpred&lt;/em&gt;: a python package for computationally efficient and methodologically consistent verification of ensemble prediction models. We rely on the &lt;em&gt;python&lt;/em&gt; software ecosystem developed by the open &lt;em&gt;pangeo&lt;/em&gt; geoscience community. We leverage &lt;em&gt;NetCDF&lt;/em&gt; metadata using &lt;em&gt;xarray&lt;/em&gt; and out-of-core computation parallelized with &lt;em&gt;dask&lt;/em&gt; to scale analyses from a laptop to supercomputer.&lt;/p&gt;&lt;p&gt;With &lt;em&gt;climpred&lt;/em&gt;, researchers can assess forecast quality from a large set of metrics (including &lt;em&gt;cprs&lt;/em&gt;, &lt;em&gt;rps&lt;/em&gt;, &lt;em&gt;rank_histogram&lt;/em&gt;, &lt;em&gt;reliability&lt;/em&gt;, &lt;em&gt;contingency&lt;/em&gt;, &lt;em&gt;bias&lt;/em&gt;, &lt;em&gt;rmse&lt;/em&gt;, &lt;em&gt;acc&lt;/em&gt;, ...) in just a few lines of code:&lt;/p&gt;&lt;p&gt;hind = xr.open_dataset('initialized.nc')&lt;br&gt;obs = xr.open_dataset('observations.nc')&lt;br&gt;he = climpred.HindcastEnsemble(hind).add_observations(obs)&lt;br&gt;# he = he.remove_bias(how='basic_quantile',&lt;br&gt;# &amp;#160; &amp;#160; &amp;#160; &amp;#160; &amp;#160; &amp;#160; &amp;#160; &amp;#160; &amp;#160; &amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160; train_test_split='unfair',&amp;#160;&lt;br&gt;# &amp;#160; &amp;#160; &amp;#160; &amp;#160; &amp;#160; &amp;#160; &amp;#160; &amp;#160; &amp;#160; &amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160; alignment='same_verif')&lt;br&gt;he.verify(metric='rmse',&lt;br&gt;&amp;#160; &amp;#160; &amp;#160; &amp;#160; &amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160; comparison='e2o',&lt;br&gt;&amp;#160; &amp;#160; &amp;#160; &amp;#160; &amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160; alignment='same_verif',&lt;br&gt;&amp;#160; &amp;#160; &amp;#160; &amp;#160; &amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160; dim='init',&lt;br&gt;&amp;#160; &amp;#160; &amp;#160; &amp;#160; &amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160;&amp;#160; reference=['persistence', 'climatology'])&lt;br&gt;&lt;br&gt;This simplified and standardized process frees up resources to tackle the large process-based unknowns in predictability research. Here, we perform a live and interactive multi-model comparison removing bias with different methodologies from NMME project hindcasts and compare against persistence and climatology reference forecasts.&lt;/p&gt;&lt;p&gt;&lt;img src=&quot;https://contentmanager.copernicus.org/fileStorageProxy.php?f=gnp.55bc62bebed162677891461/sdaolpUECMynit/22UGE&amp;app=m&amp;a=0&amp;c=f1bcc77dd44d983258ee3946a8c5b41e&amp;ct=x&amp;pn=gnp.elif&amp;d=1&quot; alt=&quot;&quot; width=&quot;487&quot; height=&quot;113&quot;&gt;&lt;/p&gt;&lt;p&gt;Documentation: https://climpred.readthedocs.io&lt;/p&gt;&lt;p&gt;Repository: https://github.com/pangeo-data/climpred&lt;/p&gt;&lt;p&gt;Reference paper: Brady, Riley X. and Aaron Spring (Mar. 2021). &amp;#8220;Climpred: Verification of Weather and Climate Forecasts&amp;#8221;. en. Journal of Open Source Software 6.59, p. 2781. https://joss.theoj.org/papers/10.21105/joss.02781&lt;/p&gt;