Package: climdex-kit
climdex-kit: compute, publish, analyseBackgroundContentTechnologiesInstallationUsageExamplesThis project contains a Python package for the parallelized
local computation ofscenario-aware
climate indices starting from input time-series
of climate projections.The package comes with an pre-compiled initial set of indices, mostly
relying on theCDOoperators.
For the more advanced ones (SPI and SPEI), theclimate-indicesPython package is used.
This set of indices can be easily customised or extended: please
check outhow to contributeif you are interested.BackgroundA climate index is information derived from one or more climate variables
(precipitation, mean temperature, etc). The index can range from a simple
conditional counting of days against a threshold, to more complex
statistical processing.The information is usually calculated spatially
on a pixel-by-pixel basis, with possibly some form of aggregation over time.
Hence the spatio-temporal domain of an index is generally a time-series
with the same spatial resolution, and with either the same or a coarser
temporal step.The availability of robust and easily interpretable information about
the spatial distribution and temporal evolution of climate related-hazards,
especially climate extremes, is an increasing need not only for the
research community but also for a wide range of sectors and applications.The European Environmental Agency has been currently supporting, in the
framework of the European Topic Centre on Climate Change impacts, vulnerability
and Adaptation (ETC-CCA), the review and selection of suitable climate-related
indices for Europe to be recommended and implemented for adaptation purposes.ContentThe project is structured as follows:src/: source folder containing the Python package implementationtest/: test suitesetc/: folder with configuration files (most notably theindices.inifile with the definition of the climate indicesnotebooks/: folder with the Jupyter notebooksdocs/: documentation folderrequirements.txt/environment.yml: package dependencies forpipandcondaenvironmentsMakefile: set of rules for building and installing the packageCONTRIBUTING.md: help for developersCHANGELOG.md: list of notable changes for each release of the projectREADME.md: this fileThe following files are required for packaging and distribution:pyproject.toml: tells what is required to build the projectsetup.cfg/setup.py: static/dynamic package metadata forsetuptoolsTechnologiesThe package is written in Python and for most of the implemented climate indices
relies on both theCDOlibrary, and theclimate-indicespackage.InstallationThe package is available as bothpipsdist/wheel andcondapackage.DependenciesThe package relies on theCDOoperators library v1.9.9. If you are usingcondaenvironments, you can install the package as follows:condainstall-cconda-forgecdo=1.9.9In alternative, you can install theclimdex-kitcondapackage as explained later in this file.Install from PyPI via pip$python3-mpipinstallclimdex-kitInstall as conda packageTODOInstall in development modeFor developers: refer to the instructions in theCONTRIBUTINGfile for the setup of the development environment instead.UsageTheclimdexPython package provides a set sub-commands for the specific actions to be takenlist/ls: list all available indicesshow/sh: show the details of a specific indexcompute/co: compute one or more indicesThere is thus a hierarchical organization of the CLI arguments.
At any level of the hierarchy, the--help/-hoption can be called to print
the help message.general argsoptiondescriptionallowed values--versionGet the version number of the program--idx-conf/-cFILEAlternative indices configuration file (.ini) (default is ./etc/indices.ini)abs/rel path--log-conf/-LFILEAlternative logging configuration file (.yaml) (default: ./etc/logging.yml)abs/rel path-dEnable debug mode(See$ python -m climdex -hfor a full synopsis){list,ls} argsThis sub-command currently does not provide any option.
Run$ python -m climdex listto get a summary of all available climate indices.{show,sh} argsoptiondescriptionallowed valuesindexthe index configuration to be visualizedsee{list,ls}sub-command(See$ python -m climdex show -hfor a full synopsis){compute,co} argsoptiondescriptionallowed values--index/-iINDEXA comma-separated list of indices to be computessee{list,ls}sub-command--multiprocessingThe CPU parallelism to be employedint>0 (N of CPUs) or one among {one,all_but_one,all}--idirDIRRoot folder where to look for input files (expected structure:$input_dir/variable/scenario/*.nc)abs/rel path--odir/-oDIRRoot folder where to store indices filesabs/rel path--scenario/-sSWhite-space separated list of scenariossub-folders of input variables--regex/-xRFilter input files with a regular expressionregex--metadata-only/-mOnly re-set the output attributes (metadata) on existing indices files (compute the index file too on non-existing file instead)--dry-run/-nOnly print jobs to output without doing anything--force/-fForce overwrite of existing output indices files (otherwise execution is stopped)(See$ python -m climdex compute -hfor a full synopsis)Data organizationThe package expects a fixed organization of the input datasets and a fixed naming scheme
in order to properly extract all the metadata.The path and name of a climate projection NetCDF starting from the $IDIR input
root directory (--idirin the command line) shall be as follows:$IDIR/{var}/{scenario}/{var}_{model}_{timeres}_{yearstart}{yearend}_{scenario}.ncBeing:{var}: the climate variable (whose label shall also coincide with the name of the
variable in the NetCDF){scenario}: the name of the emissions scenario{model}: the name of the climate model used to create the projectiontimeres: the time-step of the time-series (e.g. day, month, etc)yearstart/yearend: time range of the time-series (YYYYformat)For ancillary scenario-independent datasets (e.g. land mask),
the{scenario}/sub-folder can be omitted mandatory, and the name
of the dataset shall be{var}.nc.Analogously, given the $ODIR output specified via--odir/-oCLI argument,
each index file will be stored then as follows:$ODIR/{index}/{scenario}/{index}_{model}_{timeres}_{yearstart}{yearend}_{scenario}.ncLoggingBy default the program logs to both console (with colored output to highlight warnings and errors),
and to a file calledclimdex.login the current working directory.The configuration of both loggers can be found in./etc/logging.yaml, otherwise use
the--log-conf/-Loption to set an alternative configuration.Examples# list all avaiable indices$python-mclimdexlist# show the configuration details of the index [spei12]$python-mclimdexshowspei12# compute the frost days [fd] and 12-months SPI [spi12] index on all available climate#   projections for scenario rcp85 and by using 3 CPUs$python-mclimdex\compute\--indexamt,spi12\--multiprocessing3\--scenariorcp85\--idir$IDIR\--odir$ODIR# update the metadata of all existing indexes of scenario rcp85, and compute the missing ones anew#   using all CPUs + turn on debug mode + dry run only$python-mclimdex-d\compute\--indexall\--scenariorcp85\--multiprocessingall\--metadata-only\--idir$IDIR\--odir$ODIR\--dry-run# re-compute the [fd] and [tn] indices for the model "EUR-11_CNRM-CERFACS-CNRM-CM5_CLMcom-CCLM4-8-17_r1i1p1_v1"#   and scenario rcp45 and keep it on a separate file for comparison with existing#   + use 1 CPU (sequential execution)$python-mclimdex\compute\--indexfd,tn\--scenariorcp45\--multiprocessingone\--idir$IDIR\--odir$ODIR\--regex"*EUR-11_CNRM-CERFACS-CNRM-CM5_CLMcom-CCLM4-8-17_r1i1p1_v1*"# re-compute (and overwrite when existing) all indexes on rcp45 and rcp85 scenarios#   + using all available CPUs except one$python-mclimdex\compute\--indexall\--scenariorcp45rcp85\--multiprocessingall_but_one\--idir$IDIR\--odir$ODIR\--forceCreditsThis project is funded by theFAct CLIMAXproject at Eurac Research (Institute for Earth Observation).