Package: python-cmethods
python-cmethodsThis Python module serves as a collection of different scale- and
distribution-based bias correction techniques for climate sciences.The documentation is available at:https://python-cmethods.readthedocs.io/en/stable/Please cite this project as described inhttps://zenodo.org/doi/10.5281/zenodo.7652755.⚠️ For the application of bias corrections onlarge data setsit is
recommended to also try the command-line toolBiasAdjustCXX.Table of ContentsAboutAvailable MethodsInstallationUsage and ExamplesNotesContributionReferences1. AboutThese programs and data structures are developed with the aim of reducing
discrepancies between modeled and observed climate data. Historical data is
utilized to calibrate variables from current and future time series to achieve
distributional properties that closely resemble the possible actual values.Figure 1: Schematic representation of a bias adjustment procedureFor instance, modeled data typically indicate values that are colder than the
actual values. To address this issue, an adjustment procedure is employed. The
figure below illustrates the observed, modeled, and adjusted values, revealing
that the delta adjusted time series ($T^{*DM}{sim,p}$) are significantly more
similar to the observed data ($T{obs,p}$) than the raw modeled data
($T{sim,p}$).Figure 2: Temperature per day of year in observed, modeled, and bias-adjusted climate data2. Available Methodspython-cmethods provides the following bias correction techniques:Linear ScalingVariance ScalingDelta MethodQuantile MappingDetrended Quantile MappingQuantile Delta MappingPlease refer to the official documentation for more information about these
methods as well as sample scripts:https://python-cmethods.readthedocs.io/en/stable/Except for the variance scaling, all methods can be applied on stochastic and
non-stochastic climate variables. Variance scaling can only be applied on
non-stochastic climate variables.Non-stochastic climate variables are those that can be predicted with relative
certainty based on factors such as location, elevation, and season. Examples
of non-stochastic climate variables include air temperature, air pressure, and
solar radiation.Stochastic climate variables, on the other hand, are those that exhibit a high
degree of variability and unpredictability, making them difficult to forecast
accurately. Precipitation is an example of a stochastic climate variable
because it can vary greatly in timing, intensity, and location due to complex
atmospheric and meteorological processes.Except for the detrended quantile mapping (DQM) technique, all methods can be
applied to 1- and 3-dimensional data sets. The implementation of DQM to
3-dimensional data is still in progress.Except for DQM, all methods can be applied usingcmethods.adjust. Chunked
data for computing e.g. in a dask cluster is possible as well.For any questions -- please open an issue athttps://github.com/btschwertfeger/python-cmethods/issues3. Installationpython3-mpipinstallpython-cmethods4. Usage and Examplesimportxarrayasxrfromcmethodsimportadjustobsh=xr.open_dataset("input_data/observations.nc")simh=xr.open_dataset("input_data/control.nc")simp=xr.open_dataset("input_data/scenario.nc")# adjust only one grid cellls_result=adjust(method="linear_scaling",obs=obsh["tas"][:,0,0],simh=simh["tas"][:,0,0],simp=simp["tas"][:,0,0],kind="+",group="time.month",)# adjust all grid cellsqdm_result=adjust(method="quantile_delta_mapping",obs=obsh["tas"],simh=simh["tas"],simp=simp["tas"],n_quantiles=1000,kind="+",)# to calculate the relative rather than the absolute change,# '*' can be used instead of '+' (this is preferred when adjusting# stochastic variables like precipitation)It is also possible to adjust chunked data sets. Feel free to have a look intotests/test_zarr_dask_compatibility.pyto get a starting point.Notes:For the multiplicative techniques a maximum scaling factor of 10 is defined.
This can be changed by passing the optional parametermax_scaling_factor.Except for detrended quantile mapping, all implemented techniques can be
applied to single and multdimensional data sets by executing thecmethods.adjustfunction.Examples (see repository onGitHub)Notebook with different methods and plots:/examples/examples.ipynbThere is also an example script (/examples/biasadjust.py) that can be used to apply the available bias correction methods
on 1- and 3-dimensional data sets (see/examples/input_data/*.nc).Help:╰─python3biasadjust.py--help(1.) Example - Quantile Mapping bias correction on the provided example data:╰─python3biasadjust.py\--refinput_data/observations.nc\--contrinput_data/control.nc\--sceninput_data/scenario.nc\--kind"+"\--variable"tas"\--quantiles10\--methodquantile_mapping(2.) Example - Linear Scaling bias correction on the provided example data:╰─python3biasadjust.py\--refinput_data/observations.nc\--contrinput_data/control.nc\--sceninput_data/scenario.nc\--kind"+"\--variable"tas"\--group"time.month"\--methodlinear_scalingNotes:Data sets must have the same spatial resolutions.This script is far away from perfect - so please see it, as a starting point.
(:5. NotesComputation in Python takes some time, so this is only for demonstration. When
adjusting large datasets, you should either use chunked data using for example
a dask cluster or to apply the command-line toolBiasAdjustCXX.Formulas and references can be found in the implementations of the
corresponding functions, on the bottom of the README.md and in thedocumentation.Space for improvementsSince the scaling methods implemented so far scale by default over the mean
values of the respective months, unrealistic long-term mean values may occur
at the month transitions. This can be prevented either by selectinggroup='time.dayofyear'. Alternatively, it is possible not to scale using
long-term mean values, but using a 31-day interval, which takes the 31
surrounding values over all years as the basis for calculating the mean
values. This is not yet implemented, because even the computation for this
takes so much time, that it is not worth implementing it in python - but this
is available inBiasAdjustCXX.6. 🆕 Contributions… are welcome but:First check if there is an existing issue or PR that addresses your
problem/solution. If not - create one first - before creating a PR.Typo fixes, project configuration, CI, documentation or style/formatting PRs will be
rejected. Please create an issue for that.PRs must provide a reasonable, easy to understand and maintain solution for an
existing problem. You may want to propose a solution when creating the issue
to discuss the approach before creating a PR.7. ReferencesSchwertfeger, Benjamin Thomas and Lohmann, Gerrit and Lipskoch, Henrik (2023)"Introduction of the BiasAdjustCXX command-line tool for the application of fast and efficient bias corrections in climatic research", SoftwareX, Volume 22, 101379, ISSN 2352-7110, (https://doi.org/10.1016/j.softx.2023.101379)Schwertfeger, Benjamin Thomas (2022)"The influence of bias corrections on variability, distribution, and correlation of temperatures in comparison to observed and modeled climate data in Europe"(https://epic.awi.de/id/eprint/56689/)Linear Scaling and Variance Scaling based on: Teutschbein, Claudia and Seibert, Jan (2012)"Bias correction of regional climate model simulations for hydrological climate-change impact studies: Review and evaluation of different methods"(https://doi.org/10.1016/j.jhydrol.2012.05.052)Delta Method based on: Beyer, R. and Krapp, M. and Manica, A.:"An empirical evaluation of bias correction methods for palaeoclimate simulations"(https://doi.org/10.5194/cp-16-1493-2020)Quantile and Detrended Quantile Mapping based on: Alex J. Cannon and Stephen R. Sobie and Trevor Q. Murdock"Bias Correction of GCM Precipitation by Quantile Mapping: How Well Do Methods Preserve Changes in Quantiles and Extremes?"(https://doi.org/10.1175/JCLI-D-14-00754.1)Quantile Delta Mapping based on: Tong, Y., Gao, X., Han, Z. et al."Bias correction of temperature and precipitation over China for RCM simulations using the QM and QDM methods". Clim Dyn 57, 1425–1443 (2021). (https://doi.org/10.1007/s00382-020-05447-4)I'd like to express my gratitude to @riley-brady for initiating and
contributing to the discussion onhttps://github.com/btschwertfeger/python-cmethods/issues/47. I appreciate all
the valuable suggestions provided throughout the implementation of the
subsequent changes.