Package: s2spy
s2spy: Boost (sub) seasonal forecasting with AIA high-level python package integrating expert knowledge and artificial intelligence to boost (sub) seasonal forecasting.Why s2spy?Producing reliable sub-seasonal to seasonal (S2S) forecasts with machine learning techniques remains a challenge. Currently, these data-driven S2S forecasts generally suffer from a lack of trust because of:Intransparent data processing and poorly reproducible scientific outcomesTechnical pitfalls related to machine learning-based predictability (e.g. overfitting)Black-box methods without sufficient explanationTo tackle these challenges, we builds2spywhich is an open-source, high-level python package. It provides an interface between artificial intelligence and expert knowledge, to boost predictability and physical understanding of S2S processes. By implementing optimal data-handling and parallel-computing packages, it can efficiently run across different Big Climate Data platforms. Key components will be explainable AI and causal discovery, which will support the classical scientific interplay between theory, hypothesis-generation and data-driven hypothesis-testing, enabling knowledge-mining from data.Developing this tool will be a community effort. It helps us achieve trustworthy data-driven forecasts by providing:Transparent and reproducible analysesBest practices in model verificationsUnderstanding the sources of predictabilityInstallationTo install the latest release of s2spy, do:python3 -m pip install s2spyTo install the in-development version from the GitHub repository, do:python3 -m pip install git+https://github.com/AI4S2S/s2spy.gitConfigure the package for development and testingThe testing framework used here ispytest. Before running the test, we get a local copy of the source code and installs2spyvia the command:gitclonehttps://github.com/AI4S2S/s2spy.gitcds2spypython3-mpipinstall-e.Then, run tests:python3-mpytestGetting starteds2spyprovides end-to-end solutions for machine learning (ML) based S2S forecasting.Datetime operations & Data processingIn a typical ML-based S2S project, the first step is always data processing.  Our calendar-based package,lilio, is used for time operations. For instance, a user is looking for predictors for winter climate at seasonal timescales (~180 days). First, aCalendarobject is created usingdaily_calendar:>>>calendar=lilio.daily_calendar(anchor="11-30",length='180d')>>>calendar=calendar.map_years(2020,2021)>>>calendar.show()i_interval-11anchor_year2021[2021-06-03,2021-11-30)[2021-11-30,2022-05-29)2020[2020-06-03,2020-11-30)[2020-11-30,2021-05-29)Now, the user can load the datainput_data(e.g.pandasDataFrame) and resample it to the desired timescales configured in the calendar:>>>calendar=calendar.map_to_data(input_data)>>>bins=lilio.resample(calendar,input_data)>>>binsanchor_yeari_intervalintervalmean_datatarget02020-1[2020-06-03,2020-11-30)275.5True120201[2020-11-30,2021-05-29)95.5False22021-1[2021-06-03,2021-11-30)640.5True320211[2021-11-30,2022-05-29)460.5FalseDepending on data preparations, we can choose different types of calendars. For more information, seeLilio's documentation.Cross-validationLilio can also generate train/test splits and perform cross-validation. To do that, a splitter is called fromsklearn.model_selectione.g.ShuffleSplitand used to split the resampled data:fromsklearn.model_selectionimportShuffleSplitsplitter=ShuffleSplit(n_splits=3)lilio.traintest.split_groups(splitter,bins)All splitter classes fromscikit-learnare supported, a list is availablehere. Users should followscikit-learndocumentation on how to use a different splitter class.Dimensionality reductionWiths2spy, we can perform dimensionality reduction on data. For instance, to perform theResponse Guided Dimensionality Reduction (RGDR), we configure the RGDR operator and fit it to a precursor field. Then, this cluster can be used to transform the data into the reduced clusters:rgdr=RGDR(eps_km=600,alpha=0.05,min_area_km2=3000**2)rgdr.fit(precursor_field,target_timeseries)clustered_data=rgdr.transform(precursor_field)_=rgdr.plot_clusters(precursor_field,target_timeseries,lag=1)(for more information aboutprecursor_fieldandtarget_timeseries, check the complete example inthis notebook.)Currently,s2spysupportsdimensionality reduction approachesfromscikit-learn.Tutorialss2spysupports operations that are common in a machine learning pipeline of sub-seasonal to seasonal forecasting research. Tutorials covering supported methods and functionalities are listed innotebooks. To check these notebooks, users need to installJupyter lab. More details about each method can be found in thisAPI reference documentation.Advanced usecasesYou can achieve more by integratings2spyandliliointo your data-driven S2S forecast workflow! We have a magiccookbook, which includes recipes for complex machine learning based forecasting usecases. These examples will show you hows2spyandliliocan facilitate your workflow.DocumentationFor detailed information on usings2spypackage, visit thedocumentation pagehosted at Readthedocs.ContributingIf you want to contribute to the development of s2spy,
have a look at thecontribution guidelines.How to cite usPlease use the Zenodo DOI to cite this package if you used it in your research.AcknowledgementsThis package was developed by the Netherlands eScience Center and Vrije Universiteit Amsterdam. Development was supported by the Netherlands eScience Center under grant number NLESC.OEC.2021.005.This package was created withCookiecutterand theNLeSC/python-template.