Adaptive Bias Correction for Improved Subseasonal Forecasting
Abstract
Subseasonal forecasting—predicting temperature and precipitation 2 to 6 weeksahead—is critical for effective water allocation, wildfire management, and droughtand flood mitigation.
Recent international research efforts have advanced the subseasonal capabilities of operational dynamical models, yet temperature and precipitation prediction skills remains poor, partly due to stubborn errors in representingatmospheric dynamics and physics inside dynamical models.
To counter theseerrors, we introduce an adaptive bias correction (ABC) method that combines stateof-the-art dynamical forecasts with observations using machine learning.
Whenapplied to the leading subseasonal model from the European Centre for MediumRange Weather Forecasts (ECMWF), ABC improves temperature forecasting skillby 60-90% and precipitation forecasting skill by 40-69% in the contiguous U.S.
Wecouple these performance improvements with a practical workflow, based on Cohort Shapley, for explaining ABC skill gains and identifying higher-skill windowsof opportunity based on specific climate conditions.
Introduction
Water and fire managers rely on subseasonal forecasts 2-6 weeks in advance to allocate water, managewildfires and prepare for droughts and other weather extremes.
However, skillful forecasts for thesubseasonal regime are lacking due to the complex dependence on both local weather and globalclimate variables and the chaotic nature of weather.
Bridging the gap between short-term and seasonalforecasting has been the focus of several recent large-scale research efforts which have advancedthe subseasonal capabilities of operational physics-based models [1, 2, 3].
However, despite theseadvances, dynamical models still suffer from persistent systematic errors which limit the skill oftemperature and precipitation forecasts for longer lead times from 2 to 6 weeks ahead.
To overcome observed systematic errors of physics-based models on the subseasonal timescale, therehave been parallel efforts in recent years to demonstrate the value of machine learning and deeplearning methods in improving subseasonal forecasting [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14].
Whilethese works demonstrate the promise of statistical models for subseasonal forecasting, they alsohighlight the complementary strengths of physics- and learning-based approaches and the opportunityto combine those strengths to improve forecasting skill [6, 11].
To harness those complementary strengths, we introduce a hybrid dynamical-learning framework forimproved subseasonal forecasting.
In particular, we learn to adaptively correct the biases of dynamicalmodels and apply our novel adaptive bias correction (ABC) to improve the skill of subseasonaltemperature and precipitation forecasts.
ABC can be applied operationally as a computationallyinexpensive enhancement to any dynamical model forecast, and we use this property to substantiallyreduce the forecasting errors of eight operational dynamical models, including the state-of-the-art ECMWF model.
We couple these performance improvements with a practical workflow for explaining ABC skill gains using Cohort Shapley [15] and identifying higher-skill windows of opportunity [16] based on relevant climate variables.
To facilitate future deployment and benchmarking, we releaseour model and workflow code through the subseasonal_toolkit Python package.
Methods
We consider two prediction targets: average temperature ( ◦ C) and accumulated precipitation (mm)over a two-week period.
These variables are forecasted at two time horizons: 15-28 days ahead(weeks 3-4) and 29-42 days ahead (weeks 5-6).
We forecast each variable at 𝐺 = 376 grid pointson a 1 . 5 ◦ × 1 . 5 ◦ grid across the contiguous U.S., bounded by latitudes 25N to 50N and longitudes125W to 67W. To provide the most realistic assessment of forecasting skill [17], all predictionsin this study are formed in a real forecast manner that mimics operational use.
In particular, toproduce a forecast for a given target date, all learning-based models are trained and tuned only ondata observable on the corresponding forecast issuance date.
We evaluate each forecast accordingusing uncentered anomaly correlation skill.
For a collection of target dates, we report average skillusing progressive validation [18] to mimic operational use.
All data used in this work was obtainedfrom the SubseasonalClimateUSA dataset [19].
2.1 Operational ECMWF and CFSv2 debiasing
We bias correct a uniformly weighted ensemble of the ECMWF control forecast and its 50 ensembleforecasts following the ECMWF operational protocol [20]: for each target forecast date, we biascorrect the ECMWF 51-member ensemble using the last 20 years of reforecasts with dates within ± 6days from the target month-day combination.
The average of ensemble reforecasts on the 1 . 5 ◦ × 1 . 5 ◦degree grid are used for debiasing.
Following [21], we bias correct the 32-member CFSv2 ensemble forecast in the following way:for each target forecast date, we bias correct the CFSv2 control and ensemble forecasts using thetwelve-year period from 1999 to 2010 of reforecasts.
The average of the ensemble reforecasts on the1 .
5 ◦ × 1 . 5 ◦ degree grid are used for debiasing.
2.2 Adaptive bias correction
ABC is a uniformly-weighted ensemble of three machine learning models that we introduce in thiswork and describe below in details: Climatology ++ , Dynamical ++ , and Persistence ++ .
Supplementary algorithm descriptions can be found in Appendix B.
After averaging dynamical forecasts over a range of issuance dates and lead times, Dynamical ++debiases the ensemble forecast for each grid cell by adding the mean value of the target variableand subtracting the mean forecast over a learned window of observations around the target day ofyear.
For a given target date 𝑡 ★ and lead time 𝑙 ★ , the Dynamical ++ training set T is restricted todata fully observable one day prior to the issuance date, that is, to dates 𝑡 ≤ 𝑡 ★ − 𝑙 ★ − 𝐿 − 1 where𝐿 = 14 represents the forecast period length.
Unlike standard debiasing strategies, which employstatic ensembling and bias correction, Dynamical ++ adaptively selects the range of ensembled leadtimes L , the number of averaged issuance dates 𝑑 ★ , and the size 𝑠 of the observation window usingan automated tuning procedure.
For each target date, Dynamical ++ is run with the hyperparameter configuration that achievedthe smallest mean progressive geographic root mean squared error (RMSE) over the preceding3 years.
Here, progressive indicates that each candidate model forecast is generated using alltraining data observable prior to the associated forecast issuance date.
Every configuration with𝑠 ∈ { 0 , 14 , 28 , 35 } , 𝑑 ★ ∈ { 1 , 7 , 14 , 28 , 42 } , and L = { 29 } for the weeks 5-6 lead time and L ∈{{ 15 } , [ 15 , 22 ] , [ 0 , 29 ] , { 29 }} the weeks 3-4 lead time was considered.
Climatology ++ is a locally constant prediction rule that minimizes historical forecasting error,specified by a user-supplied loss function, over all days in a window around the target day of year.
For a given target date 𝑡 ★ and lead time 𝑙 ★ , the Climatology ++ training set T is restricted to data fully observable one day prior to the issuance date, that is, to dates 𝑡 ≤ 𝑡 ★ − 𝑙 ★ − 𝐿 − 1 where 𝐿 = 14represents
the forecast period length.
The number of training years 𝑌 and the size of the observationwindow (quantified by the span , the number of days 𝑠 included on each side of the target day of year)are determined adaptively using an automated tuning procedure, described below.
For each target date, Climatology ++ is run with the hyperparameter configuration that achieved thesmallest mean progressive geographic RMSE over the preceding 3 years.
All spans 𝑠 ∈ { 0 , 1 , 7 , 10 }were considered.
All precipitation configurations used the geographic MSE loss and all availabletraining years.
All temperature configurations used the geographic RMSE loss and either all availabletraining years or 𝑌 = 29 .
Persistence ++ fits a least-squares regression per grid point to optimally combine lagged temperatureor precipitation measurements, climatology, and a dynamical ensemble forecast.
For a given targetdate 𝑡 ★ and lead time 𝑙 ★ , the Persistence ++ training set T is restricted to data fully observable oneday prior to the issuance date, that is, to dates 𝑡 ≤ 𝑡 ★ − 𝑙 ★ − 𝐿 − 1 where 𝐿 = 14 represents the forecastperiod length.
Results
[Figure]
Figure 1 highlights the advantage of ABC over raw dynamical models when forecasting accumulatedprecipitation and averaged temperature in the contiguous U.S. Here, ABC is applied to the leadingsubseasonal model, ECMWF, and to each of seven operational models participating in the Subseasonal
Experiment [SubX, 2].
Subseasonal forecasting skill, measured by uncentered anomaly correlation,is evaluated at two forecast horizons, weeks 3-4 and weeks 5-6, and averaged over all availableforecast dates in the four-year span 2018–2021.
We find that, for each dynamical model input andforecasting task, ABC leads to a pronounced improvement in skill.
For example, when appliedto the U.S. operational model CFSv2, ABC improves temperature forecasting skill by 109-289%and precipitation skill by 165-253%.
When applied to the leading ECMWF model, ABC improvestemperature skill by 60-90% and precipitation skill by 40-69%.
Moreover, for precipitation, evenlower-skill models like CCSM4 enjoy skill comparable to the best after the application of ABC.
Overall and despite significant variability in dynamical model skill, ABC consistently reduces thesystematic errors of its input model, bringing forecasts closer to observations for each target variableand time horizon.
We next examine, in Figure 2, the spatial distribution of skill for CFSv2, ECMWF, and their ABCcorrected counterparts at three forecast horizons.
At the shorter-term horizon of weeks 1-2, both
CFSv2 and ECMWF enjoy reasonably high skill throughout the contiguous U.S.
However, skill dropsprecipitously for both models when moving to the subseasonal horizons (weeks 3-4 and 5-6).
Thisdegradation is particularly striking for precipitation, where prediction skill drops to zero or belowin the central and northeastern parts of the U.S. For temperature prediction, CFSv2 has a skill ofzero across a broad region of the East, while ECMWF produces isolated pockets of zero skill in the
West.
At these subseasonal timescales, ABC provides consistent improvements across the U.S. thateither double or triple the mean skill of CFSv2 and increase the mean skill of ECMWF by 40-90%.
[Figure]
Taking the same raw model forecasts as input, ABC also provides consistent improvements over operational debiasing protocols, tripling the precipitation skill of debiased CFSv2 and improving that of debiased ECMWF by 70%.
The average skill over all sites is displayed above each map.
Notably, ABC also improves over standard operational debiasing protocols (labeled debiased CFSv2and debiased ECMWF in Figure 2), tripling the average precipitation skill of debiased CFSv2 andincreasing that of debiased ECMWF by 70%.
An important component contributing to the overall accuracy of ABC is a reduction of the systematicbias introduced by its dynamical model input.
Figure 3 examines the spatial distribution of this biasby plotting the average difference between forecasts and observations over all forecast dates.
Theprecipitation maps reveal a wet bias over the northern half of the U.S. for CFSv2 (average bias: 8.32mm) and a dry bias over the south-east part of the U.S. for ECMWF (average bias: − 8.12 mm).
Inthis case, ABC eliminates the CFSv2 wet bias (average bias: − 0.46 mm) and slightly alleviates the
ECMWF dry bias (average bias: − 6.24 mm).
For temperature, we observe a cold bias over the easternhalf of the U.S. for CFSv2 (average bias: − 1.2°C) and notice a mixed pattern of cold and warm biasesover the western half of the U.S for ECMWF (average bias: − 0.30°C).
In this case, although ABCdoes not eliminate these biases entirely, it reduces the magnitude of the cold eastern bias by bringing
CFSv2 forecasts closer to observations (average bias: − 0 .
18 C)
and reduces the mixed ECMWF bias(average bias: − 0 .
04 C).
[Figure]
The results presented so far assess overall model skill, averaged across all forecast dates.
However,there is a growing appreciation that subseasonal forecasts can also benefit from selective deploymentduring “windows of opportunity,” periods defined by observable climate conditions in which specificforecasters are likely to have higher skill [16].
Here, we propose a practical opportunistic ABCworkflow that uses a candidate set of explanatory variables to identify windows in which ABCis especially likely to improve upon a baseline model.
This workflow is described in details in
Appendix A.
The same workflow can be used to explain the skill improvements achieved by ABC interms of the explanatory variables.
The opportunistic ABC workflow is based on the optimal credit assignment principle [22] andmeasures the impact of explanatory variables on individual forecasts using Cohort Shapley [15] andoverall variable importance using Shapley effects [23].
We use these Shapley measures to interpretthe contexts in which ABC offers improvements in terms of climate variables with known relevancefor subseasonal forecasting skill.
As a running example, we use our workflow to explain the skilldifferences between ABC-ECMWF and debiased ECMWF when predicting precipitation in weeks3-4.
As our candidate explanatory variables we use Northern Hemisphere geopotential heights (HGT)at 500 and 10 hPa, the phase of the Madden-Julian Oscillation (MJO), Northern Hemisphere seaice concentration (ICEC), global sea surface temperatures (SST), the multivariate El Niño-Southern
Oscillation index [MEI.v2, 24], and the target month.
All variables are lagged appropriately to ensurethat they are observable on the forecast issuance date.
We first use Shapley effects to determine the overall importance of each variable in explainingthe precipitation skill improvements of ABC-ECMWF.
We find the most important explanatoryvariables to be the first two principal components (PCs) of 500 hPa geopotential height, the MJOphase, the second PC of 10 hPa geopotential height, and the first PC of sea ice concentration.
These
[Figure]
a
[Figure]
P a g e o p o t e n t i
a l h e i g h t a n
[Figure]
li
[Figure]
variables are consistent with the literature exploring the dominant contributions to subseasonalprecipitation [25, 26, 27, 28].
We next use Cohort Shapley to identify the contexts in which each variable has the greatest impacton skill.
For example, Figure 4 summarizes the impact of the first 500 hPa geopotential heights PC( hgt_500_pc1 ) on ABC-ECMWF skill improvement.
This display divides our forecasts into 10bins, determined by the deciles of hgt_500_pc1 , and computes the probability of positive impactin each bin.
We find that hgt_500_pc1 is most likely to have a positive impact impact on skillimprovement in decile 1, which features a positive Arctic Oscillation (AO) pattern, and least likely indecile 9, which features AO in the opposite phase.
The ABC-ECMWF forecast most impacted byhgt_500_pc1 in decile 1 is also preceded by a positive AO pattern and replaces the wet debiased
ECMWF forecast with a more skillful dry pattern in the west.
Finally, we use the identified contexts to define windows of opportunity for operational deployment.
Indeed, since all explanatory variables are observable on the forecast issuance date, one can selectivelyapply ABC when multiple variables are likely to have a positive impact on skill and otherwise issuea default, standard forecast (e.g., debiased ECMWF).
We call this selective forecasting modelopportunistic ABC .
How many high-impact variables should we require when defining these windowsof opportunity?
Requiring a larger number of high-impact variables will tend to increase the skillgains of ABC but simultaneously reduce the number of dates on which ABC is deployed.
Figure 5illustrates this trade-off for ABC-ECMWF and shows that opportunistic ABC skill is maximized when
[Figure] [Figure]
[Figure]
two or more high-impact variables are required.
With this choice, ABC is used for approximately81% of forecasts and debiased ECMWF is used for the remainder.
Discussion
Dynamical models have shown increasing skill in accurately forecasting the weather [29], but theystill contain systematic biases that compound on subseasonal time scales and suppress forecastskill [30, 31, 32, 33].
Our proposed solution, ABC, learns to correct these biases by adaptivelyintegrating dynamical forecasts, historical observations, and recent weather trends.
When appliedto the leading subseasonal model from ECMWF, ABC improves forecast skill by 60-90% forprecipitation and 40-69% for temperature.
The same approach substantially reduces the forecastingerrors of seven additional operational models, with less skillful input models performing nearly aswell as ECMWF after correction.
This finding suggests that systematic errors in dynamical modelsare a primary contributor to observed skill differences and that ABC provides an effective mechanismfor reducing these heterogeneous errors.
Because ABC is also simple to implement and deploy inreal-time operational settings, adaptive bias correction represents a computationally inexpensivestrategy for upgrading operational models, while conserving valuable human resources.
Moreover, ABC is, by its nature, adaptive to changes in systematic biases.
As operational models areupgraded and systematic biases evolve, our ABC training protocol is designed to ingest the upgradedmodel forecasts and hindcasts reflecting those changes.
In addition, the same protocol can be adaptedto correct probabilistic subseasonal forecasts that estimate the distribution of future weather [34].
To capitalize on higher-skill forecasts of opportunity, we have also introduced an opportunistic ABCworkflow that explains the skill improvements of ABC in terms of a candidate set of environmentalvariables, identifies high-probability windows of opportunity based on those variables, and selectivelydeploys either ABC or a baseline forecast to maximize expected skill.
The same workflow canbe applied to explain the skill improvements of any forecasting model and, unlike other popularexplanation tools [e.g., 35, 36], requires no expensive model retraining, no generation of additionalforecasts beyond those routinely generated for operational or hindcast use, and allows for explanationsin terms of variables that were not explicitly used in training the model.
Overall, we find that correcting dynamical forecasts using ABC yields an effective and scalablestrategy for building the next generation of subseasonal forecasting models.
We anticipate that ourhybrid dynamical-learning framework will benefit both research and operations, and we release ouropen-source code to facilitate future adoption and development.
[Figure]
A Supplementary Method Details
Here we detail the steps of the opportunistic ABC workflow illustrated in Figure 6 using ECMWF as an example 307 dynamical input.
The same workflow applies to any other dynamical input.
Identify a set of 𝑉 candidate explanatory variables.
Here we use the temporal variables enumerated in [6,
Fig. 2] augmented with the first two PCs of 500 hPa geopotential heights and the target month.
To ensurethat the workflow can be deployed operationally, we use lagged observations with lags chosen so that eachvariable is observable on the forecast issuance date.2.
Compute the skill difference between ABC-ECMWF and debiased ECMWF for each target date in theevaluation period.3.
Use the cohortshapley Python package to compute global variable importances (measured by Shapleyeffects) and forecast-specific variable impact values explaining the skill differences.4.
For each continuous explanatory variable (e.g., hgt_500_pc2), divide the evaluation period forecasts into 10bins, determined by the deciles of the explanatory variable.
For each categorical variable (e.g., mjo_phase),divide the forecasts into bins determined by the categories (e.g., MJO phases).5.
Estimate the probability of positive variable impact in each bin and compute a 95% bootstrap confidenceinterval.
Flag all bin probabilities within the confidence interval of the highest probability bin as high impactand all bin probabilities within the confidence interval of the lowest probability bin as low impact.
Visualizeand interpret the highest and lowest impact bins.6.
Identify the forecast most impacted by the explanatory variable in the high impact bins.
Visualize theABC-ECMWF and debiased ECMWF forecasts and the associated explanatory variable for that target date.7.
For each 𝑘 ∈ { 0 , . . .
,𝑉 } , compute opportunistic ABC skill when 𝑘 or more explanatory variables fall intohigh impact bins.
Let 𝑘 ★ represent the integer at which opportunistic ABC skill is maximized.8.
At each future forecast issuance date, deploy ABC-ECMWF if 𝑘 ★ or more explanatory variables fall intohigh impact bins and deploy debiased ECMWF otherwise.
B Supplementary Model Details
This section presents the algorithm details for the three machine learning models underlying ABC: Dynamical ++ , 331 Climatology ++ , and Persistence ++ .
input test date 𝑡 ★ ; lead time 𝑙 ★ ; # issuance dates 𝑑 ★ ; span 𝑠 ; training set ground truth and dynamical forecasts ( y 𝑡 , f 𝑡,𝑙 ) 𝑡 ∈T ,𝑙 ∈L initialize days per year 𝐷 = 365 . 242199 ; # training years 𝑌 =
S = { 𝑡 ∈ T : year_diff ≔ ⌊ 𝑡 ★ − 𝑡 𝐷 ⌋ ≤ 𝑌 and day_diff ≔ 3652 −|⌊
( 𝑡 ★ − 𝑡 ) mod 𝐷 ⌋− 3652 | ≤ 𝑠 } //
Form dynamical ensemble forecast across issuance dates and lead times 𝑙 ∈ L for training and test dates 𝑡 ∈ S ∪ { 𝑡 ★ } do ¯ f 𝑡 = mean (( f 𝑡 − 𝑙 ★ − 𝑑 + 1 ,𝑙 ) 1 ≤ 𝑑 ≤ 𝑑 ★ ,𝑙 ∈L ) output ¯ f 𝑡 ★ + mean (( y 𝑡 − ¯ f 𝑡 ) 𝑡 ∈S ) input test date 𝑡 ★ ; # training years 𝑌 ; span 𝑠 ; loss ∈ { RMSE , MSE } ; training set ground truth ( y 𝑡 ) 𝑡 ∈T initialize days per year 𝐷 = 365 .
S = { 𝑡 ∈ T : year_diff ≔ ⌊ 𝑡 ★ − 𝑡 𝐷 ⌋ ≤ 𝑌 and day_diff ≔ 3652 −|⌊
( 𝑡 ★ − 𝑡 ) mod 𝐷 ⌋− 3652 | ≤ 𝑠 } output argmin y (cid:205)
𝑡 ∈S loss ( y , y 𝑡 ) input lead time 𝑙 ★ ; training set ground truth, climatology, and dynamical forecasts ( y 𝑡 , c 𝑡 , f 𝑡,𝑙 ) 𝑡 ∈T ,𝑙 ∈L initialize forecast period length 𝐿
=//
Form dynamical ensemble forecast across subseasonal lead times 𝑙 ≥ 𝑙 ★ for training dates 𝑡 ∈ T do ¯ f 𝑡 = mean (( f 𝑡,𝑙 ) 𝑙 ≥ 𝑙 ★ ) //
Combine ensemble forecast, climatology, and lagged measurements for grid points 𝑔 = 1 to 𝐺 do ˆ 𝜷 𝑔 ∈ argmin
[Table]