SlowGuess/NeurIPS_2023_OCR huggingface dataset:
N-BEATS. N-BEATS [48] is a neural network time series forecaster that obtained state-of-the-art results on the Makridakis M3 [33] and M4 [34] benchmarks for time-series forecasting.
PROPHET. The Prophet model of [62] is an additive regression model for time-series and a winning solution in the Subseasonal Forecast Rodeo II [46].
SALIENT 2.0. Salient 2.0 is an ensemble of fully-connected neural networks, trained on historical sea surface temperature (SST) data. It is based on Salient [59], a winning solution for the Subseasonal Forecast Rodeo I [46].
Ensembles Ensemble forecasts that combine the predictions of multiple models have been shown to improve the performance of long-, mid-, and short-range operational forecasting [10, 49, 23]. Here, we evaluate two ensembling strategies: Uniform ABC, which forms an equal-weighted average of the ABC model forecasts [27], and Online ABC, which uses the AdaHedgeD algorithm of [15] to choose weights adaptively to reflect relative model performance. See Appendices B.11 and B.12 for details.
We now turn to evaluating the models of Section 4 on the four subseasonal forecasting tasks of Section 3. We generate forecasts for each Wednesday in the years 2011-2020 and, for each reported period, we assess both mean RMSE relative to a baseline model and average percentage skill.
Overall Performance Table 1 summarizes model performance across the entire ten-year period 2011-2020. On each task, we find that the ABC models provide both the best RMSE and the best skill performance. For example, on the two precipitation tasks, Climatology++ alone improves upon debiased CFSv2 RMSE by  $9\%$  and skill by  $161 - 250\%$ , outperforming each of the meteorological baselines and state-of-the-art learning methods. On the two temperature tasks, CFSv2++ and Persistence++ each outperform all meteorological baselines and state-of-the-art learning methods, with CFSv2++ improving debiased CFSv2 RMSE by  $6 - 7\%$  and skill by  $30 - 53\%$ . On every task, we observe further improvements in both RMSE and skill by ensembling the predictions of the three ABC models.
One might wonder how the simple ABC models are able to outperform both the standard meteorological baselines and the state-of-the-art learning methods. We believe the answer to this question is multifaceted. First, even the leading physics-based dynamical models are subject to inaccuracies due to inexact measurement, incomplete representation of the environment, imperfect simulation, and chaos [32]. Second, many of the more elaborate machine learning models studied in this work appear to be prone to overfitting in the presence of the relatively high noise levels of subseasonal forecasting. Third, the best-performing models, while relatively simple, are hybrid physics-plus-learning models designed to leverage the strengths of an underlying dynamical model while simultaneously enhancing its predictive skill by reducing its systemic bias.
Amongst the state-of-the-art learning methods, we find that Prophet performs the best for temperature weeks 5-6 and the two precipitation tasks, while MultiLLR performs the best for temperature weeks 3-4. Amongst the neural network methods (Informer, N-BEATS, and Salient 2.0), Salient 2.0 is the top performer with skill that rivals the other learning methods and precipitation RMSE that outpaces debiased CFSv2. In the more detailed analyses to follow, we omit Informer and N-BEATS due to space constraints and their relatively poor performance overall.
Performance by Season and by Year We observe the same trends when performance is disaggregated by season or by year (Figure 3). For example, in every season, Climatology++ outperforms debiased CFSv2 and each state-of-the-art learner for the two precipitation tasks, while CFSv2++ and Persistence++ outperform debiased CFSv2 and each state-of-the-art learner each season for the two temperature tasks. Similarly and despite significant heterogeneity in all models' performances from year to year, the ABC models provide the best RMSE performance in 9 out of 10 years for temperature weeks 3-4 and in 10 out of 10 years for the two precipitation tasks. Indeed, Persistence++ alone dominates the temperature weeks 3-4 baselines and learners every year save 2019, and the more detailed RMSE summary of Appendix C.2 shows that the Uniform and Online ABC ensembles dominate the precipitation baselines and learners every year. In Figure 3, we observe nearly identical improvement patterns for skill.
Table 1: Average percentage skill and percentage improvement over mean debiased CFSv2 RMSE across 2011-2020 in the contiguous U.S. along with a  $95\%$  bootstrap confidence interval. The best performing model in each model group is bolded, and the best performing model overall is shown in green.
GROUP	MODEL	% IMPROVEMENT OVER MEAN DEB. CFSV2 RMSE	AVERAGE % SKILL
TEMPERATURE	PRECIPITATION	TEMPERATURE	PRECIPITATION
WEEKS 3-4	WEEKS 5-6	WEEKS 3-4	WEEKS 5-6	WEEKS 3-4	WEEKS 5-6	WEEKS 3-4	WEEKS 5-6
BASELINES	CLIMATOLOGY	0.13±1.33	2.93±1.23	7.79±0.55	7.51±0.48	-	-	-	-	-
DEB. CFSV2	-	-	-	-	24.94±1.88	19.12±1.94	5.77±1.11	4.28±1.09
PERSISTENCE	-109.94±5.3	-170.1±7.56	-28.27±1.47	-31.92±1.53	10.64±2.26	6.22±2.38	8.31±0.99	7.41±1.01
ABC	CLIMATOLOGY++	2.06±1.35	4.83±1.18	8.86±0.53	8.57±0.5	18.61±1.95	18.87±1.92	15.04±1.02	14.99±1.03
CFSV2++	5.94±1.08	7.09±1.02	8.37±0.5	8.06±0.45	32.38±1.75	29.19±1.76	16.34±1.03	16.09±1.07
PERSISTENCE++	6.00±1.06	6.43±0.99	8.61±0.51	7.89±0.45	32.4±1.71	26.73±1.67	13.38±0.91	9.77±0.9
LEARNING	AUTOKNN	0.93±1.33	3.22±1.25	7.73±0.56	7.33±0.49	12.43±1.67	8.56±1.52	6.66±1.23	5.93±1.33
INFORMER	-40.61±4.4	-39.57±3.89	-2.05±0.86	-2.53±0.83	0.55±2.22	0.01±2.20	6.15±1.28	5.86±1.31
LOCALBOOSTING	-0.76±1.24	-0.29±1.24	7.36±0.58	6.89±0.5	14.44±1.59	12.69±1.64	10.82±0.87	9.72±0.86
MULTILLR	2.45±1.18	2.21±1.24	7.12±0.51	6.65±0.47	24.5±1.77	16.68±1.85	9.49±1.03	7.97±1.04
N-BEATS	-46.71±2.48	-52.05±2.89	-19.19±0.92	-21.32±0.89	9.21±1.40	4.16±1.39	5.48±0.59	4.46±0.62
PROPHET	1.13±1.4	3.78±1.26	8.42±0.55	8.12±0.51	20.21±1.54	19.78±1.57	13.51±0.87	13.41±0.89
SALIENT 2.0	-6.95±1.69	-4.05±1.73	2.97±0.74	2.65±0.69	11.24±2.04	11.77±2.03	10.11±1.36	9.99±1.31
ENSEMBLES	UNIFORM ABC	6.47±1.09	7.55±0.99	9.47±0.5	9.05±0.45	33.58±1.8	30.56±1.7	18.94±0.98	18.35±1.01
ONLINE ABC	6.67±0.99	7.67±0.98	9.51±0.51	9.04±0.42	33.27±1.71	30.06±1.72	18.86±1.01	17.91±1.01
Spatial Performance Figure 4 displays how the errors of the leading models are distributed across the contiguous U.S. Here we focus on the ABC models, the best deep learning model (Salient 2.0), the best learning model (Prophet), and the best ensemble model (Online ABC) and provide RMSE improvement maps for the remaining models in Appendix C.4. At each grid point location, darker green indicates stronger improvement over debiased CFSv2, and we simultaneously witness two noteworthy phenomena. First, the improvements of each model are heterogeneous across space with the strongest improvements often occurring in the Western U.S., in Florida, or in Maine. Second, despite this heterogeneity, the ABC models consistently outperform the state-of-the-art learners.
ECMWF Comparison To compare the ABC models with the state-of-the-art ECMWF S2S dynamical model, we evaluate on the  $1.5^{\circ} \times 1.5^{\circ}$  grid and 2016-2020 twice-weekly target date range available from [12, 66]. We debias both the ECMWF control forecast and its 50-member ensemble forecast following the operational protocol described by [69]; see Appendix B.13 for more details. Table 2 summarizes model performance. Remarkably, for precipitation, Climatology++ improves upon both the skill and the RMSE of ECMWF, despite making no use of dynamical model forecasts. Meanwhile, the Uniform ABC ensemble outperforms ECMWF in both metrics for all four tasks.
Table 2: Average percentage skill and percentage improvement over mean debiased CFSv2 RMSE across 2016-2020 in the contiguous U.S. along with a  $95\%$  bootstrap confidence interval. The best performing model in each model group is bolded, and the best overall is shown in green.
GROUP	MODEL	% IMPROVEMENT OVER MEAN DEB. CFSV2 RMSE	AVERAGE % SKILL
TEMPERATURE	PRECIPITATION	TEMPERATURE	PRECIPITATION
WEEKS 3-4	WEEKS 5-6	WEEKS 3-4	WEEKS 5-6	WEEKS 3-4	WEEKS 5-6	WEEKS 3-4	WEEKS 5-6
BASELINES	CLIMATOLOGY	1.56±1.39	3.92±1.27	8.7±0.5	7.56±0.53	-	-	-	-
DEB. CFSV2	-	-	-	-	22.64±2.01	15.71±2.09	2.84±1.16	1.68±1.11
PERSISTENCE	-105.57±5.58	-169.22±7.39	-28.05±1.56	-33.43±1.65	9.12±2.48	2.27±2.48	8.11±1.07	6.21±1.04
ABC	CLIMATOLOGY++	3.88±1.38	6.44±1.13	9.79±0.53	8.61±0.51	22.09±1.89	23.2±1.91	15.34±1.05	15.06±1.07
CFSV2++	5.65±1.04	6.65±0.96	8.94±0.51	7.6±0.46	30.91±1.8	26.87±1.93	14.6±1.23	13.85±1.2
PERSISTENCE++	7.06±1.05	7.86±0.95	9.06±0.48	7.57±0.46	31.46±1.92	28.04±1.87	10.03±0.99	6.61±0.95
ECMWF	DEBIASED CONTROL	-29.05±2.39	-33.25±2.69	-30.81±1.45	-31.84±1.43	18.52±1.82	13.71±1.85	0.82±1.07	3.17±1.08
DEBIASED ENSSEMBLE	4.62±1.19	3.69±1.27	7.90±0.5	6.41±0.46	32.27±1.69	26.61±1.71	13.12±1.16	9.10±1.09
ENSEMBLES	UNIFORM ABC	7.43±1.05	8.27±0.94	10.04±0.5	8.77±0.46	32.77±1.79	29.75±1.87	16.53±1.15	15.71±1.17
ONLINE ABC	7.2±1.07	7.96±0.98	10.08±0.48	8.62±0.47	32.22±1.82	28.38±1.87	17.19±1.12	15.42±1.15
Graphcast Comparison Recently, the GraphCast deep learning model [28] was shown to outperform both the leading dynamical model and the Pangu-Weather deep learning model [5] on a range of 0 to 10-day weather forecasting tasks. While GraphCast was not developed for subseasonal forecasting, we can, as recommended by an anonymous reviewer, benchmark its performance on our subseasonal forecasting tasks. For this evaluation, we restrict our standard test set to the years 2018–2020, as GraphCast was trained on data through 2017, and report performance and additional experimental details in Appendix C.6. Consistent with the other deep learning methods benchmarked in Table 1, GraphCast outperforms debiased CFSv2 in terms of skill, underperforms debiased CFSv2 in terms of RMSE, and strongly underperforms the ABC ensemble models in both metrics when forecasting either temperature or precipitation in weeks 3-4.
RMSE improvement by season
RMSE improvement by year
Skill by season
Skill by year
Figure 3: Per season and per year average skill and improvement over mean debiased CFSv2 RMSE across the contiguous U.S. and the years 2011-2020. Despite their simplicity, the ABC models (solid lines) consistently outperform debiased CFSv2 and the state-of-the-art learners (dotted lines).
Western U.S. Competition Results Finally, we evaluate our models on the exact geographic region and target dates of the recent Subseasonal Climate Forecast Rodeo II competition. Specifically, we produce forecasts for the Western U.S. region, delimited by latitudes 25N to 50N and longitudes 125W to 93W, at a  $1^{\circ} \times 1^{\circ}$  resolution for a total of  $G = 514$  grid points. Forecasts were issued every two weeks for a yearlong period with initial issuance date October 29, 2019 and final issuance date October 27, 2020, leading to a noisier evaluation with only 26 observations.
Table 8 in Appendix C.7 compares the predictive accuracy of the models studied in this work with the accuracy of the contest baselines (debiased CFSv2, Climatology, and, for precipitation only, the Rodeo I Salient model of [59]) and the performance of the top competitors for each task. For temperature weeks 3-4, Persistence++ provides a  $16.59\%$  improvement over the mean debiased CFSv2 RMSE, outperforming the contest baselines, the state-of-the-art learning methods, and all but two of the competitors (the top three competitors improved by  $17.12\%$ ,  $16.67\%$ , and  $15.47\%$ ). For temperature weeks 5-6, CFSv2++ provides a  $9.26\%$  improvement over debiased CFSv2 and, despite its simplicity, outperforms the contest baselines, the state-of-the-art learning methods, and all of the
Figure 4: Percentage improvement over mean debiased CFSv2 RMSE in the contiguous U.S. over 2011-2020. White grid points indicate negative or  $0\%$  improvement.
competitors in the subseasonal forecasting competition (the top competitor improved by  $8.47\%$ ). On this task, the Uniform and Online ABC ensembles also outperform all competitors.
On the precipitation tasks, the Salient baseline performed strongly and ultimately placed second and fourth respectively for the weeks 3-4 and weeks 5-6 tasks. Our Salient 2.0 model also performs remarkably well, outscoring all contestants and baselines with  $12.65\%$  improvement for weeks 3-4. For comparison, the top competitors for weeks 3-4 and weeks 5-6 improved by  $11.54\%$  and  $8.63\%$  respectively. Our Uniform ABC ensemble outperforms the remaining baselines and state-of-the-art learning methods but falls short of the exceptional Salient performance. In this setting, applying the adaptive online learning ensemble to the union of the ABC models and the state-of-the-art learners (denoted by Online  $\mathrm{ABC} +$  Learning in Table 8) allows the user to exploit the irregular complementary benefits of the learning methods yielding  $12.52\%$  and  $8.18\%$  improvements in weeks 3-4 and 5-6.
In this work, we release SubseasonalClimateUSA, a dataset for subseasonal forecasting in the U.S. It is routinely updated and can be accessed as a Python package. The dataset includes a variety of features that are relevant at the subseasonal timescale, including precipitation, temperature, surface pressure, relative humidity, geopotential height, sea surface temperature, sea ice concentration, MJO, and MEI. We use this dataset to train and benchmark multifarious models, including deep learning solutions, dynamical models, and simple learned corrections, as well ensembling strategies. Our experiments with temperature and precipitation forecasting in the contiguous U.S. show that simple learning-based corrections to operational dynamical models yield low-cost strategies that are  $10\%$  more accurate and  $329\%$  more skillful than the U.S. operational CFSv2 and outperform state-of-the-art machine and deep learning methods, as well as the leading ECMWF dynamical model. Overall, we find that the SubseasonalClimateUSA dataset facilitates both the training and benchmarking of subseasonal forecasting models and hope that it will stimulate new advances in extended range forecasting.
We acknowledge the agencies that support the SubX system, and we thank the climate modeling groups (Environment Canada, NASA, NOAA/NCEP, NRL and University of Miami) for producing and making available their model output. NOAA/MAPP, ONR, NASA, NOAA/NWS jointly provided coordinating support and led development of the SubX system. This work is based on S2S data. S2S is a joint initiative of the World Weather Research Programme (WwRP) and the World Climate Research Programme (WCRP). The original S2S database is hosted at ECMWF as an extension of the TIGGE database. This work was supported by Microsoft AI for Earth (S.M. and G.F.); the Climate Change AI Innovation Grants program (S.M., P.O., G.F., J.C., E.F., and L.M.), hosted by Climate Change AI with the support of the Quadrature Climate Foundation, Schmidt Futures, and the Canada Hub of Future Earth; FAPERJ (Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro) grant SEI-260003/001545/2022 (P.O.); NOAA grant OAR-WPO-2021-2006592 (G.F., J.C., and L.M.); and the National Science Foundation grant PLR-1901352 (J.C.).
[1] Nans Addor, Andrew J Newman, Naoki Mizukami, and Martyn P Clark. The camels data set: catchment attributes and meteorology for large-sample studies. Hydrology and Earth System Sciences, 21(10):5293-5313, 2017.
[2] Troy Arcomano, Istvan Szunyogh, Jaideep Pathak, Alexander Wikner, Brian R Hunt, and Edward Ott. A machine learning-based global atmospheric forecast model. Geophysical Research Letters, 47(9):e2020GL087776, 2020.
[3] Anthony Arguez, Imke Durre, Scott Applequist, Russell S Vose, Michael F Squires, Xungang Yin, Richard R Heim Jr, and Timothy W Owen. NOAA's 1981-2010 us climate normals: an overview. Bulletin of the American Meteorological Society, 93(11):1687-1697, 2012.
[4] Clara Betancourt, Timo Stomberg, Ribana Roscher, Martin G Schultz, and Scarlet Stadtler. Aq-bench: A benchmark dataset for machine learning on global air quality metrics. Earth System Science Data, 13(6):3013-3033, 2021.
[5] Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. Accurate medium-range global weather forecasting with 3d neural networks. Nature, 619(7970):533-538, 2023.
[6] Philippe Bougeault, Zoltan Toth, Craig Bishop, Barbara Brown, David Burridge, De Hui Chen, Beth Ebert, Manuel Fuentes, Thomas M Hamill, Ken Mylne, et al. The thorpex interactive grand global ensemble. Bulletin of the American Meteorological Society, 91(8):1059-1072, 2010.
[7] Mingyue Chen, Wei Shi, Pingping Xie, Viviane BS Silva, Vernon E Kousky, R Wayne Higgins, and John E Janowiak. Assessing objective techniques for gauge-based analyses of global daily precipitation. Journal of Geophysical Research: Atmospheres, 113(D4), 2008.
[8] J. Cohen, D. Coumou, J. Hwang, L. Mackey, P. Orenstein, S. Totz, and E. Tziperman. S2S reboot: An argument for greater inclusion of machine learning in subseasonal to seasonal (S2S) forecasts. WIREs Climate Change, 10, 2018.
[9] Jeffrey J Danielson and Dean B Gesch. Global multi-resolution terrain elevation data 2010 (GMTED2010). US Department of the Interior, US Geological Survey, 2011.
[10] Jun Du, Judith Berner, Roberto Buizza, Martin Charron, Pieter Leopold Houtekamer, Dingchen Hou, Isidora Jankov, Mu Mu, Xuguang Wang, Mozheng Wei, et al. Ensemble methods for meteorological predictions. Office note (National Centers for Environmental Prediction (U.S.)), 2018.
[11] Peter D Dueben, Martin G Schultz, Matthew Chantry, David John Gagne, David Matthew Hall, and Amy McGovern. Challenges and benchmark datasets for machine learning in the atmospheric sciences: Definition, status, and outlook. Artificial Intelligence for the Earth Systems, 1(3):e210002, 2022.
[76] Pingping Xie, M Chen, and W Shi. CPC unified gauge-based analysis of global daily precipitation. In Preprints, 24th Conf. on Hydrology, Atlanta, GA, Amer. Meteor. Soc, volume 2, 2010.
[77] Pingping Xie, Mingyue Chen, Song Yang, Akiyo Yatagai, Tadahiro Hayasaka, Yoshihiro Fukushima, and Changming Liu. A gauge-based analysis of daily precipitation over east asia. Journal of Hydrometeorology, 8(3):607-626, 2007.
[78] Akio Yamagami and Mio Matsueda. Subseasonal forecast skill for weekly mean atmospheric variability over the northern hemisphere in winter and its relationship to midlatitude teleconnections. Geophysical Research Letters, 47(17):e2020GL088508, 2020.
[79] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In The Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, page online. AAAI Press, 2021.
Supplementary Material for Subseasonal ClimateUSA: A Dataset for Subseasonal Forecasting and Benchmarking
A SubseasonalClimateUSA Supplementary Details
The subseasonal_data Python package provides a detailed description of the SubseasonalClimateUSA dataset contents, sources, and processing steps at the following URL:
https://github.com/microsoft/subseasonal_data/blob/main/DATA.md
A.1 Computational Environment Details
The SubseasonalClimateUSA update pipeline runs in a Docker container on an Azure E16-4ds_v4 instance with 4 vCPUs/cores and 128 GB of RAM. The update pipeline runs in under 6 hours at a total cost of $5. All benchmarking experiments were run on the Massachusetts Institute of Technology engaging cluster (https://engaging-web.mit.edu/eofe-wiki/).
A.2 Western U.S. Competition Data Details
Following [23], for the Western U.S. competition experiments of Section 5, the sea surface temperature and sea ice concentration variables were formed by identifying the top three principal components for each variable restricted to the Pacific basin region (20S to 65N, 150E to 90W) using loadings from 1981-2010.
B Model Implementation Details
This section describes the implementation details for each learning model, including the training, hyperparameter tuning, and validation protocols. All models were implemented in Python 3.
B.1 Climatology++
Climatology++ was trained and tuned following the protocol described in [39]; see Algorithm 1. Figure 5 displays the selected window length (the span  $s$ ) and number of years  $Y$  for each target date in 2011-2020 when forecasting for the contiguous U.S. We see that the temperature models preferred fewer training years and larger windows around the target day in recent history but focused more exclusively on the target day of year (via a span of 0) in 2013-2016 and preferred more training years in 2011. Meanwhile, the precipitation models selected the largest available window (corresponding to higher bias but lower variance estimates) for nearly every target date.
Figure 5: Climatology++ hyperparameters automatically selected for each target date in 2011-2020.
B.2 CFSv2++
CFSv2++ was trained and tuned following the protocol described in [39]; see Algorithm 2. Figure 6 displays the selected window length (the span  $s$ ), lead time range, and issuance date count for each target date in 2011-2020 when forecasting for the contiguous U.S. In each task, we observe a
Algorithm 1 Climatology++
input test date  $t^\star$ ; # train years  $Y$ ; span  $s$ ; loss  $\in$  {RMSE,MSE}; training set ground truth  $(\mathbf{y}_t)_{t\in \mathcal{T}}$   
initialize days per year  $D = 365.242199$ $\mathcal{S} = \{t\in \mathcal{T}$ : year_diff :=  $\lfloor \frac{t^* - t}{D} \rfloor \leq Y$  and day_diff :=  $\frac{365}{2} - |\lfloor (t^* - t) \mod D \rfloor - \frac{365}{2}| \leq s\}$   
output  $\operatorname{argmin}_{\mathbf{y}} \sum_{t\in S} \log (\mathbf{y}, \mathbf{y}_t)$
significant amount of variability in the optimal span, lead, and date count selections, highlighting the value of adaptive ensembling and debiasing over the static ensembling and debiasing strategies employed by standard debiased CFSv2.
Figure 6: CFSv2++ hyperparameters automatically selected for each target date in 2011-2020.
Algorithm 2 CFSv2++
input test date  $t^\star$  ; lead time  $l^{\star}$  ; # issuance dates  $d^{\star}$  ; span  $s$  ; training set ground truth and CFSv2 forecasts  $(\mathbf{y}_t,\mathbf{f}_{t,l})_{t\in \mathcal{T},l\in \mathcal{L}}$    
initialize days per year  $D = 365.242199$  ; # training years  $Y = 12$ $\begin{array}{r}\mathcal{S} = \{t\in \mathcal{T}:year\_ diff\coloneqq \lfloor \frac{t^{\star} - t}{D}\rfloor \leq Y\text{and day\_diff}\coloneqq \frac{365}{2} -|\lfloor (t^{\star} - t)\mod D\rfloor -\frac{365}{2} |\leq s\} \end{array}$    
// Form CFSv2 ensemble forecast across issuance dates and lead times  $l\in \mathcal{L}$    
for training and test dates  $t\in S\cup \{t^{\star}\}$  do   
 $\bar{\mathbf{f}}_t = \mathrm{mean}((\mathbf{f}_{t - l^{\star} - d + 1,l})_{1\leq d\leq d^{\star}},l\in \mathcal{L})$    
output  $\bar{\mathbf{f}}_{t^{\star}} + \mathrm{mean}((\mathbf{y}_t - \bar{\mathbf{f}}_t)_{t\in \mathcal{S}})$
B.3 Persistence++
Persistence++ was trained and tuned following the protocol described in [39]; see Algorithm 3. Figures 7 to 10 display the learned Persistence++ regression weights for the final target date in 2020 for each of the four contiguous U.S. forecasting tasks. In each case, we observe significant spatial variation in the optimal weights used to combine lagged measurements, climatology, and CFSv2 ensemble forecasts.
Algorithm 3 Persistence++
input lead time  $l^{\star}$ ; training set ground truth, climatology, and dynamical forecasts
$$
\left(\mathbf {y} _ {t}, \mathbf {c} _ {t}, \mathbf {f} _ {t, l}\right) _ {t \in \mathcal {T}, l \in \mathcal {L}}
$$
initialize forecast period length  $L = 14$
// Form dynamical ensemble forecast across subseasonal lead times  $l \geq l^{\star}$
for training dates  $t\in \mathcal{T}$  do
$$
\bar {\mathbf {f}} _ {t} = \operatorname {m e a n} \left(\left(\mathbf {f} _ {t, l}\right) _ {l \geq l ^ {*}}\right)
$$
// Combine ensemble forecast, climatology, and lagged measurements
for grid points  $g = 1$  to  $G$  do