# Large Uncertainties When Diagnosing the Eddy Feedback Parameter and Its Role in the Signal-To-Noise Paradox

Leo Saffin, Christine M. Mckenna, Rémy Bonnet, Amanda Maycock

► To cite this version:

Leo Saffin, Christine M. Mckenna, Rémy Bonnet, Amanda Maycock. Large Uncertainties When Diagnosing the Eddy Feedback Parameter and Its Role in the Signal-To-Noise Paradox. Geophysical Research Letters, 2024, 51 (11), pp.e2024GL108861. 10.1029/2024GL108861. hal-04748442

HAL Id: hal-04748442

https://cnrs.hal.science/hal-04748442v1

Submitted on 24 Oct 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Geophysical Research Letters®

## RESEARCH LETTER

10.1029/2024GL108861

**Key Points:**

* The “eddy feedback parameter” is a highly non-stationary quantity, making reanalysis and model comparisons problematic on short time periods

* Sampling uncertainty in the eddy feedback parameter from reanalysis data is comparable to the intermodel spread

* Barotropic energy generation rate is a more stable quantity, but does not explain model spread in North Atlantic climate variability

**Supporting Information:**

Supporting Information may be found in the online version of this article.

**Correspondence to:**

L. Saffin,
l.saffin@reading.ac.uk

**Citation:**

Saffin, L., McKenna, C. M., Bonnet, R., & Maycock, A. C. (2024). Large uncertainties when diagnosing the “eddy feedback parameter” and its role in the signal-to-noise paradox. *Geophysical Research Letters*, *51*, e2024GL108861.
https://doi.org/10.1029/2024GL108861

Received 16 FEB 2024
Accepted 29 APR 2024

© 2024. The Author(s).
This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

<figure><img src="image_2.png" /></figure>

## Large Uncertainties When Diagnosing the “Eddy Feedback Parameter” and Its Role in the Signal-To-Noise Paradox

Leo Saffin¹,² ID, Christine M. McKenna¹,³ ID, Rémy Bonnet⁴ ID, and Amanda C. Maycock¹ ID

¹School of Earth and Environment, University of Leeds, Leeds, UK, ²Department of Meteorology, University of Reading, Reading, UK, ³JBA Consulting, Warrington, UK, ⁴CECI, Université de Toulouse, CERFACS/CNRS, Toulouse, France

**Abstract** A too-weak eddy feedback in models has been proposed to explain the signal-to-noise paradox in seasonal-to-decadal forecasts of the winter Northern Hemisphere. We show that the “eddy feedback parameter” (EFP) used in previous studies is sensitive to sampling and multidecadal variability. When these uncertainties are accounted for, the EFP diagnosed from CMIP6 historical simulations generally falls within the reanalysis uncertainty. We find the EFP is not independent of the sampled North Atlantic Oscillation (NAO). Within the same dataset, a sample containing larger NAO variability will show a larger EFP, suggesting that the link between eddy feedbacks and the signal-to-noise paradox could be due to sampling effects with the EFP. An alternative measure of eddy feedback, the barotropic energy generation rate, is less sensitive to sampling errors and delineates CMIP6 models that have weak, strong, or unbiased eddy feedbacks, but shows little relation to NAO variability.

**Plain Language Summary** Model forecasts on seasonal-to-decadal timescales have recently been shown to have significant skill in predicting the North Atlantic Oscillation (NAO, a large-scale pattern of variability). However, these forecasts are undermined by signal-to-noise ratios that are lower than expected given the skill, meaning the models are underconfident. This problem is known as the “signal-to-noise paradox”. Previous work has shown that models underestimate the strength of feedback from atmospheric eddies onto the midlatitude circulation, but models with a stronger eddy feedback suffer less from the signal-to-noise paradox. However, we find that the “eddy feedback parameter” (EFP) used in these studies exhibits large sampling uncertainty that has not previously been taken into account. When accounting for this sampling uncertainty, the EFP in models is generally consistent with reanalysis data. Furthermore, across samples, the EFP correlates with the variability of the NAO, meaning they are not independent, which makes the EFP problematic for understanding the causes of the signal-to-noise paradox. Samples with larger NAO variability are diagnosed with a larger EFP, even within the same dataset. An alternative measure of eddy feedback is less sensitive to sampling and better identifies models which have weak, strong, or unbiased eddy feedbacks.

### 1. Introduction

The winter North Atlantic Oscillation (NAO) has been shown to be predictable on seasonal (Scaife et al., 2014) and decadal (Smith et al., 2019) timescales. However, the predictable NAO signal in models (variability of the ensemble mean) is weaker than expected given the skill, meaning forecasts are underconfident (Scaife & Smith, 2018). This underconfidence occurs despite models having a relatively good representation of total NAO variability and has been coined the signal-to-noise paradox (Scaife et al., 2014; Scaife & Smith, 2018). This underconfidence could be a manifestation of a too-large component of forecast noise or a too-weak predictable signal (Eade et al., 2014; Scaife & Smith, 2018).

Several studies have investigated whether predictable NAO signals are poorly captured in models, including the representation of teleconnections from the tropics to the North Atlantic (O’Reilly et al., 2019; Williams et al., 2023), the response to Arctic sea ice anomalies (Smith et al., 2022), the response to North Atlantic sea surface temperature (SST) anomalies (Simpson et al., 2018), the response to solar cycle variability (Gray et al., 2013; Scaife et al., 2014) and the response to predictable tropical stratospheric variability (Andrews et al., 2019).

There are currently two main hypotheses to explain the NAO signal-to-noise problem.

1. Weak air-sea coupling in the North Atlantic. This has been shown to contribute to an underestimation of winter North Atlantic eddy-driven jet variability on multidecadal timescales (Bracegirdle et al., 2018; Simpson et al., 2018) and summer NAO variability on decadal timescales (Ossó et al., 2020).
2. Weak eddy feedbacks in midlatitudes. Eddy momentum fluxes can act to reinforce the zonal-mean flow and increase the persistence of jets (Lorenz & Hartmann, 2001, 2003) and the NAO is known to be driven by momentum forcing from synoptic and stationary eddies (Luo et al., 2007). Smith et al. (2022) calculated an "eddy feedback parameter" (EFP) to quantify the relationship between eddy forcing and the midlatitude jet (see Section 2.2.1). Smith et al. (2022) showed the EFP in present day climate correlated with the amplitude of the midlatitude zonal wind response to projected Arctic sea ice loss across a set of climate models. They showed that models underestimated the EFP compared to reanalyzes and used an emergent constraint approach to derive a constrained spread of the modeled jet shift. Hardiman et al. (2022) found that models with a weaker EFP (further from reanalysis) generally have less skill and worse signal-to-noise errors for predicting the Arctic Oscillation (AO).

Eddy momentum fluxes are both a driver of, and a response to, mean flow variability (e.g., Trenberth (1984); Karoly (1990)). Quantifying the eddy feedback on the mean flow requires a separation of these components, for example, using lagged approaches or time filtering (e.g., Lorenz & Hartmann, 2001; Lorenz & Hartmann, 2003; Simpson et al., 2013). However, the EFP as calculated in (Smith et al., 2022) does not separate the timescales of eddies and the mean flow, so it is not clear that it does diagnose eddy feedbacks.

The EFP is based on zonal-mean data and therefore blends together variability from separate regions which may make it more difficult to link to the NAO which is focused on a specific region. Therefore, we also analyze a spatially-resolved diagnostic of eddy forcing, the barotropic energy generation rate (Mak & Cai, 1989), which allows us to investigate the relationship between North-Atlantic eddies and NAO variability using time-filtered data. Although the barotropic energy generation rate includes time filtering, it also does not formally separate the eddy feedback from the response to the mean flow, so we refer to it as an eddy forcing rather than a feedback.

Most of the work on the NAO signal-to-noise problem has focused on seasonal-to-decadal timescales; it remains an open question as to whether similar issues manifest in multidecadal projections of the NAO including externally forced trends (McKenna & Maycock, 2021). The initial motivation of this work was to test the eddy feedback hypothesis in climate simulations by examining whether the EFP is related to multidecadal NAO variability. However, we found that our results were strongly affected by sampling issues with the EFP not accounted for in past studies. In this study, we address the sampling uncertainty in the EFP within reanalysis and climate model datasets, as well as the inherent relationship between the EFP and NAO characteristics within a sample.

This study is laid out as follows: Section 2 describes the datasets used in the study and methods for quantifying eddy-mean flow feedback, Section 3 presents the results and Section 4 presents a summary of the key findings.

## 2. Methods

### 2.1. Data Sets

Climate model data is taken from Phase 6 of the Coupled Model Intercomparison Project (CMIP6) (Eyring et al., 2016). We use the historical experiment (1850–2014) from 12 CMIP6 models that provide the required variables (monthly-mean mean-sea-level pressure, and daily-mean zonal (u) and meridional (v) wind on pressure levels) for at least 10 ensemble members (see Table S6 in Supporting Information S1). We select models that provide large ensembles in order to quantify sampling effects and the role of internal variability in calculating the EFP and its relationship with the NAO. Diagnostics are calculated from data regridded to the coarsest resolution climate model (CanESM5, roughly 2.8°). All diagnostics are for Northern hemisphere winter (DJF) with the year labeled by the JF (e.g., 2009/10 is labeled 2010).

We use the ERA5 (Hersbach et al., 2020) and ERA20 C (Poli et al., 2016) reanalysis datasets. The back extension of ERA5 covers the period 1940 to 1978 and the standard ERA5 covers 1979 to present. ERA20 C covers 1900–2010 and only assimilates surface pressure and surface marine wind observations. For ERA5 and ERA20 C winds, we aggregate 6 hourly data (00, 06, 12, 18) to daily means to provide an equivalent comparison to the CMIP6 data. We also use monthly-mean mean-sea-level pressure data from 20CRv3 (Slivinski et al., 2019), a reanalysis with a longer record (1836–2015) that only assimilates surface pressure, and HadSLP (Allan & Ansell, 2006), a gridded dataset produced from surface pressure observations, to calculate NAO timeseries in the supplement.
## 2.2. Diagnostics

### 2.2.1. Eddy Feedback Parameter

Smith et al. (2022) defined the eddy feedback parameter (EFP) as the squared correlation coefficient ($r^2$) between the DJF-mean zonal-mean zonal wind ($\bar{u}$) and the DJF-mean of the horizontal component of the Eliassen-Palm flux (EP-flux) divergence, calculated as a function of latitude and pressure, and then averaged over 25–72°N, and 200–600 hPa. Hardiman et al. (2022) used a similar formulation, but calculated the EFP at a single level (500 hPa) and only included the quasi-geostrophic component of EP-flux divergence, expressed as a zonal acceleration: Equation 1 from Hardiman et al. (2022),

$$ \frac{\nabla \cdot \mathbf{F}_H}{\rho a \cos(\phi)} = -\frac{1}{a \cos^2 \phi} \frac{d(\overline{u'v' \cos^2 \phi})}{d\phi}, \qquad (1) $$

where $\rho$ is density, $\phi$ is latitude, $a$ is Earth's radius. Overbars represent a zonal mean, and primes represent local deviations from the zonal mean. Here, we calculate the EFP following Hardiman et al. (2022). The differences in methodology for calculating the EFP can give a different absolute value, but give similar results for the uncertainty (see Fig. S1 in Supporting Information S1).

### 2.2.2. Barotropic Energy Generation Rate

The barotropic energy generation rate ($G$) diagnoses the exchange of energy between eddies and the large-scale flow based on an energy equation for the ageostrophic perturbation flow in quasi-geostrophic dynamics (Mak & Cai, 1989). If ($U, V$) describes the large-scale geostrophic wind and ($u', v'$) the eddies, then the barotropic energy generation rate is given by

$$ G = \mathbf{E} \cdot \mathbf{D}, \qquad (2) $$

where

$$ \mathbf{E} = \cos(\phi) \left( \frac{1}{2}(v'^2 - u'^2), -u'v' \right), \qquad (3) $$

is the E-vector, which describes the elongation of the eddy, and

$$ \mathbf{D} = \frac{1}{a \cos(\phi)} \left( \frac{\partial U}{\partial \lambda} - \frac{\partial V \cos(\phi)}{\partial \phi}, \frac{\partial V}{\partial \lambda} + \frac{\partial U \cos(\phi)}{\partial \phi} \right), \qquad (4) $$

is the deformation of the large-scale flow (Mak & Cai, 1989), where $\lambda$ is latitude. Note that we use the spherical coordinate version of these equations from Fukotomi and Yasunari (2002). We diagnose $G$ using daily-mean winds at 250 hPa that are separated into a high frequency (2–6 days) eddy component and a slowly varying (>10 days) large-scale component using Lanczos filters with a window of 61 days. We then average the daily values of $G$ to a seasonal mean.

In comparison to the EFP, $G$ is spatially-resolved, giving a measure of the local energy exchange. To provide a comparison with the EFP and relate $G$ to NAO variability, we average seasonal-mean values of $G$ over a box in the North Atlantic (60°–25°W, 30°–45°N) giving $G_{NA}$. This region is where the models and reanalysis show climatological negative values (see Fig. S2 in Supporting Information S1), indicating exchange of energy from the eddies to the large scale flow.

### 2.2.3. North Atlantic Oscillation Index

The NAO index is calculated as the difference in DJF area-averaged mean-sea-level pressure between a southern box (90°W–60°E, 20°N–55°N) and a northern box (90°W–60°E, 55°N–90°N) following Stephenson et al. (2006). From the NAO timeseries we calculate variance. Multidecadal NAO variance is also calculated by first applying a 20-year running mean.
<figure><img src="image_3.png" /><figcaption>**Figure 1.** Calculation of the EFP using ERA5. Variables used to calculate the EFP as a DJF mean, (a) zonal-mean zonal wind and (b) the acceleration of the zonal-mean zonal wind diagnosed from the quasi-geostrophic component of the horizontal EP-flux divergence. (c) The product of anomalies of (a) and (b), where the anomalies are calculated against the time mean (mean across rows) by latitude. (d) shows the same as (c), but normalized by the standard deviations, at each latitude, of (a) and (b). The time mean of (c) and (d) give the covariance and correlation as a function of latitude, respectively.</figcaption></figure>

The NAO has not been detrended, which could lead to an overestimation of NAO variance in the CMIP6 models compared to ERA5 because we are retaining longer-timescale variability. However, multidecadal variability is only a small part of the total NAO variance (see Section 3.3), so the difference in NAO variance due to including these longer timescales is small.

## 2.3. Statistics

To estimate sampling uncertainty, we recalculate the EFP in ERA5 by resampling winters with replacement (bootstrapping) using the same sample size as the input dataset (e.g., for 1940–2022, each sample is 82 years), repeating 1,000 times. We also recalculate the EFP, NAO variance, and $G_{NA}$ in ERA5 in the same way, but with a sample size matching the historical simulation length (164 years) to compare with the CMIP6 simulations. Each diagnostic is calculated using the same sample years, allowing us to assess relationships between these diagnostics due to sampling. Relationships between variables are estimated using linear least squares regression.

# 3. Results

## 3.1. Uncertainties in Reanalysis Derived Eddy Feedback Parameter

In this section, we show how the EFP is affected by sampling uncertainty and multidecadal variability. Figure 1 shows the calculation of the EFP in ERA5 broken into constituent steps. Figures 1a and 1b show the DJF-mean input variables as a function of latitude and year: $\tilde{u}$ and the acceleration of $\tilde{u}$ diagnosed from the quasigeostropic component of the horizontal EP-flux divergence. The EFP is calculated by calculating the correlation coefficient ($r$) between these two variables at each latitude and then averaging $r^2$ across latitudes. $r$ is defined as the covariance of two variables normalized by their standard deviations. To understand how different years and latitudes contribute to the EFP, Figure 1c shows the anomalies of the input variables, relative to the time mean at each latitude, multiplied together, so the time mean is the covariance as a function of latitude. Figure 1d shows the same, but normalized by the standard deviations of the input variables at each latitude, so the time mean is $r$ as a function of latitude.
<figure><img src="image_4.png" /><figcaption>**Figure 2.** Uncertainties in the EFP identified from reanalysis data. (a) The EFP calculated by resampling ERA5 over different periods. Orange lines show the median, crosses show the EFP from the original set of years, boxes show the 25%–75% range, whiskers show the 2.5%–97.5% range, and circles show points outside this range. (b) The EFP and NAO variance calculated over 23-year rolling windows for ERA5 and ERA20C data. The x-axis shows the middle year in each sample. The vertical line is for 1993–2016, the years used in Hardiman et al. (2022).</figcaption></figure>

Figure 1 reveals two potential issues with the EFP.

1. Calculating *r* at each latitude and then taking a spatial average overemphasizes latitudes with weaker variability. This can be seen by comparing Figures 1c and 1d: anomalies are weaker closer to the equator for the covariance but have a larger contribution to *r* because the standard deviation at those latitudes is smaller.

2. A single outlier season can make a large contribution to the EFP (e.g., 2009/2010 in Figure 1d). This undermines comparisons of the EFP in reanalysis data and climate models when they do not span a common period and do not sample the same internal variability. For example, if a model with inherently weak eddy feedback happens to simulate a season like 2009/2010, it may appear to have a larger EFP than a model with a strong eddy feedback that by chance does not simulate a season like 2009/2010.

Building on point 2, to quantify the sampling uncertainty we recalculate the EFP by sampling years from ERA5 with replacement (see Section 2.3). Figure 2a shows results with the resampling period varied to show the dependence of “observed” EFP on time period: the full ERA5 period (1940–2022); the pre-satellite backward extension period only (1940–1979); and the satellite period only (1979–2022). In all cases, the sampling uncertainty in the EFP (≈0.2–0.3) is comparable to the median value based on the 95% confidence interval. This sampling effect represents a substantial uncertainty that has not been acknowledged in previous studies (e.g., Smith et al. (2022); Hardiman et al. (2022); Screen et al. (2022)).

Figure 2a also shows the EFP is dependent on time period: the satellite period has a larger EFP than the pre-satellite back extension period, with no overlap of the 95% intervals. To better understand the dependence of EFP on time period, we calculate the EFP using a rolling 23-year window (consistent with the 1993–2016 period used in Hardiman et al. (2022)). ERA5 shows a systematic increasing trend in the 23-year EFP (Figure 2b). A long-term trend in the EFP could be spurious if the reanalysis is poorly constrained by observations and behaves more like the underlying atmospheric model further back in time. Figure 2b also shows the EFP from ERA20 C, which extends back to 1900. Longer-term reanalyzes that only assimilate a limited set of surface observations, such as ERA20 C, have been shown to produce unrealistic trends as the density of the observation network evolves with time (Befort et al., 2016; Bloomfield et al., 2018; Krueger et al., 2013; Oliver, 2016). However, ERA20 C actually shows a larger EFP in the 1930s/1940s when there is less observation data and reproduces the increase in EFP over the late twentieth century. This shows the apparent EFP trend is unlikely to be due to an intrinsic bias of weak eddy feedback in the model that produces ERA5 and instead is related to multidecadal variability in the input parameters.
<figure><img src="image_5.png" /><figcaption>**Figure 3.** The relationship between the EFP and NAO variance for (a), (c) CMIP6 historical simulations (1850–2014) and (b) ERA5 (full period, 1940–2022). (a) The EFP and NAO variance for CMIP6 ensemble members and mean for each model ensemble (outlined symbols). (b) EFP and NAO variance calculated using 164 years sampled from ERA5 with replacement (repeated 1000 times). The outlined dot shows the EFP and NAO variance for the full ERA5 data. (c) The same as (a), but for NAO variance calculated after applying a 20-year running-mean filter. The lines on each subfigure show linear regressions calculated from each set of data in the subfigures (see text for details). The lines from (a) are duplicated in (b) for comparison.</figcaption></figure>

Interestingly, the increase in EFP over the late twentieth century closely mirrors the positive trend in NAO variance over this period, though this common temporal behavior is weaker in the earlier period covered only by ERA20 C. It makes sense that the NAO and EFP are related. Eddy-driven jet latitude is related to the NAO (Woollings et al., 2010) and NAO predictability (Parker et al., 2019; Strommen, 2020), and zonal-mean zonal wind is one of the inputs to the EFP calculation. The EFP calculation also emphasizes large seasonal deviations in jet latitude. For example, winter 2009/10 had a strongly southward shifted jet and negative NAO (Santos et al., 2013). Figure 1d showed how the shift in jet in 2009/10 is emphasized in the correlation calculation and Figure 2b shows a step increase of almost 0.1 when 2009/10 is included in the rolling window.

The time period used by Hardiman et al. (2022) (1993–2016) is very close to the maximum EFP over the entire twentieth century due to the inclusion of 2009/10 and being close to the peak of multidecadal variability in the EFP. The results in this section show that previous studies have likely overestimated the long-term mean EFP in reanalysis data.

## 3.2. Comparison of Climate Models and Reanalysis Eddy Feedback Parameter

We next address the comparison of EFP in climate models with reanalysis data in the context of the sampling uncertainties described in the previous section. Figure 3 shows the range of EFP calculated from the CMIP6 ensembles (a, c) and from repeatedly sampling 164 years from ERA5 with replacement (b), as well as the relationship with NAO variance (discussed in the following section). In contrast to previous results, we do not find that the EFP is weaker in models than in reanalysis. The EFP diagnosed from CMIP6 models is generally within the uncertainty from ERA5, with some models potentially having too large EFP (CanESM5, CESM2, CMCC-CM2-SR5). If we only considered the EFP and its associated uncertainty from the satellite period of ERA5
(Figure 2a), then we would conclude that some CMIP6 models underestimate the EFP. This highlights the importance of considering longer-timescale variability, as well as sampling uncertainty, when quantifying the EFP and the limitation of using the EFP as a diagnostic for model performance.

### 3.3. Relationship Between the Eddy Feedback Parameter and the North Atlantic Oscillation

Section 3.1 highlighted a relationship between long-term variations in the EFP and the NAO. We next show how this relationship can lead to correlations that should be interpreted as a sample with larger NAO variability giving a larger EFP, rather than stronger eddy feedbacks leading to stronger NAO variability. Most CMIP6 models capture NAO variability well (Figure 3a) compared to ERA5 (Figure 3b). Only MIROC-ES2L is systematically too weak. Some models are potentially too weak (MIROC6, INM-CM5-0) or too strong (IPSL-CM6A-LR, CESM2), but still overlap with the uncertainty from ERA5.

The lines in Figure 3 show linear regressions calculated from the data in each panel in different ways.

1. For “ERA5” (gray line in Figure 3b) the regression is across the bootstrap samples. Because EFP and NAO variance are calculated using the same sets of sample years, this tells us how the EFP relates to NAO variability purely due to sampling.

2. “Mean” is the regression across the ensemble mean points of all models. This relates to model biases and is what would typically be used for emergent constraints (e.g., Smith et al. (2022)).

3. “Weighted” is a weighted average regression across all models. For each model, a regression is calculated across ensemble members. The average slope and intercept are then calculated from these individual model regressions, weighted by the number of ensemble members for each model. This indicates whether a sampling relationship between EFP and NAO variability is present, on average, in individual models.

4. “All” is the regression across all ensemble members of all models with each sample treated independently. This gives a mix between “Mean” and “Weighted”.

The full set of results from the linear regressions are given in the supplement (Tables S1–S5 in Supporting Information S1). Note that many of the individual model regressions in 3) are not significant due to low sample sizes and the p-value test is less meaningful for the “ERA5” and “All” regressions because the points are not independent. However, the analysis is intended to show how sampling issues with the EFP can produce spurious relationships with the NAO rather than identifying significant relationships.

All three regressions in Figure 3a show a similar relationship between EFP and NAO variance and are well reproduced by sampling ERA5 ($r = 0.34–0.55$). This means that the across model relationship between the EFP and NAO variance (“Mean”), which could have been interpreted as physically related model biases, is most likely an extension of the sampling relationship found in ERA5: a model with stronger NAO variability is diagnosed with a larger EFP.

Although total NAO variability is relatively well represented for models exhibiting the signal-to-noise paradox (Scaife & Smith, 2018), weak multidecadal NAO variability Bracegirdle (2022); Bonnet et al. (2024) could be evidence of signal-to-noise issues in climate models. However, similar relationships are found when multidecadal variability is isolated (Figure 3c), suggesting this is still only identifying sampling relationships. We haven't estimated the reanalysis relationship between the EFP and multidecadal NAO variance because ERA5 is too short for sampling and longer-timescale reanalyzes give less consistent values of NAO further back in time (see Figs. S3 and S4 in Supporting Information S1).

### 3.4. Alternative Measure of Eddy Forcing

We next show that an alternative measure of eddy forcing targeted at the North Atlantic ($G_{NA}$, see Section 2.2.2) suffers much less from the sampling issues identified for the EFP. Figure 4 shows $G_{NA}$ for ERA5 and the CMIP6 ensembles and its relationship to NAO variance and the EFP. $G_{NA}$ is better able to identify models that are weak (CanESM5, CESM2, IPSL-CM6A-LR, CMCC-CM2-SR5, INM-CM5-0, MIROC6, MIROC-ES2L), strong (MPI-ESM1-2-LR/HR), or unbiased (UKESM1-0-LL, CNRM-CM6-1, CNRM-ESM2-1) compared to ERA5 due to having much smaller sampling uncertainty.

The sampling relationship between $G_{NA}$ and NAO variability in ERA5 is much weaker ($r = 0.07$) in contrast to that of the EFP and NAO variability ($r = 0.55$). Furthermore, the relationship differs from the (nonsignificant)
<figure><img src="image_6.png" /><figcaption>**Figure 4.** The same as Figure 3, but with North-Atlantic DJF-mean barotropic energy generation rate ($G_{NA}$) on the x-axis instead of EFP and extra panels (d) and (e) with EFP on the y-axis for CMIP6 models and ERA5, respectively.</figcaption></figure>

across model relationship. Similar results are found for multidecadal NAO variability. Interestingly $G_{NA}$ shows no sampling relationship to the EFP and very little relationship across the models used here (Figures 4d and 4e). This suggests that either the EFP is capturing different aspects of eddies, due to $G_{NA}$ being more localized, or that the EFP is a poor measure of eddies due to the sampling issues shown in earlier.

$G$ shows similar uncertainty when averaged over the same region as used for the EFP (see Fig. S6 in Supporting Information S1). This suggests that the uncertainties in the EFP are due to the sensitivities in how the EFP is diagnosed (identified in Section 3.1) rather than variability in the underlying behavior of the eddies.

4. Conclusions

Previous studies have suggested that seasonal prediction systems and free running climate models systematically underestimate Northern hemisphere midlatitude eddy feedbacks (Screen et al., 2022; Smith et al., 2022), and that this bias may explain the signal-to-noise paradox (Hardiman et al., 2022; Scaife et al., 2019). However, we find that the eddy feedback parameter (EFP) used by Smith et al. (2022), Screen et al. (2022), and Hardiman et al. (2022) exhibits large sampling uncertainty which can impede model-reanalysis comparisons and makes determining physical mechanisms difficult.
We have shown that the EFP is sensitive to individual outlier seasons and also exhibits strong multidecadal variability. This makes the EFP problematic to interpret as an intrinsic property of a model or the real world because very large sample sizes are needed to produce an estimate with sufficiently small uncertainties. Previous published estimates of the EFP in modern reanalysis data are close to the maximum value derived within the 1940–2022 period because of the pronounced effect of an outlier season (2009/2010) and the phasing of multidecadal variability in the EFP. When sampling uncertainty is taken into account, the EFP in CMIP6 historical simulations is largely consistent with ERA5. Previous results using the EFP as an emergent constraint (Screen et al., 2022; Smith et al., 2022) should have much larger error bars to account for these sampling uncertainties. These uncertainties may be the reason that Screen et al. (2022) found that the reanalysis EFP in the Southern Hemisphere is roughly in the middle of the model values, while the Northern Hemisphere EFP appeared too weak in models.

We have also shown that the sample EFP correlates with sample NAO variability and this can lead to spurious across-model correlations between the EFP and NAO variability. The across model correlation could have been interpreted as a stronger model eddy feedback causing stronger NAO variability, but is actually due to a sample with stronger NAO variability being diagnosed with a stronger EFP because the EFP and NAO are not independent. The relation between the EFP and NAO makes sense because both variables have an underlying relationship with jet latitude. For example, winter 2009/10 had an anomalously southward shifted jet and negative NAO (Santos et al., 2013) and makes the largest single contribution to the EFP in ERA5. It could be argued that models with stronger eddy feedbacks would produce more years like 2009/10; however, it is clear that we need a much larger sample of data than is available for reanalyzes to determine if this is the case.

We have investigated another measure of eddy forcing, the barotropic energy generation rate *G*, which more cleanly separates eddy forcing and mean flow terms and can be calculated locally for the North Atlantic region ($G_{NA}$). $G_{NA}$ shows much smaller sampling uncertainty than the EFP and a much weaker sampling relationship with NAO variability, suggesting that it is better at describing intrinsic properties of the models and reanalysis. We find no systematic bias in $G_{NA}$, but $G_{NA}$ does better distinguish which models are too weak, too strong or unbiased.

In summary, our results raise questions about previous interpretations that weak eddy feedbacks can explain the signal-to-noise paradox. Firstly, we find that models do not systematically underestimate the EFP when accounting for sampling uncertainty or using an alternative, better constrained, diagnostic ($G_{NA}$). Secondly, the diagnosed EFP from a sample is dependent on the sample NAO variability, which makes it difficult to interpret differences associated with the EFP as being caused by eddy feedbacks rather than some confounding variable. Therefore previous results should be re-examined with a diagnostic of eddy forcing that is more robust to climate variability and where clearer causality can be determined, such as the barotropic energy generation rate or a more in depth lead-lag approach that formally isolates the feedback of eddies on the mean flow (e.g., Lorenz and Hartmann (2001)).

## Data Availability Statement

The ERA5 reanalysis data is available from the Copernicus Climate Data Store (Hersbach et al., 2023a, 2023b). ERA20C was accessed from the NCAR research data archive (European Centre for Medium-Range Weather Forecasts., 2014). The CMIP6 data used in the study was accessed from the Earth System Grid Federation ((Boucher et al., 2018; Byun, 2020; Danabasoglu, 2019; Hajima et al., 2019; Jungclaus et al., 2019; Lovato & Peano, 2020; Seferian, 2018; Swart et al., 2019; Tang et al., 2019; Tatebe & Watanabe, 2018; Voldoire, 2018; Volodin et al., 2019; Wieners et al., 2019)). The diagnostics calculated for each CMIP6 simulation are given in Supporting Information S1 (Table S6). The code used for calculating the diagnostics is available at Saffin (2024a) and the code for further processing and making the figures is available at Saffin (2024b).

## Acknowledgments

We gratefully acknowledge funding for this work by the EU Horizon 2020 project CONSTRAIN (GA 820829). We are grateful to Doug Smith and Steven Hardiman for constructive discussion about the work. We thank the two anonymous reviewers for their helpful comments and suggestions which improved this manuscript.

## References

* Allan, R., & Ansell, T. (2006). A New Globally Complete monthly historical gridded mean sea level pressure dataset (HadSLP2): 1850–2004. *Journal of Climate*, 19(22), 5816–5842. https://doi.org/10.1175/JCLI3937.1

* Andrews, M. B., Knight, J. R., Scaife, A. A., Lu, Y., Wu, T., Gray, L. J., & Schenzinger, V. (2019). Observed and simulated teleconnections between the stratospheric quasi-biennial oscillation and northern hemisphere winter atmospheric circulation. *Journal of Geophysical Research: Atmospheres*, 124(3), 1219–1232. https://doi.org/10.1029/2018jd029368

* Befort, D. J., Wild, S., Kruschke, T., Ulbrich, U., & Leckebusch, G. C. (2016). Different long-term trends of extra-tropical cyclones and windstorms in ERA-20C and NOAA-20CR reanalyses. *Atmospheric Science Letters*, 17(11), 586–595. https://doi.org/10.1002/asl.694
Bloomfield, H. C., Shaffrey, L. C., Hodges, K. I., & Vidale, P. L. (2018). A critical assessment of the long-term changes in the wintertime surface Arctic Oscillation and Northern Hemisphere storminess in the ERA20C reanalysis. *Environmental Research Letters*, **13**(9), 94004. https://doi.org/10.1088/1748-9326/aad5c5

Bonnet, R., McKenna, C., & Maycock, A. (2024). Model spread in multidecadal NAO variability connected to stratosphere-troposphere coupling. *EGUsphere*, **2024**, 1–24. https://doi.org/10.5194/egusphere-2023-3103

Boucher, O., Denvil, S., Levavasseur, G., Cozic, A., Caubel, A., Foujols, M.-A., et al. (2018). IPSL IPSL-CM6A-LR model output prepared for CMIP6 CMIP historical [dataset]. *Earth System Grid Federation*. https://doi.org/10.22033/ESGF/CMIP6.5195

Bracegirdle, T. J. (2022). Early-to-Late winter 20th century North Atlantic multidecadal atmospheric variability in observations, CMIP5 and CMIP6. *Geophysical Research Letters*, **49**(11), e2022GL098212. https://doi.org/10.1029/2022GL098212

Bracegirdle, T. J., Lu, H., Eade, R., & Woollings, T. (2018). Do CMIP5 models reproduce observed low-frequency North Atlantic jet variability? *Geophysical Research Letters*, **45**(14), 7204–7212. https://doi.org/10.1029/2018GL078965

Byun, Y.-H. (2020). NIMS-KMA UKESM1.0-LL model output prepared for CMIP6 CMIP historical [dataset]. *Earth System Grid Federation*. https://doi.org/10.22033/ESGF/CMIP6.8379

Danabasoglu, G. (2019). NCAR CESM2 model output prepared for CMIP6 CMIP historical. [dataset]. *Earth System Grid Federation*. https://doi.org/10.22033/ESGF/CMIP6.7627

Eade, R., Smith, D., Scaife, A., Wallace, E., Dunstone, N., Hermanson, L., & Robinson, N. (2014). Do seasonal-to-decadal climate predictions underestimate the predictability of the real world? *Geophysical Research Letters*, **41**(15), 5620–5628. https://doi.org/10.1002/2014GL061146

European Centre for Medium-Range Weather Forecasts. (2014). ERA-20C project (ECMWF atmospheric reanalysis of the 20th century) [dataset]. *Research Data Archive at the National Center for Atmospheric Research, Computational and Information Systems Laboratory*. https://doi.org/10.5065/D6VQ30QG

Eyring, V., Bony, S., Meehl, G. A., Senior, C. A., Stevens, B., Stouffer, R. J., & Taylor, K. E. (2016). Overview of the coupled model Inter-comparison project phase 6 (CMIP6) experimental design and organization. *Geoscientific Model Development*, **9**(5), 1937–1958. https://doi.org/10.5194/gmd-9-1937-2016

Fukotomi, Y., & Yasunari, T. (2002). Tropical-extratropical interaction associated with the 10-25-day oscillation over the Western Pacific during the northern summer. *Journal of the Meteorological Society of Japan*. Ser. II, **80**(2), 311–331. https://doi.org/10.2151/jmsj.80.311

Gray, L. J., Scaife, A. A., Mitchell, D. M., Osprey, S., Ineson, S., Hardiman, S., et al. (2013). A lagged response to the 11 year solar cycle in observed winter Atlantic/European weather patterns. *Journal of Geophysical Research: Atmospheres*, **118**(24), 13405–13420. https://doi.org/10.1002/2013JD020062

Hajima, T., Abe, M., Arakawa, O., Suzuki, T., Komuro, Y., Ogura, T., et al. (2019). MIROC MIROC-ES2L model output prepared for CMIP6 CMIP historical [dataset]. *Earth System Grid Federation*. https://doi.org/10.22033/ESGF/CMIP6.5602

Hardiman, S. C., Dunstone, N. J., Scaife, A. A., Smith, D. M., Comer, R., Nie, Y., & Ren, H.-L. (2022). Missing eddy feedback may explain weak signal-to-noise ratios in climate predictions. *npj Clim. Atmos. Sci.*, **5**(1), 57. https://doi.org/10.1038/s41612-022-00280-4

Hersbach, H., Bell, B., Berrisford, P., Biavati, G., Horányi, A., Muñoz-Sabater, J., et al. (2023a). Era5 hourly data on pressure levels from 1940 to present. [dataset]. *Copernicus Climate Change Service (C3S) Climate Data Store (CDS)*. https://doi.org/10.24381/cds.bd0915c6

Hersbach, H., Bell, B., Berrisford, P., Biavati, G., Horányi, A., Muñoz-Sabater, J., et al. (2023b). Era5 hourly data on single levels from 1940 to present. [dataset]. *Copernicus Climate Change Service (C3S) Climate Data Store (CDS)*. https://doi.org/10.24381/cds.adbb2d47

Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Muñoz-Sabater, J., et al. (2020). The ERA5 global reanalysis. *Quarterly Journal of the Royal Meteorological Society*, **146**(730), 1999–2049. https://doi.org/10.1002/qj.3803

Jungclaus, J., Bittner, M., Wieners, K.-H., Wachsmann, F., Schupfner, M., Legutke, S., et al. (2019). MPI-M MPI-ESM1.2-HR model output prepared for CMIP6 CMIP historical [dataset]. *Earth System Grid Federation*. https://doi.org/10.22033/ESGF/CMIP6.6594

Karoly, D. J. (1990). The role of transient eddies in low-frequency zonal variations of the Southern Hemisphere circulation. *Tellus*, **42**(1), 41–50. https://doi.org/10.1034/j.1600-0870.1990.00005.x

Krueger, O., Schenk, F., Feser, F., & Weisse, R. (2013). Inconsistencies between long-term trends in storminess derived from the 20CR reanalysis and observations. *Journal of Climate*, **26**(3), 868–874. https://doi.org/10.1175/jcli-d-12-00309.1

Lorenz, D. J., & Hartmann, D. L. (2001). Eddy–zonal flow feedback in the southern hemisphere. *Journal of the Atmospheric Sciences*, **58**(21), 3312–3327. https://doi.org/10.1175/1520-0469(2001)058<3312:ezfit>2.0.co;2

Lorenz, D. J., & Hartmann, D. L. (2003). Eddy–zonal flow feedback in the northern hemisphere winter. *Journal of Climate*, **16**(8), 1212–1227. https://doi.org/10.1175/1520-0442(2003)16<1212:effitn>2.0.co;2

Lovato, T., & Peano, D. (2020). CMCC CMCC-CM2-SR5 model output prepared for CMIP6 CMIP historical [dataset]. *Earth System Grid Federation*. https://doi.org/10.22033/ESGF/CMIP6.3825

Luo, D., Gong, T., & Diao, Y. (2007). Dynamics of eddy-driven low-frequency dipole modes. part iii: Meridional displacement of westerly jet anomalies during two phases of nao. *Journal of the Atmospheric Sciences*, **64**(9), 3232–3248. https://doi.org/10.1175/JAS3998.1

Mak, M., & Cai, M. (1989). Local barotropic Instability. *Journal of the Atmospheric Sciences*, **46**(21), 3289–3311. https://doi.org/10.1175/1520-0469(1989)046<3289:lbi>2.0.co;2

McKenna, C. M., & Maycock, A. C. (2021). Sources of uncertainty in multimodel large ensemble projections of the winter north atlantic oscillation. *Geophysical Research Letters*, **48**(14), e2021GL093258. https://doi.org/10.1029/2021GL093258

Olivier, E. C. J. (2016). Blind use of reanalysis data: Apparent trends in Madden–Julian oscillation activity driven by observational changes. *International Journal of Climatology*, **36**(10), 3458–3468. https://doi.org/10.1002/joc.4568

O’Reilly, C. H., Weisheimer, A., Woollings, T., Gray, L. J., & MacLeod, D. (2019). The importance of stratospheric initial conditions for winter North Atlantic Oscillation predictability and implications for the signal-to-noise paradox. *Quarterly Journal of the Royal Meteorological Society*, **145**(718), 131–146. https://doi.org/10.1002/qj.3413

Ossó, A., Sutton, R., Shaffrey, L., & Dong, B. (2020). Development, Amplification, and Decay of atlantic/European summer weather patterns linked to spring North Atlantic sea surface temperatures. *Journal of Climate*, **33**(14), 5939–5951. https://doi.org/10.1175/JCLI-D-19-0613.1

Parker, T., Woollings, T., Weisheimer, A., O’Reilly, C., Baker, L., & Shaffrey, L. (2019). Seasonal predictability of the winter North Atlantic oscillation from a jet stream Perspective. *Geophysical Research Letters*, **46**(16), 10159–10167. https://doi.org/10.1029/2019GL084402

Poli, P., Hersbach, H., Dee, D. P., Berrisford, P., Simmons, A. J., Vitart, F., et al. (2016). ERA-20C: An atmospheric reanalysis of the Twentieth century. *Journal of Climate*, **29**(11), 4083–4097. https://doi.org/10.1175/jcli-d-15-0556.1

Saffin, L. (2024a). leosaffin/constrain [software]. Zenodo. https://doi.org/10.5281/zenodo.10964829

Saffin, L. (2024b). leosaffin/eddyFeedback [software]. Zenodo. https://doi.org/10.5281/zenodo.10964831

Santos, J. A., Woollings, T., & Pinto, J. G. (2013). Are the winters 2010 and 2012 Archetypes exhibiting Extreme Opposite behavior of the North Atlantic jet stream? *Monthly Weather Review*, **141**(10), 3626–3640. https://doi.org/10.1175/MWR-D-13-00024.1
Scaife, A. A., Arribas, A., Blockley, E., Brookshaw, A., Clark, R. T., Dunstone, N., et al. (2014). Skillful long-range prediction of European and North American winters. *Geophysical Research Letters*, 41(7), 2514–2519. https://doi.org/10.1002/2014GL059637

Scaife, A. A., Camp, J., Comer, R., Davis, P., Dunstone, N., Gordon, M., et al. (2019). Does increased atmospheric resolution improve seasonal climate predictions? *Atmospheric Science Letters*, 20(8), e922. https://doi.org/10.1002/asl.922

Scaife, A. A., & Smith, D. (2018). A signal-to-noise paradox in climate science. *npj Clim. Atmos. Sci.*, 1(1), 28. https://doi.org/10.1038/s41612-018-0038-4

Screen, J. A., Eade, R., Smith, D. M., Thomson, S., & Yu, H. (2022). Net Equatorward shift of the jet Streams when the contribution from sea-ice loss is constrained by observed eddy feedback. *Geophysical Research Letters*, 49(23), e2022GL100523. https://doi.org/10.1029/2022GL100523

Seferian, R. (2018). CNRM-CERFACS CNRM-ESM2-1 model output prepared for CMIP6 CMIP historical. [dataset]. *Earth System Grid Federation*. https://doi.org/10.22033/ESGF/CMIP6.4068

Simpson, I. R., Deser, C., McKinnon, K. A., & Barnes, E. A. (2018). Modeled and observed multidecadal variability in the North Atlantic jet stream and its Connection to sea surface temperatures. *Journal of Climate*, 31(20), 8313–8338. https://doi.org/10.1175/JCLI-D-18-0168.1

Simpson, I. R., Shepherd, T. G., Hitchcock, P., & Scinocca, J. F. (2013). Southern annular mode dynamics in observations and models. Part II: Eddy feedbacks. *Journal of Climate*, 26(14), 5220–5241. https://doi.org/10.1175/JCLI-D-12-00495.1

Slivinski, L. C., Compo, G. P., Whitaker, J. S., Sardeshmukh, P. D., Giese, B. S., McColl, C., et al. (2019). Towards a more reliable historical reanalysis: Improvements for version 3 of the Twentieth Century Reanalysis system. *Quarterly Journal of the Royal Meteorological Society*, 145(724), 2876–2908. https://doi.org/10.1002/qj.3598

Smith, D. M., Eade, R., Andrews, M. B., Ayres, H., Clark, A., Chripko, S., et al. (2022). Robust but weak winter atmospheric circulation response to future Arctic sea ice loss. *Nature Communications*, 13(1), 727. https://doi.org/10.1038/s41467-022-28283-y

Smith, D. M., Eade, R., Scaife, A. A., Caron, L.-P., Danabasoglu, G., DelSole, T. M., et al. (2019). Robust skill of decadal climate predictions. *npj Clim. Atmos. Sci.*, 2(1), 13. https://doi.org/10.1038/s41612-019-0071-y

Stephenson, D. B., Pavan, V., Collins, M., Junge, M. M., Quadrelli, R., & Groups, P. C. M. (2006). North atlantic oscillation response to transient greenhouse gas forcing and the impact on European winter climate: A CMIP2 multi-model assessment. *Climate Dynamics*, 27(4), 401–420. https://doi.org/10.1007/s00382-006-0140-x

Strommen, K. (2020). Jet latitude regimes and the predictability of the North Atlantic oscillation. *Quarterly Journal of the Royal Meteorological Society*, 146(730), 2368–2391. https://doi.org/10.1002/qj.3796

Swart, N. C., Cole, J. N., Kharin, V. V., Lazare, M., Scinocca, J. F., Gillett, N. P., et al. (2019). CCCma CanESM5 model output prepared for CMIP6 CMIP historical [dataset]. *Earth System Grid Federation*. https://doi.org/10.22033/ESGF/CMIP6.3610

Tang, Y., Rumbold, S., Ellis, R., Kelley, D., Mulcahy, J., Sellar, A., et al. (2019). MOHC UKESM1.0-LL model output prepared for CMIP6 CMIP historical [dataset]. *Earth System Grid Federation*. https://doi.org/10.22033/ESGF/CMIP6.6113

Tatebe, H., & Watanabe, M. (2018). MIROC MIROC6 model output prepared for CMIP6 CMIP historical [dataset]. *Earth System Grid Federation*. https://doi.org/10.22033/ESGF/CMIP6.5603

Trenberth, K. E. (1984). Interannual variability of the Southern hemisphere circulation: Representativeness of the year of the global weather experiment. *Monthly Weather Review*, 112(1), 108–123. https://doi.org/10.1175/1520-0493(1984)112<0108:ivotsh>2.0.co;2;2

Voldoire, A. (2018). CMIP6 simulations of the CNRM-CERFACS based on CNRM-CM6-1 model for CMIP experiment historical [dataset]. *Earth System Grid Federation*. https://doi.org/10.22033/ESGF/CMIP6.4066

Volodin, E., Mortikov, E., Gritsun, A., Lykossov, V., Galin, V., Diansky, N., et al. (2019). INM INM-CM5-0 model output prepared for CMIP6 CMIP historical [dataset]. *Earth System Grid Federation*. https://doi.org/10.22033/ESGF/CMIP6.5070

Wieners, K.-H., Giorgetta, M., Jungclaus, J., Reick, C., Esch, M., Bittner, M., et al. (2019). MPI-M MPI-ESM1.2-LR model output prepared for CMIP6 CMIP historical [dataset]. *Earth System Grid Federation*. https://doi.org/10.22033/ESGF/CMIP6.6595

Williams, N. C., Scaife, A. A., & Screen, J. A. (2023). Underpredicted ENSO teleconnections in seasonal forecasts. *Geophysical Research Letters*, 50(5), e2022GL101689. https://doi.org/10.1029/2022GL101689

Woollings, T., Hannachi, A., & Hoskins, B. (2010). Variability of the North Atlantic eddy-driven jet stream. *Quarterly Journal of the Royal Meteorological Society*, 136(649), 856–868. https://doi.org/10.1002/qj.625