**JP4.27** **BRIDGING THE GAP BETWEEN WEATHER AND CLIMATE FORECASTING:**

**RESEARCH PRIORITIES FOR INTRA-SEASONAL PREDICTION**

Zoltan Toth \*^1^, Malaquias Pena Mendez ^2^, and Augustin Vintzileos
^3^

1\. Environmental Modeling Center, 2. SAIC at EMC 3. UCAR Visiting
Scientist at EMC

# 1. INTRODUCTION[^1]

On April 27, 2006, a small workshop was convened at the National Centers
for Environmental Prediction (NCEP) of the National Weather Service
(NWS), National Oceanic and Atmospheric Administration (NOAA), to
discuss the links between numerical weather and climate forecasting. The
workshop topic is a key area in broader discussions covering the
connections between weather and climate. The workshop was convened by
NOAA's THORPEX (THe Observing system Research and Predictability
Experiment) program, a World Weather Research Program under the World
Meteorological Organization, and was attended by a group of NOAA and
external experts (Table 1) along with interested NCEP scientists. This
report provides a brief summary of workshop discussions, along with a
list of open questions that the participants identified as most
important in bridging the gap between weather and climate forecasting.

# 2. BACKGROUND

The focus of NOAA's THORPEX program is to improve the skill and utility
of forecasts over the 3-14 day lead-time range. No hard barrier,
however, exists at day 14. Users require a forecast product suite that
is seamless across different time ranges; and scientifically speaking,
predicting weather at shorter ranges, or its various statistics at
longer time ranges is based on the same laws of physics.

In the past, various methodologies have been used to predict the weather
at different time ranges. Numerical prediction was first applied at
short time ranges. With improved initialization and modeling techniques,
the time range of useful NWP forecasts has been consistently expanding.
The goal of THORPEX is, in fact, to accelerate this expansion from the
current 7- to 10- day-limit out to 14 days by the introduction of a new
forecast paradigm.

Beyond the skilful range of NWP-based forecasting, an array of
statistical methods has been traditionally used for longer range
predictions. During the past decade, however, Numerical Climate
Prediction (NCP) activities have gained ground. Like the models used in
NWP, Atmospheric General Circulation Models (AGCM) are coupled with an
ocean and land surface model, for seasonal climate prediction
applications. Such NCP forecasts, after they are statistically corrected
for systematic errors, are now competitive with the best statistical
methods at predicting future climate conditions at and beyond 2 months
lead-time.

# 3. SIMILARITIES AND DIFFERENCES IN NUMERICAL WEATHER AND CLIMATE FORECASTING

The different sub-systems of the coupled Atmosphere -- Land surface --
Ocean (ALO) system continually interact with each other and at any one
time the conditions of a component depend not only on its own past but
also that of the other sub-systems. Of the three main subsystems, the
atmosphere, if uncoupled from the influence of the others, exhibits
changes on the fastest time scales (or alternatively, has the least
persistence or memory in a general sense). In contrast, the land surface
(for up to a season) and especially the ocean subsystem (yearly and
decadally) exhibit much slower time scales and more memory.
Correspondingly, the atmosphere is often referred to as the fast and the
ocean as the slow components of the coupled atmosphere -- land surface
-- ocean system.

Forecasts of future atmospheric conditions, irrespective of whether they
are for the shorter or longer (i.e., seasonal) lead-time ranges, attempt
to predict the same reality; the weather (or some statistics of it)
based on the same physical principles. In either case an initial value
problem is solved: the analyzed state of the system is projected into
the future. *An important observation is that at shorter lead times (say
for less than 7 days), the future state of the atmosphere is sensitive
mainly to the initial condition of the atmosphere, while at longer lead
times (say beyond 90 days) is sensitive to the initial condition of the
slow ocean (and possibly the intermediate land surface) component(s)
only.*

The current practice of using two different approaches for NWP and NCP
applications exploits the differences in sensitivity to initial
conditions as a function of lead times that were described above. In the
NWP application, the most accurate initial conditions are sought for the
atmosphere while ignoring or oversimplifying changes in the slowly
varying ocean conditions. In contrast, NCP applications focus on
capturing the initial conditions of the slowly varying component of the
coupled system (i.e., the ocean, land surface, and their atmospheric
response) at the expense of a poorer initialization and short range
forecasts of the rapidly varying components of the atmosphere.

# 4. OPPORTUNITIES FOR CONSOLIDATION

The current practice of using distinct approaches for weather and
seasonal climate forecasting work reasonably well for the shorter (less
than 7-day NWP) and longer (more than 90-day lead time NCP),
respectively. However, evidence is accumulating to suggest that neither
approach is tenable for the intermediate, 10-60 day, Intra-Seasonal (IS)
lead-time forecast range. Arguably, atmospheric conditions in this
intermediate lead time range, situated between the traditional weather
and seasonal climate ranges, are *influenced by initial conditions of
both the fast (atmosphere) and the slow and intermediate (ocean and land
surface) components of the coupled system*. Recognizing the limits of
both of the current approaches (NWP or NCP), one can seek further
forecast improvements by exploiting initial value information from both
the fast and slow components of the coupled atmosphere -- land surface
-- ocean system.

The 10-60 day Intra-Seasonal time range is a natural meeting ground
between scientists who have been primarily working on the shorter
weather or the longer climate applications. The weather and climate
forecast communities must work together to realize the full
predictability within the IS lead-time range. Converging and eventually
unifying weather and climate forecasting approaches that have been
developed somewhat separately over the past decades is a difficult task
not only from a scientific but also from a cultural point of view.

However, the enhanced collaboration and the ensuing closer ties between
the two communities have great potential. Beyond improving IS
forecasting, the collaboration may, at least in some situations, also
have a positive effect on short 1-7 day forecasts (e.g., improved
hurricane intensity forecasting due to more realistic ocean temperature
forecasts) and on longer than 60-day forecasts (e.g., capturing the
initiation or modulation of an ENSO cycle by an MJO). In addition, a
unified approach may also contribute to the establishment of a seamless
suite of probabilistic weather, water, and climate products, ranging
from hours to seasons ahead.

The NOAA THORPEX program will engage in collaborative research with all
interested partners in the weather and climate forecast communities to
achieve these challenging goals. Continued separate development of the
NWP and NCP approaches or the addition of yet a third new approach for
IS forecasting may in the short term bring improved performance. The
maintenance and continued development of two or three separate forecast
systems, however, has its own costs. And to realize the full potential
of weather and climate forecasting will eventually require the
development of a more unified approach. How to balance in operational
forecasting the goal of immediate skill improvements with the goal of
laying the groundwork for longer-term advances when the available
resources are limited is a challenge in itself.

The current dual approach and distinct techniques for weather and
climate forecasting reflect a knowledge gap in our understanding of the
coupled system and how it is captured in our numerical modeling systems.
To accelerate improvements in the 10-60 day forecasting range, the NOAA
THORPEX program promotes joint research by the weather and climate
communities aimed at closing the existing knowledge gap. The questions
below highlight the priority research areas for various aspects of IS
forecasting for the next 8-10 years.

# 5. OPEN SCIENCE QUESTIONS

# 5.1 *Observing systems:*

*What observations of the coupled atmosphere - land surface -- ocean
system are needed for capturing details of the initial conditions for
successful predictions in the 10-60 days lead-time range?*

Issues to be explored are which approaches are best to assess the status
of the divergent component of the atmosphere (possibly through enhanced
precipitation measurements in the tropics), the mixed layer of the ocean
(by assessing the depth of the mixed layer), the soil moisture in the
land surface system, the tropical-mid-latitude transport of energy
(heat, moisture, potential vorticity, etc), and the annual mode. More
accurate and comprehensive measurements of these processes, related to
fluxes and various balance constraints between variables in the coupled
system, can benefit both shorter and longer-range predictions. Observing
System Experiments (OSEs) and Observing System Simulation Experiments
(OSSEs) with coupled ALO models can help answer what new observations
may be most relevant for achieving these goals.

# 5.2 *Data assimilation:* 

*What are the best approaches that avoid the well-known problem of
initialization shock due to the typically strong model drift (spin-up)
characteristic of most coupled modeling systems?*

The answer to this complex question may well hold the key to successful
IS prediction. Even minor inconsistencies between the energetics of the
simulated and real sub-systems may lead to an unrealistic drift in the
behavior of the full coupled modeling system, also called
"initialization shock". Model related errors are clearly behind this
behavior. Such model behavior can destroy or mask vital information
contained in the initial condition of the coupled system, making
forecasting attempts in the IS time range futile. Novel approaches for
initializing imperfect models to capture the state of the real coupled
system on or close to the slow manifold of the model attractor may
fulfill the promise of successful IS forecasting. Can current data
assimilation methods that initialize the sub-components be separately
modified to resolve this problem? Or will new, possibly ensemble-based
methods, specifically for initializing coupled systems have to be
developed?

# 5.3 *Numerical model simulations:*

*What are the most promising ways for improving the realism of coupled
ALO models for IS prediction?*

What phenomena critical to IS forecasting are or are not represented in
numerical coupled ALO model simulations? Do NWP or NCP setups exhibit
more realism? Can simplified strategies, such as the use of mixed-layer
ocean models, play a role until the fundamental problems arising in more
complex coupled models are adequately addressed? Will better physics or
model coupling, or higher spatial resolution bring more benefits?

# 5.4 *Predictability:*

*How does information contained in the initial condition of the fast
(atmosphere) and slow (ocean) sub-components of the ALO system support
predictability in the IS range?*

A fundamental question behind IS predictability pertains to the level of
detail in the fast components of the coupled system ("weather") that
needs to be realistically represented in a forecast to successfully
predict the statistics of weather (i.e., shifts in the mean of the
system or changes in variability) at longer ranges. What is the level of
"weather noise" that needs to be simulated and/or forecast in order to
get the "climate signal" right? Since coupled model simulations do not
yet exhibit key phenomena such as the Madden-Julian waves with
convincing realism, vital information is lacking on the predictability
of such features. What the level of predictability is for these features
at different lead times, and what are the best ways of quantifying this
predictability are open questions. Is the use of complex coupled
numerical models for extended-range prediction always warranted, or can
observationally based studies, including those using empirically tuned
simple quasi-linear models, also play a role?

# 5.5 *Ensemble prediction:*

*What is the best forecast system configuration to realize the maximum
skill given the inherent limits in predictability of the coupled system
and the limitations in its numerical modeling?*

Through what procedures can the loss of skill due to imperfect initial
conditions and numerical models be captured and realistically propagated
throughout the IS forecast range? What are the relevant instabilities
responsible for the loss of predictability and skill? Does the use of
multiple models offer a solution to the problem of representing most
model related errors in an ensemble, or is there a need for
scientifically more innovative methods? What are the optimal trade-offs
between model complexity and ensemble size? Depending on the results
from predictability studies suggested above, may a cascading reduction
of model resolution with increasing lead time be a sensible way of
drastically reducing computational demand without sacrificing skill?
Without answers to these and similar questions, the full potential of IS
forecasting will not be realized.

# 5.6 *Post-processing:*

*What are the best methods of a posteriori enhancing the value of
numerical IS forecasts?*

Bias correction and spatial and temporal "downscaling" that make the
forecasts more relevant for users are important tasks. The limitations
of and potential for using statistical methods vs. running higher
resolution limited area model (LAM) simulations, forced by coarser
resolution coupled ALO models, for a posteriori enhancements of the
skill of IS forecasts are not well understood. Do LAM integrations offer
the most efficient use of computing resources, as compared to higher
resolution ALO forecasts followed by statistical corrections? What
observational and/or numerical analysis data are required for
statistical downscaling for regional or local applications? What is the
value added as the size of hind-casts for statistical post-processing is
increased? Would the cost of continually re-generating the hind-casts
with the newest model version (as compared to freezing the coupled ALO
model for a longer period of time and using a fixed set of hind-casts)
be compensated for by the incremental improvements due to continual
model upgrades? What is the added value to the users of communicating
forecasts post-processed in a generic manner? Which user groups can
benefit from additional, user specific statistical post-processing? It
is important to recognize that such post-processing requires, beyond the
actual forecast, communication of the full hind-cast dataset for proper
user specific processing. Will periodic improvements to a forecast
system associated with an appropriate hind-cast dataset pose a potential
problem in a complex user environment due to inadequate user responses
related to the updates?

# 5.7 *Socio-economic applications:*

*How can the existing gap between providers and users of IS forecasts be
narrowed?*

Despite continued efforts especially related to seasonal and longer term
forecasting, a major information barrier still exists between providers
and users of weather/climate forecasts. A typical user needs information
on multiple time scales from annual, seasonal, weekly, and daily down to
hourly lead time ranges, often first using the more skilful shorter
range forecasts. What issues need to be addressed for such a user, to
make their applications seamless? Can forecasts that are generated for
different lead time ranges by different forecast systems be made truly
seamless? What is the optimal form of forecast, considering multiple
potential applications? How can the needs of users with very different
levels of sophistication be best served? Would the use of multiple
formats (i.e., prepared generic products for basic users, to full
ensemble information for advanced users wishing to exploit the full
range of benefits from the nonlinear information present in an ensemble)
be practical and beneficial? Which are the best ways of communicating
forecast uncertainty? What tools are needed for a user to extract all
relevant information from an ensemble of forecasts?

# 5.8 *Value of forecasts:*

*What is the potential socio-economic value of improved forecast
information at different lead time ranges?*

This is an important question, especially for IS and longer lead time
forecasts with marginal skill. Who are the existing and potential users
of IS forecast information? Given limited resources, should
socio-economic applications research focus on a few key user groups such
as water management, or try to cover as broad a spectrum as possible? To
realistically assess their true value, it is critical that all forecasts
are also evaluated after post-processing, in a form as close as possible
to their actual use. Information on socio-economic value can also serve
as guidance regarding the choice of socially relevant summary
verification statistics. And to maximize value to the users, decisions
on upgrades to the system must be made based on the quality of
post-processed forecasts accessible to the user community. This is just
a partial list of open questions and critical issues, indicating the
depth of inquiries needed in the socio-economic applications research
area.

# 6. Acknowledgements

The contributions of all participants to the success of the workshop are
gratefully acknowledged. Josh Foster, John Gaynor, Melvyn Gelman, Tom
Hamill, and Hua-Lu Pan provided helpful comments on an earlier version
of this paper. The guidance and encouragement of Rick Rosen, Chair of
the NOAA THORPEX Executive Council, Louis Uccellini, Director of NCEP,
and Stephen Lord, Director of EMC, in the organization of the workshop
are also acknowledged.

+-------------+--------------------------------------------------------+
| NAME        | AFFILIATION                                            |
+-------------+--------------------------------------------------------+
| # *INVIT    |                                                        |
| ED EXPERTS* |                                                        |
+-------------+--------------------------------------------------------+
| David       | Environmental Modeling Center (EMC), NOAA/NWS/NCEP     |
| Behringer   |                                                        |
+-------------+--------------------------------------------------------+
| Ron Gelaro  | National Aeronautics & Space Administration (NASA),    |
|             | Goddard Space Flight Center                            |
+-------------+--------------------------------------------------------+
| Melvyn      | Climate Test Bed, NOAA/NWS/NCEP                        |
| Gelman      |                                                        |
+-------------+--------------------------------------------------------+
| Tom Hamill  | Physical Sciences Division, Earth System Research Lab, |
|             | NOAA                                                   |
+-------------+--------------------------------------------------------+
| Ben Kirtman | Center for Ocean-Land-Atmosphere Studies               |
+-------------+--------------------------------------------------------+
| Hua-Lu Pan  | Climate Test Bed, NOAA/NWS/NCEP                        |
+-------------+--------------------------------------------------------+
| Max Suarez  | NASA Goddard Space Flight Center                       |
+-------------+--------------------------------------------------------+
| Zoltan Toth | NOAA THORPEX program, NOAA/NWS/NCEP/EMC                |
+-------------+--------------------------------------------------------+
| Joe Tribbia | National Center for Atmospheric Research               |
+-------------+--------------------------------------------------------+
| Neil Ward   | International Research Institute for Climate and       |
|             | Society (IRI)                                          |
+-------------+--------------------------------------------------------+
| # *OTHER PA |                                                        |
| RTICIPANTS* |                                                        |
+-------------+--------------------------------------------------------+
| Jordan      | Environmental Modeling Center, NOAA/NWS/NCEP           |
| Alpert      |                                                        |
+-------------+--------------------------------------------------------+
| Josh Foster | Climate Program Office, NOAA                           |
+-------------+--------------------------------------------------------+
| John Gaynor | Office of Atmospheric Research, NOAA                   |
+-------------+--------------------------------------------------------+
| Arun Kumar  | Climate Prediction Center, NCEP                        |
+-------------+--------------------------------------------------------+
| Stephen     | Environmental Modeling Center, NOAA/NWS/NCEP           |
| Lord        |                                                        |
+-------------+--------------------------------------------------------+
| Ming Ji     | Climate Program Office, NOAA                           |
+-------------+--------------------------------------------------------+
| Ake         | SAIC at EMC/NCEP/NWS/NOAA                              |
| Johansson   |                                                        |
+-------------+--------------------------------------------------------+
| Shrinivas   | Environmental Modeling Center, NOAA/NWS/NCEP           |
| Moorthi     |                                                        |
+-------------+--------------------------------------------------------+
| Malaquias   | SAIC at Climate Prediction Center, NOAA/NWS/NCEP       |
| Pena        |                                                        |
+-------------+--------------------------------------------------------+
| Suranjana   | Environmental Modeling Center, NOAA/NWS/NCEP           |
| Saha        |                                                        |
+-------------+--------------------------------------------------------+
| Augustin    | Visiting Scientist, NOAA/NWS/NCEP/EMC                  |
| Vintzileos  |                                                        |
+-------------+--------------------------------------------------------+
| Jeff        | Physical Sciences Division, Earth System Research Lab, |
| Whitaker    | NOAA                                                   |
+-------------+--------------------------------------------------------+

Table 1. List of invited and other workshop participants

[^1]: ^\*^ *Corresponding author address:* Zoltan Toth, NOAA/NWS/NCEP,
    5200 Auth Rd., Rm. 207,

    Camp Springs, MD 20746; e-mail: Zoltan.Toth@noaa.gov