CHAPTER 3: Solutions

How does the weather enterprise implement the vision and goals set forth
in chapters one and two? This chapter describes the set of tasks needed
to fully implement uncertainty forecasts. For each of the tasks, we
provide a snapshot of how much progress the enterprise has made on this
task to date, a motivation for why more concerted effort is needed, a
concrete plan for implementing a task, and some inexpensive, short-term
steps that could be taken if the full solution cannot be implemented
given existing resources.

Three major sets of tasks will be described here, under the following
categories: (1) education, outreach, and social sciences; (2)
probabilistic forecast system development; and (3) supporting
infrastructure. The first set of tasks lays the groundwork for public
acceptance of new uncertainty guidance. A program of basic research is
proposed to inform meteorologists how to communicate uncertainty
information effectively. A process for ongoing dialog between
meteorologists and users regarding uncertainty information is described.
Modifications to university curricula to prepare forecasters and
would-be researchers for the new world of probabilistic forecasting are
proposed. Adjustments to K-12 education are proposed so that the next
generation has more of the statistical skills needed to understand, say,
a TV forecast with more probabilistic elements. Finally, a program for
training of operational forecasters is described.

The second set of tasks describes how to greatly improve uncertainty
guidance. The tasks include understanding and quantifying the
predictability of weather phenomena and providing basic research on how
to improve ensemble forecast techniques. A set of improvements to the
existing operational ensemble forecast systems is described. A program
for the regular post-processing of ensemble forecasts to ameliorate the
remaining systematic errors is also described. A robust probabilistic
verification system is proposed, as are the development of nowcasting
techniques for uncertainty forecasting. The new products developed here
should be rigorously tested and examined by potential users, and a test
bed is proposed to facilitate these interactions. The development of new
uncertainty products, of course, will not be the sole work of the public
sector; companies will also be developing tailored products. Finally,
whether we are considering television newscast graphics or a new NWS web
page, the formats in which we present the information to the public will
be crucial for their acceptance, and we describe a task dedicated to how
to present uncertainty information.

The last set of tasks describes the supporting infrastructure that must
be in place to fully implement the improved uncertainty forecasts. A
substantial investment in high-performance supercomputing is needed;
NCEP badly lags international competitors in the amount of CPU cycles
dedicated to ensemble forecast production. A comprehensive archive
facility is needed, as are improvements to the bandwidth and software
that will permit the ensemble data to be transferred efficiently.
Improvements to the NWS's AWIPS forecaster workstations will be needed
to allow forecasters to readily access and modify uncertainty guidance.
And for non-NWS users, some upgrade of their computers, storage, and
networking capabilities may be required to access and process the large
amounts of centrally produced uncertainty guidance.

These proposed tasks are meant to generally distinct from one another,
yet to solve a given problem, the work from many of these tasks must be
coordinated. Let us attempt to demonstrate how many of these tasks would
be linked to bring an important new product on line. Suppose an oil
tanker was severely damaged off the Oregon coast in a winter storm, with
several billions of dollars of ecological impact. Subsequent examination
showed that many ensemble members were able to suggest the possibility
of an extreme storm many days in advance of the deterministic forecast.
Through "education and outreach" (task 1.2), the NWS received feedback
that the implementation of uncertainty guidance related to wave heights
should be fast-tracked. The necessary changes to ensemble wave forecasts
were developed and tested in "probabilistic forecast system development"
(task 2.2), and remaining issues of reliability of the centralized
guidance are dealt with by developing appropriate statistical
post-processing methods (task 2.3). Verification software (task 2.4) was
used to validate the skill characteristics of the centralized
uncertainty guidance. Meanwhile, social scientists and meteorologists
have done the basic research to determine what may be the most effective
ways of communicating probabilistic ocean-wave information (task 1.1);
some mock display products were produced (task 2.8, "product
presentation"). Some sample forecast products for the storm in question
using the new display techniques were examined in a test-bed environment
(task 2.6) and found to be useful. Forecasters were trained on the
responsibilities of applying a quality-control check to the new
uncertainty wave guidance (task 1.5) and the deficiencies of the
automated products, and software was developed to allow a forecaster to
manually adjust the automated guidance if deemed necessary (task 3.4).
The uncertainty wave guidance was then successfully implemented.

The descriptions of the various sets of tasks now follows.

**Part 1:** **Education, Outreach, and Social Sciences**

**1.1** **Fundamental research on communication of uncertainty**

**Member**: Rebecca Morss ([**morss@ucar.edu**](mailto:morss@ucar.edu)),
others TBD.

### Overview of task:

Using best social science practices, explore/define best methods and
terminology for communicating uncertainty. Develop an understanding of
how a range of user groups interpret and use uncertainty information
communicated in different ways, and their uncertainty information
preferences and needs. Building on this understanding, develop knowledge
about how to effectively communicate forecast uncertainty (content,
terminology, format, etc.) to various audiences in different weather and
communication contexts. This will then be incorporated into the design
of operational uncertainty products (e.g., task 2.8).

### Background:

Most current weather forecast uncertainty products available in the
public domain are based on product developers' *guesses* at how
information should be communicated and/or on trial-and-error. While
uncertainty and risk communication has been studied in other contexts,
it is not apparent how this knowledge applies to communicating weather
forecast uncertainty, and limited research has been conducted on
communicating such uncertainty. Thus, we currently have limited
knowledge on effective communication of weather forecast uncertainty to
various audiences.

### Why needed:

If the weather enterprise does not know how to communicate uncertainty
effectively, audiences are unlikely to interpret and use the uncertainty
information as desired. At best, the ample forecast uncertainty
information available in the weather community will then continue to go
largely unused. At worst, uncertainty information will be misinterpreted
or misused, leading to poor decisions and negative outcomes.

### Solution:

Conducting this research will require funding to support the research
and entrain relevant experts and to develop a community of social
science, meteorology, and interdisciplinary researchers working in this
area. These goals can be reached by:

1\) NOAA, NSF, the Navy, and other government agencies a) funding calls
for proposals to examine and develop the most appropriate and effective
ways to communicate uncertainty to different audiences in different
contexts, and b) including communication of weather forecast uncertainty
as a topic in existing calls for research.

2\) AMS or universities facilitating building public/private consortiums
for funding this research

3\) A kickoff workshop to entrain new social science and
interdisciplinary researchers into this area, discussing existing
knowledge and research questions and methods, and building
collaborations

4\) AMS hosting a special conference or symposium at its annual meetings
for research community in this area to get together with each other and
forecast providers to cross-fertilize findings (perhaps through the
Policy and Socioeconomic Research Symposium and/or Board on Societal
Impacts).

5\) Researchers deliver findings to meteorological community on how
users prefer to receive uncertainty information for a spectrum of
products (rainfall, severe weather, daily temperatures, etc.)

**Quick, short-term steps**: To be filled in later.

### Linkages to other subgroups

**Needs, opportunities (subgroup 1):**

TBD

**Goals (subgroup 2):**

Supports goals related to providing products based on end user
requirements (goal 3), effectively communicating to aid decision making
(goal 4)

**Roles, responsibilities (subgroup 4):**

Funding agencies will need to provide funding for research, perhaps
private and public/academic sectors working together.

**Roadmap (subgroup 5):**

This research needs to start early, so that we can build understanding
to be applied in developing products in a few years.

### Linkages to subgroup 3 topics

1.1, Basic research on communication of uncertainty: This topic

1.2, Outreach and communication: Research in this topic can also help
develop education mechanisms (based on what users current do/don't
understand and their preferences and needs) and can inform what

1.3, Undergraduate/graduate education: Useful to incorporate knowledge
of people's understanding of uncertainty information as a small
component of curriculum, for students who will become forecasters or
product developers.

1.4, K-12 education: NA

1.5, Forecaster training: Include training on people's understanding of
uncertainty information

2.1, Basic ensemble research: NA

2.2, Ensemble forecast system development: Indirect link in that
developers will want to design forecast system so that it can provide
the uncertainty information people want and can use (based on this
research)

2.3, Statistical post-processing: Want to post-process ensemble (and
other) data so that it provides useful, wanted uncertainty information
(based on this research)

2.4, Verification: Incorporate people's needs for uncertainty
information into user-relevant verification

2.5, Nowcasting: NA

2.6, Test Beds: Test beds can provide one place where this research can
happen.

2.7, Tailored product development: Knowledge developed in this task will
help develop effective products

2.8, Product presentation: Close link between research in this topic and
learning how to present information effectively.

3.1, Supercomputers: NA

3.2, Comprehensive archive: Archive can be used to create sample
products for use in this research

3.3, Data Access: NA

3.4, Forecast preparation systems: Want forecast preparation systems
designed such that they facilitate forecasters developing products that
communicate uncertainty effectively.

3.5, User infrastructure: NA

### General notes:

**1.2** **Outreach and communication**

**Members:** Paul Heppner (3SI, Pheppner@3sinc.com), Julie Demuth
(NCAR), Tom Dulong (NOAA), Bill Bua (NOAA), Jon Ahlquist (FSU)

**Overview of Task:**

Constitute a continuing formal and informal dialog with customers and
the public to better understand customer needs for uncertainty
information, and for educating customers on new products, how they may
be used, and how well the products perform.  This dialog will inform the
development of new uncertainty products and improve the usage of
existing ones.

**Background:**

"Outreach" is the communication of ideas or principles to diverse groups
or communities. Meteorological outreach can be *within* the profession
(one discipline to another) or *between* the profession and the broader
community, including the general public, educators, scientists, and
government administrators. Policy makers and planners need to understand
uncertainties or probabilities in order to make informed decisions,
which may affect people's lives or community planning. Outreach provides
the opportunity for meteorologists to educate ***and learn from*** their
users.

An example of outreach from the meteorological and social science
community to users is the [WAS\*IS](http://www.sip.ucar.edu/wasis/)
(Weather and Society, Integrating Studies), hosted by NCAR. NCAR's
[COMET](http://www.comet.ucar.edu/) (Cooperative Program for Operational
Meteorology, Education, and Training) also provides an example of
outreach and training. One component of COMET consists of outreach from
the academic community to expose forecasters to the state-of-the-art
research in forecasting. Another part of COMET provides learning modules
put together by experts and available over the web for use in classroom
settings. Many universities also have meteorological outreach programs.
For example, the University of Wisconsin provides a [four-day
workshop](http://cimss.ssec.wisc.edu/smrwkshp/index.htm) for prospective
students to learn about the disciplines of meteorology, astronomy, land
remote sensing, and geology.

**Why Needed**

Despite the examples provided above, there is little done right now in
the way of outreach related to uncertainty forecasting. It can be
expected that many users will be unfamiliar with uncertainty products
and how they can improve decisions. While the basic research in
communicating uncertainty (task 1.1) will help meteorologists understand
some of the basic principles about how to communicate uncertainty, there
is no substitute for regular feedback: how are we doing? Are these
products satisfying your most urgent needs for uncertainty information,
or is some adjustment in the type of product or the format of the
product needed? Can we assist you in understanding what information the
product is conveying, and why the forecast is uncertain?

The biggest challenge in outreach and communication with respect to
uncertainty in weather and climate forecasts is liable to be with the
general public. This is mainly the result of their misunderstanding of
basic probability and statistics, and NWS and other providers'
probabilistic weather and climate forecast product definitions.

**Solutions**

\(1\) Certified Broadcast Meteorologists (CBMs), who act as station
scientists at TV and radio stations, should be trained to communicate
uncertainty in the forecast through the certification continuing
education process. A short course on the use of probabilistic models
should be developed through the AMS and then presented at large-venue
meetings of the CBMs. The CBMs would then serve as trainers on
probabilistic forecasting through their regular outreach to the public.

\(2\) Early versions of uncertainty products by the NWS should have,
incorporated into the web pages, a feedback form that permits users to
indicate what they understood/didn't understand. The NWS should
institute a process for gathering this information, sorting through the
comments, and determining whether changes to the products are warranted
based on these comments.

\(3\) New NWS web pages that incorporate uncertainty information can
provide links to some training material on the uncertainty information;
how it is generated, how it is to be interpreted, etc..

**Quick, short-term steps**: To be filled in later.

**Notes:**

The publication *"Completing the Forecast: Characterizing and
Communicating Uncertainty for Better Decisions Using Weather and Climate
Forecasts"* (National Reseach Council, 2006) provides useful reference
and motivation ; see chapter 4, section 4.4 in particular.

### Linkages to other subgroups

**Needs, opportunities (subgroup 1):** TBD

**Goals (subgroup 2):** TBD

**Roles, responsibilities (subgroup 4):** TBD

**Roadmap (subgroup 5):** TBD

### Linkages to subgroup 3 topics

1.1, Basic research on communication of uncertainty: Close connections
between this topic and basic research on communication of uncertainty to
frame the dialog with the public and other customers to understand
needs. Basic research will help us understand users' preferences and
needs.

1.2, Outreach and communication: This topic

1.3, Undergraduate/graduate education : Education at college level is a
form of outreach to the communicators of uncertainty.

1.4, K-12 education: Increase awareness of basics in statistics and
probability to help public understand uncertainty in weather forecasts.

1.5, Forecaster training: Outreach to forecasters on how to communicate
uncertainty to their end users using existing and new forecast systems,
and help forecasters understand how to use the probabilistic tools they
have and that are developed in the future. Additionally, need to
outreach to forecasters regarding the training that is already
available. Training developers need to be responsive to forecaster needs
for training on new and existing products.

2.1, Basic ensemble research : NA

2.2, Ensemble forecast system development: NA

2.3, Statistical post-processing : Need products that will help in
outreach and communication.

2.4, Verification: Tools need to assess that probabilistic products are
providing the best uncertainty and probability information possible.

2.5, Nowcasting: NA

2.6, Test Beds: NA

2.7, Tailored product development: Knowledge developed in this task will
help develop effective products

2.8, Product presentation: Product presentation is part of outreach and
communication.

3.1, Supercomputers: NA

3.2, Comprehensive archive: NA

3.3, Data Access: NA

3.4, Forecast preparation systems: Training covers understanding
forecast preparation systems.

3.5, User infrastructure: NA

### General notes: 

**1.3 Undergraduate and graduate education**

**Members**: Jon Ahlquist (ahlquist@met.fsu.edu), Dave Myrick

**Overview of task:**

Educate students in undergraduate and graduate geophysical science
programs on chaos theory and the fundamentals of ensemble prediction,
probabilistic forecasting, and the use of uncertainty guidance.

**Background:**

The course requirements for undergraduate and graduate degrees in
meteorology / atmospheric science vary by college/university. For
accreditation by the American Meteorological Society (AMS), bachelor
degree programs must include a suite of courses that cover a broad range
of topics in atmospheric science, as well as prerequisites in calculus,
physics, statistics, chemistry, computer science, professional writing
and oral communication ( http://tinyurl.com/5ueo9s ). Similar
requirements are necessary to obtain employment by the United States
federal government. The requirements leave a lot of room for each
college/university to develop unique course content. Thus, there are no
specific requirements related to interpreting ensemble, probabilistic
and uncertainty forecast information. At the graduate level, course
requirements are determined by the local meteorology/atmospheric
sciences department.

**Why needed**:

Newly hired meteorologists will be expected to interpret ensemble,
probabilistic and uncertainty forecast products. Students who are not
versed in these topics will be less marketable, and those that are hired
but who are uninformed will require extra training (e.g., see task 1.5,
"forecaster training"). Further, classroom education may encourage some
graduate students to consider doing research on improving uncertainty
forecasts (see task 2.1, "basic ensemble research").

**Solutions:**

Undergraduate and graduate meteorology/atmospheric science departments
should include material in their courses on understanding and
interpreting ensemble, probabilistic and uncertainty forecast products.

At the undergraduate level, while it would be highly beneficial, there
is currently no room to add an additional course to the curriculum set
forth by the AMS. It is thus necessary to find ways to assimilate new
material into courses that are currently offered. Typically,
undergraduate courses include a synoptic/weather forecasting laboratory.
Such a class would be particularly suitable for introducing students to
ensemble concepts. The course content might include a brief description
of chaos theory and model error, the basics of how ensemble prediction
methods allow estimation of uncertainty from these sources, followed by
a lab session comparing ensemble guidance from various systems for a
particular storm, or storms, with an evaluation of the strengths and
weaknesses of the ensemble guidance.

Graduate programs in atmospheric sciences, which have more flexibility
than undergraduate programs, *should require students to take a basic
probability and statistics course* because of its broad utility to both
forecasting and data analysis.  Ideally, the students would also have
the opportunity to take a course in statistical meteorology at the level
of Wilks' *Statistical Methods in the Atmospheric Sciences*. Further, if
not covered in a statistical meteorology course, a graduate-level course
in dynamic meteorology should include a section that covers chaos
theory. Numerical weather prediction courses should discuss ensemble
prediction methods, including methods for generating ensembles of
initial conditions and model error in ensemble forecasts.

To support these efforts, a platform on the internet should be built for
educators to share resources. This platform will serve as a bridge until
textbooks can be updated to cover this material. The COMET Program
([http://www.comet.ucar.edu](http://www.comet.ucar.edu/)) is one natural
place that could be used to post materials and lectures. An example of
this, using Moodle software, can be found at
\[http://courses.comet.ucar.edu\] and clicking on COMET Faculty
Multimedia Workshop. COMET modules could also be developed that are
geared directly to students that could be used to support or motivate
synoptic meteorology laboratory assignments. Such modules are already
quality controlled by multiple reviews before being allowed to be
published.

**Quick, short-term steps**: To be filled in later.

**Linkages with other solution components**:

### Linkages to other subgroups

**Needs, opportunities (subgroup 1):**

Some solution components will be directly connected to particular needs
and opportunities.

**Goals (subgroup 2):**

Goal IV, "effectively communicate and educate multiple end users..." ;
and more specifically, "hydrometeorological curricula should include
understanding and communication of risk and uncertainty (undergraduate
to graduate studies)

**Roles, responsibilities (subgroup 4):**

AMS very active in this task, as a representative of the entire
enterprise

**Roadmap (subgroup 5):**

Really need buy-in of enterprise and commitment to follow through on
this before lobbying colleges and universities

### Linkages to subgroup 3 topics

The default is NA for each of these, but in practice you will populate
many of them.

1.1, Fundamental research on communication of uncertainty: NA

1.2 Outreach and communication: Also concerned with educating the
public, K-12, on uncertainty, so materials can be shared.

1.3, Undergraduate/graduate education: NA

1.4, K-12 education: NA

1.5, Forecaster training: Can share some course training material.

2.1, Basic ensemble research : NA

2.2, Ensemble forecast system development: NA

2.3, Statistical post-processing: NA

2.4, Verification: NA

2.5, Nowcasting: NA

2.6, Test beds: NA

2.7, Tailored product development: NA

2.8, Product presentation: NA

3.1, Supercomputers: NA

3.2, Comprehensive archive: NA

3.3, Data Access: NA

3.4, Forecast preparation systems: NA

3.5, User infrastructure: NA

### General notes: 

**1.4 K-12 education**

Members: Tom Hamill (<tom.hamill@noaa.gov>); Jimmy Yun, Dan Stillman

### Overview of task:

To utilize new uncertainty information, it will be helpful to ensure
that K-12 students understand some of the background concepts such as
probability. The AMS will contact state school boards to suggest
emphasizing more probability and statistics in high-school math
curricula, and will develop and disseminate some relevant meteorology-
and uncertainty-related problems to textbook manufacturers.

**Background:**

National math education standards dictate that students in grades K-12
"should be able to develop and evaluate inferences and predictions that
are based on data" and "understand and apply basic concepts of
probability." The National Mathematics Advisory Panel recently
[recommended](http://www.ed.gov/about/bdscomm/list/mathpanel/report/final-report.pdf)
that high school algebra include material on combinatorics and finite
probability. It is likely that the topic of uncertainty and use of
probabilities in weather information only arises if students in math
class happen to be given a probability example that has to do with
weather; this is unlikely to happen unless the meteorological community
can affect the content of textbooks that are used.

Many meteorological organizations are already involved in K-12
education, and it may be useful to leverage these already existing
relationships Organizations include:

\(1\) UCAR/NCAR: Education and Outreach (<http://eo.ucar.edu/>);
COMET/MetEd (<http://www.meted.ucar.edu/comm_k12.htm>) and links
therein. Digital Library for Earth System Education (DLESE)
([http://www.dlese.org](http://www.dlese.org/)); UCAR Digital Image
Library (<http://www.fin.ucar.edu/ucardil/>).

\(2\) NSF National Science Digital Library
([http://nsdl.org](http://nsdl.org/))

\(3\) NOAA Office of Education
([http://www.oesd.noaa.gov](http://www.oesd.noaa.gov/))

\(4\) Universities: University of Illinois Urbana-Champaign Urban
Extension (<http://www.urbanext.uiuc.edu/kalani/>); Florida State
University EXPLORES! (<http://www.met.fsu.edu/explores/>)

\(5\) Private: How the Weatherworks
([http://www.weatherworks.com](http://www.weatherworks.com/)); The
Weather Channel (<http://www.weather.com/wxclass/education/>); USA Today
Web Science for Teachers (http://www.usatoday.com/weather/wteach.htm)

### Why needed:

"Completing the Forecast" recommended that uncertainty information be
incorporated into all products disseminated to the weather forecast
user. Without uncertainty education and training, many will be unable to
fully use this information. Exposure to the basic concepts of
probability and statistics as a child or adolescent, with some salient
weather examples, may help students grow into educated forecast users,
more capable of utilizing the extra uncertainty information.

### Solution:

This solution presupposes a general acceptance of the ACUF
recommendations and that the enterprise is going to follow through. If
so, then:

\(1\) The AMS would initiate contact with state school boards across the
country, providing information on how weather products were going to
change to incorporate uncertainty information, and suggesting that it
would be useful to further emphasize probability and statistics in the
high-school math curricula.

\(2\) AMS develops sample problems that illustrate the concepts of
probability and statistics in meteorology and weather forecasting; these
are put into existing libraries like DLESE.

\(3\) AMS contacts major textbook manufacturers, again indicating how
weather products are going to change to incorporate uncertainty
information. AMS encourages them to incorporate examples contained in
the repository mentioned in (2).

**Quick, short-term steps**: To be filled in later.

## Notes

### Linkages to other subgroups

**Needs, opportunities (subgroup 1):**

Some solution components will be directly connected to particular needs
and opportunities.

**Goals (subgroup 2):**

Goal IV, "effectively communicate and educate multiple end users..." ;
and more specifically, "hydrometeorological curricula should include
understanding and communication of risk and uncertainty (K-12 up to
graduate studies)

**Roles, responsibilities (subgroup 4):**

AMS very active in this task, as a representative of the entire
enterprise

**Roadmap (subgroup 5):**

Really need buy-in of enterprise and commitment to follow through on
this before lobbying school boards and textbook manufacturers on this.

### Linkages to subgroup 3 topics

The default is NA for each of these, but in practice you will populate
many of them.

1.1, Fundamental research on communication of uncertainty: NA

1.2, Outreach and communication: Also concerned with educating the
public, post K-12, on uncertainty, so materials can be shared.

1.3, Undergraduate/graduate education: NA

1.4, K-12 education: NA

1.5, Forecaster training: NA

2.1 Basic ensemble research: NA

2.2, Ensemble forecast system development: NA

2.3, Statistical post-processing: NA

2.4, Verification: NA

2.5, Nowcasting: NA

2.6, Test beds: NA

2.7, Tailored product development: NA

2.8, Product presentation: NA

3.1, Supercomputers: NA

3.2, Comprehensive archive: NA

3.3, Data Access: NA

3.4, Forecast preparation systems: NA

3.5, User infrastructure : NA

### General notes:

**1.5 Forecaster training** 

**Members:** Bill Bua (<Bill.Bua@noaa.gov>), Tom Dulong, Dave Myrick,
Daniel Nietfield

**Overview of task:** Train operational forecasters[^1] to (1)
intelligently use probabilistic forecasting tools from ensemble
prediction systems (EPS) and (2) compose probabilistic forecast products
for end users in both the private and public sectors. This training must
be done at all time and spatial scales from continental and seasonal
scales to county and short-fuse warning scales for severe local storms
and tornadoes.

**Background:** Training on the *theoretical* basis for EPS has been
developed, for example, at UCAR's Cooperative Program for Meteorological
Education and Training (COMET) \[http://www.comet/ucar/edu\], the
Meteorological Service of Canada (MSC)\[http://tinyurl.com/56j5pz\] and
the European Center for Medium-Range Forecasting (ECMWF)
\[http://tinyurl.com/57q9o7\]. The COMET training is included in the
NOAA Learning Management System as well, thus allowing for feedback to
the learner on how well they understand this theoretical material.

While some NWS forecast offices have taken initiative to develop
training cases using EPS, insufficient training has been developed for
use of EPS in the forecast process and other operational applications.
Moreover, such efforts have not typically been shared, perhaps because
they are locally based and it is assumed that these cases will not have
applicability in other locations.

Additionally, the operational forecasting culture in the NWS and
elsewhere continues to generally be deterministic, though there has been
some experimental probabilistic forecasting done in the public and
private sectors. No generally accepted format for delivering
probabilistic forecasts to the public is currently in place. This
includes the widely used National Digital Forecast Database (NDFD)
produced by NWS. Without universally accepted formats for probabilistic
forecasting in place, it is difficult to develop training in this area.
It should be noted, however, that in the past forecasters have been able
to add some value to local forecasts from their experience with local
mesoscale weather features not in the coarser resolution NWP output,
which may not be recovered by automated statistical correction and/or
downscaling.

**Why needed:** EPS represent the best application of our understanding
of atmospheric behavior to numerical weather prediction (NWP) and
probabilistic forecasting. Additionally, probabilistic forecasting,
especially beyond the nowcast range, provides the best science-based
information to the various public and private users of weather
forecasts. But even good science can be misused or ignored. The nonuse
or misuse of EPSs and the forecast tools created from them can and often
will result in poor forecasts, particularly for high-impact, extreme
weather events.

To avoid misuse of these probabilistic tools, forecasters must have
detailed knowledge of the general underlying theory behind and
performance of EPS systems, the weaknesses in the current operational
system, what can and can not be corrected with statistical
post-processing, and how all this changes as the model and
post-processing systems change.

**Solutions:** UCAR/COMET, the NWS Warning Decision Training Branch
(WDTB), and the NWS Training Center (NWSTC), along with other interested
parties, need to be involved in, and coordinate, the development of
operational forecasting-based training on the use of EPS as a tool to be
used in the forecast process. Development of training on practical uses
of EPS and related products in the forecast process for various high
impact weather phenomena would include use of the Weather Event
Simulator (WES) and other real-time simulation media.

Once a format for the expression of uncertainty is decided upon by the
NWS and others, probabilistic product training will also need to be
developed. When and where practical, develop training on expression of
uncertainty to end-users of forecast products (depends on time line for
development of such products). Access to data will be needed to develop
WES cases. Interaction with field is necessary to identify good cases
for training.

The following resources for training development and testing would be
required:

(1) Qualified instructional designers and subject matter
    experts/scientists on EPS and other probabilistic forecast tools
    from UCAR/COMET, the NWS Forecast Decision Training Branch, the
    WDTB, perhaps the NWSTC, and from academia.

(2) Reviewers and testers of training from operational environments,
    probably WFO and NWS National Centers for Environmental Prediction
    (NCEP) SOOs, and private industry training staff.

For the receivers of training, required resources would include:

(1) Time for training at WFO and in private industry settings.

(2) Sufficient budgets for

    a.  Travel to training when necessary, and for

    b.  Servers to host remote training.

(3) Access to operational tools in use for probabilistic forecast
    development, preferably as they are being developed.

(4) Benchmarking of training to other disciplines that use probabilistic
    forecasting such as insurance, marketing and finance.

**Quick, short-term steps**: To be filled in later.

**1.5 Notes**

### Linkages to other subgroups

**Needs, opportunities (subgroup 1):** TBD

**Goals (subgroup 2):** TBD

**Roles, responsibilities (subgroup 4):**

Primarily by the academic community and research laboratories.
Operational prediction facilities should facilitate visiting scientists
who have developed improved techniques offline to visit and work
alongside operational developers to further test the ideas in
operational models.

**Linkages with other solution components**

1.1, Fundamental research on communication of uncertainty: NA

1.2, Outreach and communication: understanding forecasters' and end
users' product and format needs will inform the training required for
them. Additionally, forecasters have to be aware of customer uses of
probabilistic products for economic, public safety, and other
decision-making.

1.3, Undergrad/grad education: NA

1.4, K-12 education:NA

1.5, Forecaster training: this task.

2.1, Basic ensemble research: NA

2.2, Ensemble forecast system development*:* As EPS development
continues, forecasters must understand the advantages and limitations of
new EPS methods and their implications with respect to the use of EPS in
the forecast process. Experimental guidance from the upgraded ensemble
forecast systems will be subjectively evaluated by forecasters and
users. Good evaluation requires training operational forecasters on EPS.

2.3, Statistical post-processing: Forecasters must be trained in how new
post-processing tools are developed, along with such tools' advantages
and limitations.

2.4, Verification: Experimental guidance will need to be objectively
evaluated relative to existing forecast products before implementation,
and monitored thereafter.

2.5, Nowcasting: NA

2.6, Test beds: NA

2.7, Tailored product development: NA

2.8, Product presentation: NA

3.1, Supercomputers: NA

3.2, Comprehensive archive: To develop training at the forecast office
or private enterprise level, retrospective data is often helpful. Its
availability to such entities, and how to obtain such data, is important
to training development.

3.3, Data access: NA

3.4, Forecast preparation systems. NWS forecasters and private users
rely on AWIPS and other systems in the forecast process. Training on EPS
must correspond to its availability in AWIPS and elsewhere on the
internet (e.g. web pages produced by NCEP and private enterprises).
"Knobology" on the "how to" of display of EPS products is particularly
important to AWIPS users, and must be timed with new AWIPS build
releases.

3.5, User infrastructure: NA

**Part 2:** **Probabilistic Forecast System Development**

**2.1: Basic ensemble research**

Jim Hansen, <jim.hansen@nrlmry.navy.mil>, lead. Tom Hamill and Zoltan
Toth, members.

**Overview of task:**

Perform research to (1) quantify the extent to which a phenomena at a
given lead time has information in excess of what is provided by the
climatology, and the timescale at which all predictable signal, and (2)
improve the generation of ensembles.

**Background**:

Systematic model error and the growth of initial condition errors due to
chaos each degrade the skill of ensemble forecasts. Even in a future
hypothetical scenario where the systematic errors have been rendered
negligible, the skill of ensemble forecasts for a particular phenomenon
will be constrained by the innate time scale of predictability of the
phenomenon in question, which may be a function of the scale of the
phenomenon and the synoptic and climatological conditions. Estimates of
predictability have been obtained by several methods; one common method
is to integrate multiple numerical simulations from perturbed initial
conditions. The length of predictability is estimated as the timescale
at which the differences are as large as the differences between two
model states. However, the predictability estimates may depend strongly
upon the model used. More optimistic, longer estimates of the time scale
of predictability are obtained when using simpler models computed at
lower resolutions.

Basic research has clarified some of the general concepts of how to
generate ensemble forecasts. For example, there is now a general theory
of how to generate appropriate initial conditions for ensemble
forecasts. However, continued research is needed to develop numerical
methods that are computationally efficient and yet approximate the
optimal approach indicated by theory. Similarly, methods for
incorporating the uncertainty caused by systematic model errors are in
their infancy, with rather basic approaches employed. The relative
tradeoffs between applying extra computational resources to add more
ensemble members vs. increase model resolution vs. improve model
numerics is not well understood for the gamut of possible customer
applications; more members decreases the sampling error associated with
spread estimates, but if those more members are from a lower-resolution
model, there may be a systematic underestimate of spread.

**Why needed**:

Regarding the quantification of predictability, uncertainty products are
going to be requested across a range of spatial and time scales. Before
the development of a product (e.g., the probability of a tornado at the
scale of a county at one week lead), it is important to know whether it
is theoretically feasible to produce a forecast that will have any
skill. Resources should not be allocated to projects that intend to
provide forecasts that cannot be rendered skillful. Directed
predictability research will help the community better estimate what are
the innate time scales of predictability for the phenomena of interest,
thereby improving the allocation of resources in product development.

Regarding improving the techniques for generating ensembles, the results
can be applied in the probabilistic forecast system development task
(see task 2.2 below) at operational centers, with a downstream
improvement of the raw ensemble forecast guidance.

**Solution:**

For predictability estimation, the general steps are:

\(1\) Identify phenomena that are of the greatest predictive interest;
(2) perform predictability studies to estimate the lead time at which
forecasts have no greater skill than climatology, and (3) decide what
products get developed based on (2).

For improving the generation of ensembles, the basic research may follow
a general path, such as: (1) identify a particular deficiency in
existing ensemble forecast systems; (2) generate a hypothesis for how
this deficiency could be ameliorated. (3) Develop new techniques in
accordance with this hypothesis, (4) Perform experiments, comparing the
new technique alongside existing techniques to determine if the new
technique is superior. (4) If the new technique is superior (i.e.,
hypothesis confirmed), work with operational model developers (see task
2.2) to implement the technique, otherwise refine the hypothesis and
repeat.

**Quick, short-term steps**: To be filled in later.

**2.1** **Notes**

### Linkages to other subgroups

**Needs, opportunities (subgroup 1):** TBD

**Goals (subgroup 2):** TBD

**Roles, responsibilities (subgroup 4):**

Primarily by the academic community and research laboratories.
Operational prediction facilities should facilitate visiting scientists
who have developed improved techniques offline to visit and work
alongside operational developers to further test the ideas in
operational models.

**Roadmap (subgroup 5):**

Ideally, the predictability component would be done early on, so it can
inform which uncertainty products should be developed, and in which
order; otherwise, a rank ordering of which products get developed and
which do not may be done in a seat-of-the-pants fashion.

For the generation of ensembles, this is expected to be an ongoing,
nearly continuous process over the span of time the ACUF is considering.

**Linkages with other solution components**

1.1, Fundamental research on communication of uncertainty: NA

1.2, Outreach and communication: User priorities may motivate research
in a particular area, e.g., development of hurricane ensemble
initialization techniques.

1.3, Undergrad/graduate education: Basic predictability theory, informed
by results here, should be covered in grad dynamics course.

1.4, K-12 Education: NA

1.5, Forecaster Training: Important for forecasters to be informed about
predictability theory, develop an understanding of what is predictable
(they concentrate on this) and what is not (ignore this).

2.1, Basic Ensemble Research: this task.

2.2, Ensemble forecast system development: high-resolution models that
may be used in predictability studies may be suitable for usage in
next-generation ensemble systems. Techniques developed here will be
considered for tech transfer to operations.

2.3, Statistical Post-processing: time scale at which skill scores
asymptote to zero, a result from statistical post-processing using
verification techniques (2.4) may provide another gross estimate of
predictability

2.4, Verification: Should have consistent estimates of when skill trends
to zero and when predictability vanishes...a sanity check.

2.5, Nowcasting: NA

2.6, Test beds: Experimental products from this task may be considered
in test beds, though perhaps it may be better to wait until the products
are tested by the operational prediction facilities in task 2.2.

2.7, Tailored Product Development: NA

2.8, Product presentation: NA

3.1, Supercomputers: needed for this task.

3.2, Comprehensive archive: Predictability forecast data should be
archived, but not necessarily in this comprehensive archive.

3.3, Data access: NA

3.4, Forecast preparation systems: NA

3.5, User infrastructure: NA

**2.2: Operational ensemble forecast system development**

Zoltan Toth, <zoltan.toth@noaa.gov>, lead. Members Tom Hamill, John
Schaake, Paul Nutter, Jun Du

**Overview of task:**

Improve the quality and usefulness of the operational global and
regional ensemble forecast systems. Theses may include a large
refinement of the ensemble model's resolution (followed by regular
resolution improvements thereafter); incorporating research results from
task 2.1 (basic ensemble research) into the operational systsem; greater
sharing of forecast data between operational facilities; and adding a
new, limited-area, high-resolution, high-impact event regional ensemble
system.

**Background**:

NCEP currently runs a 21-member, \~ 100-150 km grid spacing global
ensemble forecast system 4 times daily to 16 days lead (its "global
ensemble forecast system, or "GEFS"); a 32 to 45-km grid spacing,
21-member multi-model limited-area ensemble to 87 h (its short-range
ensemble forecast, or "SREF" system); and NCEP has experimentally
evaluated a 10-member small-domain \~4 km grid spacing ensemble for
severe weather applications on the Great Plains. The US Navy and Air
Force also have their own suites of operational forecast models, and
some university run quasi-operational ensemble systems (e.g., the
University of Washington). Focusing on NCEP, these ensemble forecast
systems are the NWS' primary tools for providing uncertainty estimates
that vary with the weather situation of the day. NCEP's systems are
currently much coarser than the current global standard. For comparison,
ECMWF's 2008 medium-range global ensemble forecasts are computed at
three times higher horizontal resolution, with more than double the
members and double the vertical resolution, using \~100 times the
computational resources for the raw forecast alone, not including the
data assimilation and reforecasts (see task 2.3).

**Why needed**:

Goal #1 asks the enterprise to deliver a comprehensive suite of
uncertainty products that are a reliable and skillful as practically
possible (goal #2). Except at the shortest leads, when nowcasting
strategies are more appropriate (task 2.5), ensemble forecast systems
will be the bulwark of probabilistic forecast products. Unfortunately,
the raw output from the existing ensemble forecast systems do not
provide reliable probabilistic forecasts in all circumstances. There are
several root causes, including: (1) systematic model errors. NCEP's
forecasts are computed at a coarse resolution relative to other forecast
centers, so important weather phenomena such as squall lines and
hurricanes are not properly resolved. Also, the parameterizations of
sub-gridscale physical processes (deep convection, boundary layer,
surface layer, land surface, microphysics) often provide unrealistic
estimates of the effects of the unresolved scales of motion, and their
uncertainty. (2) initialization of the ensemble with model states that
do not properly represent the uncertainty in the analysis.

**Solution:** NOAA[^2] must develop an improved suite of ensemble
forecast systems tailored to the nation's highest-impact weather
problems that reduce the ensemble system deficiencies. These systems
would incorporate improved model physical parameterizations and
initialization techniques and would use much higher-resolution models.
Specifically, NOAA should:

\(1\) Work with extramural basic researchers and NOAA lab partners that
have developed improvements in experimental ensemble forecast systems
(see task 2.1) to test these ideas in operational models.

\(2\) Develop a much higher-resolution GEFS, with improved model physics
and initialization techniques, and possibly multiple dynamical cores. To
provide state-of-the-art forecasts, NOAA should seek to increase GEFS
resolution threefold by 2011-2012 and to implement the best ensemble
initialization technique from a range of competitors. Comparisons with
the much higher resolution ECMWF ensemble forecasts suggest that such
improvements should result in a general 1-day improvement in forecast
lead (i.e., a 5-day forecast as skillful as a current 4-day forecast),
with greater improvements for many severe-weather phenomena. NOAA should
plan on doubling its ensemble forecast system horizontal resolution
thereafter approximately every 8 years, consistent with Moore's Law.

\(2\) Share global ensemble forecast model output among agencies (e.g.,
NOAA, Navy, Air Force) and governments (Canada and others) through
projects such as the North American Ensemble Forecast System (NAEFS),
National Unified Operational Prediction Capability (NUOPC) and
TIGGE/GIFS (THORPEX Interactive Grand Global Ensemble/Global Integrated
Forecast System). Recent multi-model experiments have indicated the
potential of such data sharing.

\(3\) Upgrade its SREF system threefold, to 10-20 km grid spacing by
2011-2012 with hourly output, and with a doubling of horizontal
resolution every eight years thereafter. Develop an advanced method for
perturbing the initial conditions that is consistent with the strategy
for the GEFS but which also includes realistic mesoscale differences in
the initial and lateral boundary conditions.

\(4\) Develop and implement a relocatable, high-resolution, explicit
convection (\~4-km grid spacing) limited-area ensemble forecast system
for hurricanes, severe local storms, and fire weather that can be nested
inside the SREF or GEFS systems, and appropriate methods for
initializing them.

\(5\) Upgrade its hydrologic forecast models, so that the meteorological
weather input into a hydrologic ensemble prediction system will produce
more reliable streamflow forecasts with less need of post-processing.

**Quick, short-term steps**: To be filled in later.

**2.b Notes**

This section ideally ought to include material relevant to the
development of hydrological models, also. Need input from John Schaake
and crew.

### Linkages to other subgroups

**Needs, opportunities (subgroup 1):** TBD

**Goals (subgroup 2):**

Goal 1 asks for comprehensive suite of products that realistically
cannot be achieved without updated ensemble forecast systems.

Goal 2 stresses need for reliable, skillful forecast output. Improved
ensemble systems will minimize the need for post-processing.

Goal 5 notes that the comprehensive archive of the improved forecasts is
needed to support statistical post-processing.

Goal 6 indicates that improved lead times for significant weather are
desired, with less false alarms. Improved ensemble forecast systems are
designed to help meet this.

**Roles, responsibilities (subgroup 4):**

Considering NOAA specifically, since NCEP/EMC produces the operational
forecasts, model development is expected to be primarily done at
NCEP/EMC. Basic experimental technique development will be done by the
external research community (see task 2.1), with more applied technique
development also performed within NOAA by personnel at ESRL, NSSL and
SPC, AOML, and OHD. NOAA THORPEX has been a major funding vehicle for
this research in the past few years.

**Roadmap (subgroup 5):**

In ideal world, this work would be fast-tracked, and output from
improved, stable model(s) would be available for statistical
post-processing task. (2C)

**Linkages with other solution components**

1.1, Fundamental research on communication of uncertainty: NA

1.2, Outreach and communication: There may be some suggested products
that may feed back into what priorities are applied in the ensemble
system development. For example, development of improved hurricane
ensemble forecasts may be emphasized based on feedback from users.

1.3, Undergrad/graduate education: NA

1.4, K-12 Education: NA

1.5, Forecaster Training: Forecaster will require ongoing education on
the capabilities and remaining weaknesses of the ensemble forecast
systems as they change.

2.1, Basic Predictability Research: Will improve our understanding of
predictability and the limits of ensemble forecast system development,
and may develop new forecast techniques that can be transitioned into
future operational ensemble forecast systems, e.g. stochastic physics
parameterizations.

2.2, Ensemble forecast system development: This task

2.3, Statistical Post-processing: Each major improvement to the ensemble
forecast systems will require a reworking of the statistical
post-processing, since even improved ensemble forecast systems will have
some systematic errors and will need post-hoc correction based on
discrepancies between prior forecasts and observations/analyses.

2.4, Verification: Experimental guidance will need to be objectively
evaluated relative to existing forecast products before implementation,
and monitored thereafter.

2.5, Nowcasting: Need to determine lead time when nowcasts should cease
to be used, ensemble forecasts should be used.

2.6 Test Beds: Experimental guidance from the upgraded ensemble forecast
systems will be subjectively evaluated by forecasters and users.

2.7, Tailored Product Development: Private companies will need to
monitor changes in the forecast system and how it affects the tailored
applications they develop for their customers. Through test beds (2.6),
they also have an opportunity to recommend changes.

2.8, Product presentation: NA

3.1, Supercomputers: A fifty-fold or more increase in high-performance
computing allocated to ensembles will be needed to host the improved
ensemble forecast systems. 8) 3.2, Comprehensive archive: Archives must
be planned to accommodate future resolution and output frequency
increases.

3.3, Data access: increases in resolution and temporal frequency of
ensemble forecasts will increase the required bandwidth to disseminate
to NWS forecasters and private users.

3.4, Forecast preparation systems: The NWS AWIPS-II system will need to
track the formats and resolutions of the evolving ensemble forecast
systems.

3.5, User infrastructure: TBD

## 2.3: Statistical post-processing. 

## **Members:** Tom Hamill, <tom.hamill@noaa.gov>, lead. Members Zoltan Toth, Paul Schultz, Bob Glahn.

### Overview of task:

Produce the most skillful and reliable objective uncertainty guidance
practicable through the statistical post-processing ("calibration") of
numerical forecast guidance, based primarily on discrepancies between
previous forecasts and observations/analyses. This task encompasses the
development and testing of calibration techniques appropriate across the
range of high-impact events, the production of supporting data sets
(e.g., reforecasts), and the operational implementation of these
algorithms.

### Background:

Dissemination of statistically based guidance in the NWS has a long
history, via "Model Output Statistics" (MOS). MOS utilizes regression
relationships based on prior forecasts and observations, statistically
adjusting the current numerical guidance. These regression relationships
correct for model bias and can implicitly perform a statistical
downscaling (adjusting the sub-gridscale weather to be colder and wetter
on the mountain peaks inside a model grid box, for example). Most
existing MOS products are deterministic, though implicitly the MOS
regression analyses produce uncertainty forecast information, and this
guidance could be disseminated with little additional effort as a
baseline uncertainty product. More recently, with the advent of
ensemble-prediction techniques, a variety of new extensions to the basic
MOS techniques have been developed that leverage the extra information
in the ensemble. In situations where the ensemble mean provides a more
accurate estimate than a deterministic forecast (this characteristic is
common at leads longer than a day or two), then ensemble-MOS techniques
may improve estimates of most likely outcome. If the ensemble spread
provides information on the situation- dependent uncertainty (high
spread is related to lower forecast skill, low spread to higher skill),
then ensemble-MOS techniques may provide improved, weather-dependent
uncertainty estimates as well. Ensemble-based calibration techniques are
currently being developed and tested at many NOAA facilities (for
example, MDL, NCEP, ESRL, OHD), and at many universities.

For every-day weather such as surface temperature at short forecast
leads, a variety of MOS calibration techniques appear to be relatively
competitive in their performance, and small training samples appear to
be adequate. For rare events and long-lead forecasts, more training
samples (perhaps many years or even decades of "reforecasts" from a
stable model/data assimilation system) are needed, and the calibration
technique of choice may depend upon the particular application and data
at hand. The need for large training data sets from a stable model
unfortunately conflicts with the desire to rapidly implement
improvements in ensemble forecast systems. There are two possible
compromises: (1) maintain one model that is rapidly upgraded but
unaccompanied by reforecasts and another model that is only occasionally
updated (say, every third year) but which has an accompanying reforecast
data set. Forecasters could choose which products to utilize; and (2)
conduct a more limited set of reforecasts in real time for whatever
model is being run operationally, thereby devoting a percentage of the
CPU time for the ensemble forecast system to the accompanying
reforecasts (ECMWF currently utilizes this approach).

### Why needed:

Uncertainty guidance must be reliable and skillful in order to be widely
used and accepted. There are some important limitations in using current
MOS-based uncertainty guidance and in the guidance produced directly
from unadjusted ensemble output. Existing MOS guidance is based mostly
on deterministic forecast model output rather than an ensemble and could
be improved through inclusion of ensemble information. Uncertainty
estimates from the unadjusted ensemble are liable to be sub-optimal due
to the small ensemble size, the biases inherent in the forecast model(s)
and methods for initializing the ensemble, and the comparatively coarse
grid spacing used in current ensemble prediction systems. While improved
ensemble forecast systems (task 2b) may lessen the need for
post-processing, it will never completely eliminate it. A comprehensive
program is thus needed to determine the optimal calibration techniques
across the spectrum of high-impact applications, develop routine methods
for computing the supporting data sets, and implementing the calibration
techniques.

### Solution:

\(1\) Decide which forecast variables (e.g., temperature, wind speed,
precipitation) are to be considered for calibration. This task should be
done in consultation with users (outreach, task 1a). ACUF enterprise
goal#1 provides a suggested list that will be used to initiate the
discussion. (2) Survey the literature on existing calibration techniques
for each of the parameters, as well as requirements the supporting data
sets required. (3) Perform an objective comparison of the techniques for
each forecast variable to determine which is the most appropriate. If
none of the existing calibration techniques appear to be appropriate,
develop and test new, more appropriate calibration techniques. If the
performance of the calibration technique strongly depends on the
training sample size, determine which compromises in reforecast sample
size (e.g., shrinking the number of reforecast members, or skipping days
between reforecasts) lead to the least degradation. (4) Evaluate the
prototypes of calibrated forecast products (see task 1B) using standard
verification tools (see task 2D). (5) Implement the calibration
techniques. If the calibration techniques require reforecast data sets,
implement reforecast techniques, and archive the reforecasts (task 3b).

**Quick, short-term steps**: To be filled in later.

## 2.c: Notes

### Linkages to other subgroups

**Needs, opportunities (subgroup 1):** TBD

**Goals (subgroup 2):**

Goal 2 stresses need for reliable, skillful forecast output, which
cannot be achieved regularly without calibration.

Goal 5 notes that the comprehensive archive is needed to support
statistical post-processing.

Goal 6 indicates that improved lead times for significant weather are
desired, with less false alarms. Statistical post-processing should help
achieve this.

**Roles, responsibilities (subgroup 4):**

Suggest that NOAA/MDL, NOAA/NCEP/EMC, and NOAA/ESRL, at the very least,
will play active roles in the objective comparison part of the solution
above; each group will submit candidate techniques that they believe are
appropriate. Note that NOAA THORPEX has actively funded research in this
area in the past few years, and that this task is consistent with goals
of that program. See

**Roadmap (subgroup 5):**

### Linkages to subgroup 3 topics

1.1, Fundamental research on communication of uncertainty: NA

1.2, Outreach and communication: Input from users will determine which
forecast parameters are in greatest demand for calibration.

1.3, Undergraduate/graduate education: NA

1.4, K-12 education: NA

1.5, Forecaster training: NA

2.1 Basic ensemble research: Estimates of the time scale of usable
predictability can be compared with skill results of post-processed
forecasts as a sanity check. Should decrease to zero skill at
approximately the same lead.

2.2, Operational ensemble forecast system development: Reforecasts
needed here competes with improved ensemble forecast systems at higher
resolutions. Tradeoffs that achieve greatest benefit to customer need to
be determined.

2.3, Statistical post-processing (2C): this task.

2.4, Verification: Verification tools developed in this task will be
used in the evaluation of output from prototype calibration techniques.

2.5, Nowcasting: NA

2.6, Test beds: Used for evaluation of prototypes of calibration
techniques.

2.7, Tailored product development: In some sense this is the non-NWS
component of this task. General issues are similar.

2.8, Product presentation: candidate visualization techniques applied to
post-processed forecasts will be evaluated in 2.6, test beds.

3.1 Supercomputers: Requirements for reforecast data set are an
important consideration in the sizing of future supercomputer purchases.

3.2 Comprehensive archive: This will hold the reforecasts used in this
task.

3.3, Data Access: Utilize increased bandwidth to download model
forecasts, obs.

3.4, Forecast preparation systems (3D): One concept of operations for
the forecast prep systems is that forecasters will have reforecast data
sets at their fingertips with algorithms that allow them to find past
cases with weather scenarios similar to current weather scenario.
Reforecast data sets developed in this task could be used.

3.5 User infrastructure: NA

### General notes: 

## 2.4: Verification 

## **Members:** David Myrick, <David.Myrick@noaa.gov>, lead. Dan Nietfield, Dan Stillman, Barbara Brown, Tom Hamill

### Overview of task:

Develop new uncertainty verification procedures for new uncertainty
products. Constitute a group of experts and develop an uncertainty
verification manual, describing how to implement a standardized set of
uncertainty verification methods. Develop and freely disseminate a
software library of verification and verification display procedures
appropriate to uncertainty forecast products. Implement a "verification
clearing house" where uncertainty forecast verification statistics are
collated and displayed.

### Background:

Uncertainty verification techniques are needed to evaluate uncertainty
forecasts. The statistics provide feedback to ensemble prediction system
developers, forecasters, and end-product developers, and may be used by
end users to aid in product interpretation and decision making.

Many verification methodologies are currently used to evaluate
uncertainty forecast products. These include measures of reliability
such as the rank histogram and reliability diagrams, measures that
incorporate probabilistic forecast accuracy such as the Brier Score,
Ranked Probability Skill Score, and the Relative Operating
Characteristic, and end-user relevant metrics such as Potential Economic
Value diagrams. Despite this, there is no universally agreed-upon set of
metrics that provide a complete diagnosis of uncertainty forecast
evaluation.

Availability of verification statistics varies widely. For operational
forecasts, a common but not universal practice is to post verification
scores to a web page. For ensemble prediction systems, model developers
may calculate scores they believe to be of greatest relevance.
Intercomparisons are complicated by the use of different data sets by
different model developers, verification on different grids, or
comparisons of ensembles of different sizes. Verification information
from non-governmental forecast producers typically is difficult to
obtain.

**Why needed:**

A standardized set of verification codes and agreed-upon practices is
needed in order to facilitate rapid development of improved ensemble
prediction systems and to provide forecasters and users with
quantitative information that will facilitate the proper use of the
uncertainty forecast information. New, standardized verification
procedures will be required for new uncertainty products such as those
that describe the uncertainty in the timing of an event. Without
verification, the enterprise cannot ensure that we meet the goal of
reliable, skillful forecasts.

### Solution:

\(1\) New uncertainty forecast metrics for new uncertainty products must
be developed, tested, and peer reviewed.

\(2\) A team of uncertainty verification experts should be formed to:
(a) identify a standard, scientifically defensible set of methodologies
for the evaluation of uncertainty forecast products, (b) evaluate new
metrics and determine whether they should become part of the baseline,
and (c) based on (a) and (b), develop and publish an "*uncertainty
verification manual*" that specifically indicates how each metric is to
be computed. Periodically thereafter, this team will meet to review new
verification developments and determine whether the manual should be
expanded or changed.

\(3\) Software developers should build a standardized library of
routines based on the specifications in the uncertainty verification
manual, as well as software for the convenient display of verification
data. This software will be made publicly available as freeware.

\(4\) Relevant parts of the weather enterprise (model developers,
forecasters, applications developers) use the toolkit for verification,
and make the verification data publicly available.

\(5\) Institute a verification "clearing house" where the synthesized
verification results from various components (raw ensemble guidance,
objectively post-processed guidance, official forecasts, and end
products) are all gathered in one place and made available.

## 2.4: Notes

### Linkages to other subgroups

**Needs, opportunities (subgroup 1):**

TBD

**Goals (subgroup 2):**

Goal 2 requires standardized uncertainty verification in order to ensure
that products are as skillful and reliable as possible.

**Roles, responsibilities (subgroup 4):**

Funding from NOAA, Navy/Air Force, and/or NSF?.

Building the team of experts and "clearing house" should be a
collaborative effort between National Centers (NOAA/NCEP, CMC, UKMet,
ECMWF, Navy, Air Force) and other experts (e.g., WMO verification
programs such as the Joint Working Group on Verification,
http://www.wmo.int/pages/prog/arep/wwrp/fvr.html). Potentially this may
be a role that could be coordinated by the WMO project THORPEX, as other
countries would be just as interested in the techniques and software
library as the US enterprise.

Will need software developers to actually develop the verification tools
and display library; whether this is done at NOAA, shared among US
agencies, or perhaps done internationally via THORPEX is TBD. Also need
to find organizations who will run the "clearing house".

**Roadmap (subgroup 5):**

\(1\) Need to get some idea early on about the scope of new uncertainty
products, such as those related to timing of events, and to quickly fund
work to develop and test verification metrics appropriate to these new
products.

\(2\) Team of experts should get to work relatively quickly, circulate
draft manual perhaps at T+1 year after start of whole project.

\(3\) Software development perhaps T+1.5 years to T+3 years.\
(4) Plans will need to be made as to how the "clearing house" is built
and funded.

### Linkages to subgroup 3 topics

1.1, Fundamental research on communication of uncertainty: NA

1.2, Outreach and communication: NA

1.3, Undergraduate/grad education: NA

1.4, K-12 education: NA

1.5, Forecaster training: Forecasters will want to see verification
statistics for new and experimental systems to help them understand
systematic errors, when they need to modify forecasts.

2.1, Basic ensemble research:

2.2, Operational ensemble forecast system development: need to pump
existing, experimental forecasts through verification package to
evaluate.

2.3, Statistical post-processing: Verification a crucial part of whether
stat post-processing is having a beneficial effect.

2.4, Verification: This task.

2.5, Nowcasting: will need to be verified, too.

2.6, Test beds: All users and developers of ensemble and probabilistic
forecast information will need to be trained on how to interpret
verification information.

2.7, Tailored product development: these products need verification,
too, and verification routines should be general enough to work for the
unusual products as well as the usual.

2.8, Product presentation: NA

3.1, Supercomputers: Modest amount of computing resources will need to
be allocated to conduct verification calculations.

3.2, Comprehensive archive: "Clearing House" probably should archive
verification stats alongside the other archive materials (forecasts,
observations) needed for verification.

3.3, Data Access: Sufficient bandwidth will be required to large obtain
forecast/obs datasets.

3.4, Forecast preparation system: Forecasters will want access to
verification stats.

3.5, User infrastructure (3E): NA

### General notes: 

## 2.5: Nowcasting 

## Tom Hamill, <tom.hamill@noaa.gov>, others TBD.

### Overview of task:

Develop observationally based probabilistic forecast tools that are
useful from minutes to several hours forecast lead, at timescales
numerical ensemble prediction techniques may be less useful. Since the
usefulness of these observationally based techniques can be expected to
drop off with increasing lead (relative to ensemble-based products),
this task also includes the development of approaches for variably
weighting observationally based and dynamically based tools as a
function of forecast lead.

### Background:

Much progress has been made in numerical weather prediction and ensemble
forecasting, to permit their use in short-range forecasting, including
the use of higher-resolution models, initialization with radar data, and
the diabatic initialization of forecasts. Nonetheless, for the immediate
future model guidance can be expected to be subject to "spin-up"
problems, whereby the first few hours of a forecast may be inaccurate as
the forecast goes through an adjustment period of developing internally
consistent vertical motions. At slightly longer leads (beyond several
hours), there is increasing evidence that high-resolution ensemble
forecast systems may be able to provide useful forecast information.

Several research groups have developed more observationally based tools
for making nowcasts. For severe weather, these tools may involve the
detection of significant features using radar or other remotely sensed
data; the extrapolation of this data using wind or derived motion
vectors; and perhaps some development/decay mechanism. An example of
this is NCAR/RAL's AutoNowcaster. Another nowcast system in the western
US makes short-range extreme precipitation forecasts using expected wind
and moisture flux information and thermodynamic stability, estimating
how much precipitation may occur as a column interacts with the complex
terrain along the US west coast. As demonstrated with these two
examples, typically the nowcast tools may be application- and
geographically specific.

Unfortunately, little work has been performed to date to develop
probabilistic nowcast tools; most have a more deterministic cast to
them, though there is some scant literature on the application of
Bayesian statistical methods to nowcasting.

**Why needed:**

### The NRC report recommends the inclusion of uncertainty information throughout the spectrum of forecast products, including those at the shortest leads. Unfortunately, in the near, term the techniques that are appropriate at 24 h lead may not be appropriate at 2 min or 2 h lead, so development of probabilistic nowcast techniques is required. 

### Solution:

\(1\) Develop probabilistic nowcasting tools that rely heavily on recent
observed data. Perhaps tools like the AutoNowcaster can be modified to
permit other discrete realizations based on realistic changes to the
estimated propagation speed or rate of development/dissipation. In this
manner, an ensemble of possible future scenarios is obtained. If there
are systematic biases in the output, perhaps statistical post-processing
(task 2.3) can improved these in the same manner that they improve the
raw ensemble forecast guidance. As mentioned above, it is unlikely that
there will be any one-size-fits-all nowcast scheme; perhaps different
methods will be developed tailored to different customers and different
locations.

\(2\) Develop tools for blending together the nowcast and numerically
based guidance as forecast lead increases. At the earliest leads,
nowcasts would be heavily weighted, and at longer leads the numerical
guidance would receive increasingly large weight.

**Quick, short-term steps**: To be filled in later.

## 2.d: Notes

### Linkages to other subgroups

**Needs, opportunities (subgroup 1):**

**Goals (subgroup 2):**

**Roles, responsibilities (subgroup 4):**

Best done locally.

**Roadmap (subgroup 5):**

### Linkages to subgroup 3 topics

1.1, Fundamental research on communication of uncertainty: NA

1.2, Outreach and communication: NA

1.3, Undergraduate/grad education: NA

1.4, K-12 education: NA

1.5, Forecaster training: NA

2.1, Basic ensemble research: NA

2.2, Operational ensemble forecast system development: NA.

2.3, Statistical post-processing: NA.

2.4, Verification: NA.

2.5, Nowcasting: This task.

2.6, Test beds: NA.

2.7, Tailored product development: NA.

2.8, Product presentation: NA

3.1, Supercomputers: NA.

3.2, Comprehensive archive: NA.

3.3, Data Access: NA.

3.4, Forecast preparation system: NA.

3.5, User infrastructure (3E): NA

### General notes:

## 2.6: Test Beds 

**Members**: Bill Bua (UCAR/COMET; bill.bua@noaa.gov), Julie Demuth
(NCAR/ISSE), Rebecca Morss (NCAR), Tom Hamill (NOAA/ESRL)

### Overview of task:

Implement an ensemble product test bed, a place where model developers,
forecasters, and users can interact prior to product implementation to
discuss the efficacy of experimental uncertainty products and
visualization techniques. Feedback from the test bed would be collated
by an independent third party and may result in an "implement"
recommendation for a new product, or a "needs more refinement." Such
objective feedback is especially crucial for the new suite of
uncertainty products, where standards and expectations are not yet
developed.

### Background:

Only a very *informal* test bed now exists at NCEP for examination and
feedback on experimental ensemble prediction systems (EPS) and related
products pre-implementation. This test bed consists of NCEP making data
available for review, with data provided either via the web (graphical
products) or pulled in through ftp by participants. Data is not yet
available via NWS forecaster workstations (AWIPS). Operational
forecasters at NWS national centers and some WFOs participate. The
testing and evaluation is not monitored by an objective third party
independent of operations.

**Why needed:**

The NRC report "*Completing the Forecast*" recommended that uncertainty
information should be provided across the whole suite of weather and
climate forecast products. The rewards that may be gained from effective
use of this additional uncertainty information are likely to be fully
realized only when users understand the product, how to use it (see
tasks 1.1 and 1.2), and have a chance to evaluate experimental products
and determine whether it suits their needs.

There is currently no facility that permits users (e.g., operational NWS
and private sector forecasters, emergency managers, other officials
responsible for public safety, utility companies, general public) to
conveniently evaluate and critique experimental products. A test bed
avoids the hazards of testing in a *live* production environment, and
provides a forum for feedback among all providers and users *before*
operational implementation.

### Solution:

Two general possibilities exist for a test bed: (1) an on-site facility
or, (2) a virtual test bed, with users and developers interacting
remotely. If the on-site approach is desired, a facility must be
maintained that includes fast workstations that have a large amount of
storage capacity, a laboratory environment that facilitates interaction,
and projection displays. For remote capabilities, meeting software must
be procured. For either on-site or remote approaches, software must be
developed to access and display the experimental forecasts for a large
number of cases. The software should also be able to display verifying
observations, verification statistics, and perhaps the underlying data
such as the unprocessed ensemble forecasts. For both approaches, a small
staff of meteorologists and social scientists will be needed to develop
test software, facilitate the interaction between developers,
forecasters, and users, and to implement a formalized method for
evaluating uncertainty products, collating feedback, and making
impartial recommendations.

A test bed experiment would be initiated several months prior to a
product becoming ready for evaluation. Software would be tested on the
experimental forecasts, a test date set, and test bed staff would
arrange for a suite of users to be available on that date. Experimental
products and other data sets would then be made available for a wide
variety of cases. The technique developer would begin by making a
presentation on the experimental product, providing some comparison
against existing baseline products, if available. Users would then have
the chance to examine experimental forecast products over a large number
of cases, e.g., across a representative sample of synoptic situations
and times of the year. The users would be asked to provide formalized
feedback (e.g., a 1-5 rating of the product) and less formal feedback,
such as written comments on what was liked/disliked, and suggestions for
improvement. The test bed would conclude with a discussion between users
and developers and the issuance of a formal recommendation.

## 2.d: Notes

### Linkages to other subgroups

**Needs, opportunities (subgroup 1):**

**Goals (subgroup 2):**

Goals are to provide a comprehensive suite of uncertainty products
(goal 1) ... so lots of products to run through the test bed! Test bed
can help confirm (goal 2) that products are deemed to be reliable and
skillful. Test bed ensures that products will be delivered in formats
based on user requirements and in wide array of formats convenient to
them (goals 3, 4).

**Roles, responsibilities (subgroup 4):**

For the basic NWS product suite, the development of products and
staffing of the test bed may be a collaborative effort between
NOAA/NCEP, MDL, NOAA OAR labs, NCAR/UCAR, and universities, perhaps
through NCAR/UCAR connections. If private companies wanted to have their
products evaluated in the test bed, they would be collaborators as well.
Users would come from a wide variety of sources, from NWS forecasters to
the general public.

**Roadmap (subgroup 5):**

### Linkages to subgroup 3 topics

1.1, Fundamental research on communication of uncertainty: Developers
should have incorporated lessons learned from this task prior to
submitting experimental products to the test bed.

1.2, Outreach and communication: Feeds into the development of the
products to be submitted to the test bed.

1.3, Undergraduate/grad education: Grad students might make good,
informed guinea pigs for test bed.

1.4, K-12 education:

1.5, Forecaster training: complementary to test beds; forecasters who
get exposed to experimental products in test bed will have a leg up in
training.

2.1, Basic ensemble research: NA

2.2, Operational ensemble forecast system development: Generates the
ensembles that feed into test bed products.

2.3, Statistical post-processing: Generates the derived statistical
products that feed into the test bed.

2.4, Verification: Stats should be made available for experimental
products to the test bed.

2.5, Nowcasting: These products should also be evaluated in test bed.

2.6, Test beds: This task.

2.7, Tailored product development: If private sector wishes, their
products could be run through such a test bed, providing they are
willing to have data shared publicly, results made publicly available.

2.8, Product presentation: This provides the look and feel to the
products to be evaluated in the test bed, filtered through 2.2, 2.3.

3.1, Supercomputers: NA.

3.2, Comprehensive archive: Will supply experimental forecasts.

3.3, Data Access: Will be needed to transfer experimental forecasts.

3.4, Forecast preparation systems: Human-QC'ed products generated here
should be evaluated alongside the objective guidance.

3.5, User infrastructure (3E): NA

### General notes:

## 2.7: Tailored product development

### **Members**: Ross Hoffman (rhoffman@aer.com), Paul Heppner, Bill Bua

### Overview of task:

Develop specialized uncertainty products to meet the needs of specific
forecast user communities needs, such as the energy industry or
television broadcasters. Also, provide novel visualization approaches
for public or specialized web pages (see 2F). For the most part,
commercial products will be tailored by the private sector specific to
their commercial clients.

### Background:

Most tailored forecasts are still deterministic, but some probabilistic
offerings are now available. For example, at AER, Inc. ensemble
forecasts are post-processed for energy traders. Tailored probabilistic
forecasting is an area set to take off. Novel visualizations include,
for example, the NHC cone of uncertainty, below:

![](media/image1.png){width="3.004166666666667in"
height="2.3993055555555554in"}

### Why needed:

Many customers will not have their uncertainty forecast requirements met
by the baseline set of products that will be produced by the NWS. For
these customers to make optimal decisions, reliable and skillful
probability forecasts are required that are tailored to match the
required inputs of their decision processes. Visualizations tailored for
specific applications can aid in quick, accurate decisions.

### Solution:

Solution methods are likely to be somewhat specific to each particular
provider and customer, and each will require close communication to
define a viable interface for communicating uncertainty.

The AMS or similar organization can facilitate the more rapid
development of tailored products by gathering and promoting the
publication of best practices and success stories in this field.

Private-sector products are likely to be based upon numerical ensemble
forecast information provided by the NWS. Since private customers may
want products that do not change radically in characteristics from one
day to the next, the NWS will need to provide advanced warning of
modeling system changes. Ideally the NWS would make available training
data such as reforecasts in advance of implementation.

**Quick, short-term steps**: To be filled in later.

## 2E: Notes

### Linkages to other subgroups

**Needs, opportunities (subgroup 1):**

[TBD.]{.mark}

**Goals (subgroup 2):**

Tailored products should be skillful and reliable (goal 2) and meet end
user requirements (goal 3) using formats catered to the end user (goal
4).

**Roles, responsibilities (subgroup 4):**

This activity will occur in private industry, and perhaps in some other
government agencies (e.g., USDA).

**Roadmap (subgroup 5):**

[TBD.]{.mark}

### Linkages to subgroup 3 topics

1.1, Fundamental research on communication of uncertainty. Lessons
learned here apply as much in private as public sector.

1.2, Outreach and communication: Private sector will do their own sort
of outreach, i.e., marketing.

1.3, Undergraduate/graduate education: Trained individuals needed to
staff this activity.

1.4, K-12 education: NA

1.5, Forecaster training : NA

2.1, Basic ensemble research: NA

2.2, Ensemble forecast system development: Improved numerical guidance
should improve tailored products. Will need to track changes in this for
new tailored product opportunities.

2.3, Statistical post-processing: Will need to track changes in this for
new tailored product opportunities. Reforecast data sets produced in
this task may be beneficial.

2.4, Verification: Will utilize these tools to demonstrate utility of
products.

2.5, Nowcasting: NA

2.6, Test beds: NA

2.7, Tailored product development: THIS TOPIC

2.8, Product presentation: The best graphical techniques should be
applicable regardless of whether product is public/private sector.

3.1, Supercomputers (3A): NA

3.2 Comprehensive archive: Will make use of historical data for testing
and development.

3.3, Data Access: Fast access a necessity for real time data. The
ability to parse subsets for smaller locations of interest will be a big
plus.

3.4, Forecast preparation systems: NA

3.5, User infrastructure: Must accommodate ensemble data to enable this
activity at the tailoring institution.

A major requirement for this activity is readily accessible archives
including most data (see 3.2). Historical reforecast data are needed
because for particular cases the ensemble post-processing that was done
by NOAA will be redone with specific definitions of optimality and with
customer-specific verification data.

A second major requirement is fast and fair access since business
decisions can be extremely time sensitive.

### General notes:

These questions are more general than 2E, but they came up in
discussions here.

a.  How do we deal with the requirement of commercial entities to keep
    some data and methods proprietary?

b.  Would publications be limited because of proprietary issues in some
    cases for the private sector? Can the basic science/results be
    presented while maintaining "trade secrets"?

c.  How would this relate to the "test bed" section for experimental
    products?

    -   The test bed should allow for experimentation by the private
        sector.

    -   As a simulation of a real system the test bed should allow users
        to download and post-process data at their own facility.

##  

## 2.8: Product presentation

**Members:** Gina Eosco (<gme7@cornell.edu>) and Jon Ahlquist
(<ahlquist@met.fsu.edu>) leads. Members Thomas LeFebvre, R. Weeks, N.
Johnson, Paul Heppner, Dan Nietfield.

### Overview of task:

Determine specific formats for uncertainty products that do the best job
possible of conveying the breadth of uncertainty information iconically,
graphically, and/or numerically. Also, for NWS applications, design a
general "look-and-feel" for presentation of the products so that there
is consistency, to the extent possible, common across diverse locations,
product types, etc.. This task leverages the research performed in task
1.1, "fundamental research on communication of uncertainty," and uses
test beds, task 2.6, for product testing.

### Background:

For general weather forecast information, the NWS has designed a
standard weather forecast page format; see <http://tinyurl.com/29wvor>
for an example. This page contains a mix of iconic, numeric, text,
imagery, and even a small amount of probabilistic information, conveyed
through the PoP. There is also generally a consistent format across many
of the NCEP Climate Prediction Center products, e.g., the color scheme
for 6-10 day forecast uncertainty products at <http://tinyurl.com/34ocd>
is the same general format as for a 3-month forecast. While the NWS home
pages currently provide only a small amount of uncertainty information,
its consistency from one location to the next makes it easy for users.
This consistency should be emulated as these pages are upgraded to
provide additional uncertainty information.

The NRC report "*Completing the Forecast*" provided some ideas about how
probabilistic information could be conveyed effectively. The World
Meteorological Organization issued a publication entitled "Guidelines
for Communicating Forecast Uncertainty" (<http://tinyurl.com/676dyd>).
These documents are a starting point for a complex process of designing
appealing new web pages and web services for uncertainty products.

### Why needed:

Effective product presentation formats will be crucial to generate
acceptance and widespread usage of the new products. Similarly, a
consistent format allows a user that becomes familiar with one product
to more easily interpret a second product.

### Solution:

Ideally, work on product presentation would be preceded by extensive
research on the communication of uncertainty (see task 1.1), which would
help define the best methods and terminology for communicating
uncertainty. This task would leverage that knowledge. Lacking that, work
proceeds from existing information like that compiled in the NRC and WMO
reports.

We consider here the NWS's standard weather forecast web page for
purposes of illustration; for other government sectors or private
industry, envision an analogous process. The process would start by
convening a team of forecasters, graphical designers, and communications
experts to work together. The team would be supplied some a set of
requirements, e.g., assume that automated guidance or a forecaster will
provide forecasts of the 10^th^/50^th^/90^th^ percentiles of the
probability distribution for all forecast elements currently on the NWS
web page. The initial task of the team would then be to develop several
prototypes of redesigned web pages. These formats would be critiqued in
a test bed environment (see task 2.6), which would indicate which of the
prototypes, or which combination of prototypes, is preferred. This
information is then conveyed to the NWS web page designers, who redesign
the web pages to incorporate the additional information. In addition to
some appealing graphical format, the revised web page should also allow
sophisticated users to obtain more quantitative information, such as
numerical tables of probabilities that could readily be entered into
decision-making software. Since the information is bound to be somewhat
difficult to interpret at first for many users, the revised web page
should some sort of help page for the user to go to in order to learn
about how to interpret the new information being conveyed.

The NWS may also wish to have some way of allowing the public to comment
on the new products after implementation; the small sample of a focus
group on a test bed may not uncover all of the ambiguities the larger
public will have.

If the product is something different, say, a commercial uncertainty
graphic for a television newscast, the process would still be similar;
build a team to develop some prototype graphics, run some sample
newscasts that use the various prototypes by a focus group, and
implement the one(s) that tested the best.

**Quick, short-term steps**: To be filled in later.

## Notes

### Linkages to other subgroups

**Needs, opportunities (subgroup 1):**

Some solution components will be directly connected to particular needs
and opportunities.

**Goals (subgroup 2):**

Effective displays needed across comprehensive suite of products (goal
1). Formats should meet needs/requirements of users (goals 3, 4)

**Roles, responsibilities (subgroup 4):**

For NWS products, will need talents of web designers, forecasters.
Communication experts may need to be entrained from academia, or as
consultants from private sector.

**Roadmap (subgroup 5):**

Work of task 1.1, research on communication of uncertainty, ideally
would precede this task.

### Linkages to subgroup 3 topics

The default is NA for each of these, but in practice you will populate
many of them.

1.1, Fundamental research on communication of uncertainty. Ideally, this
task accomplished prior to this product presentation task. Feeds
directly into it.

1.2, Outreach and communication: used to indicate which products is
there the greatest need/desire for probabilistic products to be
developed and implemented. After implementation of new products, this
task will handle the education of users on how to effectively utilize
this information.

1.3, Undergraduate and graduate education: NA.

1.4, K-12 education: NA

1.5, Forecaster training: NA

**Probabilistic forecast system development (topic 2):**

2.1, Basic predictability research: NA

2.2, Ensemble forecast system development: NA

2.3, Statistical post-processing: NA

2.4, Verification : NA

2.5, Nowcasting: NA

2.6, Test beds: used for critique of prototypes of presentation methods
developed here.

2.7, Tailored product development (2E): May utilize some of the
presentation techniques developed here.

2.8, Product presentation (2F): This task

**Operational technology requirements (topic 3):**

3.1, Supercomputers: NA

3.2, Comprehensive archive: NA

3.3, Data Access : NA

3.4, Forecast preparation systems:

3.5, User infrastructure: Design of products must be cognizant of what
typical infrastructure users will have, e.g., numerical information
conveyed in industry-standard formats so that users can readily utilize
the information.

Other notes:

Companies that design weather graphics need to know about ACUF effort,
about change to uncertainty guidance.

We also recognize that the development of mature visualization products
will take much longer than the lifetime of this *ad hoc* committee. It
also involves many people outside the meteorological community.

##  

## **Part** 3: Supporting Infrastructure

## 3.1: Supercomputers

**Members**: Tom Hamill, <tom.hamill@noaa.gov> & others?

### Overview of task:

Procure and install the high-performance supercomputing resources
necessary to perform the advanced predictability and ensemble
development studies, operational ensemble predictions, advanced data
assimilation, and statistical post-processing necessary to disseminate
skillful, reliable uncertainty guidance.

### Background:

NCEP currently runs a suite of ensemble forecast systems; see task 2.2
for a description of the current suite. However, in comparison to other
operational centers, NCEP devotes a much smaller number of CPU cycles to
their ensembles. For example, ECMWF currently runs a larger global
ensemble (51 members, vs. 21 for NCEP), at approximately three times
higher resolution (T399 in week 1 vs. T126), and includes the regular
production of real-time reforecasts that can be used for calibration
(however, NCEP runs its system 4 times daily to ECMWF's twice daily).
Overall, ECMWF dedicates approximately 50 times more computational
resources to the production of its global medium-range ensemble than
does NCEP. Without a computer upgrade, some improvement in uncertainty
products in the US may be possible by sharing ensemble forecast data
with other countries and with the US military (see description of NUOPC
and GIFS in task 2B). Accordingly, NCEP has worked out cooperative
agreements to share ensemble forecast data with Canada and hopes to
share forecast data between NCEP and the US Navy and Air Force in the
coming years.

### Why needed:

Despite the advances that may be possible by sharing multi-model
ensemble forecast data, the production of skillful, reliable probability
products cannot be achieved in full without a massive increase in
computational resources dedicated to the production of improved
uncertainty forecasts.

### Solution:

\(1\) Determine the CPU cycles necessary for NOAA to run the global,
regional, and extreme-event systems envisioned in task 2B, as well as
the reforecasts necessary for calibration in task 2C. Also, NOAA's
high-performance research computers will need a similar magnitude of
upgrade so that they are capable of testing future ensemble forecast
systems. The upgrade should also not be considered a one-time-event;
after this major upgrade, NOAA should regularly upgrade its computers
roughly in accordance with Moore's Law (a doubling of CPU power
approximately every 2 years) (2) Procure and install the supercomputers
sufficient to carry out tasks 2B and 2C.

**Quick, short-term steps**: To be filled in later.

## 3.1 Notes

### Linkages to other subgroups

**Needs, opportunities (subgroup 1):** NA currently.

**Goals (subgroup 2):**

The comprehensive suite of uncertainty products (goal 1) that are as
reliable and skillful as practicably possible (goal 2) requires the
supercomputer investment described here to carry out the tasks related
to improved ensemble forecast systems (tasks 2.2 and 2.3)

**Roles, responsibilities (subgroup 4):**

NCEP/NCO and and NOAA's HPC office, currently led by Bill Turnbull, are
important points of contact. Hamill has talked with David Michaud in HPC
office about this (29 Feb 08).

**Roadmap (subgroup 5):**

Implementation of the crucial tasks 2.2 and 2.3 require completion of
this task; this task must be completed early on in the process.

### Linkages to subgroup 3 topics

1.1, Fundamental research on communication of uncertainty: NA

1.2, Outreach and communication: NA

1.3, Undergraduate/grad education: NA

1.4, K-12 education: NA

1.5, Forecaster training: NA

2.1, Basic ensemble research: An upgrade to research computers in and
out of NOAA (NSF? Navy?) should be supported to support this research.

2.2, Operational ensemble forecast system development: Depends upon this
task being completed.

2.3, Statistical post-processing: Depends on this task being completed.

2.4, Verification:

2.5, Nowcasting: Algorithm development may need substantial computing
resources.

2.6, Test beds:

2.7, Tailored product development:

2.8, Product presentation: NA

3.1, Supercomputers: This task

3.2, Comprehensive archive: Fast interconnects necessary between
supercomputers and archives.

3.3, Data Access: NA

3.4, Forecast preparation systems: NA

3.5, User infrastructure: NA

### General notes:

NA

## 3.2: Comprehensive archive

**Members**: Ross Hoffman, <rhoffman@aer.com>; members Dan Stillman,
Zoltan Toth, Bill Bua.

### Overview of task:

Build a readily accessible public archive of past ensemble forecasts and
verification statistics for operational forecast models to facilitate
the calibration (statistical adjustment) of ensemble forecasts, the
ensemble technique development process, and for use in training.

### Background:

[NOMADS](http://nomads.ncdc.noaa.gov/), the NOAA Operational Model
Archive and Distribution System, is NOAA's current system for storing
numerical forecast guidance. NOMADS has been storing all NCEP
operational ensemble outputs (in GRID format) since 2007. As guidance is
received, the files are stored in fast-access storage for a period of
time. Eventually the files are removed from fast-access, but they can be
retrieved from off-line storage (see task 3C). NOAA has a cooperative
agreement with the Meteorological Service of Canada (MSC) to share
ensemble forecast information and derived products through a program
called GIFS (the Global Interactive Forecast System). Through this
program, MSC ensemble data is available on NOMADS as well. NOAA is also
attempting to develop cooperative agreements to share forecasts with the
US Navy and Air Force through NUOPC (National Unified Operational
Prediction Capability).

The THORPEX Interactive Grand Global Ensemble (TIGGE) currently archives
a base set of global medium-range ensemble forecast and analysis
information from nine different forecast centers worldwide. TIGGE
archive facilities are currently located at ECMWF, NCAR, and CMA (China
Meteorological Agency). Recent forecasts are available via fast access,
with older forecasts available from tape with a modest delay. A
limitation of TIGGE is that the forecasts are only available two days
after their production to the general user.

Note that currently reforecast data sets are not part of the NOMADS or
TIGGE archive

### Why needed:

Information from past forecasts allows for identifying and correcting
forecast errors.

Thus the primary requirement for a comprehensive archive is to make
readily available the training data needed for statistical
post-processing (task 2C) and tailored product development (task 2E).
For these purposes past forecasts, observations, and analyses, are
required. In some cases a few weeks of data are sufficient but to make
useful adjustments for high-impact, but rare events, years or decades of
data from stable models are required (see task 2C). The archive will
also be useful for providing data for sufficient case studies to improve
ensemble prediction systems (task 2B) and to support predictability
research (task 2A). Finally, an archive provides past cases that can be
used to educate university students (task 1C), customers (task 1E), and
forecasters (task 1F). For example, forecasters commonly learn how to
improve their forecasts of high-impact events by studying the model
performance and systematic error characteristics of similar past cases.

Statistical post-processing works best if the same system (observation,
model, data assimilation) is used to prepare the historical data as in
the system providing the current data. For this purpose then it should
not be necessary to maintain archive data that corresponds to a model or
model version that is no longer operational. However, for forensic
purposes, all the data that was used to prepare actual forecasts should
be archived, but the timeliness requirements for to access such data are
considerably relaxed.

### Solution:

It is anticipated that the NOMADS system will be extended to accommodate
the extra forecast and reforecast data. Analysis and observation data
are available elsewhere, but should be accessible thru NOMADS to provide
seamless ordering and delivery.

The first step in extending NOMADS will be a requirements definition
phase. This will determine the archival size needed to accommodate the
anticipated raw numerical guidance and past forecasts and reforecasts,
and their anticipated growth in time as model resolution increases.
Ideally, the full models states would be archived at high temporal
resolution, especially since post-processing depends on the end-users'
definition of what is optimal. Since it is likely that the full archive
cannot be maintained on fast storage, relevant members of the community
(see tasks 2A, 2B, 2E) provide inputs to determine what subset of data
must be kept on fast storage (e.g., what temporal resolution is
acceptable -- 1, 3 or 6-hourly?, what model fields -- 500 hPa but not
550 hPa?). Members of the community should also be queried on the
variety of ways they are likely to access the data; some users may want
to access time series of horizontal fields; others may want vertical
columns of data for a given model grid point. NOAA will then need to
determine the hardware and software resources (and an anticipated growth
path), obtain these resources, write software, and install the system.

Since the community use of this resource is expected to grow and since
the data base itself is expected to grow, the archive must be robust,
flexible, and extensible.

## 3.2: Notes

### Linkages to other subgroups

**Needs, opportunities (subgroup 1):**

TBD

**Goals (subgroup 2):**

Current goal 5 addresses comprehensive archive.

**Roles, responsibilities (subgroup 4):**

Assume NOAA will take the lead role in this task. Will need to form
advisory committee to determine what data to archive; advisory committee
should span the weather enterprise.

Or will TIGGE repositories take over this function. \[
http://tigge.ucar.edu :: http://tigge.ecmwf.int ::
http://wisportal.cma.gov.cn/tigge \]

**Roadmap (subgroup 5):**

TBD

### Linkages to subgroup 3 topics

1.1, Fundamental research on communication of uncertainty: NA

1.2, Outreach and communication: NA

1.3, Undergraduate/graduate education: This archive needed to support
research, class case studies.

1.4, K-12 education (1D): NA

1.5, Forecaster training: Historical cases accessed from this archive
needed.

2.1 Basic ensemble research (2A): Archives may be useful for assessing
predictability limits and how estimates of these change as forecast
model changes.

2.2, Ensemble forecast system development (2B): Needed to diagnose
current system's deficiencies.

2.3, Statistical post-processing: Archive definitely needed for all
aspects.

2.4, Verification: Verification data should be in the archive, will
access forecast/obs from archive

2.5, Nowcasting: Will access observations from archive.

2.6, Test beds: Historical cases needed for demonstration.

2.7, Tailored product development: Needed for any statistical
adjustments, as in 2.3

2.8, Product presentation: Some canonical cases from the archive may be
useful for developing and demonstrating visualization techniques.

3.1, Supercomputers: Operational supercomputers should have a fast
connection to the archive.

3.2, Comprehensive archive: This topic.

3.3, Data Access: Serving historical data could have a huge impact on
data access system sizing and operation.

3.4, Forecast preparation systems: Access to historical cases helpful ?

3.5, User infrastructure: A well functioning archive will remove the
need for individual users to develop infrastructure to host large
amounts of the historical data. Because of the size of the archive,
university and private sector investigators will need remote resources
(ie, capabilities at the archive site) to make such research possible.

### General notes:

Ensemble forecast data can be thought of as hyper-dimensional data
"cubes". Ensemble data from a single model have seven
dimensions---latitude, longitude, height, verification time, initial
time, variable, and ensemble member (= x, y, z, t, t~0~, v, k).

## 3.3: Data Access

**Members**: Ross Hoffman, <rhoffman@aer.com>; members Jordan Alpert and
Mohan Ramamurthy.

### Overview of task:

Data access systems must be developed that are capable of transferring
very large amounts of data from provider to client, and that allow these
data to be parsed into subsets, transformed, and reformatted prior to as
part of the transfer to the client. For some applications the important
transformation will be interpolating the data cube to an observational
set.

### Background:

A number of current projects­ are exploring facets of ensemble data
access. These include

-   The NOAA National Operational Model Archive and Distribution System
    (NOMADS) is a Web-services based project providing both real-time
    and retrospective format-independent access to climate and weather
    model data \[http://nomads.ncdc.noaa.gov\]. NOMADS provides
    applications/web services for slicing, dicing and area sub-setting.

-   Linked Environments for Atmospheric Discovery (LEAD) makes
    meteorological data, forecast models, and analysis and visualization
    tools available. \[https://portal.leadproject.org\]

-   The Unidata Local Data Manager (LDM) is a collection of cooperating
    programs that select, capture, manage, and distribute arbitrary data
    products. \[http://www.unidata.ucar.edu/software/ldm\]

-   The THORPEX Interactive Grand Global Ensemble (TIGGE) will likely
    use LDM (see 3b). A goal is to provide real time access to ensembles
    from several centers via common web interface.

-   The Global Interactive Forecasting System (GIFS) plans to build on
    real time data access capabilities and provide real time probability
    products based on ensembles stored in TIGGE and possibly other
    systems. Some of these products may be generated in real time in
    response to online requests, requiring capabilities like NOMADS.

-   The Unidata Internet Data Distribution (IDD) is a peer-to-peer (P2P)
    system designed for disseminating near real-time earth observations
    via the Internet. IDD is based on LDM, is designed for real-time
    distribution, not archival, but could be considered a prototype P2P
    system for observational archives.
    \[http://www.unidata.ucar.edu/software/idd\]

Current trends include the distribution of services on scalable servers,
and the provision of slice and dice service applications (e.g., OPENDAP)
for user convenience and to conserve bandwidth. In terms of
applications, NOMADS already allows the user to calculate the
probability of a particular weather event (like frost, gale wind or
anything else) and setting alarms---all on the server side.
\[<http://nomads.ncdc.noaa.gov/data.php?name=ensembles>\].

### Why needed:

Without adequate access, data in archives will not be used.

### Solution:

Data access system requirements are sufficient bandwidth and compute
power, and well defined/implemented interfaces for the ordering and
delivery of data. Multiple "mirror" sites for redundancy and scalability
are desired to make the system robust. The continued evolution of
services should help to conserve bandwidth. The requirements definition
phase of this task should analyze expected data volume growth and
expected data volume requests, and then compare with existing and
currently anticipated resources. Existing distributions systems should
collect information on data requests to guide future "NOMADS-like"
developments. Depending on the application, data access solutions will
be best satisfied by varying combinations of speed and agility.

**Quick, short-term steps**: To be filled in later.

## 3.3: Notes

The idea here is to track items connected with the current topic that
should be resolved someplace but not necessarily in the current topic or
that appear in the current topic, but should also be touched upon
elsewhere. Don't eliminate any linkages below just say "NA" for now
because someone later may think of a linkage.

### Linkages to other subgroups

**Needs, opportunities (subgroup 1):**

TBD

**Goals (subgroup 2):**

TBD

**Roles, responsibilities (subgroup 4):**

Forecast centers; TIGGE distribution centers; observation data
providers.

**Roadmap (subgroup 5):**

Growth of data access resources must match various links below.

### Linkages to subgroup 3 topics

1.1, Fundamental research on communication of uncertainty: NA

1.2, Outreach and communication: NA

1.3, Undergraduate/grad education: Limited but flexible access is
needed.

1.4, K-12 education: NA

1.5, Forecaster training: Training developers need access to the archive
since older cases are often needed.

2.1, Basic ensemble research: Easy access to all ensemble data is needed
to facilitate development of new approaches.

2.2, Operational ensemble forecast system development: Thru testbeds
(1B).

2.3, Statistical post-processing: Developers outside the data providers
will need access to large data volumes.

2.4, Verification: Access to data subsets corresponding to observations
will be needed.

2.5, Nowcasting: Access to data subsets corresponding to observations
will be needed.

2.6, Test beds: Easy access to all ensemble data is needed to facilitate
testing of new approaches

2.7, Tailored product development: Fast, easy access required for the
development and delivery to end-customers.

2.8, Product presentation: In some cases the data transport system will
provide results as graphics.

3.1, Supercomputers: Serving large numbers of time sensitive requests
can be burdensome.

3.2, Comprehensive archive: Provides the means to disseminate the
archive.

3.3, Data Access: THIS TOPIC

3.4, Forecast preparation systems: Will receive ensemble data this way.

3.5, User infrastructure: The users must install computer resources
commensurate with the data that will be downloaded.

### General notes:

TBD

## 3.4: Forecast preparation systems

**Member**: Paul Schultz, <Paul.J.Schultz@noaa.gov>

### Overview of task:

To provide forecasters with the tools to generate and disseminate
probabilistic weather forecast products. The NWS practice of operations
is to provide field forecasters with various guidances, which are
forecast information the forecaster can use to help generate official
forecasts. Specifically, forecasters in the field use the Advanced
Weather Information Processing System (AWIPS) workstation to receive
guidance produced by the National Centers for Environmental Prediction
(NCEP), visualize the guidance and other relevant information, and
generate official forecasts on a 5-km grid. These grids are collected
centrally and provided to the public via the National Digital Forecast
Database (NDFD). This is the current practice for delivering
single-value (deterministic) forecast services. None of this is expected
to change with the expansion of services to include probabilistic
expression of forecast uncertainty: NCEP will provide guidance in the
form of probabilities, forecasters will use AWIPS to review guidance and
produce forecasts, and the probabilistic forecast products will be
posted to NDFD. At this time, the intent is to post the median value of
the forecast probability distribution function (PDF), which is
equivalent to today's deterministic value, along with the values
representing the 10% and 90% positions on the PDF. This applies to
variables such as temperature and wind; for precipitation the plan is to
post the probability of precipitation in excess of multiple thresholds.

### Background:

The National Weather Service forecast preparation systems are in the
process of a systematic upgrade that will merge the capabilities of all
such systems into Version 2 of the Advanced Weather Information
Processing System, or AWIPS II. This workstation will incorporate the
current capabilities for probabilistic data processing, which exist only
on the platform used by the National Centers (Hydrometeorological
Prediction Center, Storm Prediction Center, Aviation Weather Center,
etc.), and new capabilities to support the field forecasters at the 100+
NWS Warning and Forecast Offices (WFOs) across the US. It is important
to note that *resources for developing and testing probabilistic
forecast techniques for AWIPS II have not been committed.*

### Why needed:

Forecasters add value to centrally produced guidance by applying
understanding of, and statistical information on, how various ensemble
members err or succeed in high-impact local weather events. New forecast
preparation systems, specifically AWIPS II in the case of the NWS, must
enable the forecasters by providing access to model ensembles,
verification tools, calibration algorithms, and interactive manipulation
of gridded information.

### Solution:

To support the NWS forecaster in the task of preparing probabilistic
forecasts, AWIPS II must provide the following:

-   The ability to acquire guidance products and ensemble model members

-   Versatile access to climatological data

-   A complete verification record of past performance of the various
    sources of guidance

-   A variety of displays to visualize and manipulate the guidance
    products and other forecast inputs, such as: \"spaghetti\" plots
    displaying a single (or few) contours from multiple models; plume
    diagrams representing time series plots from multiple models, valid
    for a single point; \"postage stamp\" displays presenting miniature
    image or contour plots from many models arranged in a rectangular
    matrix; a blender capability that allows forecasters to
    interactively and subjectively weight certain ensemble members for
    the local calculation of estimated probability distribution
    functions (PDFs); and an interactive tool allowing the forecaster to
    manually edit PDFs and then propagate these adjustment to adjacent
    points in space and/or time

-   Statistical tools for computing PDF attributes the ability to post
    final forecast products a local database of gridded forecast
    products, and ultimately to the NDFD

In the envisioned forecast process, each WFO maintains lists of
high-impact weather event cases; these historical cases must be
reforecast using the predictor models used in real time. In anticipation
of certain events, the forecaster can review these cases, compute
verification statistics relevant to the situation, and then make
well-informed decisions on which ensemble members to emphasize or
de-emphasize in the blending process. Examples: 1) Forecasters in
Buffalo NY may determine that the RSM-based members of the SREF ensemble
consistently underestimate the Bowen ratio in cases of November lake
effect snowstorms, leading to over prediction; in this case the RSM
members would be de-emphasized in the blender. 2) Forecasters in Denver
may determine that the WRF-EM core is particularly accurate in cases of
strong down slope windstorms, and on that basis choose to weight the
WRF-EM model members more heavily.

It is expected that the guidance provided to WFOs will be of
sufficiently high quality that for most benign weather scenarios it will
be used without editing by the forecaster to populate forecast product
grids that are then passed to the NDFD. This concept of operations is
intended to allow forecasters to focus on high-impact weather events.
The probabilistic forecast process is illustrated in figure 2.

![](media/image2.png){width="5.996527777777778in"
height="3.0027777777777778in"}

**Figure 1: Flow diagram represents the operational concept of the WFO
probabilistic forecast process in the AWIPS II era. MOS is Model Output
Statistics. SREF is the Short-Range Ensemble Forecast. NAEFS is the
North American Ensemble Forecast System. GFE is the Graphics Forecast
Editor. NDFD is the National Digital Forecast Database.**

Once the forecast products are posted they become immediately available
to emergency managers and other users via a small set of user tools to
extract appropriate subsets of the probabilistic forecast data, and then
perhaps use those as inputs to automated decision support methodologies.
Responsibility for the decision support function lies with the private
sector, except in the case of emergency managers, who will interact
directly with NWS personnel.

## 3.4: Notes

The idea here is to track items connected with the current topic that
should be resolved someplace but not necessarily in the current topic or
that appear in the current topic, but should also be touched upon
elsewhere. Don't eliminate any linkages below just say "NA" for now
because someone later may think of a linkage.

### Linkages to other subgroups 

**Needs, opportunities (subgroup 1):**

**Goals (subgroup 2):**

Each solution component should help to satisfy some goal.

**Roles, responsibilities (subgroup 4):**

Anything to do with where the activity should take place and who should
do it. *The required combination of AWIPS experience and
ensemble/probabilistic forecasting research resides uniquely in the NOAA
Earth System Research Laboratory.*

**Roadmap (subgroup 5):**

Anything to do with timelines, dependencies should be captured here.
*The necessarily iterative process of AWIPS workstation technique
development, exposure to forecasters in a test environment, and
redevelopment, will require at least three years to "harden", train
forecasters, and implement in the field. The clock starts with funding.*

### Linkages to subgroup 3 topics

1.1, Fundamental research on communication of uncertainty: NA

1.2, Outreach and communication: NA

1.3, Undergraduate/grad education: NA

1.4, K-12 education: NA

1.5, Forecaster training: ): Forecasters must have formal academic
training in statistics and probabilistic methods. COMET training in the
specific methods and datasets used in the NWS must be developed.

2.1, Basic ensemble research: Helps forecasters understand which
variables to concentrate on; don't bother modifying forecasts beyond the
inherent predictability timescale.

2.2, Ensemble forecast system development: tight coordination with AWIPS
development is required ... software components shared where possible
and appropriate

2.3, Statistical post-processing: *--* again, tight coordination with
AWIPS is required

2.4, Verification: It is incumbent on any provider of forecast
information to verify its forecasts, and to maintain an archive of those
forecasts. This is critical for three purposes. First, it allows the
routine quantitative assessment of service quality, suitable for
reporting to the U.S. Congress. Second, it enables post-disaster
analysis of service quality and societal response to the forecast
information. And third, it enables the assessment of new candidate
methods for improving current capabilities.

2.5, Nowcasting: NA

2.6, Test beds: NA

2.7, Tailored product development: NA

2.8, Product presentation: NA

3.1, Supercomputers: *Reforecasting "all" the predictors will be
expensive.*

3.2, Comprehensive archive: NA

3.3, Data Access: NA

3.4, Forecast preparation systems: NA

3.5, User infrastructure: NA

### General notes:

We seek confirmation or correction from our non-government cooperators
on the ACUF committee that our current path is the most reasonable and
efficient way to achieve the goal of excellent probabilistic forecast
services from the NWS.

##  

## 

## 

## 3.5: User infrastructure

**Members**: Ross Hoffman, <rhoffman@aer.com>; Mohan Ramamurthy.

### Overview of task:

Define and implement adequate compute power, storage, network resources,
software systems outside of NOAA and other data providers. User
infrastructure enables users to make use of what is received through
data access (3C).

### Background:

Universities, private sector meteorologists, and consumers all have made
significant and continuing investments in infrastructure. Technological
advances keep increasing capabilities for the same price. Current
software systems are mostly oriented towards a single deterministic
forecast.

### Why needed:

Will the existing and planned infrastructure be adequate to make use of
the ensemble forecast data deluge. Software systems that deal with a
single forecast and no probabilistic information will ignore the new
data streams.

### Solution:

The major need is to educate users on the size of the problem early
enough so the planning process will result in infrastructure
commensurate with future needs. The biggest problem will be that many
software systems will require a thorough redesign.

Any solution will require fast network access. However current trends
towards providing commercial digital media (e.g., high def TV shows)
suggests that typical high end home internet capability will keep up
with evolving demands of ensemble forecast data.

## 3.5: Notes

Contact: Ross Hoffman, rhoffman@aer.com

### Linkages to other subgroups

**Needs, opportunities (subgroup 1):**

TBD.

**Goals (subgroup 2):**

TBD.

**Roles, responsibilities (subgroup 4):**

Involves the whole enterprise outside of the data providing centers.

**Roadmap (subgroup 5):**

An early start is desired.

### Linkages to subgroup 3 topics

1.1, Fundamental research on communication of uncertainty: NA

1.2, Outreach and communication: NA

1.3, Undergrad/graduate education: Large data sets will be hosted in
labs.

1.4, K-12 education: NA

1.5, Forecaster training: NA

2.1, Basic ensemble research: NA

2.2, Ensemble forecast system development: NA

2.3, Statistical post-processing: NA

2.4, Verification: NA

2.5, Nowcasting: NA

2.6, Test Beds: NA

2.7, Tailored product development: Infrastructure required here.

2.8, Product presentation: High dimensional visualization.

3.1, Supercomputers: NA

3.2, Comprehensive archive: NA

3.3, Data Access: User infrastructure must be matched to data access
methods. Robust, flexible data access will allow smaller user
infrastructure investments.

3.4, Forecast preparation systems: Special case of user infrastructure
not included here.

3.5, User infrastructure: THIS TOPIC.

### General notes:

TBD.

[^1]: These operational forecasters include those in both the public/NWS
    and private sectors.

[^2]: NOAA recommendations are emphasized here, though similar
    recommendations are applicable to other agencies (Navy, Air Force)
    for their own specialized regional modeling requirements.