# Presentation: 176743

## Short-Term Ensemble (STE)Concept of OperationsRevised Draft

- NOAA/NWS/OHD/HL/HSMB &      DOH Science Steering Team
- Jul 26, 2006

- 7/12/2006

## This slideshow

- Summarizes the concept of operations (CONOPS) for short-term ensemble (STE) forecasting, jointly developed by OHD/HL/HSMB/HEP and DSST
- Depicts the envisioned ensemble forecast system that may best support the CONOPS, while building and capitalizing on the existing and planned capabilities

- 7/12/2006

## STE CONOPS Attributes

- _[Forecast operation](slide4.xml)_
- Blending with long-term ensemble forecast
- _[Uncertainty accounting](slide7.xml)_
- _[Multi-model ensemble forcing](slide10.xml)_
- _[Capturing variability](slide12.xml)_
- _[Down- & re-scaling](slide13.xml)_
- _[Data availability](slide14.xml)_
- _[Relationship with deterministic forecast](slide15.xml)_
- _[Hydraulic routing & reservoirs](slide18.xml)_
- _[Forecast environment](slide19.xml)_

- _[Visualization & MOD](slide20.xml)_
- _[Product generation](slide21.xml)_
- _[DA & MOD](slide22.xml)_
- _[Transparency of forecast system](slide23.xml)_
- _[Verification](slide24.xml)_
- _[Products](slide25.xml)_
- _[Data storage & management](slide27.xml)_
- _[Training](slide28.xml)_
- _[Computing power](slide29.xml)_
- _[Keeping up with science evolution](slide30.xml)_

- 7/12/2006

## Forecast operation

- STE shall be used to issue forecasts for all forecast points for all situations (see next slide)
- At RFCs: To issue all but long-term water supply forecasts
- At WFOs: To produce primary guidance for flood watches and outlooks

- 7/12/2006

## Examples of STE graphical products

**A**

**B**

**B**

- Same as B above , but for hydrometeorological forcings

**NWS-NDGD gridded uncertainty product**

**A**

- 7/12/2006

## Blending with long-term ensemble forecast

- STE (Day 1 to 14) shall be seamlessly blended with long-range ensembles (Day 15 and beyond) into a single forecast
- With Climate Forecast System (CFS) ensemble forecast in addition to, or in place of, monthly/seasonal outlook forecast
- Science issues associated with blending shall be addressed

- 7/12/2006

## Uncertainty accounting

- STE forecast shall reflect uncertainties from all sources
- Source-specific accounting and quantification of uncertainties (see next slide) associated with initial conditions, observed boundary conditions, future boundary conditions, model parameters, rating curves, etc.
- Aggregate statistical modeling of residual uncertainties
- STE forecast shall reflect uncertainties that are conditional on environmental conditions
- Some storms (strong cold front crossing the state) are much easier to forecast than others (cut-off low just off the coast)
- The statistics and uncertainties during transitions or storms tend to shift from the climatology
- Wide range of potential river responses is not always pleasing due to large uncertainties in precipitation forecasts

- 7/12/2006

## Uncertainties in streamflow forecast

- Uncertainty in initial conditions

- Parametric uncertainty

- Lead Time

- Uncertainty

- Meteorological/Input uncertainty

- Reduced uncertainty due to pre-processing

- Reduced uncertainty due to calibration

- Reduced uncertainty due to data assimilation

- 7/12/2006

## Uncertainty accounting (cont.)

- STE shall:
- Account for dependence of uncertainty on forecast lead time
- Account for uncertainty in forecasts of interrelated variables (e.g., QPF and QTF)
- Account for uncertainty in forecaster MODs
- Be able to process multiple forecasts of the same variables, and provide information on relative skill among them for forecaster decision-making

- 7/12/2006

## Multi-model ensemble forcing

- STE shall:
- Allow use of alternate forcing sources (see next slide)
- Meteorological forcing for hydrologic models originates from multiple sources; the final product is a combination of these procedures
- Sources include HPC, MDL, WFO ISC/GFE grids, HAS, the numerical model of the day
- Help forecasters select model(s) of choice (see next slide)

- 7/12/2006

## Multi-model ensemble forcing (cont.)

- GFS

- HPC

- HAS

- MOS

- Performance measures for Individual and multi-model forcings

- 7/12/2006

## Capturing variability

- STE shall capture uncertainty in QPF due to:
- Spatial and temporal variability
- Individual forecaster bias
- STE shall account for space-time variability of precipitation
- Is a precipitation value at one basin linked to a precipitation value at a basin nearby?
- Is trace for one location linked to a trace at the downstream location?
- STE shall account for and allow adjustment of co-variability relationships between precipitation and temperature
- Science for capturing variability shall be improved beyond Schaake Shuffle

- 7/12/2006

## Down- & re-scaling

- STE applications shall down- or re-scale forcings as necessary
- Meteorological forcing for the hydrologic models arrive at the RFC in many different scales which are then converted to a base scale used in the hydrologic models
- Adequacy of the current data networks shall be addressed for downscaling in STE applications
- Science for scaling shall be improved toward a universal methodology

- 7/12/2006

## Data availability

- Data requirement and availability (both quantity and quality) for STE applications and operation shall be rigorously assessed
- Recent historical QPF and MAP are limited, and have been dynamic with time
- Basin boundary changes and addition of new basins require recomputing of MAPs and QPFs
- STE applications shall utilize reforecast data as necessary
- In order to downscale we need several years, if not decades, of forecast data, which requires a re-forecast process of NWP model
- If NWP model is changed, e.g., to use better physics, a new re-forecast data set would be necessary to determine new downscaling relationships

- 7/12/2006

## Relationship with deterministic forecast

- STE forecast shall:
- Be generated in addition to deterministic/control/single-value/best-guess forecast
- Replace deterministic forecast once scientifically and operationally proven
- Have clear, consistent and easy-to-understand relationship with deterministic forecast for both casual and sophisticated users
- STE operation shall generate ensembles whose median, i.e., 50%, is close to deterministic forecast (see next slides)
- STE operation shall provide a measure of reliability that is easily understood by users

- 7/12/2006

## If ensemble mean (~median) is close to truth,

- 97.5th

- 85.0th

- 50.0th

- 25.0th

- If ensemble mean (~median) is close to truth,

- 7/12/2006

## we have reliable, i.e. probabilistically unbiased, ensembles at all ranges of flow

- 7/12/2006

## Hydraulic routing & reservoirs

- STE shall account for and propagate uncertainties associated with natural and man-made hydrologic and hydraulic systems, including:
- regulations, diversions, debris, blockages and ice jams
- STE applications shall address;
- Hydraulic routings complicates the ability to operate at the segment level
- Ensembles on autopilot can introduce unrealistic results for reservoirs
- Consumptive use is a source of large uncertainty, particularly during draining stage
- STE operation shall have tools to analyze and visualize results from hydrologic, hydraulic and/or statistical models

- 7/12/2006

## Forecast environment

- STE forecast shall be generated interactively under IFP
- ESP shall run interactively from any point in time and for individual components of a forecast group
- The notion of carryover shall be reexamined
- Generation and viewing of ensemble forecasts shall be controlled by a switch to allow basic tuning of the model before introducing ensembles into the picture
- Balance rivers without ensemble forecasts
- Switch on ensemble forecasting and look at rivers again
- If deterministic forecast is, e.g., higher than 2% of ensemble, modify deterministic and/or ensemble for a particular location
- IFP shall allow, through a switch, selection of MODs for deterministic, ensemble or both
- STE forecast process shall utilize a mix of proven automatic DA and forecaster MODs
- Automatic DA reduces the apparent conflicts of wanting control over the output (tweaking the results) vs. maintaining ensembles that are dynamically and statistically consistent
- Capitalize on CHIPS for development and operation of the above functions

- 7/12/2006

## Visualization & run-time modification (MOD)

- Visualization tools shall be used to provide ensemble and probabilistic information within the IFP framework
- Displays of forecast ensembles shall include an overlay of historical ensembles and ranges
- Forecasters need them to get some idea of possible adjustments to the distribution plots
- Drawing tools shall be used to redraw forecasts in problem cases
- To adjust distributions operationally when models are deficient
- Necessary for timeliness and expediency

- 7/12/2006

## Product generation

- STE operation shall have the software that will produce graphics and text output in a user friendly manner (e.g., xsets/hydrograph creation software)
- STE operation shall allow the users to create their own products on the fly
- STE shall address OFS not writing information back to the processed DB until the segment is exited
- This prevents product and information generation while looking at the IFP display

- 7/12/2006

## Data assimilation (DA) & MOD

- DA shall be used to provide guidance for MODs
- STE operation shall have:
- ability to trace what DA did
- capability to undo or nudge differently
- STE operation shall re-examine the notion of and complementarity between DA and MODs, and their impacts on forecast products
- There will always be times when a forecaster needs to drive the model to the appropriate outcome
- As the models and forecast systems get more complex, forecasters do not have the time to do all the driving and need more guidance to do the driving
- Implementation of DA capabilities shall be based on extensive verification of them
- RFCs need to be in the decision to buy off on it

- 7/12/2006

## Transparency of forecast system

- STE applications shall be “white boxes”
- The system shall have full transparency for the forecasters
- The system shall provide summary views of inputs and model internals but with the capability to drill down and view detailed information
- Essential for quality control and debugging
- STE applications shall include smart tools to assist in diagnosis of the ensemble inputs, DA, and results

- 7/12/2006

## Verification

- STE shall provide and improve verification capabilities for both ensemble and deterministic forecasts
- Without verification, one cannot say whether, e.g., ensemble mean is any better or worse than deterministic forecast
- STE shall be verified using the Ensemble Verification Program (EVP) as the primary tool
- STE shall be verified against climatology and other reference forecasts
- Verification outputs shall be translated into lay person’s terms

- 7/12/2006

## Products

- STE forecast shall be generated in both graphical and text products
- STE shall address the need for customer buy-in of the products
- “So how good are you guys anyway?” (see next slide)
- STE shall address the need for a measure of reliability that is easily understood by users (e.g., 50% or median)
- STE shall improve reliability of forecasts
- Is, e.g., the 10% exceedance number really exceeded only 10% of the time?
- STE shall develop meaningful displays for end-users to illustrate the model inputs such as precipitation and model QPF (focus group requirement)
- STE operation shall generate SHEF output for selected confidence levels as tabular information in relational database at WFOs (or provide other means of having data readily available in AWIPS database)
- STE forecasts shall be easily accessible from the AHPS website

- 7/12/2006

## Conveying past performance in STE product by appending verification plots

- Reliability

- Resolution

*Ensemble Hindcaster (EH)*

*Ensemble Verification Program (EVP)*

- 7/12/2006

## Data storage & management

- STE operation shall keep its forecasts distinct, which will require new data storage and handling techniques
- STE operation shall manage ensemble and probabilistic information for generation of STE products and guidance
- STE shall seek new and innovative approaches to deal with large increases in data volume

- 7/12/2006

## Training

- Training shall be provided to the producers and users of STE forecasts
- Tools shall be developed for self-learning

- 7/12/2006

## Computing power

- STE applications shall have computing power necessary to support:
- Interactive use (e.g., be able to rerun small pieces of forecast groups in a few seconds)
- Large-volume data processing
- Ensemble hindcasting
- Ensemble DA
- Service backup

- 7/12/2006

## Keeping up with science evolution

- STE applications shall be flexible and modular to be able to adapt relatively easily to the changes in science (e.g., model changes) and operational procedures

- 7/12/2006

## Envisioned ensemble forecast system

- The next 3 slides show;
- The envisioned ensemble forecast system that:
- Interfaces with IFP, OFS and DA, and runs ESP in the forecast mode
- Interfaces with OFS and DA, and runs ESP in the hindcast mode
- Capitalize on CHPS (to be determined)
- The existing ensemble forecast system for comparison
- The existing deterministic forecast system for reference

- 7/12/2006

## Targeted Ensemble forecast system Forecast mode – ESP interfaced w/ IFP, OFS & DA Hindcast mode – ESP interfaced w/ OFS & DA Capitalize on CHPS

- reservoir models

- Historical data

- Analysis

- Analysis

- Real time data

- MA(P,T,PE)
- FMA(P,T,PE)

- Parameters

- Carryover

- River-
- flow

- Products

***ESP Analysis & Display Program (ESPADP)***

- Post-processor

***Ensemble Post-Processor***

***Interactive Fcst Program (IFP)***

***Ensemble Pre-Processor (EPP)***

- Ensemble Verification Program (EVP)

***Operational Fcst System (OFS)***

***Targeted Ensemble forecast system ****Forecast mode – ESP interfaced w/ IFP, OFS & DA Hindcast mode – ESP interfaced w/ OFS & DA Capitalize on CHPS*

***Data Assimilator (DA)***

- 7/12/2006

## Existing ensemble forecast system

- reservoir models

- Historical data

- Analysis

- Analysis

- Real time data

- MA(P,T)
- FMA(P,T)

- Parameters

- Carryover

- River-
- flow

- Products

***Ensemble Stream Prediction (ESP)***

- Post-processor

***Ensemble Post-Processor***

***EPP parameter estimator***

***EPP ensemble generator***

***Calibration System (CS)***

***Operational Forecast System (OFS)***

***Existing ensemble forecast system***

***Batch operation***

***ESP Analysis & Display Program (ESPADP)***

***Ensemble Pre-Processor (EPP)***

- 7/12/2006

## Existing deterministic forecast system

- reservoir models

- Historical data

- Analysis

- Analysis

- Real time data

- MA(P,T,PE)
- FMA(P,T,PE)

- Parameters

- Carryover

- River-
- flow

***Calibration System (CS)***

***Interactive Forecast Program (IFP)***

***Operational Fcst System (OFS)***

***Existing deterministic forecast system***

- Products

- 7/12/2006

## End of slides

- 7/12/2006