# Presentation: 176745

## DSST Short-Term Ensemble Planning (STEP) Call #2

- RFC Presentations and Discussion

## Agenda

- Introduction (DJ)
- RFC presentations and Q/A
- AB
- AP
- CB
- CN
- MA
- MB
- NC
- NE
- NW
- Comments by other RFCs
- Discussion (All)
- Towards developing (near-) consensus operations concept and identifying overarching needs and issues (service, operations and science)
- What next? (DJ)

## ABRFC

## Operations Concept

- I see the ABRFC using short term ensembles (STE) as the "official" way to issue almost all of our forecasts. That includes our 27 daily forecast, as well as our forecasts we issue for the other 200+ flood forecast points we have.  Only our long term water supply forecasts would be different. If I had our way, I would have implemented the STE as the "AWIPS" requirement fulfillment long ago. 98% of our users do not want long term ESP forecasts!
- I picture that we would issue the forecasts mainly in a graphical method, much like the examples that this office has provided (see below).   Instead of issuing just deterministic forecasts like we do now, I see STE as a replacement/addition to this.

## Operations Concept (cont.)

- Another use of short-term ensemble (STE) forecasts is by the WFOs as a basis for triggering a river flood watch
- If we got the procedures efficient enough, we could issue our STE forecasts routinely for all of our forecast points, not just for flood situations. This way, with experience and possibly local WFO procedures, the WFOs could use the STE forecasts as the main part of the decision process in determining when to issue flood watches and outlooks.

## Needs and Issues

- One thing that NEEDS be to accomplished is for us to determine or define a relationship between probabilistic forecasts and a single deterministic forecast. I think all STE forecasts should also include the office’s best guess of a deterministic forecast, and this deterministic forecast should be located near the 50% probability line. We have to define how deterministic forecasts relate to probabilistic forecasts before we can go any further.
- Many of our users just want one forecast, i.e. the deterministic forecast, while other more sophisticated users will enjoy the probabilistic forecast. Issuing a STE with both data would be the best bet I think!
- I also think that the STE info could be presented in a text format, but the exact format could be determined later.

## Needs and Issues (cont.)

- Operationally, I see MANY significant things that need to be accomplished. Once you get the science down, we need software that will produce the graphics/text output of this data in a user friendly manner (i.e. xsets/hydrograph creation software).
- We need training for those producing and using these forecasts.
- We need a solid verification set of data proving that the science behind STE is reliable and sound. I have yet to see any verification (including that which I have done) that convinces me that the science behind the STE is producing reliable and accurate short term forecasts.
- As far as the DA goes, I still am not convinced it works. The little I have heard from WGRFC users is that they were not satisfied with it. I think we need some extensive verification of it, and HICS/DOHS/RFCS need to be in the decision to buy off on this. At this point, I have not seen any convincing evidence that it works.

## Needs and Issues (cont.)

- Overall, I am excited about STE, but still am not convinced that the science behind our current configuration works.
- I do however foresee a future where the vast majority of forecasts issued from the ABRFC are STE, which will allow for a great advance in the hydrologic program.

## APRFC

## APRFC HAS ConOps

- Ability to select model(s) of choice
- Ability to evaluate 6-hour QPF for a set of stations
- Ability to apply post processing to get pcpn range for each station
- Ability to review/edit pcpn ranges
- Ability to apply “Mountain Mapper” functionality to get MAPs
- Ability to review/edit MAP value ranges
- Ability to repeat process for temperatures

## APRFC Hydro ConOps

- Ability to review forecast and range of MAP and MAT values
- Ability to run IFP and see resulting hydrographs for range of inputs
- Ability to adjust MAP/MAT relationships
- Ability to run post processing to get most likely and range of hydrographs

## APRFC Summary Comments

- We think that short term ensembles have equal or greater value than long term ensembles for our users
- We have concerns about
- the ability to quantify the uncertainty in QPF given its significant spatial and temporal variability and individual forecaster bias
- the ability to quantify the uncertainty of the interrelated QPF and QTF variables and the resulting impact on the ensemble of hydrographs
- the apparent conflicts of wanting control over the output (tweaking the results) vs. maintaining unbiased statistical ensembles

## CBRFC

## CBRFC Short-term Ensemble

**Opportunity:  We can improve our short to medium range esp forecasts by using numerical model predictions in place of historical data.**

## But...  We can not  directly use output from a weather model, even if it is in ensemble form.  Our conceptual model does not necessarily want to see reality.  It wants to see its own twisted version of reality.

**But...  We can not  directly use output from a weather model, even if it is in ensemble form.  Our conceptual model does not necessarily want to see reality.  It wants to see its own twisted version of reality.**
**We get that twisted version by determining a relationship between what the weather model predicts and what our river model has been calibrated with.  Otherwise known as downscaling.**

## Project Area: 27 Segments Above Cameo, 

_**Project Area: 27 Segments Above Cameo, **_
_**Colorado River**_

- All recently recalibrated
- and set up For ESP.

## Downscaling

- MRF Variables:
- 2m air temp
- Precipitation
- 700mb Relative Humidity
- Sea Level Pressure
- 10m Vector Wind
- Total Column Precipitable Water

- Basin Scale Variables:
**Mean Areal  **
** ****Temperature**
**Mean Areal   **
**Precipitation**

## In order to downscale we need several years, if not decades of forecast data.  This requires a re-forecast process of the weather model.

**In addition, for operational use, we need that weather model output to stay consistent with the relationships we have determined.  If it is changed to use better physics, for example, we would then need a re-forecast data set and determine new downscaling equations.**

## Downscaling Results

- MRF is colder than normal in this case.

## Input into ESP

_**Input into ESP**_

- 	MRF derived MAT/MAPs are attached to historical years (“ensembles”) and ‘fed’ to ESP.

## ESP peak flow

- 	Smaller peaks because MRF is colder for first 14 days causes less melt.

## Operational challenges
Scale – it may be too large for precipitation.  In general, smaller scale should be better
Availability – we need commitment to use this operationally
Re-forecast process – any changes in model physics or scale requires a re-forecast and new downscaling equations.  The good news is the weather community is starting to see the value in statistical correction.
Presentation – we should keep these forecasts distinct which will require new data storage and handling techniques

**Scale – it may be too large for precipitation.  In general, smaller scale should be better**
**Availability – we need commitment to use this operationally**
**Re-forecast process – any changes in model physics or scale requires a re-forecast and new downscaling equations.  The good news is the weather community is starting to see the value in statistical correction.**
**Presentation – we should keep these forecasts distinct which will require new data storage and handling techniques**

## CNRFC

## CNRFC comments on short term ensemble

- Need customer buy-in,  “So how good are you guys anyway?”
- 50% or median....need a measure of reliability that is easily understood by users
- 90% 10% bounds....is the 10% exceedance number really exceeded only 10% of the time?
- A reasonable range of expected values should be very useful to the emergency folks
- Need to adequately explain how uncertainty increases as you move into the future
- Conditional uncertainty....Some storms (strong cold front crossing the state) are much easier to forecast that others (cut-off low just off the coast)  We should be able to modulate the uncertainty accordingly.

## A Vision for Operational Hydrologic Short-Term Ensemble Forecasting - Rob Hartman (AHPS Theme Team, June 2005)

- For many years, NWS customers have benefited from probabilistic long-range water supply forecasts in the Western U.S.  The potential benefits of accurate short-term probabilistic flood forecasts are very significant.  This becomes obvious when one considers the cost of local emergency management activities.
- For several years now, OHD has been working on short-term ensemble prototypes.  These efforts have been concentrated on developing ensemble inputs through model downscaling or simulation based on the joint distribution of forecasts and observations.  Additional activities such as post processing and data assimilation (DA) have been identified.  To date, however, the operational environment for short-term ensemble based probabilistic forecasts has not been described.
- The time-line for AHPS implementation of short-term probabilistic forecasts is such that the forecasting environment will certainly still involve OFS and the IFP.  As such, operational ensemble forecasts must function in this environment.  This will require several changes to the ESP and OFS architecture.  Here is a typical scenario:

*The RFC is experiencing moderate flooding in several watersheds.  A forecast update is due out at 03Z.  Data come in after 00Z for the second period of the day, the 5-day QPF is updated by the HAS function.  Input data are QC’d and the forecaster starts his/her IFP run to update guidance.*

- This scenario identifies several issues that are not currently supported with OFS and ESP.  These include:
- 1.	The notion of carryover must to be re-examined so that ESP can be run interactively from any point in time and for individual components of a forecast group.  Ensemble generation must become interactive rather than a batch process.  Performance must support interactive use (rerun small pieces in a few seconds).
- 2.	The ESP process must thoroughly re-examine the notion of MODs and their impacts on short-term ensembles.  Some believe that DA and automatic state updating is the only solution to avoid MODs.   In the midst of forecasting, this is unrealistic.  There will always be times when a forecaster needs to drive the model to the appropriate outcome.  That’s why we have forecasters.
- 3.	Statistical post-processing techniques must to be fast and interactive.
- 4.	Visualization tools must to be developed within the IFP framework to support ensemble and probabilistic information.
- 5.	Ensemble and probabilistic information must be managed to facilitate the generation of products and guidance.
- 6.	OFS does not write information back to the processed DB until the segment is exited.  This prevents product and information generation while looking at the IFP display.
- Without doubt, we’ll find lots of other issues as we attempt operational implementation of short-term probabilistic forecasts.  As such, it is important to being addressing the issues and developing an operational prototype.

## MARFC

## Shorter Term Probabilistic Forecasts 

- 7-day probabilistic river forecasts
- PQPF/PQTF
- Demonstration of short-term approach
- Juniata & Schuylkill Basins
- 18 points issued daily

## 7-Day Probabilistic River Forecasts

- Current Basin Conditions
- Short term probabilistic precipitation and temperatures (PQPF/PQTF)
- 48-hour PQPF merged with 5 days of climo
- QPF scenarios based on comparison of historic forecast and observed MAPs
- 3 graphics generated daily for 18 basins in PA-Juniata (CTP) and Schuylkill (PHI) Basins

## WFO CTP AHPS Page

## Frankstown Br. Juniata River at Williamsburg, PA

## Difficult to Find on AHPS Web Page

## 7-Day Expected Value Plot 

## 7-Day PQPF Traces 

## Lessons Learned 

- 7-day PQPF/PQTF forecasts are good contingency forecasts
- Describe a range of outcomes that help address HSA questions...”What if..”
- Wide range of potential river responses is not always pleasing...depicts difficulty in forecasting precipitation

## Lessons Learned

- Only limited use by WFO’s and cooperators  (much less than 30-day ESP forecasts)
- Difficult for “non-technical” audience
- Software runs well and can generate graphics in a “hands-off” mode
- Once rain is on the ground we have considerable modeling uncertainties not incorporated in this method

## Lessons Learned

- Await results of OHD verification work to  assess validity of products
- MARFC would like to generate probabilistic forecasts beyond day 2 based on more than climatology
- Potential collaboration with WFO CTP and SREF ensembles

## MBRFC

## Define User Requirements

- Have user create own on the fly if capable
- Graphics within hydrograph plot if adequate description provided

## Vision of Operations

- Balance rivers without ensemble forecasts
- Switch on ensemble forecasting and look at rivers again
- Data assimilation used as guides to mods
- Displays of historical precip and streamflow to compare distributions

## Concerns

- How are segments linked?  Is a precip value at one basin linked to a precip value at a basin nearby?  Is trace for one location linked to a trace at the downstream location?
- Users will think uncertainty distribution in hydrograph is really a set of hydrographs
- Recent historical QPF and MAP are limited and have been dynamic with time

## Concerns (cont.)

- Basin boundary changes and addition of new basins require recomputing MAPs and QPFs
- Inconsistency between 14-day short term and 90-day long term
- How do you handle forecast locations that are model deficient and require manual override? Blending of regulation modeling?
- Use of mods in the future?

## Concerns (cont.)

- How can distributions be adjusted operationally?
- What can forecaster view to get some idea of possible needed adjustments to the distribution plots?  Are ensemble data out of bounds?

## NCRFC

## Operations Concept – Ideas

- Need the ability to operate and generate forecasts at the sub-basin level (segment)
- Ensemble generation and viewing should be controlled by an option switch (on/off/default) to allow for interaction with the model for basic tuning before introducing complications of ensembles (assumes integration w/ IFP)

## Operations Concept – Ideas, cont.

- System needs full transparency for the forecaster – Summary views of inputs and model internals but with the capability to drill down and view detailed information
- Essential for quality control and debugging
- Smart tools – to assist in diagnosis of the ensemble inputs, data assimilation, and results

## Operations Concept – Ideas, cont.

- Probabilistic hydrograph displays should include an overlay of historical ranges
- Historical perspective of Model inputs
- Selectable MODS – parameter to designate a MOD for deterministic, ensemble or both
- Drawing tool to redraw forecasts – necessary for timeliness and expediency in problem cases

## Issues and Challenges

- Hydraulic routings – complicates the ability to operate at the segment level
- Model BIAS correction – error model
- DA – ability to trace what it did and provide capability to undo or nudge differently
- Reservoirs – ensembles on autopilot can introduce unrealistic results

## Slide 50

## Issues and Challenges, cont.

- Ability to use alternate model sources (inputs)
- Need to develop meaningful displays for end-users to illustrate the model inputs (focus group requirement)
- Hand-off / Receive mechanism for upstream / downstream RFCs

## NERFC

## NERFC Requirements

- At least 3-5 days of precip & temperature data input
- Generate hydrograph ensembles out 5-7 days using graphics that have been reviewed by customers
- Generate SHEF output for selected confidence levels (or other means of having data readily available in AWIPS database)

## NERFC Concerns

- How do you generate 72+ hours of statistical ensemble input when normal operations never exceed 48 hrs of QPF?
- WFO Customer likes “what-if” contingency scenarios that are based upon known models
- Training for NWS staff needs to be included as part of development

## NWRFC

## The Shooting Target...

- Meteorological forcing for the hydrologic models originates from multiple sources. The final product is a combination of these procedures.
- Sources: HPC, WFO ISC/GFE grids, HAS, the numerical model of the day.
- Can the methodology be developed to determine the uncertainty and address all the inputs to the hydrologic model?  How do we adapt these processes to the change in science and operational procedures?

## Scale...

- Meteorological forcing for the hydrologic models arrive at the RFC in many different scales. These are then converted to a base scale used in the hydrologic models.
- Can the current data networks adequately support the proposed level of downscaling? Will the uncertainty results be beneficial?
- Do we see a universal methodology to define the uncertainties generated by the downscaling?

## Transitions are where the money is...

- Defining the uncertainties is based on understanding the statistics between the forecast and the observation. Are we applying the uncertainties appropriately?
- Transitions or storms are where short-term forecasts make their money. The statistics and uncertainties during these events tend to shift from the climatology. Can we expect to provide a true picture of the uncertainties during the events that are most beneficial to short-term ensembles techniques?

## How does it work in a network...

- How do uncertainties translate from point to point in a hydrologic system?

## déjà vu...

- Short- and long-term ESP needs to address the man made influences; such as diversions and regulations.
- In the NWRFC area, over 90% of the forecast points are influenced by reservoir regulation and diversions. ESP is unable to work on a variable set of rules used by water managers.
- How do we quantify the uncertainty of forecaster MODS?

## Benefits...

- Hydrologic users like uncertainty information to assess the confidence in the forecast. Is there benefit and how is it determined?
-  what is the timeline of short-term ESP being operational?
-  will the uncertainty represent the true knowledge of the forecast error?
-  need a system to develop and keep track of the uncertainty component to the forecast.
-  NWS needs the ability deliver/provide more ESP derived information.
-  requires a verification and validation system.

## 1)

- 2) Similar displays or information as number 1, but for meteorological forcings.
- 3) Verification outputs which are converted to layman terms.

- Products Don wants...

## End of RFC presentations