# Presentation: 629540

## SUMMARY OF & RECOMMENDATIONS FROM THE4TH ENSEMBLE USER WORKSHOP

- Zoltan Toth
- Environmental Modeling Center
- NOAA/NWS/NCEP
- Acknowledgements:
- Doug Hilderbrand, Louis Uccellini, Steve Lord &
- Workshop Participants
- http://wwwt.emc.ncep.noaa.gov/gmb/ens/index.html

## 4th ENSEMBLE USER WORKSHOP

- Logistics
- May 13-15 2008, Laurel, MD, 70+ participants
- NWS Regions (7), Headquarters (15), NCEP (31)
- OAR (4), other government agencies (2), private (4), academic (4) sectors & international (3)
- For further info, see: http://wwwt.emc.ncep.noaa.gov/gmb/ens/UserWkshop_May2008.html
- Main Theme
- How to support NWS in its transition from single value to probabilistic forecasting
- Goal is to convey forecast uncertainty in user relevant form
- 28 presentations
- Significant progress since last workshop
- Reports from NCEP Service Centers and Regions
- E.g., first numerical ensemble-based 2-day tornado and week 3-4 MJO outlook
- Working groups
- Operational Requirements		-   Corporate outreach
- Ensemble forecast systems		-   Statistical post-processing
- Data access / distribution		-   Display / manipulation tools
- Role of forecasters			-   Training
- Outcome / Recommendations
- Prepared draft blueprint for transition from single value to probabilistic forecasting
- Plan for immediate steps (interim solution to be implemented in 2-3 years)
- Outline for long term solution and resource requirements (5-10 years)
- WGs to refine & execute plan
- All activities to be coordinated under NWS Forecast Uncertainty Program (NFUSE)

## BACKGROUND

- NRC Report on Completing the Forecast (2006)
- All major recommendations embraced
- NWS Forecast Uncertainty Program (NFUSE, 2007-)
- Corporate commitment and coordination
- AMS Ad Hoc Committee on Uncertainty in Forecasts (ACUF, 2007-)
- Enterprise-wide planning and coordination

**CURRENT STATUS**

- Scientific and technical preparedness
- High due to major advances in past 50 years
- Past initiatives
- MDL PoP and other probabilistic products since 1971
- NCEP ensemble forecasting since 1992
- Numerous other guidance & products by WFOs & NCEP
- Corporate policy
- Good vision:  Vision 2005 – NWS Strategic Plan (1998)
- Deliver Better Products and Services
- Provide weather, water, & climate forecasts in probabilistic terms (2005)
- Sporadic efforts - No coordinated execution
- Example: Major NWS forecast database (NDFD) lacks basic uncertainty information
***Need overarching plan & Coordinated execution***

## DRAFT PLAN

**Goal**
- Capability to answer any question related to future weather, climate, & water conditions, including ***forecast uncertainty***
- Example
*What is joint probability of heavy precipitation and strong wind?*
**Resource limitations**
- Computing power & storage, telecommunication, workforce training, etc, etc
**Two-stage approach**
- Interim stage
- Limited capability with approximations
- Achievable with reallocation of projected resources in 2-3 years
- Major enhancement of forecast system, leading toward long term solution
- Final stage
- Full capability
- Built on Stage-1 achievements
- Requires long term budget planning and commitments

## RECOMMENDATIONS – OPERATIONAL REQUIREMENTS

- Institute official requirement for probabilistic information for all NWS warnings & forecasts
***Expand NDFD midpoint estimate with 10 & 90 percentile values of forecast distribution***
- Other single value probabilistic information derived via fitted distribution
- Provide 20+ member ensemble trajectories to
- Drive professional user applications
- Derive any weather, water, climate foecast information
- Maintain user relevant verification statistics (with level of uncertainty incl.)

## RECOMMENDATIONS – ENSEMBLE FORECAST SYSTEMS

- Recognize ensembles as backbone of probabilistic forecast system
- Significantly increase HPCS computing capacity for operations & research
- Increase THORPEX funding for related research, development, transition
- Encourage NSF funding of related research (eg, model uncertainty)
- Develop high-impact event local ensemble forecast system for operations
- Variety of applications (hurricane, severe storm, floods, fire, etc)
- Consider uncertainties related to boundary conditions
- Improve ensemble methods for coupled systems
- Ocean/Atmosphere/Land ssytem, eg, MJO application
- Link initial perturbation techniques with analysis uncertainty information
- Explore methods for representing model related uncertainty
- Continue use of multi-model approach (SREF, NAEFS)
- Explore alternative methods (stochastic approach, parameter uncertainty, etc)
- Continue hind-cast related activities
- Operationally implement limited but affordable system
- Carry out research on benefits of expanding hindcasts vs. improving frcst system
- Study optimal configuration for operational systems
- Possibly terminate single high resolution forecast beyond quasi-linear regime
- ~8 days for GFS?
- Provide 1/3/6 hourly output out to 1/8/35 days

## RECOMMENDATIONS – STATISTICAL POST-PROCESSING

- Analyze/compare performance of different methods
- Suggested approach
- Use common data / metrics
- Special focus on high impact events
- Components for evaluation
- Bias correction of model forecasts (relationship between analyzed & forecast model variables, including ensemble distribution)
- Downscaling (perfect prog relationships between model variables and observed quantities)
- Generation of observationally based analysis (used as a proxy for truth)
- Improve methods to increase forecast reliability while maintaining sharpness
- Study sensitivity/requirement of different user groups to/for various levels of forecast reliability
- Evaluate what sample size is required for desired level of reliability
- Develop methods to combine information from diverse sources into single pdf
- Ensemble, hires forecasts, climatology, latest observations, regime dependent biases, etc

## RECOMMENDATIONS – DATA STORAGE / ACCESS

**Data types**
- Expanded NDFD data – 10/50/90 percentiles (3-fold increase in volume, short term)
- Multi-model ensemble – raw & bias corrected (~100-fold increase in volume, long term)
**Architecture**
- Distributed data access with mirrored sites (WOC-type)
- All ensemble data archived at NCDC
**Access**
- On demand pull (NOMADS-type, in contrast with on schedule push)
- Public access – NOMADS
- AWIPS-2 – Test with NOMADS until new software developed
- Link with display / interrogation tools below

## RECOMMENDATIONS – DISPLAY / MANIPULATION TOOLS

- Develop / adapt tools for probabilistic / ensemble guidance
- Graphical display of ensemble members / probabilistic info (spaghetti, stamps, etc)
- Migrate existing NAWIPS ensemble/probabilistic functionalities
- Statistical manipulation of uni-variate distributions
- 10/50/90 percentiles => Fitted distribution => Derive probability for any threshold & vice versa
- Choice of analytical distribution based on climatology
- Forecaster modification of 10/50/90 percentile fields
- Use existing, & develop more sophisticated ensemble-based field modification software
- Adjust ensemble distribution based on forecaster modification
- Use increment between distributions fitted to guidance vs. modified 1/90/90 percentiles
- Ensemble database interrogation
- To derive joint probabilities (and any other information)
- Diagnostics, perfect prog relationships (CAPE, helicity, etc),
- Verification software / results
- Use standardized software throughout forecast process / across organization
- Deploy necessary software in AWIPS2

## RECOMMENDATIONS – ROLE OF FORECASTERS

- Forecast process steps
- Ensemble guidance
- Statistical post-processing
*Forecaster modification*
*Product derivation*
*Interpretation*
*Outreach*
- Assess where forecaster can have maximum positive impact
- Which part of forecast process?
- High impact events?
- Use common internal format for probabilistic forecasts
- Internal format can be different from external products
- Common verification for evaluating value added at each step important
- Elevate role of forecaster
- Traditional role
- Make / edit forecasts
- New role
- Direct / QC forecast process
- Focus on high impact events
- Adaptively assign resources
- Interpretation of probabilistic forecasts, outreach

## RECOMMENDATIONS – TRAINING

**Universities**
- Build into existing courses (hard to add new courses)
**On the job** (for forecasters and sophisticated external users)
- Prepare inventory of, and provide easy access to existing training material
- NCEP, Regions, WFOs, COMET, MSC, WMO, etc.
- Develop online curriculum with multiple entry points and levels of advancement (WFO, RFC, Nantional Centers)
- Chaos, statistics, ensemble techniques/data, modification & interpretation of probabilistic guidance, user applications (eg., emergency management – WCM)
- Case studies of high impact, large anomaly events for different regions
- Hosted by NWS, COMET, NOAA Learning Center?
- User feedback via blog?
- Provide mechanism for incorporation of locally developed material
- Periodically update training material
- Track training as part of official job requirement

## RECOMMENDATIONS – CORPORATE OUTREACH

- Use **ACUF** as NOAA’s “official” interface with enterprise
**Utilize other channels**
- NWS Partner’s meeting, AMS/NWA conferences, workshops, journal articles, customer education
- Broadcaster conferences
- Assess **end-user operational requirements**
- Challenge as requirements evolve with availability of new products
- Survey focused on Emergency Management and Transportation needs
- Build on success of NURO survey
- Use WCMs expertise to bridge the gap between
- Expanding product/service suite &
- Evolving user needs

## 2-STAGE PLAN

**Stage 1 – Interim capability (2-3 yrs)**
- Reliable 10/50/90 in NDFD database
- Forecasters edit fields
- “Official forecast”
- All questions on uni-variate distributions can be answered via AWIPS-2 tools
- “Guidance” info available for
- Forecasters
- Web
**Stage 2 – Full calability (5-10 yrs)**
- ~100 bias corrected ensemble members in NOAA database
- Forecasters edit 10/50/90 corresponding to ensemble
- Forecaster edits back-propagated to ensemble
- All questions on future weather/climate can be answered via AWIPS-2 tools
- Ensemble trajectories consistent with official forecast for user applications

## NEXT STEPS AFTER WORKSHOP

- Finalize recommendations
- Prepare short workshop report (BAMS)
- Report to Forecast Uncertainty oversight board
- Carry on WG activities
- Develop coordinated plan for next 2-3 yrs
- WG leads to coordinate activities within and across WGs under NFUSE
- Seek long term funding through PPBES
- Engage with ACUF – example products?
- Plan for next workshop - Oct 2009

## ISSUES FOR FURTHER DISCUSSION

**Midpoint estimate**
- 50%, mean, or mode?
**Where to store forecast uncertainty info?**
- NDFD
- Separate probabilistic NDFD dataset?
**Official forecast vs. guidance**
- 10,90 part of expanded “official” forecast? Human reviewed / edited if needed
- Rest of derived products “guidance”?
**Forecaster edits**
- 10,90 – official forecast
- 25,75 – May be easier to learn/train?
**Assembled training material**
- Hosted by NWS, COMET, NOAA Learning Center?

## BACKGROUND