# Presentation: 476411

## A New NWP Course:  Effective Use of NWP

- Bill Bua and Stephen Jascourt, UCAR/COMET NWP Team
- 9 December 2008

## Presentation Outline

- Why a new course, what’s different?
- Approach - follow AWOC model
- Needs assessment survey, results
- Course outline/structure
- Course development
- Open issues

## Why a new course? Why forecast process?

- NWP training 1999-2008 mostly model-specific, focus on model
- 		- Meets many needs, however...
- SOOs and ISST ask for more context-driven training
- SOOs and ISST want WES and practical exercises
- Forecasters want training more applied to a forecast problem
- Primary NWS forecaster product using NWP: grids.
-  	Paradigm shift – NWP products not just in D2D but also in GFE
-  	Forecasters want help in using NWP to more efficiently make better grids
- NSTEP for FY2008 and FY2009 identified #1 NWP priority as “Effective use of NWP”

## 
Place NWP into the context of the NWS WFO 

Make use of AWIPS or AWIPS-like output wherever possible

Focus on where/when forecasters can add value to NWP

Help forecasters effectively use NWP in performing their daily tasks

- Place NWP into the context of the NWS WFO
- Make use of AWIPS or AWIPS-like output wherever possible
- Focus on where/when forecasters can add value to NWP
- Help forecasters effectively use NWP in performing their daily tasks

## Survey, design, collaborate (SOOs, etc.), consolidate – 
			“1-stop shopping”

SOOs, ISST highly recommend Winter AWOC-type structure:
WES cases
Contributions from folks in the field who are leading the way
Resources all co-located and well organized
Not so happy with 1-size-fits-all, want more flexibility, not all lectures

WDTB visit – learn what they did:
Needs assessment survey  training topics, priorities
Needs assessment survey  experts – collaborators/contributors
Prepare how to manage a project with so many contributors
Kick-off meeting to get everyone on same page, set expectations, goals
Carry it out, remind, edit, keep on focus, keep on track...

- 			“1-stop shopping”
- SOOs, ISST *highly *recommend Winter AWOC-type structure:
- WES cases
- Contributions from folks in the field who are leading the way
- Resources all co-located and well organized
- Not so happy with 1-size-fits-all, want more flexibility, not all lectures
- WDTB visit – learn what they did:
**Needs assessment survey**  training topics, priorities
**Needs assessment survey**  experts – collaborators/contributors
- Prepare how to manage a project with so many contributors
- Kick-off meeting to get everyone on same page, set expectations, goals
- Carry it out, remind, edit, keep on focus, keep on track...

- Advanced Warning Operations Course (AWOC) Model

## Survey – sample (slide 1 of 2)

- Name (optional):                                          Office (optional):
- 1. What aspects of using models and model guidance products in the operational forecast process need improvement in your office (or region)? Please explain if necessary.
- 2. What aspects of using models and model guidance products in the operational forecast process are not in much need of improvement in your office (or region)? Please explain if necessary.
- 3. Prioritize the following topics of instruction from 1 to 8 (with 1 being the highest) in terms of a perceived need for training over the next 2 to 3 years. Please check the box in the right column next to those topics that are particularly urgent.

| Topic | Priority | Urgent? |
| --- | --- | --- |
| Use of NWP at global-continental scales |  |  |
| Synoptic to meso-a |  |  |
| Meso-a to county scale |  |  |
| NDFD/FPS scale |  |  |
| Model data sources/characteristics, including post-processing |  |  |
| Formulating the forecast problem as a testable hypothesis using available model runs |  |  |
| Using NWP to assess forecast uncertainty |  |  |
| Probabilistic NWP |  |  |
| (add up to 3 suggestions of your own in the fields below) |  |  |
|  |  |  |
|  |  |  |

## 4. Briefly describe up to 3 potential new instructional topics that would be useful in your office or region for improving operational use of NWP. 

5. What innovative tools (software applications, forecasting techniques, etc.) are used in your office (or region) for model evaluation and assessment during the forecast process?

6. Is there a local training module, such as a WES case, that would potentially provide good training on best practices in the use of NWP? If so, please provide a brief description and let us know if you would consider allowing us to adapt it for a national audience.
7. Have you or someone in your office developed, or, is interested in developing, a training module on NWP best practices? If so, please provide a brief description and let us know if you would consider allowing us to adapt it for a national audience.

8. Do you think the forecast funnel approach (evaluating largest to smallest scales) is the best approach in the NWS operational environment? Are there other approaches we should consider? If so, please provide a brief description.

- 5. What innovative tools (software applications, forecasting techniques, etc.) are used in your office (or region) for model evaluation and assessment during the forecast process?
- 6. Is there a local training module, such as a WES case, that would potentially provide good training on best practices in the use of NWP? If so, please provide a brief description and let us know if you would consider allowing us to adapt it for a national audience.
- 7. Have you or someone in your office developed, or, is interested in developing, a training module on NWP best practices? If so, please provide a brief description and let us know if you would consider allowing us to adapt it for a national audience.
- 8. Do you think the forecast funnel approach (evaluating largest to smallest scales) is the best approach in the NWS operational environment? Are there other approaches we should consider? If so, please provide a brief description.

- Survey – sample (slide 2 of 2)

## Survey Results – who responded

- 42 responses
- 11 anonymous, no location
- 27 WFOs shown on map in white
- no OCONUS (unless anonymous)

- 2 later responses so far, all comments overlap the 42 analyzed surveys

## Survey Results – question 1

**What aspects of using models and model guidance products in the operational forecast process need improvement in your office (or region)?**

- Voluminous responses, of great variety! Categorizing and summarizing...
- _Ensemble/probabilistic_
-  “Effective use”, “know how to make good use”, etc.  - SREF, GEFS, NAEFS
-  Data digestion – how to quickly assess, better AWIPS access, better display
-  Assessing uncertainty and probabilistic forecasting (e.g., POP)
- _MOS_ – how use in forecast process, ensemble MOS, gridded MOS
- _High-resolution models_ (say ****x < 5 km)
-  Use in forecast process, differences from meso/synoptic scale models
-  Interpretation, simulated reflectivity, convective mode
-  Local initialization (1 response)
- _Basic NAM/GFS/RUC/etc._
-  Strengths, weaknesses, keeping up with changes, convection
-  Model bias – too fast/slow, p-type, regime-based errors, etc.
-  Comparisons – which is best for a particular phenomenon/scenario
-  Interpretation – believability, predictability

## Survey Results – question 1, continued

**What aspects of using models and model guidance products in the operational forecast process need improvement in your office (or region)?**

- _QPF_ – general (singled out among forecast parameters)
- _Forecast process_
-  Gridded forecast generation
-  Observations – pay attention, check model initializations, everything starts with obs
-  Where can human add value, what to focus on, how good is “good enough”
-  Verification
-  Model comparisons, “model of the day” (or not), how determine which to use
-  Warn-on forecast (1 response)
- _Aviation_ – especially ceiling and visibility grids (1 response)
- _AWIPS-related and diagnostics_
-  Diagnostics (q-vectors, frontogenesis, etc.), what useful, how best utilize
-  GFE downscaling vs. model-resolvable features, downscaling, GFE
- _Organization of information _– concise, one-stop shopping

## Survey Results – question 2

**What aspects of using models and model guidance products in the operational forecast process are **_**not**_** in much need of improvement in your office / region)?**

- Only 28 responses – 9 said none or not sure!
- Remainder mostly were large-scale, knobology, decoding text bulletins.
- Several specifically said large-scale aspects of the forecast are not what the forecasters spends most time on.

## Survey Results – question 3

- Prioritize the following topics of instruction from 1 to 8 (with 1 being the highest) in terms of a perceived need for training over the next 2 to 3 years. Please check the box in the right column next to those topics that are particularly urgent.

- 57% total
- 1 or more topics

## Survey Results – question 4

**Briefly describe 3 potential new instructional topics useful to your office or region for improving operational use of NWP**

- Categorizing and summarizing...
- _Ensemble/probabilistic_ (most popular topic area, wide variety of focus areas)
-  Probabilistic forecast products, conveying uncertainty to users
-  Anomalies, extreme events
-  COMAP topics (2008)
-  Ensemble clusters, ensemble probabilities, new tools
- _Initial condition assessment, data assimilation_ – general, practical exercises
- _High-resolution models_ (say ****x < 5 km)
-  Interpreting, making sense of output
-  When is it better/worse than coarser resolution models
- _Forecast methods/techniques_ – QPF, downscaling, identifying when models deficient
- _Model architecture_
-  Keeping up with changes
-  Boundary layer
-  Complex terrain
- A few misc. other topics (MOS, tropical convection effect on long wave pattern, etc.)

## Survey Results – question 5

**What innovative tools (software applications, forecasting techniques, etc.) are used in your office for model evaluation/assessment during the forecast process?**

- 10 said none. 27 described something...
- _Ensemble/probabilistic_
-  Online (many, various sites)
-  Populate GFE using blend of last 2 SREF runs
-  Using NAEFS downscaled T and wind in GFE
- _MOS_ – trend assessment, database for tracking biases
- _Smart tools_
-  BOIverify – lots of responses
-  Local p-type tool
- _Other_
-  D2D procedures to compare model initializations
-  Model bias – too fast/slow, p-type, regime-based errors, etc.
-  Local models – LAPS hot start, MODIS SST

## Survey Conclusions

- Results showed:
-  NWP training still vitally needed in forecast offices
-  Training on effective NWP use *in the context of the forecast process* will meet many needs
-  Training needs group into four categories
- Forecast process (including “pay attention to observations, analyze/think!”)
- Core model topics (bias, diagostics, what’s in the model, etc.)
- Uncertainty/probabilistic (focusing heavily on use of ensembles)
- Local-scale forecasting for NDFD (high-resolution models, downscaling, etc.)
- Proposed course on effective use of NWP was (re)designed based on survey responses
- Some needs can be met by existing core NWP Team activities such as model matrix updates

## Course Content

***IC1***: Overview and orientation [short]
***IC2***: The NWP Model  [updating existing COMET NWP Course]
***IC3***: NWP in the Forecast Process
- 	     [adding value to NWP, diagnostic analysis of NWP, etc.]
***IC4***: Moving Toward High-Resolution and Probabilistic Forecasting
- 	    [interpretation of hi-res model data, local models, using uncertainty in the         forecast process, probability and extreme events, etc.]
***IC5***: NWP in the IFPS Era [model bias, downscaling, local features, etc.]
***IC6****:* New Forecast Paradigms
*	**   *[AWIPS2, Probabilistic forecasting on NDFD grids, hypothesis testing,     finding useful data in the data spigot, etc.]

**FY2009****	****FY2010-2011****	**
**(IC=Instructional Component)**

## NWP Course FY2009 Deliverables and Learning Path

**IC1: Course Overview/orientation:**
-  Integrated into forecast process, IFPS
-  moving toward hi-res and probabilistic NWP
-  new paradigms for using NWP products

**IC2: The NWP model** (an update of previous NWP course material)

**IC3: NWP in the forecast process**
** **Lesson 1: Observations and the fcst process
-  Lesson 2: Optimizing use of operational NWP products*
-  Lesson 3: When does the human add most value?
-  Lesson 4: When is “Model of the Day” valid? (or not!)
-  Lesson 5: Gleaning more information from NWP 	  models and EPS: Model diagnostics @*

**IC 4: Moving toward high-resolution and probabilistic forecasting**
-  Lesson 1: Effective use of hi-res models
* *Lesson 2: Using local models to forecast microscale features
-  Lesson 3: Forecasting in complex terrain and coastal regions
-  Lesson 4: Using NWP and EPS to assess forecast uncertainty @
-  Lesson 5: Using probabilistic and uncertainty information in the forecast process
-  Lesson 6: Using probability and uncertainty information in extreme events @

- Other resources:
- NWP/Ens Matrix
- (ongoing
- development)

**Needs legend:**
- Need WES case(s) only
- @Need *help* w/module

**Notes:**
Commitments from:

IC3:
Todd Lericos, CAR (WES case for lesson 2)
Bill Bua (module for lesson 4)
Eric Thaler, BOU (Module on QG for lesson 5)

IC4:
Brett McDonald, RIW (Module and WES case for lesson 2)
Warren Blier, MTR (WES case for lesson 3)
Rich Grumm, CTP (WES case for lessons 4 and 6, Bill Bua to develop modules for lessons 4 and 6)
Ken Cook, ICT (Module for lesson 5) and Chris Smallcomb, RNV (WES for lesson 5)

## NWP Course FY2010 and 2011 Deliverables and Learning Path

- FY2010
**IC 5: NWP in the IFPS Era**
-  Lesson 1: Forecast funnel in the NDFD era†
-  Lesson 2: Model bias and NWP model architecture†
-  Lesson 3: Model bias correction and its limitations*
-  Lesson 4: Downscaling large-scale model data to the local area*
-  Lesson 5: Starting points for IFPS grids*
-  Lesson 6: Fine spatial scales at long time ranges †
-  Lesson 7: Using local data to initialize local models

- Development waits until topics mature, might be 2011
**IC 6: New forecast paradigms**
-  Lesson 1: Quick assessment of model data – wading through data spigot**‡**
-  Lesson 2: Formulating forecast problem as testable hypothesis**‡**
-  Lesson 3: Probabilistic forecasts on the NDFD grid (prerequisite IC5, lesson 4)**‡**
-  Lesson 4: Warn-On Forecast**‡**

- FY2010 and FY2011 and every year:
- Other resources:
- NWP/Ens Matrix
- (ongoing
- development)

**Needs legend:**
-  Need WES case only**† **Need WES and module
**# **Need Module only
**‡**** **Requires research

- from FY2009 ...

## Future training possibilities

- AWIPS II
- Pull capability (IOC 1/2011, FOC 4/2012)
- WFOs decide what to pull in based on situation
- 3-D visualization (IOC 12/2013)
- What capabilities will 3-D visualization bring to WFO?  Another training opportunity
- NCEP paradigm shift to “ENVIRONMENTAL prediction” in a real way
- Aerosols, O3, sea ice, coupled models of various kinds (e.g. ocean), etc.

## Course Development IC3,4,5,6

- NWS field contributors: content and reviews (modules and WES cases)
- The contributors are a vital component
- Without contributors, will lose operational context
- COMET produces modules, makes into *learning* materials
- WDTB produces WES cases (limited staff time)
***May*** need some help from modeling personnel, but amount expected to be similar to recent years, limited

## Winter AWOC: 	2 1⁄2 -day in-person meeting, key to success
	Reduced development/delivery time ~ 33% (WDTB estimate)

	Served many functions
Raise importance and value of project
Demonstrate top-down commitment from the NWS 
Stimulate discussion among contributors, helping define and refine course learning objectives and content
Train contributors how to design effective learning modules (i.e. not just create a lecture) 
Explain collaborative development system, given the complex nature of the whole project with so many contributors 

NWP Course: 1-day virtual meeting – no travel $ or time, similar goals

- 	***Reduced development/delivery time ~ 33% ***(WDTB estimate)
- 	Served many functions
- Raise importance and value of project
- Demonstrate top-down commitment from the NWS
- Stimulate discussion among contributors, helping define and refine course learning objectives and content
- Train contributors how to design effective learning modules (i.e. not just create a lecture)
- Explain collaborative development system, given the complex nature of the whole project with so many contributors
**NWP Course:** 1-day virtual meeting – no travel $ or time, similar goals

- Contributors’ Kick-Off Meeting

## Remaining Open Issues...

- We need SSDs to help us identify additional contributors, especially in the development “out years” (FY10 and beyond)
*“*Buy-in” from MICs so contributors can spend time making their contributions
- Want contributors to feel valued, that resulting course will be worthwhile, unique, useful, and of  top quality.
- Need promotion of NWP course by top brass including you