# Presentation: 668174

## Advances in Uncertainty Guidance:Understanding why the uncertainty looks the way it does

**Jim Hansen**
- NRL MRY
- Probabilistic-prediction Research Office
- _[Jim.Hansen@nrlmry.navy.mil](mailto:Jim.Hansen@nrlmry.navy.mil)_
- Jim Goerss, NRL
- Buck Sampson, NRL
- LCDR Ken Wallace, NPS
- Dan Gombos, MIT

**Notes:**
I’m a new NRL employee who was lured to Monterey from academia.

This is a proposal about using forecasting as a tool to better understand the underlying system (the atmosphere, in this case), and represents the continuation of 6.1-ish work that I did last year at NRL MRY during my sabbatical

## Objectives

- Introduce GPCE-AX
- TC track skewness guidance
- Using wind speed probabilities for TCCOR guidance
- Ensemble-based sensitivities for identifying the forecast problem of the day

## GPCE-AX

## Goerss Prediction Consensus Error (GPCE)

- Produces state-dependent, isotropic track error estimates.
- Multivariate linear regression
- Predictors include
- Ensemble spread
- Initial intensity
- Predicted longitudinal displacement
- Displays 70% probability isopleth
- Reliable, and sharp relative to climatology

## GPCE

- GUNA

- Consensus

- Verification

## GPCE Along/Across (GPCE-AX)

- Produces state-dependent, anisotropic track error estimates (ellipses).
- Eigenvectors are across-track/along-track directions
- Includes bias correction
- Multivariate linear regression
- Predictors include
- Across/along ensemble spread
- Initial intensity
- Initial longitude
- Predicted longitudinal displacement
- Displays 70% probability isopleth
- Reliable, and sharp relative to GPCE

## GPCE-AX

- GUNA

- Consensus

- Verification

## GPCE

- GPCE-AX

- GUNA

- Consensus

- Verification

## Reliability and sharpness

- GUNA, 2 predictors, trained using 2002-2005, tested using 2006

| Lead | GPCE
70% reliability | GPCE-AX 70% reliability | Fractional difference between 70% areas | Sample
size |
| --- | --- | --- | --- | --- |
| 12hr | 0.80 | 0.77 | 0.31 | 193 |
| 24hr | 0.77 | 0.76 | 0.13 | 173 |
| 36hr | 0.81 | 0.73 | 0.19 | 157 |
| 48hr | 0.82 | 0.77 | 0.10 | 136 |
| 72hr | 0.82 | 0.80 | 0.11 | 98 |
| 96hr | 0.79 | 0.65 | 0.10 | 68 |
| 120hr | 0.84 | 0.70 | 0.12 | 50 |

## Probability of left/right/fast/slow (P-LRFS)

- Logistic regression
- Use 70% as a warning threshold.
- Simple tests indicate utility.
- Not clear how to communicate.

- Predictor

- Predictand

- 0

- 1

## Slide 11

## Slide 12

## Slide 13

## Slide 14

## There is a greater than 70% chance

- that verification will fall left of center.

## Slide 16

## GPCE conclusions

- GPCE-AX outperforms GPCE.
- P-LRFS provides value for hedging (skewness-like information).

## Probabilistic TCCOR guidance

## Example: TCCOR

**TCCOR 4** - Destructive winds are _*possible*_ within **72** hrs.
**TCCOR 3** - Destructive winds are _*possible*_ within **48** hrs.
**TCCOR 2 **- Destructive winds are _*anticipated*_ within **24 **hrs.
**TCCOR 1 **- Destructive winds are _*occurring or anticipated*_
- 		within **12 **hrs.

- Not entirely sure how to quantify *destructive winds*,
*possible,* or *anticipated*

**Notes:**
TCCORs are Tropical Cyclone Conditions Of Readiness, and going to different TCCORs means that bases have to spend money to do things to prepare for the storm.

This is the way TCCORS are currently defined.  We aren’t necessarily provided with information about what “possible”, “anticipated”, and “occuring or anticipated” mean.  It’s probably for good reason.  Different locations are going to have different definitions.

## Example: TCCOR

**TCCOR 4** – Probability of winds exceeding 50kts within 		**72** hrs is greater than 20%.
**TCCOR 3** – Probability of winds exceeding 50kts within 		**48** hrs is greater than 20%.
**TCCOR 2 **– Probability of winds exceeding 50kts within 		**24** hrs is greater than 50%.
**TCCOR 1 **– Probability of winds exceeding 50kts within 		**12**hrs is greater than 80%.

*Destructive winds **=> 50kts*
*Possible **=> 20%*
*Anticipated **=> 50%*
*Occurring or anticipated **=> 80%*

- I made
- these up

**Notes:**
Here is one example of how probabilistic forecast information could be used to help aid the setting of TCCORs.

I made these up, but imagine that “destructive winds” means 50kts.  And imagine that “possible” means 20%, “anticipated” means 50%, and “occuring or anticipated” means 80%.  These are made up, and we expect that they should be different for different locations.

Now NMFC forecasters can be alerted when probabilistic wind forecasts indicate that different thresholds have been crossed and can decide if the TCCOR recommendation should be changed.

## How decide on probability threshold values?

- If we assume that the forecast probabilities are correct, then the threshold for action is given by the ratio of the cost of taking action (evacuation, etc.) to the loss associated with a bad thing happening if no action is taken (deaths, damage, etc.).
- Typically, COST << LOSS, so the probability threshold is low.
- Implies thresholds will be different for different locations.
- In reality, the relevant LOSS appears to be the CO’s career, and so COST/LOSS is infinite!
- Instead, take a retrospective climatological approach.

**Notes:**
What about these thresholds?  How might they be set?  It turns out there is a right answer and it is given by the cost-loss ratio.

The cost is the cost incurred by taking action to protect against a bad thing happening, while the loss is the loss incurred if a bad thing happens and you didn’t take action to protect yourself.

For something like TCs, the cost of taking action is expected to be much much less than the loss associated with not taking action, so the cost-loss ratio is small and you take action when the probability of occurrence is small.

This also provides a nice frame work in which different locations will have different thresholds.  It may cost more or less to protect at different bases, and the potential losses may be more or less.

## Four locations of interest

**in WestPac:**
**1. Identify storms **that resulted in TCCOR at each location for as many years
- as possible (’02 – ’07)
**2.** **Collect TCCOR times**
- from any available sources (log books, storm reports, etc.)
**3.** **Collect observations** from
- each site corresponding to the
- TCCOR times (storm reports, AFCCC, etc.).
**4.** **Compare Observed conditions with historical TCCORs.**

**Yokosuka**

**Sasebo**

**Guam**

**Kadena**

- 2004

- Wallace, NPS

## TY 14W “NABI”

- Kadena Wind P = 69.7%
- Sasebo Wind P = 10.53%
- 	090306Z

## TY 14W “NABI”

- Sasebo Wind P = 50.63%
- 	090406Z

## TY 14W “NABI”

- Kadena Wind P = 0.0%
- Sasebo Wind P = 69.1%
- 	090500Z

## TY 14W “NABI”

- Kadena Wind P = 0.0%
- Sasebo Wind P = 77.09%
- 	090512Z

## Relative

- frequency

- 5 day 50kt wind speed probability when TCCOR was set

## TCCOR guidance conclusions

- Cost/loss models for selecting relevant wind speed probabilities are problematic.
- Retrospective climatological approach provides the thin end of the wedge.

## Ensemble sensitivities

- Lots of high-level interest in ensembles
- NRC “Completing the Forecast”
- Little motivation for using ensembles
- Lower resolution than deterministic model
- Time constraints
- Training, assessment, and majority of products are deterministic

## Ensemble sensitivities

- Often hear scientists complain that forecasters “don’t understand ensembles”.
- Unlikely. I’ve not yet found a forecaster who doesn’t understand that errors in initial conditions and, especially, models mean that the future state of the atmosphere is uncertain.
- In my view, the real issue seems to be that uncertainty guidance is of limited value without information about why the uncertainty looks the way it does.

## Ensemble track

- Landfall location

- Actual track

**Notes:**
Here is an example of how we can use ensemble information to tell us not only about uncertainty, but also about *why* the uncertainty looks the way it does.

This is an example of an ensemble forecast of tropical cyclone tracks.  The black dots show the track of the storm over three days.  The green lines show the ensemble forecast tracks from the forecast launched at the time when the storm was the furthest to the SE.  The magenta dots indicate the locations at which each of those ensemble members made landfall.

As you can see there is a lot of uncertainty in where the storm will make landfall.  A forecaster might be thankful that they had this information, but they are also going to want to know *why* all that uncertainty is there.

This information can be provided.

## Ensemble mean 500mb at 6hrs

- Sensitivity of latitude of landfall to
- 500mb heights at hour 6 of forecast

**Notes:**
The plot on the left is the ensemble mean 500mb height at hour 6 of the forecast (the location of the second black dot on the previous slide).  The big red blob is high pressure sitting over Japan.  The blue blob over the Philippines is the storm itself.

The picture on the right is the sensitivity of the latitude of landfall (happening around day 3) to the 500mb height field at hour 6.

The colors are sensitivities, not 500mb heights themselves!

Blue colors means there is a negative correlation between landfall latitude and the 500mb heights.  That means as the 500mb heights get smaller, the latitude of landfall gets bigger.

Red colors means there is a positive correlation between landfall latitude and 500mb heights.  That means as the 500mb heights get smaller the latitude of landfall also gets smaller.

What this plot tells us is that *if* the 500mb heights associated with the forecast high at hour 6 of the forecast are smaller than expected, then the latitude of landfall is expected to be large.  On the other hand, if the 500mb heights associated with the forecast high at hour 6 of the forecats is larger than expected, then the latitude of landfal is expected to be low.

The positive (red) sensitivities in the right plot indicate that if the storm is stronger at hour 6 than is expected by the mean (lower heights), then the latitude of landfall is also expect to be low.

The idea of this guidance product is that it provides the forecaster at a glance what the forecast problem of the day is.

## TC Landfall Location

**TC Landfall Location**

**6hr 500mb height at 122E, 16N **

## Slide 34

## Typhoon Sepat

- Ensemble track

- Landfall location

- Actual track

- Ensemble mean 500mb at 6hrs

- Sensitivity of latitude of landfall to
- 500mb heights at hour 6 of forecast

**Notes:**
The combination of these three types of images 

O The ensemble associated with the event/parameter of interest
O The ensemble mean field of interest
O The sensitivity of the event to the field

could be sent to forecasters over MetCast or ATCF. 

## Ensemble sensitivity conclusions

- Provides opportunity to identify why the uncertainty looks the way it does.
- Provides the opportunity to hedge forecast based on new observations.

## Conclusions

- Can usefully add across/along and skewness information to track guidance.
- Can use retrospective climatological information to identify probability thresholds.
- Can use ensemble sensitivities to identify forecast problems of the day.