# Presentation: 321603

## Model Jumpiness and Lagged Average forecasts

*Richard Grumm*
*And *
*Bill Bua*
*United States National Weather Service*

## OBJECTIVES

- Get a fix on some aspects of uncertainty and be able to recognize uncertainty in forecasts.
- Provide  examples of uncertainty in the NCEP GFS and its impact on ensemble prediction systems via examples of model jumpiness.
- Jumpiness often viewed in a single model or a single deterministic forecast.
_***Define model jumpiness as cycle-to-cycle changes in the forecast from a single deterministic model***_
- Lagged Average forecasts (LAF) and single model uncertainty

## Define model jumpiness

*the changes or differences of forecasts of features and parameters from cycle-to-cycle runs from a single numerical model. These inconsistencies may span intensity, gradient, and location of a feature or parameter.*
*jumpiness is normally caused by changes in the initial conditions from one cycle to the other. Thus, it serves as a simple ensemble forecast system alerting the user to possible forecast problems related to initial conditions.  *

## Model Jumpiness in a model 

- How we visualize _**uncertainty**_ in deterministic models on a daily basis. Some common things include:
*Significant run-to-run differences of distinct features such as*
*A change the track and intensity of a cyclone, anticyclone or frontal system.*
*Shift in the precipitation shield north, south, east or west*
*Timing changes *
*The problem is typically *
*worse at longer forecast range though not always.*
*A function of scale and mesoscale details can be quite changeable*
*Like the rain snow line or area for heavy snow/rain*
*Remember errors grow rapidly with smaller scale features*
*Most likely the results/impact of uncertainties in initial conditions (IC’s)*
*Not to mention differences between different models!*

## Leveraging Jumpiness

- Jumpiness unavoidable when dynamic models use imperfect initial conditions.
- Jumpiness is an indication of uncertainty.
- A potentially significant feature is impacting the forecast.
- Most recent forecasts are considered more accurate
- Use information to acknowledge uncertainty and lower confidence in the forecast. The inconsistency is a source of information
- Do not be fooled by the lack of jumpiness either.
- Examine an ensemble prediction system.

## Jumpiness History Lagged Average Forecasting

- Developed in early 1980s
- Prior to ability to run EPS
- Hoffman and Kalnay 1983 (Tellus) presented the Technique
- Used older forecasts (T-12,T-24,T-36) to construct ensemble
- Compute mean and spread
- Need to weight most recent forecast higher
- Also looked at forecast valid at same time (***dProg/dt)***
***Quickly can high light areas of high/low uncertainty***
***Hamill (2003) ***
***used 23-years of data from a fixed model and lagged forecasts to statistically demonstrate that extrapolation of a forecast trend (dProg/dt) had little or no forecast value.***
*Did not mitigate information of the jumpiness just ability to project it into the future**** trends can change!*

## Example of Model Jumpiness

- Sorry this is case of the Eastern United States
- Surely, they occur everywhere this was an extreme case and thus worth a quick show.
- Some are subtle and some are not so subtle.
- We will see this in a South African Case
- At finer scales jumpiness is more pronounced
- The devil is in the details.

## 4 GFS Runs  Big cyclone disappears?All images valid the same time!

- Big cyclone most of PA might be rain.

- Weak storm...PA is now cold...snow in SE?

**Notes:**
Lets get right to it for a case. Our goal is multi-fold we want to show how model trends change and you should not attempt to anticipate them to correct a models forecast. And we want to show how much uncertainty there can be in one model.

Here we compare 4 forecasts from the GFS initialized at 12-hour increments all valid at 0000 UTC 5 Feb. We start out with a big storm east of the mountains from the 0000 UTC 31 Jan 2006 forecast. But 12 hours later...its gone and we have a weak coastal low. 

We show MSLP forecasts and the departures of these forecasts in standard deviations from normal. Thus we see a very anomalous storm which move eastward and weakens with successive forecasts.

Big impacts here on QPF types and amounts! Rain implied along the coast for the oldest forecast then snow. Not to mention the changes in sensible weather elements like winds. Imagine the impact on the MOS forecasts from these 4 forecasts!

## Return of the cyclone?all images valid the same time!

**Notes:**
We are focused on the same valid time but have moved our initialization times foreward. 

Initially, we see the trend for weaker cyclone along the coast. Then...suddenly, a stronger storm...tracking farther west is forecast. The impact on winds in the Northeast and Mid-Atlantic are potentially significant. Not to mention the distribution of precipitation which we do not show here, but can infer.

Note that the trend for a weaker more eastern track reversed. 

## Things cannot get so bad so fast!Or can they as Robin might say “Holy short-wave Batman”

**Notes:**
Our trend continues only now we have a return of the western tracking and deeper cyclone. Note the big difference in the cyclone position and intensity in the two lower panels, initialized just 12 hours apart.

The 1200 UTC 2 Feb forecast looks surprisingly similar to the 31 January forecast we saw in the first slide for this case!

## Getting closer to event time...still lots of uncertainty

**Notes:**
We moved the valid time back 12 hours to 12Z 4 Feb. But note the trend for a more westward cyclone with a lower central pressure.

Imagine the impact on your forecasts of temperatures, dew points, winds and clouds if you were forecasting for say  Richmond, VA or New York, NY.

MOS from the GFS had to have been rather inconsistent and sensible weather forecasts must of varied markedly ...been quite jumpy....as each new run came in and was used in the forecast process.

## Well...it passed to our West!Warm windy winter rain

**Notes:**
These forecasts are valid at 0000 UTC 5 Feb. They focus on the storm now safely passing to the west of most of the east coast cities. In the winter, this implied a warm rain in most areas. And quite windy. Though the upper left panel implied winds nearly 180 degrees out of phase and much lighter over the east coast relative to the other forecasts.

## “Jump” right into some points

**The GFS showed run-to-run inconsistencies**
*These inconsistence ****uncertainty.*
*Causes**** same model each time suggests uncertainty in the initial conditions. The need for multiple sets of IC’s*
**Significant impacts on sensible weather elements**.
*Areas and amounts of rain or early on, snow*
*POPS and temperatures *
- Winds to include direction changes of over 180 degrees!
**We need to acknowledge, visualize, and be deal with uncertainty and quantify it.**
**Do you think this case is unique?** *It happened within 7 day of this event* and it ***does all the time***!

**Notes:**
Sum up the slide above. This case is and was not unique!

## “Jump” right into more points

**Run-to-Run inconsistencies **
**even at 6-hr increments- impacts of IC’s**
**Shorter range forecasts******** Big Storm**
**But we had problems with the location and intensity**
**Still hard to get the details nailed down **
**Winds direction and rain snow line looked elusive**
**Did not show QPF but it too must was hard to nail down.**
**At the smaller scales, Province and city levels, the details due to jumpiness still remain elusive.**
**Value of an ensemble********probabilities and highlighting areas of high uncertainty.**

**Notes:**
Sum up the slide above. This case is and was not unique!

## 30 August GFS forecast heavy rains-Front and Ernesto

## 31 August GFS Heavy rain forecastsaxis/location heavy rain and end time

## More relevant examples 

- We will tie jumpiness to the LAF concept
- Compute the mean of 4 successive forecasts
- Show the mean and the spread of these forecasts
- We will focus on two examples over the southern portion of the African continent
- This will localize the examples
- Shows this can and of course, does occur anywhere
- Shows value of this poorman’s ensemble and the value of the high resolution model.
- The need for a good ensemble suite of data

##  Review: Lagged Average Forecasts

- A quick ensemble of previous forecasts from the same model
- Data all valid at the same time
- Older forecasts can be used need to be used
- More recent forecast considered more skillful
- Compute
- Mean
- Spread see where the differences are
- Can be due to speed, propagation and intensity errors.
- Value in the areas of uncertainty
- Uncertainty like related to impacts of initial conditions.
- We limited our mean/spread to 4 recent forecasts.
- At longer ranges there are more forecasts available
- But remember longer ranges are generally less skillful

## Computing the LAF

- We will use 4 consecutive forecasts
- We will look at only a few variables
- Though the impact is on many variables
- Time forces us to limit our discovery
- We will compute
- The mean from all 4 members
- And the spread about the mean
- We will also look at the comparative forecasts in 4 panel format

## MSLP valid 12Z 18 Aug 2007

## Corresponding 4 Member LAF valid 12Z18Aug 2007

## It was obvious of course?

## 850 hPa temperatures 

## Things are a bit harder to see

**Things are a bit harder to see**

## Frontal zones are nearly always areas of high uncertainty! 

## 500 hPa heightstrough with tight gradients are good guesses for high uncertainty

## The LAF shows the key areas of uncertainty 

## The dilemma

- Tight gradients are often where there is uncertainty
- Tight gradients often are where we have the significant weather
- Thermal boundaries
- Moisture boundaries
- Areas of strong low-level jets
- Probably need more probabilitistic and uncertainty feel were critical gradients and weather interact

## PW uncertainty for same time

- Implicatoins on QPFs

## 4 comparative forecasts

## Deep cyclone from same cycles

## QPF example

## Errors around the cyclone in the gradients

## Another exampleMay 2007 system

- Key feature was a strong frontal system
- Key problems were cold and winter season like weather
- Key area of focus in southern Africa

## ColdForecast 16-22 May

## 850 forecasts

## 850 frontal system

## Sharp 850 hPa frontal boundary

## Uncertainty in the baroclinic zone

## MSLP inconsistencies hard to visualize can subtle

## Uncertainty with the trough/front

## PWAT with cold surge and in warm air

## PWAT LAF

## Lots of uncertainty with the deep low

## May 2007

- Time frame shown showed:
- Some uncertainty but small spread
- These were pretty high confidence forecasts of a relatively well forecast front
- There were some differences
- Most near the 850 hPa front
- Near the surface trough
- Uncertainty is a function of scale and mesoscale features show this to us.
- Visualization takes more than comparing
- Need to see the mean and spread about the mean
- This is a key uncertainty tool in all EPS data!

## Conclusions

**There is considerable uncertainty in weather forecasting**
*Model jumpiness is a signal**** impact of IC’s*
*Ensembles help us identify these signals*
**Model uncertainty**
- Due to errors in initial conditions of observed data
- Also due to model physics.
- We deal with them at various forecast lengths, and meteorological scales.
- We see these problems on a daily basis.

**Notes:**
Summarize the points

## Hubrisoverbearing pride or presumption; arrogance: 

- Single model successes and give us great confidence in that particular model
- These event give forecasters confidence a model
- How big a success was it?
*Lucky at the scales presented?*
*There still are storms/events that are hard to predict?*
*Were the mesoscale details really correct?*
- High confidence and precise forecasts are quite likely hubris.

**Notes:**
One big success got everyone excited....but we have a lot to learn. 

....don’t let hubris consume you into bad decisions!

## Questions?

- Have a good case let me know the date and we can  use it as an example?
- Thoughts?
- Hoffman, R. N., E Kalnay, 1983: Lagged average forecasting, an alternative to Monte Carlo forecasting. Tellus,35A,100-118.