# Presentation: 644136

## Everything You Wanted to Know About MOS* * But Were Afraid to Ask

**Joe Maloney**
**Statistical Modeling Branch, MDL**
_**[Joseph.Maloney@noaa.gov](mailto:Joseph.Maloney@noaa.gov)**_
**(301) 713-0024 x151**

## Outline

*** *****MOS Basics**
**What is MOS?  **
**MOS Properties**
**Predictand Definitions**
**Equation Development**
**Guidance post-processing**
**MOS Issues**
*** *****Future Work**
**New Packages**
**Gridded MOS**

## Model Output Statistics (MOS)

**MOS relates observations of the weather element to be**
**predicted (**_**PREDICTANDS**_**) to appropriate variables**
**(**_**PREDICTORS**_**) via a statistical method**
**Predictors can include:**
** ****NWP model output interpolated to observing site**
** ****Prior observations**
** ****Geoclimatic data – terrain, normals, lat/lon, etc.**
**Current statistical method:  Multiple Linear Regression (forward selection)**

## MOS Properties

** ****Mathematically simple, yet powerful technique**
** ****Produces probability forecasts from a single run of the underlying NWP model**
** ****Can use other mathematical approaches such as logistic regression or neural networks**
** ****Can develop guidance for elements not directly forecast by models; e.g. thunderstorms**

## MOS Guidance Production

**Model runs on NCEP IBM mainframe**
**Predictors are collected from model fields, “constants” files, etc.**
**Equations are evaluated**
**Guidance is post-processed**
- Checks are made for meteorological and statistical consistency
- Categorical forecasts are generated
**Final products disseminated to the world**

## MOS Guidance

** ****GFS (MAV) 4 times daily (00,06,12,18Z)**
** ****GFS Ext. (MEX) once daily (00Z)**
** ****Eta/NAM (MET) 2 times daily (00,12Z)**
** ****NGM (FWC) 2 times daily (00,12Z)**
** ****Variety of formats:  text bulletins, GRIB and BUFR messages, graphics ... **

## Slide 7

## Slide 8

## GFS/NAM MOS v. NGM MOS

** ****MORE STATIONS: now at ~1700 forecast sites **
** ****MORE FORECASTS: available at projections of 6 - 84 hours, GFS at 06Z and 18Z cycles **
** ****BETTER RESOLUTION:**
***GFS predictors on 95.25 km grid;  Eta on 32 km***
***Predictor fields available at 3-h timesteps***
***Predictors available beyond 48-h projection ***
****** No extrapolative forecasts ***
** ****BUT:  DEPENDENT SAMPLE NOT “IDEAL”:**
***Fewer seasons; non-static underlying NWP model***

## MOS Development Strategy

** ****CAREFULLY define your predictand**
** ****Stratify data as appropriate**
** ****Pool data if needed (Single Station / Regional) **
** ****Select predictors for equations**
** ****AVOID OVERFITTING!**

## Predictand Strategies

**Predictands always come from meteorological data and a variety of sources:**
** ****Point observations (ASOS, AWOS, Co-op sites)**
** ****Satellite data (e.g., SCP data)**
** ****Lightning data (NLDN)**
** ****Radar data (WSR-88D)**
**It is very important to quality control predictands before performing a regression analysis...**

## Suitable Observations?

**Appropriate Sensor?**

**Good siting?**

**Real ?**

**Photos Courtesy W. Shaffer**

**Suitable Observations?**

## Predictand Strategies

**(Quasi-)Continuous Predictands: best for variables with a relatively smooth distribution**
- Temperature, dew point, wind (u and v components, wind speed)
- Quasi-continuous because temperature available usually only to the nearest degree C, wind direction to the nearest 10 degrees, wind speed to the nearest m/s.
**Categorical Predictands: observations are reported as categories**
- Sky Cover (CLR, FEW, SCT, BKN, OVC)

**Notes:**
There are four different types of predictands we use.  Most weather observations can be considered quasi-continuous, but for reasons I’ll discuss shortly, there are only two groups that we use in a continuous form—temperature and wind.

## Predictand Strategies

**“****Transformed” Predictands: predictand values have been changed from their original values**
- Categorize (quasi-)continuous observations such as ceiling height
- Binary predictands such as PoP (precip amount _>_ 0.01”)
- Non-numeric observations can also be categorized or “binned”, like obstruction to vision (FOG, HAZE, MIST, Blowing, none)
- Operational requirements (e.g., average sky cover/P-type over a time period, or getting 24-h precip amounts from 6-h precip obs)
**Conditional Predictands: predictand is conditional upon another event occurring**
- PQPF: Conditional on PoP
- PTYPE: Conditional on precipitation occurring

**Notes:**
Predictands are transformed for a variety of reasons: (1) the distribution is heavily skewed [like ceiling height and precipitation amount]; (2) the observations are non-numeric [like “weather” or “obstruction to vision”]; (3) the desired predictand is not observed directly but can be derived from available observations.  Conditional predictands are used when forecasts are useful only in certain circumstances.  For example, precipitation type is not important unless it will precipitate, and precipitation amount is meaningless unless there is PoP.

## MOS Predictands

**Temperature**
- Spot temperature (every 3 h)
- Spot dew point (every 3 h)
- Daytime maximum temperature [0700 – 1900 LST] (every 24 h)
- Nighttime minimum temperature [1900 – 0800 LST] (every 24 h)
**Wind**
- U- and V- wind components (every 3 h)
- Wind speed (every 3 h)
**Sky Cover**
- Clear, few, scattered, broken, overcast [binary/MECE] (every 3 h)

## MOS Predictands

**PoP/QPF**
- PoP:  accumulation of 0.01” of liquid-equivalent precipitation in a {6/12/24} h period [binary]
- QPF:  accumulation of {0.10”/0.25”/0.50”/1.00”/2.00”*} CONDITIONAL on accumulation of 0.01” [binary/conditional]
- 6 h and 12 h guidance every 6 h; 24 h guidance every 12 h
- 2.00” category not available for 6 h guidance
**Thunderstorms**
- 1+ lightning strike in gridbox [binary]
**Severe**
- 1+ severe weather report in gridbox [binary]

## MOS Predictands

**Ceiling Height**
- CH < 200 ft, 200-400 ft, 500-900 ft, 1000-1900 ft, 2000-3000 ft, 3100-6500 ft, 6600-12000 ft, > 12000 ft [binary/MECE]
**Visibility**
- Visibility < 1⁄2 mile, < 1 mile, < 2 miles, < 3 miles, _<_ 5 miles, _<_ 6 miles [binary]
**Obstruction to Vision**
- Observed fog (fog w/ vis < 5/8 mi), mist (fog w/ vis _>_ 5/8 mi), haze (includes smoke and dust), blowing phenomena, or none [binary]

## MOS Predictands

**Precipitation Type**
- Pure snow (S); freezing rain/drizzle, ice pellets, or anything mixed with these (Z); pure rain/drizzle or rain mixed with snow (R)
- Conditional on precipitation occurring
**Precipitation Characteristics (PoPC)**
- Observed drizzle, steady precip, or showery precip
- Conditional on precipitation occurring
**Precipitation Occurrence (PoPO)**
- Observed precipitation _on the hour_ – does NOT have to accumulate

## Stratification

**Goal:  To achieve maximum homogeneity in our developmental datasets, while keeping their size large enough for a stable regression**
**MOS equations are developed for two seasonal stratifications:**
** ****COOL SEASON:  October 1 – March 31**
** ****WARM SEASON:  April 1 – September 30**
***EXCEPT Thunderstorms (Oct. 16 – Mar. 15, Mar. 16 – Jun. 30, July 1 – Oct. 15)**

## Pooling Data

** ****Generally, this means REGIONALIZATION: collecting nearby stations with similar climatologies**
** ****Particularly important for forecasting RARE EVENTS:**
***QPF Probability of 2”+ in 12 hours***
***Ceiling Height < 200 ft; Visibility < 1⁄2 mile***
** ****Regionalization allows for guidance to be produced at sites with poor, unreliable, or non-existent observation systems**
*** ******All MOS equations are regional except temperature and wind***

## Example of Regions

**GFS MOS PoP/QPF Region Map,**
**Cool Season**
** ****Note that each element has its own regions, which usually differ by season**

**Notes:**
Important to find a balance between larger region sizes (larger pool of data, more stable regression) and smaller region sizes (local effects better modeled)

## MOS Development Strategy

**MOS equations are *****multivariate***** of the form:**
**Y = c****0**** + c****1*****X****1**** + c****2*****X****2**** + ... + c****N*****X****N**
**C’s are constants, X’s are predictors**
**N is the number of predictors in the equation and is specified when the equations are developed.**
**Setting N too high is an easy way to OVERFIT your regression to your developmental dataset.**

**Notes:**
Now that we have our predictands settled, and our predictors to offer to the regression, we can talk about developing some MOS equations.

## MOS Development Strategy

**Forward Selection ensures that the “best” or most STATISTICALLY IMPORTANT predictors are chosen first.**
** ****First predictor selected accounts for greatest reduction of variance (RV)**
** ****Subsequent predictors chosen give greatest RV in conjunction with predictors already selected**
** **_**STOP**_** selection when max # of terms reached,  or when no remaining predictor will reduce variance by a pre-determined amount**

## MOS Equations

_**GFS 00Z Warm Season, 12-h PoP, F48, Gulf Coast Region**_
**  ****003220508         0        48  254001230**
** ****003041508   8500500        42  700002230**
** ****004100008       500        36        230**
** ****003210508         0        42  127001230**
** ****003220508         0        48  254000230**
** ****0.2516278E+00**
** ****0.5221328E+00**
** ****0.3407131E-01**
** ****0.1199076E+00**
** ****0.2503718E+00**
** ****0.4982985E-02**
** ****0.2315209E+00**
** ****0.1204374E+00**

**A useful format for the equation evaluator program, but certainly NOT for human eyes!**

## Sample Linear RegressionKATL, June 2005

## Sample Linear RegressionKATL, June 2005

**MEAN**

**Notes:**
The dashed line represents the mean of the 30-day sample.  A forecast of simply the mean is a skill-less forecast.

## Sample Linear RegressionKATL, June 2005

**RV**

**Unexplained**
**Variance**

**MEAN**

**Notes:**
I have now added the best-fit regression line using the least-squares approach.  This predictor alone (2m temperature) reduces the variance by 69% (correlation of about 82.8%).  Reduction of Variance (RV) = (variance – standard error) / variance.  Standard error (RMSE) is 3.3°F.

## MOS Predictor Strategy

** ****Important to offer predictors which describe the physical processes associated with event:**
- PoP: model precip, vertical velocity, moisture divergence, RH
** ****Avoid irrelevant predictors:**
- PoP: 1000-500 mb thickness, tropopause height
** ****High-resolution geophysical data (terrain), site-specific relative frequencies help with local forcing effects**
** ****Non-linear transformations of predictors are useful, particularly when the predictand is non-linear (e.g., binary predictand)**

**Notes:**
Additionally, observations (aka PERSISTANCE) is sometimes a helpful predictor.

## TransformPoint Binary Predictor

**FCST: F24 MEAN RH         PREDICTOR CUTOFF = 70%****INTERPOLATE;  STATION RH ≥ 70% ,   SET BINARY = 1;**** BINARY = 0,  OTHERWISE*****  ***

- 96

- 86

- 89

- 94

- 87

- 73

- 76

- 90

- 76

- 60

- 69

- 92

- 64

- 54

- 68

- 93

**RH ≥70%  ;  BINARY AT KBHM =  1**

- •

**KBHM**

**(71%)**

## TransformGrid Binary Predictor

**FCST: F24 MEAN RH         PREDICTOR CUTOFF = 70%****WHERE RH ≥ 70% , SET GRIDPOINT VALUE = 1, OTHERWISE = 0**** INTERPOLATE TO STATIONS*****  ***

- 1

- 1

- 1

- 1

- 1

- 1

- 1

- 1

- 1

- 0

- 0

- 1

- 0

- 0

- 0

- 1

**0 **_**<**_** VALUE AT KBHM **_**<**_** 1**

- •

**KBHM**

**(0.21)**

## Transform – Logit Fit

**Transform – Logit Fit**

- KPIA (Peoria, IL)   0000 UTC; 18-h projection

## Binary Predictands

**If your predictand is BINARY, MOS equations yield estimates of event PROBABILITIES...**
**MOS Probabilities are:**
*** *****Unbiased – the average of the probabilities over a period of time equals the long-term relative frequency of the event**
** ****Reliable – conditionally (“piece-wise”) unbiased over the range of probabilities**
** ****Reflective of predictability of the event – range of probabilities narrows and approaches relative frequency of event as predictability decreases, for example, with increasing projections or with rare events**

**Notes:**
This is called REEP, or Regression Estimation of Event Probabilities.

## Post-Processing MOS Guidance

** ****Meteorological consistencies – SOME checks**
**T **_**>**_** Td; min T **_**<**_** T **_**<**_** max T; dir = 0 if wind speed = 0**
**BUT no checks between PTYPE and T, between PoP and sky cover**
** ****Statistical consistencies – again, SOME checks**
**Conditional probabilities made unconditional**
**Truncation (no probabilities < 0, > 1)**
**Normalization (for MECE elements like sky cover)**
**Monotonicity enforced (for elements like QPF)**
**BUT temporal coherence is only partially checked**
*** *****Generation of “best categories”**

## Unconditional Probabilities from Conditional

**If event B is conditioned upon A occurring:**
** ****Prob(B|A)=Prob(B)/Prob(A) **
** ****Prob(B) = Prob(A) × Prob(B|A) **
**Example:  **
**Let A = event of **_**>**_** .01 in., and B = event of **_**>**_** .25 in., then if:**
**Prob (A) = .70, and **
**Prob (B|A) = .35, then **
**Prob (B) = .70 × .35 = .245**

**U**

**A**

**B**

## Truncating Probabilities

**0 **_**<**_** Prob (A) **_**<**_** 1.0**
**Applied to PoP’s and thunderstorm probabilities**
**If Prob(A) < 0, Prob****adj**** (A)=0 **
**If Prob(A) > 1, Prob****adj**** (A)=1.**

## Normalizing MECE Probabilities

**Sum of probabilities for exclusive and exhaustive categories must equal 1.0**
**If Prob (A) < 0, then sum of Prob (B) and Prob (C) = D, and is > 1.0.**
**Set:  Prob****adj**** (A) = 0,**
**Prob****adj**** (B) = Prob (B) / D,**
**Prob****adj**** (C) = Prob (C) / D**

## Monotonic Categorical Probabilities

**If event B is a subset of event A, then:**
**   ****Prob (B) should be **_**<**_** Prob (A).**
**Example:  B is **_**>**_** 0.25 in; A is      **_**>**_** 0.10 in**
**Then, if Prob (B) > Prob (A)**
**set Prob****adj**** (B) = Prob (A).**
**Now, if event C is a subset of event B, e.g., C is **_**>**_** 0.50 in, and if Prob (C) > Prob (B),**
**set Prob****adj**** (C) = Prob (B)**

## Temporal Coherence of Probabilities

**Event A is **_**>**_** 0.01 in. occurring from 12Z-18Z**
**Event B is **_**>**_** 0.01 in. occurring from 18Z-00Z**
** ****A ********B is **_**>**_** 0.01 in. occurring from 12Z-00Z**
**Then P(A********B) = P(A) + P(B) – P(A********B)**
**Thus, P(A********B) should be:**
_**<**_** P(A) + P(B) and**
_**>**_** maximum of P(A), P(B)**

**A**

**B**

**C**

## Temporal Coherence - Partially Enforced

**Thus, P(A********B) should be:**
_**<**_** P(A) + P(B) – ****coherence not checked**
_**>**_** maximum of P(A), P(B) – ****coherence checked**
**SAN DIEGO           **
** ****KMYF   GFS MOS GUIDANCE   12/28/2004  1200 UTC                      **
** ****DT /DEC  28/DEC  29                /DEC  30                /DEC  31 **
** ****HR   18 21 00 03 06 09 12 15 18 21 00 03 06 09 12 15 18 21 00 06 12**
** ... **
** ****P06        79    ****71****   ****100 ****   ****68****     ****5****     6    14     9    16 21 28 **
** ****P12                   ****100          68****          19          25    32 **
** ****Q06         4     3     5     2     0     0     0     0     0  0  1 **
** ****Q12                     5           2           0           0     0 **
** ****T06      ****9****/ 0 ****30****/ 2 22/ 4  9/ 0  0/ 0  0/ 0  0/ 0  0/ 0  1/ 0  0/ 0 **
** ****T12           ****47****/ 3       29/ 4        0/ 0        0/ 0     3/ 0****    **
** **

## MOS Best Category Selection

**MOS Best Category Selection**

- NO

**PROBABILITY (%)**

- NO

- NO

- NO

- 4

- 1

- 6

- 3

- 2

- 5

- 0

- YES

- YES

**THRESHOLD**

**EXCEEDED?**

**TO MOS GUIDANCE MESSAGES**

**An example with QPF...**

## Other Possible Post-Processing

** ****Computing the Expected Value**
- used for estimating precipitation amount
** ****Fitting probabilities with a distribution**
- Weibull distribution used to estimate median or other percentiles of precipitation amount
** ****Reconciling meteorological inconsistencies**
-  Not always straightforward or easy to do
-  Inconsistencies are minimized somewhat by use of NWP model in development and application of forecast equations

## MOS Weaknesses / Issues

** ****MOS can have trouble with some local effects (e.g., cold air damming along Appalachians, and some other terrain-induced phenomena)**
** ****MOS can have trouble if conditions are highly unusual, and thus not sampled adequately in the training sample**
**But, MOS can and has predicted record highs & lows**
*** *****MOS typically does not pick up on mesoscale-forced features**

## MOS Weaknesses / Issues

** ****Like the models, MOS has problems with QPF in the warm season (particularly convection along sea breeze fronts along the Gulf and Atlantic coasts)**
** ****Model changes can impact MOS skill**
** ****MOS tends toward climate at extended projections – due to degraded model accuracy **
** ****CHECK THE MODEL...MOS will correct many  systematic biases, but will not “fix” a bad forecast.  GIGO (garbage in, garbage out).**

## MOS Weaknesses / Issues

** ****We rely on our customers and end-users to help us identify vagaries and oddities which occasionally pop up in the guidance.**
** ****If you ever see anything in the MOS guidance which seems WRONG, and you see nothing in the model output to help explain it, PLEASE PLEASE PLEASE let us know!**

## Future Work

** ****New 12Z GFS Extended MOS (MEX) package**
***Coming September 2005***
** ****New stations in MAV/MET/MEX**
***Approximately 80 new sites, majority in Texas***
** ****Eta/NAM MOS**
***New Visibility & Obstruction to Vision Guidance (soon)***
***NAM is changing from Eta to WRF – Need to evaluate impacts of model change on NAM MOS guidance***
** ****Gridded MOS...**

## Why do we need Gridded MOS?

**Because forecasters have to**
**produce products like this for the NDFD...**

## Traditional MOS Guidance

**KSFO   GFSX MOS GUIDANCE   7/26/2005  0000 UTC                       **
** ****FHR  24| 36  48| 60  72| 84  96|108 120|132 144|156 168|180 192      **
** ****TUE  26| WED 27| THU 28| FRI 29| SAT 30| SUN 31| MON 01| TUE 02 CLIMO**
** ****X/N  75| 56  74| 58  73| 58  74| 58  74| 57  74| 56  75| 57  74 54 72**
** ****TMP  69| 57  69| 59  69| 59  70| 59  69| 58  69| 56  69| 57  69      **
** ****DPT  54| 53  55| 54  53| 54  54| 53  54| 54  54| 52  53| 53  54      **
** ****CLD  PC| CL  PC| PC  PC| PC  PC| PC  PC| PC  PC| PC  PC| PC  PC      **
** ****WND  16| 16  13| 15  13| 14  16| 18  18| 18  18| 19  18| 18  17      **
** ****P12   1|  0   1|  3   2|  3   3|  1   2|  1   2|  3   2|  1   1  0  2**
** ****P24    |      1|      7|      3|      2|      4|      3|      2     2**
** ****Q12   0|  0   0|  0   0|  0   0|  0   0|  0   0|  0    |             **
** ****Q24    |      0|      0|      0|      0|      0|       |             **
** ****T12   7|  0   3|  4   1|  3   0|  1   0|  0   0|  0   0|  0   0      **
** ****T24    |  7    |  4    |  3    |  1    |  1    |  0    |  0 **

**But the guidance available doesn’t even come close to the resolution of the NDFD.**

## Traditional MOS Graphics

**This is better, but still lacks most of the detail in the Western U.S.**

## Objectives

** ****Produce MOS guidance on high-resolution grid (2.5 to 5 km spacing)**
** ****Generate guidance with sufficient detail for forecast initialization at WFOs**
** ****Generate guidance with a level of accuracy comparable to that of the station-oriented guidance**

## Approach

** ****High-resolution geoclimatic variables**
** ****Diverse observational networks **
** ****Appropriate MOS equation development**
** ****Analysis on high-resolution grid**

## Western CONUS

**Notes:**
Western tile (in red) over the CONUS grid – 5km resolution output grid

## Slide 52

**Notes:**
Our current station map is shown again; note in particular the gaps of coverage over the Western third of the nation.

## Slide 53

**Notes:**
We have added observations from COOP stations, bouys and C-MAN sites, data supplied by RFC at selected spots, and Mesowest observations supplied to us by the University of Utah.  It’s still not 5-km resolution but it’s much better.

## BCDG Analysis

** ****Method of successive corrections**
** ****Land/water gridpoints treated differently**
** ****Elevation (“lapse rate”) adjustment**

## MOS Max Temperature Forecast

## NDFD Max Temperature Forecast

## Future of Gridded MOS

** ****Evaluation (objective & subjective)**
** ****Expansion (area & elements)**
** ****Improvement**
** ****Use of remote-sensing observations**
** ****Dissemination (Fall 2005, June 2006)**

## Any Questions?

**Web site:  **_**[http://weather.gov/mdl/synop/](http://weather.gov/mdl/synop/)**_
**E-mail:  **_**[Joseph.Maloney@noaa.gov](mailto:Joseph.Maloney@noaa.gov)**__** **_
**or MDL_MOS.Webmaster@noaa.gov**