Forecast Model Uncertainty; 

MM5 Accuracy Over Utah 

by Barbara Sauter 

ARL-TR-3170 March 2004 

Approved for public release; distribution is unlimited. 
NOTICES 

Disclaimers 

The findings in this report are not to be construed as an official 

Department of the Army position, unless so designated by other 

authorized documents. 

Citation of manufacturers’ or trade names does not constitute an 

official endorsement or approval of the use thereof. 
Army Research Laboratory 

White Sands Missile Range, NM 88002-5501 

ARL-TR-3170 March 2004 

Forecast Model Uncertainty; 

MM5 Accuracy Over Utah 

Barbara Sauter 

Computational and Information Sciences Directorate (CISD) 

Approved for public release; distribution is unlimited.
ii 

REPORT DOCUMENTATION PAGE Form Approved 

OMB No. 0704-0188 

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the 

data needed, and completing and reviewing the collection information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing the 

burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. 

Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid 

OMB control number. 

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. 

1. REPORT DATE (DD-MM-YYYY) 

March 2004 

2. REPORT TYPE 

Final 

3. DATES COVERED (From - To) 

5a. CONTRACT NUMBER 

5b. GRANT NUMBER 

4. TITLE AND SUBTITLE 

Forecast Model Uncertainty; MM5 Accuracy Over Utah 

5c. PROGRAM ELEMENT NUMBER 

5d. PROJECT NUMBER 

5e. TASK NUMBER 

6. AUTHOR(S) 

Barbara Sauter 

5f. WORK UNIT NUMBER 

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 

U.S. Army Research Laboratory 

Computational and Information Sciences Directorate 

Battlefield Environment Division (ATTN: AMSRL-CI-EB) 

White Sands Missile Range, NM 88002-5501 

8. PERFORMING ORGANIZATION 

 REPORT NUMBER 

ARL-TR-3170 

10. SPONSOR/MONITOR'S ACRONYM(S) 

ARL 

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 

U.S. Army Research Laboratory 

2800 Powder Mill Road 

Adelphi, MD 20783-1145

11. SPONSOR/MONITOR'S REPORT 

 NUMBER(S) 

ARL-TR-3170 

12. DISTRIBUTION/AVAILABILITY STATEMENT 

Approved for public release; distribution is unlimited. 

13. SUPPLEMENTARY NOTES 

14. ABSTRACT 

Researchers are investigating various approaches to determine and portray weather forecast uncertainty. Methods requiring extensive 

computing power on the battlefield or massive data communications to the end user will not be implemented in the near future. This study 

investigated weather forecast uncertainty through a compilation of forecast errors in the Penn State/National Center for Atmospheric Research 

fifth-generation Mesoscale Model (MM5) over 50 winter days in Utah. The percentage of forecasts meeting the Army’s stated accuracy 

requirements for temperature, dew-point temperature, wind speed, and wind direction is highlighted. 

15. SUBJECT TERMS

Weather Forecast Uncertainty; Forecast Accuracy; MM5; Utah MesoWest 

16. SECURITY CLASSIFICATION OF: 

19a. NAME OF RESPONSIBLE PERSON 

Barbara Sauter 

a. REPORT 

U 

b. ABSTRACT 

U 

c. THIS PAGE 

U 

17. LIMITATION 

OF 

ABSTRACT 

SAR 

18. NUMBER 

OF 

PAGES

46 19b. TELEPHONE NUMBER (Include area code) 

(505) 678-2840 

Standard Form 298 (Rev. 8/98) 

Prescribed by ANSI Std. Z39.18
iii 

Contents 

Preface vii

Summary 1

1. Introduction 2

2. Methodology 3

2.1 MM5 Model Runs ...........................................................................................................3 

2.2 Forecast Accuracy Analysis ............................................................................................4 

3. Results 5

3.1 Forecast Errors Associated with the Month ....................................................................5 

3.2 Forecast Errors Associated with the Forecast Time........................................................7 

3.3 Forecast Errors Associated with the Observed Value of the Parameter........................11 

3.4 Forecast Errors Associated with the Forecast Value of the Parameter .........................13 

3.5 Forecast Errors Associated with the Station Elevation .................................................19 

3.6 Forecast Errors for All Cases ........................................................................................22 

4. Conclusions 25

4.1 Relating Error Amounts to User Forecast Uncertainty Calculations ............................25 

4.2 Recommendations .........................................................................................................26 

References 28

Acronyms 29

Distribution List 30
iv 

List of Figures 

Figure 1. Map of MM5 Inner Nest Model Domain Over Northern Utah.......................................4 

Figure 2. Percentage of Temperature Forecasts with Specified Error Amounts by Time of 

Day. .................................................................................................................................8 

Figure 3. Percentage of Wind Speed Forecasts with Specified Error Amounts by Time of 

Day...................................................................................................................................9 

Figure 4. Percentage of Wind Direction Forecasts with Specified Error Amounts by Time of 

Day. ...............................................................................................................................10 

Figure 5. Percentage of Forecasts Satisfying Required Accuracy by Forecast Time...................11 

Figure 6. Percentage of Temperature Forecasts with Specified Error Amounts by Observed 

Temperature Value. .......................................................................................................12 

Figure 7. Percentage of Forecasts Satisfying Required Accuracy by Observed Values. .............13 

Figure 8. Percentage of Temperature Forecasts with Specified Error Amounts by Forecast 

Temperature Value. .......................................................................................................14 

Figure 9. Percentage of Wind Speed Forecasts with Specified Error Amounts by Forecast 

Wind Speed Value. ........................................................................................................15 

Figure 10. Percentage of Wind Direction Forecasts with Specified Error Amounts by 

Forecast Wind Direction Value. ..................................................................................16 

Figure 11. Bar Chart Version of Percentage of Wind Direction Forecast Error Categories by 

Forecast Wind Direction Value for December. ...........................................................16 

Figure 12. Percentage of Forecasts Satisfying Required Accuracy by Forecast Value................19 

Figure 13. Percentage of Temperature Forecasts with Specified Error Amounts by Station 

Elevation......................................................................................................................20 

Figure 14. Percentage of Dew-Point Temperature Forecasts with Specified Error Amounts 

by Station Elevation.....................................................................................................20 

Figure 15. Percentage of Wind Speed Forecasts with Specified Error Amounts by Station 

Elevation......................................................................................................................21 

Figure 16. Percentage of Wind Direction Forecasts with Specified Error Amounts by Station 

Elevation......................................................................................................................22 

Figure 17. Percentage of Temperature Forecasts with Specified Error Amounts for all 

Stations and Forecast Times Combined. .....................................................................23 

Figure 18. Percentage of Dew-Point Temperature Forecasts with Specified Error Amounts 

for all Stations and Forecast Times Combined............................................................23 

Figure 19. Percentage of Wind Speed Forecasts with Specified Error Amounts for all 

Stations and Forecast Times Combined. .....................................................................24 

Figure 20. Percentage of Wind Direction Forecasts with Specified Error Amounts for all 

Stations and Forecast Times Combined. .....................................................................24 
v 

List of Tables 

Table 1. Forecast Error Statistics by Month for Temperature and Dew-Point Temperature..........6 

Table 2. Forecast Error Statistics by Month for Wind Speed and Wind Direction. .......................7 

Table 3. Percent of the Time the Wind was Actually Observed From Each Compass 

Direction for Each Forecast Direction. ..........................................................................18 

Table 4. User-Selected Confidence Desired to Generate a Forecast Range Based on the 

Frequency of Forecast Errors.........................................................................................25 

Table 5. Staff Weather Officer Confidence Assigned to Generate a Forecast Range Based on 

the Frequency of Forecast Errors. ...................................................................................26 
vi 

INTENTIONALLY LEFT BLANK. 
vii 

Preface 

In August, 2002, a group of atmospheric science researchers participated in an Army-Scale 

Meteorology Workshop at White Sands Missile Range, NM. The U.S. Army Research 

Laboratory Battlefield Environment Division sponsored the workshop to obtain input on where 

the Army should invest its atmospheric research resources. One of the participants emphasized 

that the Army should have information relating to forecast uncertainty. 

Some Army decision makers receive weather intelligence from an Air Force Staff Weather 

Officer, who may use meteorological expertise to provide a range of values or probability 

information relating to the weather forecast parameters of interest. Often, however, users must 

rely on the single deterministic value provided by a model such as the Pennsylvania State 

University/National Center for Atmospheric Research Mesoscale Model version 5 (MM5). The 

Integrated Weather Effects Decision Aid (IWEDA) provides automated red/amber/green 

guidance on weather impacts on specific military systems based solely on the deterministic 

model values. Commanders may be able to make better tactical decisions if they are provided 

information on the amount of certainty associated with a particular forecast. 

Several organizations are making forecast uncertainty calculations based on running ensembles 

of forecast models. However, ensembles are not being run in support of battlefield operations. 

In response to the workshop recommendation, a limited study was performed to associate 

forecast model uncertainty with forecast accuracy. 

The author acknowledges the contributions of Dr. Teizi Henmi, who set up the MM5 model and 

observation domain over Utah for a previous study. The existing results from that study included 

many forecasts of basic surface weather parameters. These forecasts were validated against 

observations to quantify the occurrences of different error amounts. This report documents the 

forecast errors seen, with the partially addressed goal of relating the forecast errors to model 

uncertainty in a way useful to Army decision makers. 
viii 

INTENTIONALLY LEFT BLANK. 
1 

Summary 

The meteorological community is recognizing the need for incorporating weather forecast 

uncertainty along with the forecasts provided to users. A consensus has formed that a forecast 

range or a confidence level associated with a single forecast value can lead users to make better 

decisions. On the battlefield, weather intelligence and decision aids are frequently automated, 

based on a single deterministic value produced by a numerical weather forecast model. 

Researchers are investigating various approaches to determine and portray weather forecast 

uncertainty. However, some of these approaches are not currently viable for use on the 

battlefield because of the amount of computing power or data communications required. This 

study investigated weather forecast uncertainty in a compilation of forecast errors in the Penn 

State/National Center for Atmospheric Research fifth-generation Mesoscale Model (MM5). 

MM5 forecast outputs were compared to observations of basic surface weather parameters at 50-

70 Utah MesoWest sites. The forecasts covered 24 hourly forecasts for each of 50 days during 

the winter of 2002. The frequency of occurrence of the resulting error amounts was tabulated 

and is presented in this report. Some effort was included to relate changes in error frequencies to 

readily available information on the battlefield, such as the month, the forecast time, the 

observed value, the forecast value, and the station elevation. 

These relationships either did not appear to be significant, or the limited time and space of the 

data included do not allow for useful generalizations to be made. The error data presented may 

be useful in relaying information to MM5 users on the general model accuracy seen for one 

specific and difficult forecast location. Emphasis is given to the percentage of forecasts meeting 

the U.S. Army’s stated accuracy requirements for temperature, dew-point temperature, wind 

speed, and wind direction. The results for the percentage of forecasts that were correct for all the 

times and stations combined were: 25 percent for temperature within 1° C; 24 percent for dewpoint temperature within 1° C; 73 percent for wind speed within 2.5 m/s; and 7 percent for wind 

direction within 5o. The percentages given for correct wind speed and wind direction forecasts 

may be misleading. Users will often want wind speed forecasts to be more accurate than plus or 

minus 2.5 m/s. On the other hand, wind direction forecasts will often be acceptable at less 

stringent limits than plus or minus 5°. 

This report presents a simple approach for using historical error amounts to modify the single 

forecast value generated by the model to a range of values. This can be accomplished by a user 

specifying the level of confidence desired in a particular instance or by a human forecaster 

tagging a specific day’s model output with an estimate of high, medium, or low confidence. 
2 

1. Introduction 

Weather forecast users understand that the forecast may be wrong. When decisions must be 

made using imperfect weather information, it would be helpful to know the amount of 

uncertainty associated with the predicted weather. The meteorological community is attempting 

to address this goal through a variety of approaches, including: 

x Categorizing forecast confidence levels based on specific situations and experience. 

x Providing a probability or range forecast rather than a single deterministic value. 

x Running an ensemble of separate model outputs using different models or different initial 

conditions for a single model. 

An expert weather forecaster may be familiar with a particular model’s bias and error tendencies, 

but these trends often pertain only to a particular place and time. Many users do not have any 

quantitative knowledge about probable forecast model errors. 

On the battlefield, the primary weather forecast model used in support of Army operations is the 

Penn State/National Center for Atmospheric Research fifth-generation Mesoscale Model (MM5) 

(1). This study quantifies the MM5 forecast errors seen in an existing data set generated in a 

previous nowcasting research effort (2). The results cannot be generalized to other times or 

locations based on this limited analysis, but they do provide a single set of quantified forecast 

error occurrences. 
3 

2. Methodology 

2.1 MM5 Model Runs 

Each MM5 model run was initialized with 30 hours of Global Forecast System (GFS) model 

output data (3). The initial 6 hours served as spin-up time prior to the 24 hourly forecast 

validation times. The MM5 was then run in 3 nests on the Army High Performance Computing 

Resource Center’s Cray computer: 

x The outer nest contained 55 x 55 grid points at 45 km grid point spacing for a 2430 x 2430 

km domain. 

x The middle nest consisted of 55 x 55 grid points at 15 km grid point spacing, covering a 

810 x 810 km domain. 

x The inner nest included 85 x 85 grid points at 5 km grid point spacing, resulting in a 420 x 

420 km domain. 

Hourly output from the inner nest was saved each day for 01Z through 00Z, equating to 6 p.m. 

through 5 p.m. the following day, local Utah time. 

The area of the inner nest is shown by the green square in figure 1. The red dots indicate 

locations registered by the Utah MesoWest cooperative as having surface observing stations, but 

many of the locations do not report the required data on the hour to be used for these evaluations 

(4). The smaller blue rectangles indicate areas with appropriate stations reporting at the 

beginning of the study. 
4 

Figure 1. Map of MM5 Inner Nest Model Domain Over Northern Utah. 

MM5 forecasts of temperature, dew-point temperature, and u- and v-wind components at the grid 

points were bilinearly interpolated to the surface station locations. Well over 100 different 

stations within the MM5 domain reported observations on some of the various days covered. 

Typically, however, approximately 50-70 stations would be available at a forecast time. 

The model forecast runs were performed beginning mid December 2002 through February 2003 

for days when the GFS model and the surface observations were available. During this period, a 

total of 50 days was used, including 8 in December, 18 in January, and 24 in February. 

2.2 Forecast Accuracy Analysis 

The forecast value of the basic weather parameter provided by the MM5 model was compared to 

the observed value. Since the ultimate goal is to be able to provide information beneficial to 

Army decision makers, the error ranges were generally grouped relative to the accuracy 

requirement stated by Army intelligence experts (5). The desired forecast accuracy for both 

temperature and dew-point temperature is within 1°C. An analysis of relative humidity might be 

more meaningful to many users than dew-point temperature, but the surface observation stations 

used in the study reported dew-point temperature and not relative humidity. The Army has 

specified that wind speed forecasts are needed within an accuracy of 5 kts. An accuracy 

requirement stated as a percentage of the actual wind speed value would often reflect a more 

realistic specification than the static value given, but for this analysis the value was simply 

MM5 inner nest 

 general areas with 

 stations reporting 
5 

rounded to 2.5 m/s. Wind direction data were omitted for any observation with wind speed less 

than 1 m/s to reduce the misleading errors associated with light and variable winds. The Army’s 

wind direction accuracy requirement of 5o is highlighted in this analysis, but in general a more 

reasonable range of 45° is included. 

One of the aims in investigating weather forecast uncertainty is to be able to provide information 

to decision makers on when the forecast is expected to be more or less accurate. The data set 

available for this study did not contain information on the actual weather synoptic situations, and 

no attempt has been made here to associate forecast errors with specific weather scenarios. A list 

of readily available factors that might be associated with forecast error amounts was devised, to 

include: 

x month 

x forecast time 

x observed value of parameter 

x forecast value of parameter 

x station elevation 

The following section will provide the forecast errors seen as a function of these variables, 

although some are covered only briefly. 

3. Results 

3.1 Forecast Errors Associated with the Month 

All of the model runs were performed during a single winter, with 8 days in December, 18 in 

January, and 24 in February. Basic error statistics are provided separately for each of these three 

months for temperature and dew-point temperature in table 1 and for wind speed and wind 

direction in table 2. 
6 

Table 1. Forecast Error Statistics by Month for Temperature and Dew-Point Temperature. 

The left column labeled OBS under each weather parameter contains statistics of the observed 

data highlighting the large range of values seen, including some erroneous observations. Some 

bad observations were deleted before running the statistics, but other bad data points were not 

found until after all the statistical analyses had been calculated. The small number of incorrect 

reported observations should not have a significant impact on the average error values based on 

many thousands of validations. The temperature and dew-point temperature forecasts reflect a 

warm bias in each month, but by differing amounts. The average absolute error values are 

slightly higher in January than in December or February. 

 TEMPERATURE (°C) DEW-POINT (°C) 

 OBS BIAS ABS 

ERR OBS BIAS ABS ERR 

AVG -3.1 1.0 2.3 -7.5 1.8 2.9 

STDEV 5.4 2.7 1.8 5.0 3.7 3.0 

MAX 11.7 12.6 12.6 3.9 25.4 25.4 

MIN -23.3 -8.0 0 -35.2 -10.1 0 

DEC 

COUNT 12000 12000 

AVG 0.5 2.1 3.0 -5.3 2.4 3.4 

STDEV 4.9 3.3 2.6 5.9 4.4 3.7 

MAX 15.0 29.5 29.5 15.0 50.1 50.1 

MIN -23.0 -17.0 0 -51.1 -13.2 0 

JAN 

COUNT 28000 28000 

AVG -2.1 1.0 2.5 -7.1 1.1 2.9 

STDEV 6.0 3.2 2.2 6.1 4.1 3.1 

MAX 14.8 42.5 42.5 11.8 65.3 65.3 

MIN -35.6 -13.6 0 -59.1 -25.4 0 

FEB 

COUNT 37000 37000 
7 

Table 2. Forecast Error Statistics by Month for Wind Speed and Wind Direction. 

The number of validations for wind direction totals about one-quarter to one-third the number for 

wind speed. This drop reflects the large percentage of wind observations with speed less than 1 

m/s, which were not used in the wind direction statistics. The predominance of light wind speeds 

contributed to relatively low wind speed errors. The average errors in wind speed and wind 

direction were virtually identical in the January and February results. 

Comparing the forecast accuracies across these three specific months shows that there can be 

significant differences in the bias or absolute error occurrences. It is impossible to draw any 

conclusions about long-term error trends associated with a particular month from this limited 

study. It seems logical that any change in systemic biases might be related to different seasons, 

but with data from only a single season in a single year, no further investigations relating to 

monthly differences would be worthwhile. 

3.2 Forecast Errors Associated with the Forecast Time 

Another factor potentially associated with varying forecast error amounts is the time of day. 

Since each of the forecast model runs was initialized at 01Z (6 p.m. local time), the following 

results cannot treat the time of day independently from the lag time of the hourly forecasts from 

1-24 hours. In spite of this limitation, the December forecasts were analyzed by time of day by 

grouping the results in six 4-hour intervals. The average forecast errors were calculated for the 

following time groups: 

 WIND SPEED (m/s) WIND DIRECTION (deg) 

 OBS BIAS ABS 

ERR OBS BIAS ABS ERR 

AVG 3.5 1.0 2.4 183 6 47 

STDEV 3.0 2.9 2.0 91 63 43 

MAX 19.8 11.9 12.3 359 180 180 

MIN 0 -12.3 0 0 -180 0 

DEC 

COUNT 11000 8000 

AVG 2.2 0.4 1.7 187 9 61 

STDEV 1.8 2.2 1.5 102 78 49 

MAX 14.5 14.8 14.8 359 180 180 

MIN 0 -10.5 0 0 -180 0 

JAN 

COUNT 24000 16000 

AVG 2.7 0.3 1.8 190 10 61 

STDEV 2.4 2.4 1.6 105 77 49 

MAX 23.6 11.4 14.6 359 180 180 

MIN 0 -14.6 0 0 -180 0 

FEB 

COUNT 32000 24000 
8 

x evening (forecast hours 1-4, equating to 6 p.m. - 9 p.m. local time) 

x night1 (forecast hours 5-8, 10 p.m. - 1 a.m.) 

x night2 (forecast hours 9-12, 2 a.m. - 5 a.m.) 

x morning (forecast hours 13-16, 6 a.m. - 9 a.m.) 

x midday (forecast hours 17-20, 10 a.m. - 1 p.m.) 

x afternoon (forecast hours 21-24, 2 p.m. - 5 p.m.) 

Figure 2 shows the results for these time periods for temperature forecasts from the 8 days in 

December. Points on the graph correspond to the percentage of the forecasts falling within the 

error category listed on the x-axis. The categories are bands covering 2° C error ranges, 

extending from an underforecast of 7 to 9o at the left side of the chart to an overforecast of 11 to 

13o at the right side. The midday and afternoon forecasts contain slightly less of a warm bias 

than the other times, but the differences between the different groups of forecast times are not 

very large. The percentage of the forecasts meeting the Army’s stated requirement of 

temperature forecasts within plus or minus 1° C ranges from 32 percent of the evening forecasts 

to 27 percent of the afternoon forecasts. 

Figure 2. Percentage of Temperature Forecasts with Specified Error Amounts by Time of Day. 

A similar chart for wind speed forecast errors is provided in figure 3. The documented Army 

requirement for wind speed forecasts within plus or minus 2.5 m/s is very broad, particularly in 

low wind cases. Rather than use consistent 5 m/s error bands providing minimal information, 

these results show 2.5 m/s error ranges. The percentage of forecasts meeting the Army goal for 

Temperatures

All December Forecast Hours 1-24

0.0% 0.5%

4.6%

17.3%

29.4%

25.7%

13.5%

6.9%

1.6%

0% 0.4% 0.1%

10%

20%

30%

40%

-9 to -7

-7 to -5

-5 to -3

-3 to -1

-1 to 1

1 to

3

3 to

5

5 to

7

7 to

9

9 to 11

11 to 13

Forecast Error amount

Percent of Occurrences

evening

night1

night2

morning

midday

afternoon

TOTAL

Underforecast Overforecast

Time Forecast Valid

(fcst hr 1 = 6 p.m. LST)
9 

accuracy can still easily be determined by adding the -2.5 to 0 m/s error occurrences with the 0 to 

+2.5 m/s errors. The morning forecasts were most likely to contain wind speeds higher than 

those actually observed. This bias was prevalent throughout all the time periods, based on the 

generally low wind speeds observed. The percentage of forecasts within the required accuracy 

varied from 58 percent in the midday forecasts to 68 percent in the evening forecasts. 

Figure 3. Percentage of Wind Speed Forecasts with Specified Error Amounts by Time of Day. 

While the wind speed accuracy requirement of plus or minus 2.5 m/s may be broader than 

desired, the wind direction forecast goal of plus or minus 5° seems exceedingly stringent. This 

study incorporates error bands covering 45o, but the errors falling within the -5 to +5° range are 

totaled separately, in order to specify the percentage of forecasts actually meeting the 

requirement as it stands. Note that in figure 4, the x-axis values to the left and right of this -5 to 

5° point incorporate ranges of 40o each, so summing the errors falling within these three ranges 

gives the percentage of forecasts accurate within plus or minus 45°. The remaining points 

include 45° ranges, where of course an error of -180o is identical to an error of +180o. The wind 

direction error in these cases exhibits a slightly greater bias in the counterclockwise direction in 

the evening and in the clockwise direction by the afternoon forecast. 

Wind Speeds

All December Forecast Hours 1-24

0.0% 0.4%

2.2%

7.9%

23.3%

37.9%

19.7%

6.7%

1.7%

0% 0.2%

10%

20%

30%

40%

50%

-12.5 to -10

-10 to -7.5

-7.5 to -5

-5 to -2.5

-2.5 to

0

0 to

2.5

2.5 to

5

5 to

7.5

7.5 to 10

10 to 12.5

Forecast Error amount

Percent of Occurrences

evening

night1

night2

morning

midday

afternoon

TOTAL

Underforecast Overforecast

Time Forecast Valid

(fcst hr 1 = 6 p.m. LST)

Wind Speeds

All December Forecast Hours 1-24

0.0% 0.4%

2.2%

7.9%

23.3%

37.9%

19.7%

6.7%

1.7%

0% 0.2%

10%

20%

30%

40%

50%

-12.5 to -10

-10 to -7.5

-7.5 to -5

-5 to -2.5

-2.5 to

0

0 to

2.5

2.5 to

5

5 to

7.5

7.5 to 10

10 to 12.5

Forecast Error amount

Percent of Occurrences

evening

night1

night2

morning

midday

afternoon

TOTAL

Underforecast Overforecast

Time Forecast Valid

(fcst hr 1 = 6 p.m. LST)
10 

Figure 4. Percentage of Wind Direction Forecasts with Specified Error Amounts by Time of Day. 

One aspect to the question, “How good is the forecast model?” is how often it meets the required 

accuracy. This is certainly too simplistic, especially since the required accuracy depends on the 

situation rather than being a constant fixed value. However, since this study easily provides the 

data on how often the required accuracy is satisfied, several charts highlighting this information 

are included. 

Figure 5 summarizes this information by forecast time. The wind speed forecasts show a 

significant decrease in percent accuracy from the evening forecasts to the morning through 

afternoon forecasts. The temperature forecasts reflect a small decrease in accuracy by the 

midday forecasts, although the dew-point temperature forecasts are accurate slightly more often 

at those times. The wind direction forecasts that fall within 5° of the observed wind direction are 

depicted by the light green circles, while those within 45o are the dark green circles specified as 

WD+ in the legend. The 45° criterion is met most frequently during the morning and midday 

forecast times. 

Wind Directions

All December Forecast Hours 1-24

2.5%

4.5%

9.7%

24.3%

9.3%

27.2%

13.2%

5.7%

3.5%

0%

10%

20%

30%

-180 to -135

-135 to -90

-90 to -45

-45 TO -5

-5 to 5

5 to 45

45 TO 90

90 TO 135

135 TO 180

Forecast Error amount

Percent of Occurrences

evening

night1

night2

morning

midday

afternoon

TOTAL

Counterclockwise Clockwise

Time Forecast Valid

(fcst hr 1 = 6 p.m. LST)

Wind Directions

All December Forecast Hours 1-24

2.5%

4.5%

9.7%

24.3%

9.3%

27.2%

13.2%

5.7%

3.5%

0%

10%

20%

30%

-180 to -135

-135 to -90

-90 to -45

-45 TO -5

-5 to 5

5 to 45

45 TO 90

90 TO 135

135 TO 180

Forecast Error amount

Percent of Occurrences

evening

night1

night2

morning

midday

afternoon

TOTAL

Counterclockwise Clockwise

Time Forecast Valid

(fcst hr 1 = 6 p.m. LST)
11 

Figure 5. Percentage of Forecasts Satisfying Required Accuracy by Forecast Time. 

These results do not show consistent differences in the forecast accuracy between the various 

times of day. Some of the apparent differences may be attributed to the model start time rather 

than the relative capability of the model to forecast for different times of the day or night. 

However, a drop-off in forecast accuracy from the 1-hour to the 24-hour forecasts is not evident 

to the extent expected. Since the results did not appear promising for providing useful 

information to decision makers, this analysis by forecast time was not performed for the entire 

data set after acquiring the January and February data. 

3.3 Forecast Errors Associated with the Observed Value of the Parameter 

Another analysis considered with the initial December results involved correlating temperature 

forecast errors with the observed values. Figure 6 reveals these errors as a well-defined function 

of the actual temperature value reported. 

* T&DP: + 1oC WS: + 2.5 m/s WD: +5 deg WD+: +45 deg

evening night1 night2 morning midday afternoon

* T&DP: + 1oC WS: + 2.5 m/s WD: +5 deg WD+: +45 deg

evening night1 night2 morning midday afternoon
12 

Figure 6. Percentage of Temperature Forecasts with Specified Error Amounts by Observed 

Temperature Value. 

Although the number of cases with observed temperatures in the most extreme categories is 

small, the trend for the coldest temperatures to be overforecast and for the hottest temperatures to 

be underforecast is very clear. Figure 7 shows this substantial drop in the percent of forecasts 

satisfying the plus or minus 1° C accuracy requirement. The percent of forecasts seen as 

accurate in these December cases was 37 percent when the observed temperature fell between 5 

below and 5 above 0° C, but dropped by 10 percent or more in each 5o observed temperature 

range category moving away from these more moderate values. 

Temperatures

All December Forecast Hours 1-24

0.0% 0.5%

4.6%

17.3%

29.4%

25.7%

13.5%

6.9%

1.6%

0% 0.4% 0.1%

10%

20%

30%

40%

-9 to -7

-7 to -5

-5 to -3

-3 to -1

-1 to 1

1 to 3

3 to 5

5 to 7

7 to 9

9 to 11

11 to 13

Forecast Error amount

Percent of Occurrences

-25 to -20

-20 to -15

-15 to -10

-10 to -5

-5 to 0

0 to 5

5 to 10

10 to 15

TOTAL

 25

129

1203

2733

4085

3344

 515

 27

12061

Underforecast Overforecast

Observed Temp Range

with number of occurrences 
13 

Figure 7. Percentage of Forecasts Satisfying Required Accuracy by Observed Values. 

After considering these results, it was decided that the approach of relying on observed values 

would not be of much benefit to users, since they would not know the observed value ahead of 

time. It is generally intuitive that extreme cases are more likely to be missed, so no further 

analyses were performed using this method for the other weather parameters or the January or 

February data. 

3.4 Forecast Errors Associated with the Forecast Value of the Parameter 

The association of extreme observations with greater forecast error may be similar to the one 

expected for extreme forecast values. Human weather forecasters generally believe that “going 

out on a limb” with a forecast well outside the normal values is usually riskier than a more 

moderate forecast. Numerical weather forecast models do not temper calculated values that fall 

far from climatological or persistence-based expectations. The investigations considering 

forecast values described in this section include the data from all three months used in this study. 

As can be seen in figure 8, the bias for predicting too warm temperatures shows up even in the 

small number of hottest forecasts. Incorporating some sort of bias correction to these forecasts 

was considered, using the average bias from the preceding month. A quick look at the average 

bias amounts listed in table 1 indicates that correcting January temperature and dew-point 

temperature forecasts based on the December bias amounts would have decreased the average 

error amount. On the other hand, using the January average bias to modify the February 

forecasts for these parameters would have simply transformed a 1° C warm bias into a 1° C cold 

bias. Since the data period used in this study was quite limited, no bias correction method was 

pursued. 

Forecast Accuracy by Observed Values December

37.0% 37.0%

21.0%

0.0%

22.0%

13.0%

3.0%

0%

10%

20%

30%

40%

-20 to -15 -15 to -10 -10 to -5 -5 to 0 0 to 5 5 to 10 10 to 15

Observation Ranges (deg C)

Percent of Forecasts with

Required Accuracy

Temp

Temp Avg.

+ 1 deg C

29.4%
14 

Figure 8. Percentage of Temperature Forecasts with Specified Error Amounts by Forecast 

Temperature Value. 

The great majority of wind speed forecasts were for wind speeds below 2.5 m/s or for wind 

speeds between 2.5 and 5 m/s. These forecasts were predominantly within the 2.5 m/s accuracy 

requirement. Figure 9 shows the wind speed errors associated with the wind speed forecast 

categories. Not surprisingly, the highest wind speed forecasts were much more likely to be too 

high than too low. 

Temperatures (deg C)

Winter

0%

10%

20%

30%

40%

-9 to -7

-7 to -5

-5 to -3

-3 to -1

-1 to 1

1 to 3

3 to 5

5 to 7

7 to 9

9 to 11

11 to 13

Forecast Error amount

Percent of Occurrences

-15 to -10

-10 to -5

-5 to 0

0 to 5

5 to 10

10 to 15

Forecast Temp Range

with number of occurrences

Underforecast Overforecast

 2000

8100

19400

25100

11700

500
15 

Figure 9. Percentage of Wind Speed Forecasts with Specified Error Amounts by Forecast 

Wind Speed Value. 

Providing useful information relating to general wind direction error tendencies can be more 

difficult than with other basic weather parameters. Wind direction is frequently listed as one of 

the most important variables for Army users. However, wind direction rules driven by the 

numerical forecast are not commonly included in automated decision aids, since the rules 

typically specify a cross wind or some relative direction rather than a fixed compass direction. 

Figure 10 consists of forecast errors in wind direction in a similar fashion as previous charts. 

The eight forecast wind direction range categories are labeled by compass direction name, with 

each category encompassing 22.5° on either side of the compass point. The average bias amount 

of wind direction error is a relatively smaller percentage of total error than seen in the other 

parameters. 

Decision makers may not be able to easily use information on wind direction bias. A bar chart is 

provided in figure 11, incorporating only the December results as an example of an alternate way 

to view this information. Some people may find the errors stand out somewhat more clearly than 

in the overlapping line chart, but the chart still requires considerable explanation and study to 

determine the desired information. In the bar chart, the forecast wind direction categories 

displayed on the x-axis are equivalent to those used as separate lines in figure 10. The forecast 

error categories associated with each individual forecast direction are portrayed in a separate 

color. However, the error categories themselves are not identical, because in the bar chart the 

errors are counted according to observed compass direction categories as opposed to strictly the 

number of degrees difference in the forecast and observed wind values. A small error could 

occasionally result in a validation in the neighboring category. The information contained in the 

bars in figure 11 is described after the figure. 
16 

Figure 10. Percentage of Wind Direction Forecasts with Specified Error Amounts by Forecast Wind 

Direction Value. 

Figure 11. Bar Chart Version of Percentage of Wind Direction Forecast Error Categories by Forecast Wind 

Direction Value for December. 

N NE E SE S SW W NW all
17 

x For each forecast direction, the percentage of observations from that same direction is 

indicated by a striped bar. 

x The bars to the immediate left of the striped bar for each specific color include the percent 

of cases where the observed wind was one 45o wind category counter-clockwise to the 

forecast category. 

x Likewise, the bars to the immediate right of the striped bars include the total cases where 

the observed wind was one 45° wind category clockwise from the forecast category. 

x The remaining bars to the left and right within each group show the percentage of time the 

observed wind direction is from categories increasingly distant from the forecast category 

in the counter-clockwise and clockwise directions, respectively. 

The highest percentages of accurate forecasts occur when the forecast is for wind direction from 

the northwest, south, or north. When the forecast wind direction is north, the observed wind 

cases are very predominantly from the north or northwest. A similar statement can be made for 

northeast forecast winds, with the validating wind most often from the northeast or the adjacent 

counter-clockwise direction of north. However, northeast forecasts do not verify correctly as 

often as north wind forecasts, and forecasts for east winds show the lowest accuracy of any 

direction. This bar chart is included for illustrative purposes since it is only based on the eight 

forecast days in December. Another option for illustrating all the wind direction data is provided 

in table 3, which also includes only the December cases. Color coding can highlight trends for 

an initial inspection, but a thorough analysis of the results included in a table like this can still be 

time-consuming. The following list summarizes a few conclusions from the December wind 

direction forecasts. 

x The forecasts with an easterly component were somewhat less accurate. 

x Forecasts for south or northwest winds verified accurately most often, but south winds 

were still forecast more frequently than they occurred. 

x Actual wind directions show up opposite from the direction forecast in about 5 percent of 

the cases, ranging from 3 percent when the forecast direction is north to 8 percent when the 

forecast direction is southeast. For this study, no information is provided on whether or not 

frontal passages occurred on the included dates, potentially leading to timing errors on 

wind direction shifts. 

x The observed wind direction was within the correct or adjacent compass wind direction the 

majority of the time, ranging from 49 percent of the forecasts for east winds to at least 70 

percent of the cases when the model forecast winds were from the northwest, south, or 

north. 
18 

Table 3. Percent of the Time the Wind was Actually Observed From Each Compass Direction for Each 

Forecast Direction. 

 OBSERVED WIND DIRECTION Number of Fcsts 

from the Given 

Wind Direction 

 N NE E SE S SW W NW 

N 0.31 0.12 0.05 0.04 0.03 0.05 0.09 0.31 667 6% 

NE 0.23 0.23 0.10 0.07 0.04 0.05 0.10 0.18 601 6% 

E 0.16 0.16 0.19 0.14 0.12 0.06 0.05 0.11 490 5% 

SE 0.06 0.07 0.15 0.22 0.25 0.12 0.06 0.08 1264 12% 

S 0.05 0.05 0.08 0.24 0.35 0.14 0.05 0.04 4073 38% 

SW 0.05 0.04 0.07 0.10 0.23 0.27 0.16 0.07 1846 17% 

W 0.06 0.08 0.07 0.07 0.13 0.16 0.28 0.15 1046 10% 

FORECAST 

WIND DIRECTION 

NW 0.18 0.08 0.07 0.06 0.04 0.06 0.15 0.37 617 6% 

 

Number of Obs 973 777 921 1703 2458 1504 1133 1165 

from the Given 

Wind Direction 9% 7% 9% 16% 23% 14% 11% 11% 

10634 

 

 < 10% 

 10-20%

 20-30% 

For the given forecast wind direction, the cell 

color depicts the percent of time the observed 

wind is from the noted direction 

 30-40% 

 Cells enclosed in double borders indicate the observed wind is from 

the same direction as the forecast wind 

 0.00 Red text indicates the observed wind is from the opposite direction 

as the forecast wind 

Returning to the complete results for all three months, figure 12 depicts the relationship between 

the percent of accurate forecasts and the forecast value for temperature, dew-point temperature, 

wind speed, and wind direction. 
19 

Figure 12. Percentage of Forecasts Satisfying Required Accuracy by Forecast Value. 

As with the observed values, when the forecast value is for an infrequent occurrence, it tends to 

be less accurate. 

x Obviously, the most notable trend is the steep drop in accuracy as wind speeds become 

higher. 

x A lesser but still significant difference exists between wind forecasts for north to east winds 

being accurate about 10 percent less frequently than from other directions. These numbers 

are not equivalent to the ones presented in figure 11 or table 3 because those included only 

the December model runs. Also, the previous discussions were based on discrete compass 

direction categories, while figure 12 wind direction accuracy percentages rely on a straight 

calculation of the difference between observed wind direction and forecast wind direction 

being less than 45o. 

x Only a slight impact can be seen on the hottest or coldest forecast temperatures and dewpoint temperatures. 

3.5 Forecast Errors Associated with the Station Elevation 

Forecast models are frequently presumed to have difficulty handling complex terrain. The 

elevation may not be resolved well at specific station locations, and the scale of local orographic 

effects may not be represented by the model physics. The northern Utah area used in this study 

consists of widely varying terrain, so it seems logical to look for a correlation between station 

elevation and forecast accuracy. The surface stations were arbitrarily divided into categories of 

low (below 1500 m), middle (1500-2200 m), and high (above 2200 m). 

Forecast Accuracy by Forecast Values Winter

87.3%

70.5%

43.4%

33.8%

24.4%

18.3%

51.9%

58.1%

51.6%

39.4% 42.1% 40.8%

53.5% 54.3%

19.0%

25.3% 25.4% 28.0% 25.7%

22.2%

24.5% 21.7% 22.9% 22.0% 23.5% 25.6%

6.6% 6.4% 4.8% 6.5% 8.3% 7.1% 7.4% 7.8%

0%

20%

40%

60%

80%

- 0 to 2.5 2.5 to 5 5 to 7.5 7.5 to 10 10 to 12.5 12.5 to 15 -

N NE E SE S SW W NW

-20 to -15 -15 to -10 -10 to -5 -5 to 0 0 to 5 5 to 10 10 to 15 -

Forecast Ranges

Percent of Forecasts with

Required* Accuracy

Wind Spd

WS Avg.

WD+

WD+ Avg.

Temp

Temp Avg.

DP

DP Avg.

WD

WD Avg.

Wind Speed (m/s)

Wind Direction (45 deg ranges)

Temp & DP (deg C)

+ 2.5 m/s *

+ 1 deg C *`

73.0%

24.3%

+ 45 deg

+ 5 deg *

7.2%

50.5%

25.7%

+ 1 deg C *

Forecast Accuracy by Forecast Values Winter

87.3%

70.5%

43.4%

33.8%

24.4%

18.3%

51.9%

58.1%

51.6%

39.4% 42.1% 40.8%

53.5% 54.3%

19.0%

25.3% 25.4% 28.0% 25.7%

22.2%

24.5% 21.7% 22.9% 22.0% 23.5% 25.6%

6.6% 6.4% 4.8% 6.5% 8.3% 7.1% 7.4% 7.8%

0%

20%

40%

60%

80%

- 0 to 2.5 2.5 to 5 5 to 7.5 7.5 to 10 10 to 12.5 12.5 to 15 -

N NE E SE S SW W NW

-20 to -15 -15 to -10 -10 to -5 -5 to 0 0 to 5 5 to 10 10 to 15 -

Forecast Ranges

Percent of Forecasts with

Required* Accuracy

Wind Spd

WS Avg.

WD+

WD+ Avg.

Temp

Temp Avg.

Forecast Accuracy by Forecast Values Winter

87.3%

70.5%

43.4%

33.8%

24.4%

18.3%

51.9%

58.1%

51.6%

39.4% 42.1% 40.8%

53.5% 54.3%

19.0%

25.3% 25.4% 28.0% 25.7%

22.2%

24.5% 21.7% 22.9% 22.0% 23.5% 25.6%

6.6% 6.4% 4.8% 6.5% 8.3% 7.1% 7.4% 7.8%

0%

20%

40%

60%

80%

- 0 to 2.5 2.5 to 5 5 to 7.5 7.5 to 10 10 to 12.5 12.5 to 15 -

N NE E SE S SW W NW

-20 to -15 -15 to -10 -10 to -5 -5 to 0 0 to 5 5 to 10 10 to 15 -

Forecast Ranges

Percent of Forecasts with

Required* Accuracy

Wind Spd

WS Avg.

WD+

WD+ Avg.

Temp

Temp Avg.

DP

DP Avg.

WD

WD Avg.

Wind Speed (m/s)

Wind Direction (45 deg ranges)

Temp & DP (deg C)

+ 2.5 m/s *

+ 1 deg C *`

73.0%

24.3%

+ 45 deg

+ 5 deg *

7.2%

50.5%

25.7%

+ 1 deg C *
20 

The temperature errors for each of the three elevation categories are shown in figure 13. The 

forecast bias toward too warm temperatures is most prevalent at the higher stations. 

Figure 13. Percentage of Temperature Forecasts with Specified Error Amounts by Station Elevation. 

The dew-point temperature accuracies are similar to those for the temperature forecasts, but the 

low stations show a greater improvement over the middle stations, as seen in figure 14. 

Figure 14. Percentage of Dew-Point Temperature Forecasts with Specified Error Amounts by Station 

Elevation.

Temperatures (deg C)

Winter

0%

10%

20%

30%

40%

-9 to -7

-7 to -5

-5 to -3

-3 to -1

-1 to 1

1 to 3

3 to 5

5 to 7

7 to 9

9 to 11

11 to 13

Forecast Error amount

Percent of Occurrences

Low

Mid

High

 21600

33600

17800

Underforecast Overforecast

29.4%

26.5%

21.5%

Station Elevation

with number of 

occurrences
21 

Figure 15 portrays the wind speed forecast accuracy by station elevation. These appear only a 

little less accurate at the high stations. 

Figure 15. Percentage of Wind Speed Forecasts with Specified Error Amounts by Station Elevation. 

On the other hand, figure 16 reveals wind direction forecasts significantly more accurate at the 

high stations. Cases with wind speeds below 1 m/s are omitted in these results, but it is still 

likely that greater wind direction variability at the lower stations causes more difficulty in 

forecasting this parameter. 
22 

Figure 16. Percentage of Wind Direction Forecasts with Specified Error Amounts by Station Elevation. 

This study indicated that varying forecast error amounts may be associated with station 

elevation, based on data from many surface stations in each of the selected elevation categories 

separated by 1500 and 2200 m (4900 and 7200 ft). A check of the lowest and highest subset of 

the stations below 1300 m and above 2500 m did not highlight any great differences from the 

less stringent definition used for low and high stations. The forecast error trends as a function of 

station height are summarized below: 

x Temperatures and dew-point temperatures were more likely to be over-forecast at the high 

stations. 

x Since the dew-point error tends to be in the same direction as the temperature error, relative 

humidity forecasts might not reflect as much difference between the higher and lower 

stations. 

x Wind speeds were somewhat less accurate at the high stations, but not by a lot considering 

they can be expected to be stronger with more room for error. 

x The wind direction forecasts were accurate more often at the high stations. 

3.6 Forecast Errors for All Cases 

In this study, it appears that consideration of the station elevation may provide some insight into 

the frequency of various errors in the MM5 forecasts. These differences cannot be generalized to 

other locations or times. Since the forecast errors don’t appear to vary significantly with several 

of the factors considered, it seems worthwhile to view the total occurrence of the error amounts 

for all the cases together. However, it is still true that these findings cannot be assumed to 

represent MM5 errors for other times or places. 
23 

Figures 17 through 20 show the frequency of the various error amounts for temperature, dewpoint temperature, wind speed, and wind direction, respectively, for all the forecasts and stations 

combined. 

Figure 17. Percentage of Temperature Forecasts with Specified Error Amounts for all Stations 

and Forecast Times Combined. 

Figure 18. Percentage of Dew-Point Temperature Forecasts with Specified Error Amounts 

for all Stations and Forecast Times Combined. 
24 

Figure 19. Percentage of Wind Speed Forecasts with Specified Error Amounts for all Stations and 

Forecast Times Combined. 

Figure 20. Percentage of Wind Direction Forecasts with Specified Error Amounts for all Stations and 

Forecast Times Combined. 
25 

4. Conclusions 

4.1 Relating Error Amounts to User Forecast Uncertainty Calculations 

The previous charts all take a fair amount of study to determine the probability of various 

forecast errors. Table 4 presents a simplified method for users to incorporate previously 

observed errors into forecast uncertainty calculations. A user would choose the confidence level 

desired for the weather forecast, based on its criticality. Then the single forecast value output by 

the MM5 model would be turned into a range of values by adding and subtracting the amount 

listed for the confidence level chosen and the parameter of interest. 

Table 4. User-Selected Confidence Desired to Generate a Forecast Range Based on the Frequency 

of Forecast Errors. 

CONFIDENCE LEVEL TEMP DP TEMP WND SPD WND DIR 

DESIRED (°C) (m/s) (degrees) 

 

5% 0.2 0.2 0.1 3 

10% 0.4 0.4 0.2 7 

15% 0.6 0.6 0.4 11 

20% 0.7 0.8 0.5 15 

25% 0.9 1.0 0.6 19 

30% 1.1 1.2 0.8 23 

35% 1.4 1.5 0.9 28 

40% 1.6 1.7 1.1 33 

45% 1.8 1.9 1.2 38 

50% 2.1 2.2 1.4 44 

55% 2.3 2.4 1.6 51 

60% 2.6 2.7 1.8 59 

65% 2.9 3.0 2.1 68 

70% 3.3 3.4 2.3 78 

75% 3.7 3.9 2.6 91 

80% 4.2 4.4 3.0 105 

85% 4.9 5.1 3.4 121 

90% 5.7 6.0 4.1 137 

95% 

Determine 

a Forecast 

Range by 

Using the 

MM5 

Forecast 

Value 

plus 

and 

minus 

the amount 

indicated

7.0 8.2 5.0 154 

Alternatively, an experienced staff weather officer (SWO) could determine a high, medium, or 

low confidence in the model run based on knowledge of the weather situation and the particular 

forecast involved. Table 5 shows values that could be used to modify the single forecast value in 

these cases. The ranges associated with these high, medium, and low confidence levels are 

similar to the user’s desired 25 percent, 50 percent, and 75 percent confidence levels, 

respectively. 
26 

Table 5. Staff Weather Officer Confidence Assigned to Generate a Forecast Range Based on the 

Frequency of Forecast Errors. 

CONFIDENCE LEVEL TEMP DP WND SPD WND DIR 

ASSIGNED BY SWO (°C) (m/s) (degrees) 

 

HIGH 1 1 0.5 20 

 

MEDIUM 2 2 1.5 45 

 

LOW 

Forecast 

Value 

+

Amount 

Indicated 4 4 2.5 90 

NOTE: Red cells include ranges greater than specified in Army requirements. 

Bias trends have been ignored in the forecast ranges calculated from the values in tables 4 and 5. 

Correcting for biases, if known, would result in smaller forecast ranges for each confidence 

level. In the forecasts used for this study, the improvements based on correcting for bias errors 

would be most pronounced for temperature and dew-point temperature. Cells shaded red in table 

5 highlight forecast ranges larger than the Army’s stated accuracy requirement. Note that a 

model forecast at the high confidence level would still result in a total wind direction forecast 

range of 40°. If users truly find this amount of variability to be unacceptable, they certainly 

wouldn’t be satisfied with the vast majority of wind direction forecasts. 

4.2 Recommendations 

This study approached weather forecast uncertainty as a function of historical model errors. 

Many thousands of MM5 forecast values were compared to observations. However, this 

“history” was limited in time to three months in a single winter season, and in space to a single 

geographic area around northern Utah. Even within this limited domain, it is difficult to clearly 

identify variables that can be associated with more or less forecast uncertainty. The most 

promising variable appeared to be station elevation. Many more cases would need to be 

examined over different times and locations to determine if the forecast errors seen in this study 

are representative of overall MM5 performance. 

The statistics generated and presented in this and similar studies could be used to answer a 

general question of the typical accuracy of a forecast model. But decision makers on the 

battlefield would benefit much more by having information on the specific uncertainty associated 

with the individual day’s forecast. Additional efforts to quantify model forecast error will be 

useful. 
27 

In addition to expanding the time and space coverage of model accuracy statistics, they may be 

used to determine and correct for model biases. Other efforts hold more promise for providing 

insight into the expected error on a particular day. A possible follow-on study to the one 

presented in this paper could find how closely errors from the previous day or two’s model runs 

correlate to an individual day’s forecast error from the same model. It seems logical to assume 

that when a forecast turns out to be wrong one day, users will have a lower confidence in its 

accuracy the next day. 

Statistics on the length or repetitiveness of poorer-than-average model performance may be 

useful. Although model ensembles are not currently available on the battlefield, information 

from ensembles may someday be used to estimate weather forecast uncertainty in tactical 

situations. Methods to provide uncertainty information to decision makers will need to be 

automated and easy to interpret. Additional information only has value if it leads to better 

outcomes. 

Both civilian and military agencies are pursuing efforts to facilitate better decision making in the 

face of weather forecast uncertainty. The U.S. Army Research Laboratory (ARL) is currently 

sponsoring Small Business Innovation Research contracts with this goal in mind. In addition, the 

Department of Defense (DoD) Center for Geosciences/Atmospheric Research at Colorado State 

University will continue to coordinate with ARL to address Army needs in atmospheric research, 

including the use of model ensembles. 
28 

References 

1. Grell, G.A.; Dudhia, J.; Stauffer, D.R. A description of the fifth-generation Penn 

State/NCAR Mesoscale Model (MM5). NCAR/TN-398+STR; NCAR Tech. Note, 1995, pp 

122. 

2. Sauter, B.; Henmi, T. Reality-Based Nowcasting: The Utah Winter Episode. Proceedings 

of the Battlespace Atmospheric and Cloud Impacts on Military Operation (BACIMO) 

Conference 2003, September 2003. 

3. Hogan, T.F.; Rosmond, T.E. The Description of the Navy Operational Global Atmospheric 

Prediction System’s Spectral Forecast Model. Mon Wea. Rev. 1991, 199, pp1786-1815. 

4. National Center for Environmental Prediction Global Forecast System: The GFS 

Atmospheric Model (status as of November 01, 2002). 

http://www.emc.ncep.noaa.gov/modelinfo/index.html (accessed November 2003). 

5. Weather Support Team, U.S. Army Intelligence Center: Weather and Environmental Data 

Element Requirements. Internal Report, Futures Directorate: Fort Huachuca, Arizona, May 

1998. 
29 

Acronyms 

ARL U.S. Army Research Laboratory 

CISD Computational and Information Sciences Directorate 

DoD Department of Defense 

IWEDA Integrated Weather Effects Decision Aid 

MM5 Penn State/National Center for Atmospheric Research fifth-generation 

Mesoscale Model 
30 

Distribution List 

 Copies 

DTIC 1 

8725 JOHN J KINGMAN RD 

STE 0944 

FT BELVOIR VA 22060-6218 

HEADQUARTERS DEPT OF ARMY 1 

DAMI POI 

WASHINGTON DC 20310-1067 

US ARMY RESEARCH LABORATORY 1 

AMSRD ARL D 

2800 POWDER MILL ROAD 

ADELPHI MD 20783 

US ARMY RESEARCH LABORATORY 1 

AMSRD ARL CI 

ATTN: J GANTT 

2800 POWDER MILL ROAD 

ADELPHI MD 20783-1197 

US ARMY RESEARCH LABORATORY 1 

AMSRD ARL CI E 

BATTLEFIELD ENVIR DIV 

ADELPHI MD 20783-1197 

US ARMY RESEARCH LABORATORY 1 

AMSRD ARL CI EB 

BATTLEFIELD ENVIR DIV 

WSMR NM 88002-5501 

US ARMY RESEARCH LABORATORY 1 

ATTN AMSRD ARL CI IS R 

MAIL& RECORDS MGMT 

ADELPHI, MD 20783-1197 

US ARMY RESEARCH LABORATORY 1 

AMSRD ARL CI LL 

ADELPHI MD 20783-1197 
31 

US ARMY RESEARCH LABORATORY 1 

AMSRD ARL CS 

RECORD COPY 

2800 POWDER MILL ROAD 

ADELPHI MD 20783 

US ARMY RESEARCH LABORATORY 1 

NBC EFFECTS BRANCH (EDGEWOOD) 

CHEMICAL BIOLOGY NUC EFFECTS DIV 

AMSRD ARL SL BN 

APG MD 21010-5423 

US ARMY RESEARCH LABORATORY 1 

ATTN SFAE-C3T-IE-II 

ROBERT DICKENSHIED 

WSMR, NM 88002 

US ARMY RSRC OFC 1 

ATTN AMXRO GS DR BACH 

PO BOX 12211 

RTP NC 27009 

US ARMY RESEARCH OFFICE 1 

DRXRO GS 

ATTN DR W FLOOD 

PO BOX 12211 

RTP NC 27709 

OPTEC TECHNICAL LIBRARY 1 

4501 FORD AVENUE SUITE 820 

ALEXANDRIA VA 22302-1458 

US ARMY ARDEC 1 

AMSRD AAR TA AR IMC BLDG 59 

INFO RESEARCH CENTER 

PICATINNY ARSENAL NJ 07806-5000 

US ARMY CECOM 1 

INFORMATION & INTELLIGENCE 

WARFARE DIRECTORATE 

ATTN AMSEL RD IW IP 

FORT MONMOUTH NJ 07703-5211 
32 

US ARMY CECRL 1 

CECRL GP 

ATTN DR DETSCH 

HANOVER NH 03755-1290 

US ARMY CHEM RES DEV & ENGR CTR 1 

SMCCR OPA 

ATTN MR PENNSYLE 

APG MD 21010-5423 

US ARMY DUGWAY PROVING GROUND 1 

STEDP MT 

DUGWAY UT 84022 

US ARMY FIELD ARTILLERY SCHOOL 1 

ATSF TSM TA 

FT SILL OK 73503-5600 

US ARMY INFANTRY 1 

ATSH CD CS OR 

ATTN DR E DUTOIT 

FT BENNING GA 30905-5090 

US ARMY MATERIEL SYST 1 

ANALYSIS ACTIVITY 

AMSXY 

APG MD 21005-5071 

US ARMY MISSILE COMMAND 1 

AMSMI RD TE F 

ATTN MET TEAM 

REDSTONE ARSENAL AL 35898 

US ARMY MISSILE COMMAND 1 

REDSTONE SCI INFO CTR 

AMSMI RD CE R DOCUMENTS 

REDSTONE ARSENAL AL 35898-5253 

US ARMY MISSILE CMND 1 

REDSTONE SCI INFO CTR 

AMSMI RD CS R DOC 

REDSTONE ARSENAL AL 35898-5241 
33 

US ARMY OEC 1 

CSTE EFS 

PARK CENTER IV 

4501 FORD AVE 

ALEXANDRIA VA 22302-1458 

US ARMY TOPO ENGR CTR 1 

CETEC ZC 1 

FT BELVOIR VA 22060-5546 

US ARMY TRADOC ANALYSIS COMMAND 1 

ATCD FA 

FT MONROE VA 23651-5170 

US ARMY TRADOC ANALYSIS CTR 1 

ATTN ATRC WSS R 

WSMR NM 88002-5502 

US ARMY TRAIN & DOCT CMND 1 

ATCL FA 

FT MONROE VA 23651-5170 

WSMR TECH LIBRARY BR 1 

STEWS IM IT 

WSMR NM 88002 

AFMC DOW 1 

WRIGHT PATTERSON AFB OH 45433-5000 

AFRL/VSBL 1 

29 RANDOLPH RD 

HANSCOM AFB MA 01731 

DEPT OF THE AIR FORCE 1 

HQ WEATHER WING MAC 

5WWDN 

LANGLEY AFB VA 23665-5000 

DEPT OF THE AIR FORCE 1 

OL A 2D WEATHER SQUAD MAC 

HOLLOMAN AFB NM 88330-5000 

HQ AFWA/DNX 1 

106 PEACEKEEPER DR STE 2N3 

OFFUTT AFB NE 68113-4039 
34 

PL WE 1 

KIRTLAND AFB NM 87118-6008 

TACDOWP 1 

LANGLEY AFB VA 23665-5524 

USAF ROME LAB TECH 1 

CORRIDOR W STE 262 RL SUL 

26 ELECTR PKWY BLD 106 

GRIFFISS AFB ROME, NY 13441-4514 

NAVAL RESEARCH LABORATORY 1 

MARINE METEOROLOGY DIVISION 

7 GRACE HOPPER AVENUE, STOP 2 

MONTEREY, CA 93943-5502 

NAVAL SURFACE WEAPONS CTR 1 

CODE G63 

DAHLGREN VA 22448-5000 

PACIFIC MISSILE TEST CTR 1 

GEOPHYSICS DIV 

ATTN CODE 3250 

POINT MUGU CA 93042-5000 

NASA MARSHALL SPACE FLT CTR 1 

ATMOSPHERIC SCIENCES DIV 

CODE SD01 

ATTN DR FITCHL 

HUNTSVILLE AL 35812 

NCAR LIBRARY SERIALS 1 

NATL CTR FOR ATMOS RSCH 

PO BOX 3000 

BOULDER CO 80307-3000 

US ARMY RESEARCH LAB 5 

AMSRD ARL CI EB (B SAUTER) 

WSMR NM 88005 

US ARMY RESEARCH LABORATORY 1

ATTN: AMSRD ARL CI IS R (A SMITH) 

MAIL & RECORDS MGMT 

ADELPHI MD 20783-1197
35 

ADMNSTR 1 

DEFNS TECHL INFO CTR 

ATTN: DTIC OCP (ELECT CPY) (W SMITH) 

8725 JOHN J KINGMAN RD STE 0944 

FT BELVOIR VA 22060-6218 

US ARMY RESEARCH LABORATORY 2 

AMSRD ARL CI OK TL 

ATTN: K RAPKA 

2800 POWDER MILL ROAD 

ADELPHI MD 20783-1197 

TOTAL 53 
36 

INTENTIONALLY LEFT BLANK.