Evaluation Study of Mesoscale Models MM5 and BFM Over 

the Model Domains of Utah Using Surface Meteorological 

Data by Mesowest 

by Teizi Henmi 

ARL-TR-2928 March 2003 

Approved for public release; distribution is unlimited. 
NOTICES 

Disclaimers 

The findings in this report are not to be construed as an official Department of the Army position unless 

so designated by other authorized documents. 

Citation of manufacturer’s or trade names does not constitute an official endorsement or approval of the 

use thereof. 

Destroy this report when it is no longer needed. Do not return it to the originator. 
Army Research Laboratory 

White Sands Missile Range, NM 88002-5513

ARL-TR-2928 March 2003 

Evaluation Study of Mesoscale Models MM5 and BFM Over 

the Model Domains of Utah Using Surface Meteorological 

Data by Mesowest 

Teizi Henmi 

Computational and Information Sciences Directorate, ARL 

Approved for public release; distribution is unlimited.

iii 

Contents 

Executive Summary 1

1. Introduction 2

2. Description of the Models and Model Domains 4

2.1 MM5................................................................................................................................4 

2.2 BFM.................................................................................................................................7 

3. Meteorological Data 9

4. Forecast and Evaluation Methods 11

4.1 Forecast Method............................................................................................................11 

4.1.1 MM5..................................................................................................................11 

4.1.2 BFM...................................................................................................................11 

4.2 Evaluation Method ........................................................................................................12

4.2.1 Data Comparison Method .................................................................................12 

4.2.2 Statistical Parameters ........................................................................................12 

5. Results 14

5.1 Statistical Parameters of Daily Data..............................................................................14 

5.2 MM5 Forecast and Observation Data Comparison.......................................................15 

5.3 Statistical Parameters of Entire Forecasting Periods.....................................................21 

5.4 Time Series of the Statistical Parameters......................................................................22 

6. Conclusions 27

References 29

Acronyms 79

Report Documentation Page 80
iv 

Appendix A. Statistical Diagrams Displaying the Difference Between MM5 

Forecast Data and Observation Data for the Period of January–March, 2002 31 

Appendix B. Statistical Diagrams Displaying the Difference Between MM5 

Forecast Data and Observation Data for the Period of April–June, 2002 43 

Appendix C. Statistical Diagrams Displaying the Difference Between BFM 

Forecast Data and Observation Data for the Period of January–March, 2002 55 

Appendix D. Statistical Diagrams Displaying the Difference Between BFM 

Forecast Data and Observation Data for the Period of April–June, 2002 67 

List of Figures 

Figure 1. Geographical extents of the four computational domains................................................5 

Figure 2. Terrain contours of MM5 domain 3. ................................................................................6 

Figure 3. Terrain contours of MM5 domain 4. ................................................................................7 

Figure 4. Terrain contours of the BFM model domain (2.5 km grid increments). ..........................8 

Figure 5. Terrain contours of the BFM model domain (5 km grid increments). .............................9 

Figure 6. Difference between MM5 forecast and observation: temperature 

(domain 3)................................................................................................................................16 

Figure 7. Difference between MM5 forecast and observation: temperature 

(domain 4)................................................................................................................................16 

Figure 8. Difference between MM5 forecast and observation: dew point temperature 

(domain 3)................................................................................................................................17 

Figure 9. Difference between MM5 forecast and observation: temperature 

(domain 4)................................................................................................................................17 

Figure 10. Difference between MM5 forecast and observation: wind vector-x 

component u (domain 3). ........................................................................................................18 

Figure 11. Difference between MM5 forecast and observation: wind vector-x, 

component u (domain 4). .........................................................................................................18 

Figure 12. Difference between MM5 forecast and observation: wind vector-y, 

component v (domain 3). ........................................................................................................19 

Figure 13. Difference between MM5 forecast and observation: wind vector-y, 

component v (domain 4). ........................................................................................................19 

Figure 14. Difference between MM5 forecast and observation: wind speed 

(domain 3)................................................................................................................................20 

Figure 15. Difference between MM5 forecast and observation: wind speed 

(domain 4)................................................................................................................................20 
v 

Figure 16. Time series of statistical parameters for temperature 

(April-June 2002, MM5 domain 4)..........................................................................................23 

Figure 17. Time series of statistical parameters of temperature 

(April-June 2002, BFM 2.5 km grid increment).....................................................................23 

Figure 18. Time series of statistical parameters for dew point temperature 

(April-June 2002, MM5 domain 4).........................................................................................24 

Figure 19. Time series of statistical parameters for temperature 

(April-June 2002, BFM 2.5 km grid increment).....................................................................24 

Figure 20. Time series of statistical parameters for wind speed 

(April-June 2002, MM5 domain 5)..........................................................................................25 

Figure 21. Time series of statistical parameters for wind speed 

(April-June 2002, BFM 2.5 grid increment)...........................................................................25 

Figure 22. Time series for RMSVE, MWDDF, and RMSE of wind direction 

difference (April-June 2002, MM5 domain 4). ......................................................................26 

Figure 23. Time series for RMSVE, MWDDF and RMSE of wind direction 

difference (April-June 2002, BFM 2.5 km grid increment).....................................................26

List of Tables 

Table 1. Portion of the daily data...................................................................................................10 

Table 2. Dates of archived data sets and model calculations for 24-h forecasting........................11 

Table 3. Statistical parameters calculated for surface temperature................................................14 

Table 4. Statistical parameters calculated for wind vector. ...........................................................15 

Table 5. Statistical parameters of temperature for different model domains (JanuaryMarch 2002).............................................................................................................................21 

Table 6. Statistical parameters of dew pt temperature for different model domains 

(January-March 2002)..............................................................................................................21 

Table 7. Statistical parameters of wind vector for different model domains 

(January-March 2002)..............................................................................................................21 

Table 8. Statistical parameters of temperature for different model domains 

(April-June 2002).....................................................................................................................21 

Table 9. Statistical parameters of dew point temperature for different model domains 

(April-June 2002).....................................................................................................................22 

Table 10. Statistical parameters of wind vector for different model domains 

(April-June 2002).....................................................................................................................22 
1 

Executive Summary 

The forecast skills of the Penn State/National Center for Atmospheric Research 

Mesoscale Model version 5 (MM5), and the Battlescale Forecast Model (BFM) were 

statistically evaluated by comparing the model forecasting data with surface observation 

data obtained from the University of Utah Mesowest cooperative. The evaluation study 

was made for two different periods: January-March 2002, and April-June 2002. 

The MM5, with quadruple-nests of 67.5, 22.5, 7.5, and 2.5 km grid increments, all of 

which have 51x51 grid points centered near Salt Lake City, UT, was set for computation 

on the U.S. Army High Power Computer. The forecast data of domains 3 and 4, with grid 

increments of 7.5 and 2.5 km (respectively), are compared with the observation data in 

each domain. The BFM forecasting calculations were made separately over two model 

domains with grid increments of 5 and 2.5 km, both centered on Salt Lake City, UT. 

The major findings of this study include the following: 

1. There are no significant statistical differences in forecast results between MM5 

domains 3 and 4. Likewise, the forecast results of the BFM for the two grid 

increments of 5 and 2.5 km are statistically similar. 

2. Both MM5 and the BFM produce better forecast results for surface temperature 

than for dew point temperature, and have difficulties producing good forecast 

results for wind vectors. 

3. For all of the surface meteorological parameters, MM5 produced statistically 

better forecast results than the BFM over the complex terrain of Utah. 

4. Both MM5 and the BFM showed better forecasting statistics for the period of 

April-June 2002 than for the period of January-March 2002. 
2 

1. Introduction 

The Battlescale Forecast Model (BFM), developed at the U.S. Army Research Laboratory 

(ARL), has been extensively used to make short-range forecasts of atmospheric 

conditions as a component of the Integrated Meteorological System (IMETS). The BFM 

uses, for prognostic calculation, a hydrostatic model called the Higher Order Turbulence 

Model for Atmospheric Circulation (HOTMAC), which was developed by Yamada and 

Bunker (1989). The HOTMAC was used for prognostic calculation because of its 

numerical stability, which allows operational forecasting calculations to be made without 

failure. HOTMAC was also chosen because of its short computational time requirement. 

A 24-h forecast calculation over a battlescale area, such as 500x500 km2

, could be 

completed by BFM within one hour on a workstation computer having a single processor. 

With advancements in both computer hardware and software, such as the availability of a 

parallel computer with distributed memory and Message Passage Interface (MPI), 

applications like non-hydrostatic models with many different physics options are 

becoming possible. Thus, it is important to evaluate how a non-hydrostatic model, such 

as the Pennsylvania State University/National Center for Atmospheric Research (NCAR) 

Mesoscale Model version 5 (MM5), performs over areas of complex terrain. For 

comparison, the BFM is also evaluated in this study. 

In a previous study, the forecast skills of the BFM were compared to those of MM5 by 

applying the models to the domain of White Sands Missile Range (WSMR), NM, which 

covers an area of 167x167 km, 51x51 grid points with grid spacing of 3.33 km (Henmi, 

2000). Meteorological parameters forecasted by the models were compared with 

observed data. The comparison study showed that the forecast skills of the BFM are 

comparable to those of the MM5. Surface temperature forecasted by both the BFM and 

MM5 agreed well with observed values. Both the BFM and the MM5 showed difficulties 

for forecasting the relative humidity. In forecasting wind parameters, both models tended 

to predict wind speed less than that observed, but BFM calculations produced lower wind 

speed than the MM5. The BFM resulted in better forecasts than the MM5 for wind 

direction. 

In operational mode on the IMETS, the BFM has been extensively used over the model 

domain of 500x500 km2

 with grid spacing of 10 km. A statistical evaluation of the BFM 

in operational mode was conducted for cases during a 30-day period over three different 

model domains (Colorado, Washington and Florida), which had different terrain 

complexities and climates (Henmi, 2000). The model calculations were initialized with 

three different sets of initial conditions: 
3 

1. NOGAPS + upper air data + surface data 

2. NOGAPS + upper air data 

3. NOGAPS 

Forecast data for 24-h periods were statistically compared with surface observation data 

by calculating parameters such as mean difference (MD), absolute difference (AD), root 

mean square error (RMSE), root mean square vector error (RMSVE), and correlation 

coefficient (CC). 

For all three model domains, the temperature fields of the BFM were statistically better 

when initialized with (1) and (2) than when initialized with (3). For Colorado and 

Washington model domains, the BFM showed clear tendencies of forecasting dew point 

temperature lower than those observed throughout the 24-h forecast period. However, for 

the Florida model domain, forecasts of dew point temperature were higher than observed. 

The three different types of initialization data did not produce significantly different wind 

fields throughout the 24-h forecast period. The value of MD for wind speed is in the 

range of 0 to 1 m/sec. The values of AD are also between 0 and 1 m/sec for the three 

model domains throughout the 24-h forecast period. 

For the Colorado and Washington model domains, where the terrain is more complex 

than in Florida, the use of the BFM improves temperature forecasts over those of the 

Navy Operational Global Atmospheric Prediction System (NOGAPS) and the Navy 

Operational Regional Atmospheric Prediction System (NORAPS). For the Florida 

domain, no significant improvements in temperature forecasts are found. Similarly, the 

BFM produces better wind fields than NOGAPS and NORAPS over Colorado and 

Washington. 

In the recent study of an objective multiyear evaluation of the University of Washington 

real-time MM5 forecasts, in which forecasts from the 36, 12, and 4 km domains were 

verified at surface observation locations over western Washington state, Mass et al. 

states: “Reduction from 36 to 12 km grid increments allows the definition of the major 

topographic features of the region and their corresponding atmospheric circulations, 

resulting in a beneficial effect on the evaluation. However, there are only small 

improvements in verification statistics as grid spacing decreased from 12 to 4 km.” 

(2002). 

The purpose of this report is to describe the results of a statistical evaluation of surface 

forecast data from MM5 domains with grid increments of 7.5 and 2.5 km, and from BFM 

domains with grid increments of 5 and 2.5 km, all of which are centered near Salt Lake 

City, UT, and use Mesowest data. The ARL High Power Computer (HPC) parallel 

computer is extensively used for this study. 
4 

2. Description of the Models and Model Domains 

2.1 MM5 

The model used in this study is the distributed-memory version of the PSU-NCAR MM5. 

The source code of MM5 was obtained from the MM5 Home page (2001). 

The MM5 is based on non-hydrostatic dynamics and features multiple-nest capabilities 

and many physics options. Details of this modeling system can be found in Dudhia 

(1993), Grell et al. (1994) and Warner et al. (1992). 

For the present study, the following physics options are employed: 

1. Planetary Boundary Layer (PBL). The PBL technique that is used in the MediumRange Forecast (MRF) model of the National Center for Environmental 

Prediction (NCEP) by Hong and Pan (1996). 

2. Precipitation parameterization. A simple treatment of cloud microphysics based 

on Dudhia (1989), in which both ice and liquid phases are permitted for cloud and 

precipitation, but mixed phases are not permitted. 

3. Cumulus parameterization. Grell’s scheme that is based on rate of destabilization 

or quasi-equilibrium, simple single-cloud scheme with updraft and downdraft 

fluxes, and compensating motion determining heating and moistening profile 

(1994). 

4. Radiation parameterization. Dudhia’s scheme in which long wave and short wave 

radiation interact with the clear atmosphere, clouds, precipitation, and the ground 

(1989). 

5. Ground temperature scheme. Multilayer soil temperature model. 

The MM5 quadruple-nested computational grids are depicted in figure 1. All four 

computational domains have a mesh size of 55x55 grid points, and grid increments of 

67.5, 22.5, 7.5, and 2.5 km, respectively, for domains 1, 2, 3, and 4. For the present study, 

the forecast fields only for domains 3 and 4 are compared with observed data. 
5 

Figure 1. Geographical extents of the four computational domains. 

Terrain height contours of domains 3 and 4 are shown, respectively, in figures 2 and 3. 

Terrain, vegetation/land use, and soil type data for the model domains are obtained from 

data archived in the MM5 Internet site. 
6 

Figure 2. Terrain contours of MM5 domain 3. 
7 

Figure 3. Terrain contours of MM5 domain 4. 

2.2 BFM 

The BFM is an operational mesoscale forecast model developed by the ARL (Henmi and 

Dumais, 1998) at WSMR, NM. It has been extensively used to make short-range 

forecasts of atmospheric conditions as a component in the IMETS. The BFM uses, for 

prognostic calculation, the HOTMAC, developed by Yamada and Bunker (1989). In 

operational mode on the IMETS, the BFM has been extensively used over a model 

domain of 500x500 km with grid increments of 10 km. So far, there has been no 

evaluation study of the BFM over the domain with grid increment smaller than 5 km for 

an extended period. In this study, the BFM was used over two domains with 2.5 and 5 km 

grid increments, centered at Salt Lake City, UT. 

Figure 4 shows the terrain contours of the 2.5 km grid increment domain. Figure 5 shows 
8 

the terrain contours of the 5 km grid increment domain. An inner squared area of 51x51 

grid points with 125 km2 and 250 km2 are used for the model calculations, respectively, 

for figures 4 and 5. In these figures, the inner, squared areas represent the BFM model 

domains. The spots denoted by “N” are the locations of the NOGAPS grid points that are 

used for initial and time-dependent boundary values, and the spots denoted by “*” 

represent the Mesowest data points. For model evaluation, data within the squared area is 

used. 

Figure 4. Terrain contours of the BFM model domain 

 (2.5 km grid increments). 
9 

Figure 5. Terrain contours of the BFM model domain 

 (5 km grid increments). 

3. Meteorological Data 

There is a large amount of surface observation data available for the model domains that 

are shown in figures 4 and 5. The data is freely available from the University of Utah 

Mesowest cooperative. 

A Mesowest data file, called “total.dat,” contains data in 15-min intervals for a 24-h 

period for the western United States. A detailed description of Mesowest data can be 

found in Horel et al. (2002). 

A daily data file that contains data only for the even hours of a 24-h period (starting at 

00:00 UTC (universal time coordinate)) was created for the areas to be studied. Table 1 

displays a portion of the daily data file. Table 1 shows the following (from left to right): 

station name, day, hour, minute, latitude, longitude, wind direction, wind speed (knot), 

temperature (°F), dew point temperature (°F), relative humidity (%), and 24-h 

accumulated precipitation (in this example –9999.0 zero or no report). This data was 

compared to hourly model forecast data. 
10 

 Table 1. Portion of the daily data. 

BGRUT 2 516 0 0 40.93 -112.56 90.00 7.80 62.00 35.50 37.00-9999.00 

 BR4 2 516 0 0 41.44 -112.16 148.00 0.90 65.00 27.40 24.00-9999.00 

 DVB 2 516 0 0 40.62 -111.48 248.00 2.60 56.00 19.70 24.00-9999.00 

EMPUT 2 516 0 0 40.61 -111.53 245.00 4.30 45.00 20.30 37.00-9999.00 

 FPK 2 516 0 0 41.03 -111.84 323.00 13.00 38.00 8.50 29.00-9999.00 

 FWP 2 516 0 0 40.66 -112.20 244.00 3.50 43.00 17.20 35.00-9999.00 

 GNI 2 516 0 0 41.33 -112.85 96.00 8.70 62.00 29.40 29.00-9999.00 

 HAT 2 516 0 0 41.07 -112.59 42.00 7.00 62.00 23.80 23.00-9999.00 

 LDS1 2 516 0 0 40.77 -111.89 359.00 3.50 63.00 17.50 17.00-9999.00 

 LMR 2 516 0 0 41.06 -112.89 104.00 0.00 60.00 22.10 23.00-9999.00 

 LMS 2 516 0 0 41.70 -112.85 148.00 2.60 67.00 20.80 17.00-9999.00 

 MBY 2 516 0 0 40.61 -111.48 282.00 9.60 48.00 16.50 28.00-9999.00 

 PCB 2 516 0 0 40.65 -111.51 347.00 1.70 60.00 24.10 25.00-9999.00 

 PCS 2 516 0 0 40.65 -111.52 266.00 1.70 55.00 20.80 26.00-9999.00 

 PRP 2 516 0 0 41.26 -112.44 341.00 6.10 53.00 10.60 18.00-9999.00 

 SB1 2 516 0 0 41.19 -111.84 312.00 3.50 55.00 13.50 19.00-9999.00 

 SNH 2 516 0 0 40.55 -111.85 340.00 4.30 65.00 26.30 23.00-9999.00 

 SNI 2 516 0 0 41.20 -111.86 20.00 0.90 50.00 15.60 25.00-9999.00 

 SNV 2 516 0 0 40.62 -111.51 269.00 0.90 52.00 20.70 29.00-9999.00 

 SNX 2 516 0 0 41.04 -112.23 48.00 4.30 62.00 27.70 27.00-9999.00 

 SNZ 2 516 0 0 40.88 -111.87 339.00 0.00 66.00 25.00 21.00-9999.00 

 TPC 2 516 0 0 40.43 -111.71 192.00 4.30 52.00 23.10 32.00-9999.00 

 WBU 2 516 0 0 40.71 -111.56 271.00 4.30 61.00 25.00 25.00-9999.00 

 WCR 2 516 0 0 40.55 -111.32 261.00 7.80 57.00 16.30 20.00-9999.00 

 KMS 2 516 0 0 40.56 -111.13 253.00 3.50 57.00 18.50 22.00-9999.00 

 SKY 2 516 0 0 39.58 -111.25 249.00 8.70 52.00 16.30 24.00-9999.00 

 UT1 2 516 0 0 41.20 -111.11 225.00 10.50 56.10 16.20 20.00-9999.00 

 UT12 2 516 0 0 40.64 -111.90 0.00 5.80 66.00 21.90 18.00-9999.00 

 UT20 2 516 0 0 40.75 -111.90 337.50 6.80 66.70 20.80 17.00-9999.00 

 UT23 2 516 0 0 40.72 -111.90 0.00 7.20 67.60 20.70 16.00-9999.00 

 UT5 2 516 0 0 40.71 -111.80 270.00 6.00 64.90 17.20 15.00-9999.00 

 UT7 2 516 0 0 40.48 -111.90 0.00 12.60 65.10 19.90 17.00-9999.00 

 UT9 2 516 0 0 40.68 -112.26 22.50 15.60 63.10 23.40 22.00-9999.00 

 SND 2 516 0 0 40.37 -111.59 167.00 7.00 55.00 19.80 25.00-9999.00 

 QSA 2 516 0 0 40.83 -112.01 5.00 7.00 63.00 22.50 21.00-9999.00 

 KEM 2 516 0 0 41.76 -110.58 270.00 16.50 54.00 25.60 33.00-9999.00 

 LGP 2 516 0 0 41.71 -111.71 288.00 10.40 36.00 14.60 41.00-9999.00 

 CASD 2 516 0 0 39.22 -111.02 257.40 8.80 74.10 28.20 18.20-9999.00 

 ELMO 2 516 0 0 39.40 -110.81 286.60 5.90 77.00 20.60 12.10-9999.00 

 FERR 2 516 0 0 39.09 -111.09 359.30 11.60 68.80 17.70 14.10-9999.00 

 GUNN 2 516 0 0 39.16 -111.81 333.20 11.20 74.60 27.00 17.00-9999.00 

 LITW 2 516 0 0 39.32 -111.33 339.80 7.20 67.50 33.90 28.80-9999.00 

 MOLE 2 516 0 0 39.06 -111.14 202.50 10.50 70.90 18.20 13.30-9999.00 

 CUPC 2 516 0 0 40.36 -111.90 333.40 6.30 67.20 27.40 22.30-9999.00 

 CUPE 2 516 0 0 40.49 -111.47 255.40 6.30 65.80 25.90 22.00-9999.00 

 CUPF 2 516 0 0 40.22 -111.11 273.60 11.70 54.60 31.00 40.40-9999.00 

 CUPG 2 516 0 0 40.37 -111.03 355.30 7.30 54.40 30.30 39.50-9999.00 

 CUPI 2 516 0 0 40.56 -110.70 354.40 0.00 55.80 14.60 19.40-9999.00 

MSI01 2 516 0 0 40.72 -111.86 325.00 5.60 69.70 25.60 19.00-9999.00 

 BBN 2 516 0 0 40.89 -111.85 330.00 2.60 63.00 17.50 17.00-9999.00 

ARAU1 2 516 0 0 40.59 -113.02 40.00 6.10 64.00 13.00 14.00-9999.00 

BRAU1 2 516 0 0 40.88 -110.83 90.00 4.30 54.00 19.00 26.00-9999.00 

NOGAPS data, which is available for every 1° grid point for the entire world, is used for 

initialization and time-dependent lateral boundary conditions for both the MM5 and the 

BFM. The forecast data for the periods of 0, 12, and 24-h, initialized at 00:00 UTC, were 

obtained at 13 pressure levels from 1000 mb to 100 mb. The data was obtained through 

the Defense Modeling and Simulation Office, Master Environmental Library Web site. 
11 

Upper-air sounding data was obtained from the University of Wyoming Web site. 

For this study, these three different data sets for the winter and spring of 2002 were 

collected, and model calculations for 24-h forecasting were performed for both seasons 

(Table 2). 

 Table 2. Dates of archived data sets and model calculations for 24-h forecasting. 

Winter, 2002 Spring, 2002 

January 15, 22, 23, 24, 25, 28, 29, 30 April 17, 22, 23, 24, 30

February 4, 5, 6, 7, 8, 11, 12, 13, 14, 19, 

 20, 21, 22, 26, 27,28 

May 1, 2, 6, 8, 13, 14, 16, 20, 21, 30 31

March 4, 5, 6, 7, 8, 11, 12, 14 June 3, 4, 5, 10, 11, 17, 18, 19, 20 

4. Forecast and Evaluation Methods 

4.1 Forecast Method 

4.1.1 MM5 

The standard method of MM5 initialization (REGRID) is used, which is discussed in the 

MM5 user’s manual (MM5 Home Page, 2001). GRIB-formatted NOGAPS data on 

pressure levels is read and interpolated to the MM5 grid. For the MM5 calculations, only 

NOGAPS data is used. The program, INTERP, conducts vertical interpolation, diagnostic 

computation and data reformatting to generate model initial and lateral boundary 

conditions, and a lower boundary condition. The data sets are generated separately for 

four different nested grids. 

4.1.2 BFM 

From the NOGAPS data, horizontal wind vector components, temperature, dew point 

temperature and geopotential height at 13 different pressure levels for the forecast periods 

of 0, 12, and 24-h are interpolated to the 81x81 grid with grid increments of 2.5 or 5 km 

at each pressure level for the above forecast periods. The data is then vertically 

interpolated from the pressure levels to the BFM’s height levels to produce 4-d fields of 

the input data for initialization and lateral boundary conditions. In this study, all model 

calculations were initiated at 00:00 UTC. The 0-h NOGAPS data is composed with upper 

air sounding data obtained at Salt Lake City. The details of the initialization method are 

described in Henmi and Dumais (1998). 
12 

4.2 Evaluation Method 

4.2.1 Data Comparison Method 

After the forecast calculation is completed, the following bilinear interpolation is 

conducted to obtain model forecast data at the surface observation locations. 

Suppose a surface observation location (x’ and y’) is surrounded by four model grid 

points. An interpolated value, φ’, of an arbitrary variable, φ at (x’, y’), is calculated using 

a bilinear interpolation method, such as: 

 φ1=φ(i, j)+(x’ –x)[φ(i+1, j) - φ(i, j)] 

 φ2= φ(i, j+1) +(x’ –x)[φ(i+1, j+1) - φ(i, j+1)] 

 φ’(x’,y’)=φ1 + (y’–y)[φ2 - φ1] (1) 

where 

(i, j) is the southwest grid point of the four grid points surrounding a surface 

observation location (x’, y’), and (x, y) is the location for the grid point (i, j): 

φ (i, j) is an arbitrary variable at (x, y). 

4.2.2 Statistical Parameters 

The following statistical parameters between model forecast data and surface observation 

data are calculated using the data available in the model domains. The parameters are 

calculated hourly for the entire period. Statistical parameters are calculated for 

temperature, relative humidity, wind speed and horizontal wind vector components u and 

v. 

Mean Difference (MD): 

(2) 

where 

the subscript o represents observation and p represents prediction. The 

subscript i represents the i

th surface station, and the subscript j represents 

the j

th forecast day. n is the number of surface stations, and m is the total 

number of forecast days. 

A nonzero MD indicates bias. For instance, if the MD value is positive, it 

indicates that the model tends to over-forecast. 

mn

MD

m

j

n

i

∑∑ x p i j xo i j = =

−

= 1 1 , , , , ( )
13 

Mean Absolute Difference (AD): 

(3) 

Root Mean Square Error (RMSE): 

(4) 

Better agreements between observation and forecast are, in general, related to 

smaller values of AD and RMSE. 

Root Mean Square Vector Error (RMSVE): 

(5) 

This parameter measures the differences of both wind speed and direction. Good 

agreements of wind vectors are related to small values of the RMSVE. 

Correlation Coefficient (CC): 

(6) 

 where 

 

xo and xp are the means of observed and forecasted values, respectively. 

mn

AD

m

j

n

i

∑∑ x p i j xo i j = =

−

= 1 1 , , , ,

mn

RMSE

xo i j xo i j

m

j

n

i

2

, , , , 1 1

( − )

=

∑∑= =

mn

RMSVE

m

j

n

i

∑∑ uo i j up i j vo i j v p i j = =

− + −

= 1 1

2

, , , ,

2

, , , , [( ) ( ) ]

∑∑ ∑∑

∑∑

== ==

= = = m

j

n

i

m

j

n

i o i j p i j

m

j

n

i o i j p i j

y y

y y

CC

11 11

2

, ,

2

, ,

1 1 , , , ,

y xo i j xo o i j

= − , , , ,

p i j p p i j y = x − x , , , ,
14 

5. Results 

The following section presents statistical parameters that were calculated between model 

forecast data and surface observation data. For this study, all of the MM5 forecast 

calculations are done with a 6-h spin-up period. So for the MM5, forecast data is only 

available between 06:00 and 24:00 UTC. For the BFM, 24-h forecast data between 00:00 

and 24:00 UTC is available. 

5.1 Statistical Parameters of Daily Data 

Table 3 shows the statistical parameters (CC, MD, and RMSE) of surface temperature 

calculated for the daily data from the period of January-March 2002. Table 4 shows the 

statistical parameters (CC, MD and the mean wind direction difference between model 

and observation data (MWDDF)) of wind vector calculated for the daily data from the 

period of January-March 2002. The results that are displayed in Tables 3 and 4 are for 

MM5 domain 4 and the BFM of the grid increment of 2.5 km. 

 Table 3. Statistical parameters calculated for surface temperature. 

 MM5 BFM 

Date CC MD RMSE CC MD RMSE 

1-15 .06 .7 3.0 .30 2.6 4.4 

1-22 .82 .0 2.7 .72 2.6 4.8 

1-23 .73 -.7 3.5 .78 2.0 3.6 

1-24 .59 -1.6 4.2 .73 1.1 3.4 

1-25 -.25 1.9 3.9 .47 1.2 3.2 

1-28 .68 1.4 3.1 .78 2.6 3.8 

1-29 .63 -.5 3.2 .80 3.2 4.0 

1-30 .35 -.7 3.9 .63 2.5 4.3 

2-04 .40 .8 4.2 .48 1.8 4.1 

2-05 .55 .7 4.5 .56 1.6 4.0 

2-06 .53 2.1 4.8 .59 2.3 4.4 

2-07 .44 4.0 6.3 .62 .8 3.8 

2-08 .63 -1.2 4.1 -.02 -5.2 8.4 

2-11 .52 1.1 4.6 .64 .56 3.6 

2-12 .58 -.2 3.3 .51 4.2 5.8 

2-13 .56 -.3 4.7 .57 6.4 7.7 

2-14 .61 .0 2.2 .54 1.4 2.8 

2-17 .78 -1.8 3.5 .64 .23 3.4 

2-20 .73 -1.0 2.9 .54 -1.3 3.7 

2-22 .54 2.0 3.8 .51 -.6 3.6 

2-26 .78 -.9 3.9 .65 7.5 8.6 

2-27 .81 -1.9 2.6 .69 2.1 3.6 

2-28 .70 -.6 3.1 .59 1.6 3.9 

3-04 .88 .36 3.0 .69 4.3 5.9 

3-05 .81 .6 3.4 .77 -1.0 3.8 

3-07 .86 -.7 2.7 .23 -3.5 6.6 

3-11 .88 .4 2.6 .77 -.5 3.4 
15 

 Table 3. Statistical parameters calculated for surface temperature (continued). 

3-12 .77 -.6 2.6 .40 -2.3 4.9 

3-14 .87 -2.1 3.3 .69 3.3 4.6 

 Table 4. Statistical parameters calculated for wind vector.

 MM5 BFM 

Date CC RMSVE MWDDF CC RMSVE MWDDF 

1-15 .20 3.6 56.9 .15 4.3 64.1 

1-22 .11 5.6 43.1 -.18 7.3 55.9 

1-23 .21 2.6 65.6 .00 2.9 62.9 

1-24 .17 3.3 64.8 .00 3.2 73.6 

1-25 .39 4.4 55.0 .13 3.4 55.0 

1-28 .20 6.3 52.2 .11 5.8 50.7 

1-29 .29 2.8 46.3 -.01 2.6 42.6 

1-30 .00 4.2 62.8 .03 2.9 52.8 

2-04 .40 2.3 34.5 .07 2.3 41.8 

2-05 .06 2.9 58.6 -.04 2.8 45.6 

2-06 .26 2.3 70.5 .00 2.6 70.0 

2-07 .24 3.9 57.2 .03 2.8 62.3 

2-08 .23 7.0 45.2 .07 5.7 69.8 

2-11 .31 3.1 54.3 .09 2.7 55.0 

2-12 .48 2.8 35.9 .10 3.5 37.6 

2-13 .10 3.7 60.7 .03 4.2 79.4 

2-14 .51 3.4 42.2 .18 4.0 61.2 

2-19 .29 3.0 49.3 .02 4.0 75.0 

2-20 .31 4.7 60.8 -.03 4.9 77.5 

2-22 .50 3.4 48.8 -.03 3.5 70.0 

2-26 .23 3.6 57.3 -.08 4.5 67.4 

2-27 .21 4.0 31.1 .08 4.7 53.1 

2-28 .41 4.5 50.8 .17 4.6 70.3 

3-04 .51 2.6 66.4 .02 3.4 66.2 

3-05 .26 4.0 45.2 .10 3.2 61.0 

3-07 .19 5.7 43.2 .03 4.0 65.7 

3-11 .45 3.29 46.7 -.05 3.9 59.8 

3-12 .50 4.0 32.1 .01 9.8 73.2 

3-14 .12 4.0 28.6 -.13 4.7 45.8 

The values of daily statistical parameters for both temperature and wind vary 

considerably between days. For surface wind forecasts, the CC values for MM5 are 

greater than those for the BFM, and the MWDDF values for MM5 are smaller than those 

for the BFM throughout the period (Table 4). These results indicate that MM5 performed 

better than the BFM in wind forecasts over the complex terrain of Utah. However, for 

surface temperature forecasts, statistical parameters for both models are compatible 

(Table 3). As shown in the appendix, similar results are obtained between MM5 domain 3 

and the BFM with 5 km grid increments. 

5.2 MM5 Forecast and Observation Data Comparison 

Figures 6–15 are scatter diagrams that illustrate the difference between the MM5 forecast 

data and the observation data for the period of January-March 2002. For each set of 
16 

figures that follow, MM5 domain 3 with grid increments of 7.5 km is the figure displayed 

at the top of the page, and MM5 domain 4 with grid increments of 2.5 km is the figure 

displayed at the bottom. 

Diagrams for the period of April-June 2002 can be found in Appendix B. Figures 

representing the BFM for the periods of January-March and April-June 2002 are included 

in Appendixes C and D. 

Figure 6. Difference between MM5 forecast and 

observation: temperature (domain 3). 

Figure 7. Difference between MM5 forecast and 

observation: temperature (domain 4). 
17 

Figure 8. Difference between MM5 forecast and 

observation: dew point temperature (domain 3). 

Figure 9. Difference between MM5 forecast and 

observation: temperature (domain 4). 
18 

Figure 10. Difference between MM5 forecast and 

observation: wind vector-x component u 

(domain 3). 

Figure 11. Difference between MM5 forecast and 

observation: wind vector-x, component u 

(domain 4). 
19 

Figure 12. Difference between MM5 forecast and 

observation: wind vector-y, component v 

(domain 3). 

Figure 13. Difference between MM5 forecast and 

observation: wind vector-y, component v 

(domain 4). 
20 

Figure 14. Difference between MM5 forecast and 

observation: wind speed (domain 3). 

Figure 15. Difference between MM5 forecast and 

observation: wind speed (domain 4). 

Figures 6–15 were obtained by plotting all of the data accumulated including 18 forecast 

hours during a period of 30 days in January-March 2002. For instance, the figure 

representing the temperature of MM5 domain 4 (figure 7) contains more than 27,000 

points. Similar figures for the BFM can be found in Appendixes C and D. 
21 

5.3 Statistical Parameters of Entire Forecasting Periods 

Statistical parameters are calculated for the data inclusive of all forecast hours and days 

during the two seasons. Tables 5, 6, and 7 are calculated for the period of January-March 

2002, and Tables 8, 9, and 10 are calculated for the period of April-June 2002. Tables 5 

and 8 contain CC, MD, AD, and RMSE of temperature for four different model domains. 

Tables 6 and 9 contain CC, MD, AD, and RMSE of dew point temperature. Tables 7 and 

10 contain the statistical parameters for wind: CC, MD, and AD of wind speed, and 

RMSVE and MWDDF. 

 Table 5. Statistical parameters of temperature for different model domains (January-March 2002). 

Parameters MM5 D3 

7.5 km grid 

MM5 D4 

2.5 km grid 

BFM 

5 km grid 

BFM 

2.5 km grid 

CC .79 .81 .71 .69 

MD .30 -.02 1.5 1.3 

AD 3.3 2.9 3.7 3.6 

RMSE 4.3 3.9 4.9 4.8 

# of Data 43,192 27,042 56,300 35,807 

 Table 6. Statistical parameters of dew pt temperature for different model domains (January-March 2002)

Parameters MM5 D3 

7.5 km grid 

MM5 D4 

2.5 km grid 

BFM 

5 km grid 

BFM 

2.5 km grid 

CC .66 .64 .61 .63 

MD -1.6 -1.9 -2.1 -2.8 

AD 4.5 4.6 4.6 4.8 

RMSE 5.7 5.9 5.6 5.8 

# of Data 42,864 26,830 57,181 36,388 

 Table 7. Statistical parameters of wind vector for different model domains (January-March 2002). 

Parameters MM5 D3 

7.5 km grid 

MM5 

2.5 km grid 

BFM 

5 km grid 

BFM 

2.5 km grid 

CC of speed .37 .42 .09 .09 

MD of speed .7 .5 -.4 .0 

AD of speed 1.9 1.9 1.9 2.1 

RMSVE 4.1 4.1 4.7 5.0 

MWDDF 51.2 49.4 62.7 61.2 

# of Data 37,177 22,594 47,706 29,420 

 Table 8. Statistical parameters of temperature for different model domains (April-June 2002). 

Parameters MM5 D3 

7.5 km grid 

MM5 

2.5 km grid 

BFM 

5 km grid 

BFM 

2.5 km grid 

CC .86 .87 .70 .70 

MD -.62 -.64 -.5 -.4 

AD 3.3 3.2 4.5 4.5 

RMSE 4.2 4.0 6.1 6.1 

# of Data 19,338 13,520 28,007 19,636 
22 

 Table 9. Statistical parameters of dew point temperature for different model domains (April-June 

2002). 

Parameters MM5 D3 

7.5 km grid 

MM5 

2.5 km grid 

BFM 

5 km grid 

BFM 

2.5 km grid 

CC .56 .53 .33 .32 

MD 4.0 3.8 -2.3 -2.5 

AD 5.5 5.4 4.9 5.0 

RMSE 7.0 6.9 6.5 6.6 

# of Data 19,429 13,573 28,211 19,757 

 Table 10. Statistical parameters of wind vector for different model domains (April-June 2002). 

Parameters MM5 D3 

7.5 km grid 

MM5 

2.5 km grid 

BFM 

5 km grid 

BFM 

2.5 km grid 

CC of speed .28 .25 .07 .02 

MD of speed .37 .50- -.2 .1 

AD of speed 2.0 2.2 2.3 2.5 

RMSVE 4.3 4.4 4.7 5.0 

MWDDF 57.1 54.4 59.5 61.1 

# of Data 17,077 11,737 24,354 16,934 

From the data displayed in Tables 5–10, the following can be inferred: 

1. Both MM5 and the BFM yielded better forecasting statistics in the period of AprilJune 2002 than in the period of January-March 2002. This may be caused by the 

following: over the middle latitude area, the forcing by synoptic weather patterns is 

more dominant than the forcing by local surface heating/cooling in cold months than 

in warm months. Synoptic weather forcing is inputted into model calculation as initial 

and lateral condition values from the NOGAPS forecast data, and local forcing effects 

are calculated by surface physics routines of the models. 

2. There are no significant statistical differences between MM5 domain 3 (grid 

increments of 7.5 km) and domain 4 (grid increments of 2.5 km). Similarly, the 

statistical parameters for the BFM of grid increments 5 km and 2.5 km are similar. 

Reducing the grid increment does not produce a significant improvement of forecast 

skills for either MM5 or the BFM. 

3. For all the meteorological parameters of temperature, dew point, and wind vector, the 

statistical parameters for MM5 are better than those for the BFM. Note that the 

amount of data is different for MM5 and the BFM. 

4. Both MM5 and the BFM show better forecast skills for temperature than they show 

for dew point temperature and wind vector. 

5. Over the complex terrain of Utah, both MM5 and BFM show low forecast skills for 

wind vector. 

5.4 Time Series of the Statistical Parameters 

Every statistical parameter is calculated at each forecast hour using all available data for 

the period. In this section, figures 16–23 display the time series for MM5 domain 4, and 

the BFM of 2.5 km grid increment for the period of April-June 2002. For MM5, the data 
23 

for the time period of 06–23 h are plotted. For the BFM, the data is plotted from 00–24 h. 

Similar figures are shown in Appendixes A, B, C, and D for other model domains and 

periods. 

Figure 16. Time series of statistical parameters for temperature (April-June 2002, MM5 domain 4). 

Figure 17. Time series of statistical parameters of temperature (April-June 2002, BFM 2.5 km grid 

 increment). 
24 

Figure 18. Time series of statistical parameters for dew point temperature (April-June 2002, MM5 

domain 4). 

Figure 19. Time series of statistical parameters for temperature (April-June 2002, BFM 2.5 km grid 

increment). 
25 

Figure 20. Time series of statistical parameters for wind speed (April-June 2002, MM5 domain 5). 

Figure 21. Time series of statistical parameters for wind speed (April-June 2002, BFM 2.5 grid 

increment). 
26 

Figure 22. Time series for RMSVE, MWDDF, and RMSE of wind direction difference (April-June 

2002, MM5 domain 4). 

Figure 23. Time series for RMSVE, MWDDF and RMSE of wind direction difference (April-June 

2002, BFM 2.5 km grid increment). 
27 

From the information displayed in figures 11–18, the following can be seen: 

1. For temperature, both the MM5 and the BFM forecast data produced statistically 

good agreement with observed data, with the AD values around 2 °C and the CC 

greater than 0.9. 

2. Dew point temperatures forecasted by MM5 tend to be greater than the dew point 

temperatures of the observed data. Dew point temperatures forecasted by the 

BFM tend to be less than the dew point temperatures of the observed data. For 

MM5, the modeled atmosphere near the surface tends to be more humid than the 

observed atmosphere. For BFM, the modeled atmosphere near the surface tends to 

be less humid than the observed atmosphere. The forecast skills of both MM5 and 

the BFM, with regards to dew point temperature, are statistically less than those 

for temperature. 

3. The AD values of wind speed for both MM5 and the BFM are between 2 and 3 

m/sec, and the RMSVE values are between 4 and 5 m/sec. In the previous study 

(Henmi, 2000), the AD values of three different climatological areas (Colorado, 

Washington, and Florida) over a 500x500 km2

 area with grid increment of 10 km 

were smaller than 1 m/sec, and the RMSVE values were between 2 and 3 m/sec. 

Lower statistical skills of both MM5 and the BFM for wind vector forecasts over 

the Utah domains in this study may be attributed to the fact that the observed data 

used for comparison in Henmi (2000) was obtained at sites such as airports, where 

the topography surrounding the sites is flat. Thus, the BFM might have simulated 

wind vector fields over the previous domains better than over the Utah domains. 

4. The CC values of MM5 (approximately 0.3) and of the BFM (less than 0.2), and 

the mean wind direction differences for both models (greater than 60o) are 

statistically poor. 

Similar findings can be found in the statistics of MM5 domain 3 and the BFM with grid 

increments of 5 km (see appendixes). 

6. Conclusions 

Forecast calculations by the MM5 and the BFM mesoscale models were made over the 

model domains of Utah for two different periods (January-March, April-June 2002). 

The mesoscale model, MM5, has quadruple-nested computational grids. All four domains 

have a mesh size of 55x55 points, and grid increments of 67.5, 22.5, 7.5, and 2.5 km, 

respectively, for the domains 1,2, 3, and 4. Forecast computations were carried out in 

parallel modes using 16 processors. The U. S. Army HPC computer was used for this 

study. For this evaluation study, the forecast data for domains 3 and 4 were compared 

with observed data. 

BFM calculations were made independently over two model domains with grid 

increments of 5 and 2.5 km, both centered at Salt Lake City, UT. Surface meteorological 
28 

data forecasted by the models were statistically compared with Mesowest network data. 

The major findings of this study included: 

1. There are no significant statistical differences in forecast skills between MM5 

domain 3 (7.5 km grid increment) and domain 4 (2.5 km grid increment). 

Similarly, the forecast skills of two BFM (5 and 2.5 km) grid increments are 

statistically similar. It can be concluded that reducing the grid increment from 7.5 

to 2.5 km for MM5, and from 5.0 to 2.5 km from BFM does not produce 

significant improvement of forecast results. 

2. Both MM5 and the BFM produced more accurate forecast results for surface 

temperature than for dew point temperature and wind vector. The AD values of 

temperature (approximately 2 o

C ) produced by both models are compatible with 

the values reported by Mass et al. (2002). Over the complex terrain of Utah, both 

MM5 and the BFM have difficulties in producing good forecast results for wind 

vector. For both models, the MWDDF between forecast and observation was 

greater than 60o

, and absolute wind speed differences were around 2 m/sec for the 

entire forecast hour. In a similar study over western Washington, Mass et al. 

(2002) reported that the operational MM5, triply-nested with 36, 12, and 4 km 

grid increments, produced values of approximately 40o for MWDDF, and 

approximately 4 knots (~ 2 m/sec) for absolute difference. The terrain 

complexities of the model domains over Utah may be the reason for these 

differences. 

3. For all of the surface meteorological parameters of temperature, dew point 

temperature and wind vector, MM5 produced statistically better forecast skills 

than the BFM. 

4. Both MM5 and the BFM showed better forecasting statistics during the period of 

April-June 2002 than during the period of January-March 2002. During the 

warmer period of April-June 2002, local heating and cooling effects might have 

been more significant, influential factors for surface meteorological parameters 

than during the colder period of January-March 2002. A similar study for the 

summer season is planned. 
29 

References 

Dudhia, J. A Nonhydrostatic Version of the Penn State-NCAR Mesoscale Model:

 Validation Tests and the Simulation of an Atlantic Cyclone and Cold Front. 

Monthly Weather Review 1993, 121, 1493-1513. 

Dudhia, J. Numerical Study of Convection Observed During the Winter Monsoon 

Experiment using a Mesoscale Two-Dimensional Model. Journal of Atmospheric 

Science 1989, 46, 3077-3107. 

Grell, G. A.; Dudhia, J.; Stauffer, D. R. A Description of the Fifth Generation Penn 

State/NCAR Mesoscale Model (MM5); NCAR/TN 398+STR; National Center for 

Atmospheric Research, 1994. 

Henmi, T. Comparison and Evaluation of Operational Mesoscale Models MM5 and BFM 

over WSMR; ARL-TR-1476; U.S. Army Research Laboratory: WSMR, NM, 

2000. 

Henmi, T. Evaluation Study of an Operational Mesoscale Forecast Model over Three 

Climatologically Different Areas; ARL-TR-1034; U.S. Army Research 

Laboratory: WSMR, NM, 2000. 

Henmi, T.; Dumais, R. Jr. The Battlescale Forecast Model (BFM); ARL-TR-1032; U.S. 

Army Research Laboratory: WSMR, NM, 1998. 

Hong, S. Y.; Pan, H.L. Nonlocal Boundary Layer Vertical Diffusion in a Medium-Range 

Forecast Model. Monthly Weather Review 1996, 124, 2322-2339. 

Horel, J.; Splitt, M.; Dunn, L.; Pechmann, J.; White, B.; Ciliberti, C.; Lazarus, S.;

 Slemmer, J.; Zaff, D.; Burks, J. MOSOWEST: Cooperative Mesonets in the 

Western United States. Bulletin of the American Meteorological Society 2002, 83, 

211- 225. 

Mass, C. F.; Ovens, D.; Westrick, K.; Colle, B. A. Does Increasing Horizontal Resolution 

Produce More Skillful Forecasts? Bulletin of the American Meteorological 

Society 2002, 82, 407 – 430. 

Master Environmental Library. http://mel.dmso.mil (accessed April-June, 2002), Defense 

Modeling and Simulation Office. 

Mesowest. fttp//:met.utah.edu (accessed April-June, 2002), University of Utah. 

MM5 Home Page. http://www.mmm.ucar.edu/mm5 (accessed 2001), Mesoscale and 

Microscale Meteorology, National Center for Atmospheric Research. 
30 

Upper Air Data. http://weather.uwyo.edu/upperair (accessed April-June, 2002), 

University of Wyoming, College of Engineering, Department of Atmospheric 

Science. 

Warner,T. T.; Kuo, Y. H.; Doyle, J. D.; Dudhia, J.; Stauffer, D. R.; Seaman, N. L. 

Nonhydrostatic, Mesobeta-Scale Real-Data Simulations with the Penn State 

University/National Center Atmospheric Research Mesocale Model. Meteorology 

and Atmospheric Physics 1992, 49, 209-227. 

Yamada,T.; Bunker, S. Anumerical Model Study of Nocturnal Drainage Flows with 

Strong Wind and Temperature Gradients. Journal of Applied Meteorology 1989,

28, 545-554. 
31 

Appendix A Statistical Diagrams Displaying the Difference Between 

MM5 Forecast Data and Observation Data for the Period of 

January-March 2002. 

This appendix is part of ARL-TR-2928, Evaluation Study of Mesoscale Models MM5 

and BFM Over the Model Domains of Utah Using Surface Meteorological Data by 

Mesowest. 

Figure A1. Difference between MM5 forecast data and 

observation data for temperature (domain 3). 
32 

Figure A2. Difference between MM5 forecast data and 

observation data for temperature (domain 4). 

Figure A3. Difference between MM5 forecast data and 

observation data for dew point temperature (domain 3). 
33 

Figure A4. Difference between MM5 forecast data and 

observation data for dew point temperature (domain 4). 

Figure A5. Difference between MM5 forecast data and observation 

data for wind vector x, component u (domain 3). 
34 

Figure A6. Difference between MM5 forecast data and observation 

data for wind vector x, component u (domain 4). 

Figure A7. Difference between MM5 forecast data and observation 

data for wind vector y, component v (domain 3). 
35 

Figure A8. Difference between MM5 forecast data and observation 

data for wind vector y, component v (domain 3). 

Figure A9. Difference between MM5 forecast data and observation 

data for wind speed (domain 3). 
36 

Figure A10. Difference between MM5 forecast data and observation 

data for wind speed (domain 4). 

Figure A11. Time series of statistical parameters between MM5 temperature 

forecast data and surface observation data (domain 3). 
37 

Figure A12. Time series of statistical parameters between MM5 temperature 

forecast data and surface observation data (domain 4). 

Figure A13. Time series of statistical parameters between MM5 dew point temperature forecast 

data and surface observation data (domain 3). 
38 

Figure A14. Time series of statistical parameters between MM5 dew point temperature 

forecast data and surface observation data (domain 4). 

Figure A15. Time series of statistical parameters between MM5 wind vector x, component u 

forecast data and surface observation data (domain 3). 
39 

Figure A16. Time series of statistical parameters between MM5 wind vector x, component u 

forecast data and surface observation data (domain 4). 

Figure A17. Time series of statistical parameters between MM5 wind vector y, component v forecast 

data and surface observation data (domain 3). 
40 

Figure A18. Time series of statistical parameters between MM5 wind vector y, component v 

forecast data and surface observation data (domain 4). 

Figure A19. Time series of statistical parameters between MM5 wind speed forecast data and surface 

observation data (domain 3). 
41 

Figure A20. Time series of statistical parameters between MM5 wind speed forecast data and surface 

observation data (domain 4). 

Figure A21. Time series of statistical parameters between MM5 rmsve and mean wind direction 

forecast data and surface observation data (domain 3). 
42 

Figure A22. Time series of statistical parameters between MM5 rmsve and mean wind direction 

forecast data and surface observation data (domain 4). 
43 

Appendix B Statistical Diagrams Displaying the Difference Between 

MM5 Forecast Data and Observation Data for the Period of AprilJune 2002. 

This appendix is part of ARL-TR-2928, Evaluation Study of Mesoscale Models MM5 

and BFM Over the Model Domains of Utah Using Surface Meteorological Data by 

Mesowest. 

Figure B1. Difference between MM5 forecast data and 

observation data for temperature (domain 3). 
44 

Figure B2. Difference between MM5 forecast data and 

observation data for temperature (domain 4). 

Figure B3. Difference between MM5 forecast data and 

observation data for temperature (domain 3). 
45 

 Figure B4. Difference between MM5 forecast data and 

 observation data for temperature (domain 4). 

Figure B5. Difference between MM5 forecast data and 

observation data for wind vector x, component u 

(domain 3). 
46 

Figure B6. Difference between MM5 forecast data and 

observation data for wind vector x, component u 

(domain 4). 

Figure B7. Difference between MM5 forecast data and 

observation data for wind vector y, component v 

(domain 3). 
47 

Figure B8. Difference between MM5 forecast data and 

observation data for wind vector y, component v 

(domain 4). 

Figure B9. Difference between MM5 forecast data and 

observation data for wind speed (domain 3). 
48 

Figure B10. Difference between MM5 forecast data and 

observation data for wind speed (domain 4). 

Figure B11. Time series of statistical parameters between MM5 temperature forecast data 

and surface observation data (domain 3). 
49 

Figure B12. Time series of statistical parameters between MM5 temperature forecast data 

and surface observation data (domain 4). 

 Figure B13. Time series of statistical parameters between MM5 dew point temperature 

 forecast data and surface observation data (domain 3). 
50 

Figure B14. Time series of statistical parameters between MM5 dew point temperature 

forecast data and surface observation data (domain 4). 

Figure B15. Time series of statistical parameters between MM5 wind vector x, 

 component u forecast data and surface observation data (domain 3). 
51 

 Figure B16. Time series of statistical parameters between MM5 wind vector x, 

 component u forecast data and surface observation data (domain 4). 

Figure B17. Time series of statistical parameters between MM5 wind vector y, 

 component v forecast data and surface observation data (domain 3). 
52 

 Figure B18. Time series of statistical parameters between MM5 wind vector y, 

 component v forecast data and surface observation data (domain 4). 

 Figure B19. Time series of statistical parameters between MM5 wind speed forecast data 

 and surface observation data (domain 3). 
53 

Figure B20. Time series of statistical parameters between MM5 wind speed forecast data 

and surface observation data (domain 4). 

Figure B21. Time series of statistical parameters between MM5 rmsve and mean wind 

direction forecast data and surface observation data (domain 3). 
54 

Figure B22. Time series of statistical parameters between MM5 rmsve and mean wind 

direction forecast data and surface observation data (domain 4). 
55 

Appendix C Statistical Diagrams Displaying the Difference Between 

BFM Forecast Data and Observation Data for the Period of 

January-March 2002. 

This appendix is part of ARL-TR-2928, Evaluation Study of Mesoscale Models MM5 

and BFM Over the Model Domains of Utah Using Surface Meteorological Data by 

Mesowest. 

Figure C1. Difference between BFM forecast data and 

observation data for temperature (5 km grid 

increment). 
56 

Figure C2. Difference between BFM forecast data and 

observation data for temperature (2.5 km grid 

increment). 

Figure C3. Difference between BFM forecast data and 

observation data for dew point temperature (5 

km grid increment). 
57 

Figure C4. Difference between BFM forecast data and 

observation data for dew point temperature 

(2.5 km grid increment). 

Figure C5. Difference between BFM forecast data and 

observation data for wind vector x component, 

u (5 km grid increment). 
58 

Figure C6. Difference between BFM forecast data and 

observation data for wind vector x component, 

u (2.5 km grid increment). 

Figure C7. Difference between BFM forecast data and 

observation data for wind vector y component, v (5 

km grid increment). 
59 

Figure C8. Difference between BFM forecast data and 

observation data for wind vector y component, v (2.5 

km grid increment). 

Figure C9. Difference between BFM forecast data and 

observation data for wind speed (5 km grid 

increment). 
60 

Figure C10. Difference between BFM forecast data and 

observation data for wind speed (2.5 km grid 

increment). 

Figure C11. Time series of statistical parameters between BFM temperature forecast data and 

surface observation data (5 km grid increment). 
61 

Figure C12. Time series of statistical parameters between BFM temperature forecast data and 

surface observation data (2.5 grid increment). 

Figure C13. Time series of statistical parameters between BFM dew point temperature 

forecast data and surface observation data (5 grid increment). 
62 

Figure C14. Time series of statistical parameters between BFM dew point temperature forecast 

data and surface observation data (2.5 grid increment). 

Figure C15. Time series of statistical parameters between BFM wind vector x component u forecast 

data and surface observation data (5 grid increment). 
63 

Figure C16. Time series of statistical parameters between BFM wind vector x component u 

forecast data and surface observation data (2.5 grid increment). 

Figure C17. Time series of statistical parameters between BFM wind vector y component v forecast 

data and surface observation data (5 grid increment). 
64 

Figure C18. Time series of statistical parameters between BFM wind vector y component v 

forecast data and surface observation data (2.5 grid increment). 

Figure C19. Time series of statistical parameters between BFM wind vector speed forecast data 

and surface observation data (5 grid increment). 
65 

Figure C20. Time series of statistical parameters between BFM wind vector speed forecast data 

and surface observation data (2.5 grid increment). 

Figure C21. Time series of statistical parameters between BFM rmsve and mean wind direction 

forecast data and surface observation data (5 grid increment). 
66 

Figure C22. Time series of statistical parameters between BFM rmsve and mean wind direction 

forecast data and surface observation data (2.5 grid increment). 
67 

Appendix D Statistical Diagrams Displaying the Difference Between 

BFM Forecast Data and Observation Data for the Period of AprilJune 2002. 

This appendix is part of ARL-TR-2928, Evaluation Study of Mesoscale Models MM5 

and BFM Over the Model Domains of Utah Using Surface Meteorological Data by 

Mesowest. 

Figure D1. Difference between BFM forecast data and 

observation data for temperature (5 km grid 

increment). 
68 

Figure D2. Difference between BFM forecast data and 

observation data for temperature (2.5 km grid 

increment). 

Figure D3. Difference between BFM forecast data and 

observation data for dew point temperature 

(5 km grid increment). 
69 

Figure D4. Difference between BFM forecast data and 

observation data for dew point temperature 

(2.5 km grid increment). 

Figure D5. Difference between BFM forecast data 

and observation data for wind vector x 

component u (5 km grid increment). 
70 

Figure D6. Difference between BFM forecast data and 

observation data for wind vector x 

component u (2.5 km grid increment). 

Figure D7. Difference between BFM forecast data and 

observation data for wind vector y component v 

(5 km grid increment). 
71 

Figure D8. Difference between BFM forecast data and 

observation data for wind vector y component 

v (2.5 km grid increment). 

Figure D9. Difference between BFM forecast data and 

observation data for wind speed (5 km grid 

increment). 
72 

Figure D10. Difference between BFM forecast data and 

observation data for wind speed (2.5 km grid 

increment). 

Figure D11. Time series of statistical parameters between BFM temperature forecast data and surface 

observation data (5 km grid increment). 
73 

Figure D12. Time series of statistical parameters between BFM temperature forecast data and surface 

observation data (2.5 km grid increment). 

Figure D13. Time series of statistical parameters between BFM dew point temperature forecast data 

and surface observation data (5 km grid increment). 
74 

Figure D14. Time series of statistical parameters between BFM dew point temperature forecast data 

and surface observation data (2.5 km grid increment). 

Figure D15. Time series of statistical parameters between BFM wind vector x component u forecast 

data and surface observation data (5 km grid increment). 
75 

Figure D16. Time series of statistical parameters between BFM wind vector x component u forecast 

data and surface observation data (2.5 km grid increment). 

Figure D17. Time series of statistical parameters between BFM wind vector y component v forecast 

data and surface observation data (5 km grid increment). 
76 

Figure D18. Time series of statistical parameters between BFM wind vector y component v forecast 

data and surface observation data (2.5 km grid increment). 

Figure D19. Time series of statistical parameters between BFM wind speed forecast data and surface 

observation data (5 km grid increment). 
77 

Figure D20. Time series of statistical parameters between BFM wind speed forecast data and surface 

observation data (2.5 km grid increment). 

Figure D21. Time series of statistical parameters between BFM rmsve and mean wind direction 

forecast data and surface observation data (5 km grid increment). 
78 

Figure D22. Time series of statistical parameters between BFM rmsve and mean wind direction 

forecast data and surface observation data (2.5 km grid increment). 
79 

Acronyms 

AD absolute difference 

ARL U.S. Army Research Laboratory

BFM Battlescale Forecast Model 

CC correlation coefficient 

HOTMAC Higher Order Turbulence Model of Atmospheric Circulation 

HPC High Power Computer 

IMETS Integrated Meteorological System 

MD mean difference 

MM5 Mesoscale Model version 5 

MPI Message Passage Interface 

MRF Medium Range Forecast 

MWDDF mean wind direction difference 

NCAR National Center for Atmospheric Research 

NCEP National Center for Environmental Prediction 

NOGAPS Navy Operational Global Atmospheric Prediction System 

NORAPS Navy Operational Regional Atmospheric Prediction System 

PBL Planetary Boundary Layer 

RMSE root mean square error 

RMSVE root mean square vector error 

UTC universal time coordinate 

WSMR White Sands Missile Range 
80 

REPORT DOCUMENTATION PAGE Form Approved 

OMB No. 0704-0188 

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing 

data sources, gathering and maintaining the data needed, and completing and reviewing the collection information. Send comments regarding this burden estimate 

or any other aspect of this collection of information, including suggestions for reducing the burden, to Department of Defense, Washington Headquarters Services, 

Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware 

that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a 

currently valid OMB control number. 

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. 

1. REPORT DATE (DD-MM-YYYY)

2003 

2. REPORT TYPE 

Final 

3. DATES COVERED (From - To) 

Oct. 1 2001- Sep. 30, 2002 

5a. CONTRACT NUMBER 

5b. GRANT NUMBER 

4. TITLE AND SUBTITLE 

Evaluation Study of Mesoscale Models MM5 and BFM Over the Model 

Domains of Utah Using Surface Meteorological Data by Mesowest 

5c. PROGRAM ELEMENT NUMBER 

5d. PROJECT NUMBER 

5e. TASK NUMBER 

6. AUTHOR(S) 

Teizi Henmi 

5f. WORK UNIT NUMBER 

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 

U.S. Army Research Laboratory 

Computational Information Sciences Directorate 

Battlefield Environment Division (ATTN: AMSRL-CI-BE) 

White Sands Missile Range, NM 88002-5501

8. PERFORMING ORGANIZATION 

 REPORT NUMBER 

ARL-TR-2928 

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR'S ACRONYM(S) 

U.S. Army Research Laboratory 

2800 Powder Mill Road 

Adelphi, MD 20783-1145

11. SPONSOR/MONITOR'S REPORT 

 NUMBER(S) 

ARL-TR-2928 

12. DISTRIBUTION/AVAILABILITY STATEMENT 

Approved for public release; distribution unlimited.

13. SUPPLEMENTARY NOTES 

None 

14. ABSTRACT 

The forecast skills of the Penn State/National Center for Atmospheric Research Mesoscale Model version 5 (MM5), and the 

Battlescale Forecast Model (BFM) were statistically evaluated by comparing the model forecasting data with surface 

observation data. The study was done for the periods of January-March 2002, and April-June 2002. The MM5 used quadruplenests of 67.5, 22.5, 7.5, and 2.5 km grid increments, all of which have 51x51 grid points centered near Salt Lake City, UT. The

forecast data of domains 3 and 4 were compared with the observation data. The BFM calculations were made separately over 

two model domains of grid increments of 5 and 2.5 km, both centered at Salt lake City, UT. Findings included: No significant 

statistical differences in forecast skills between the two MM5 domains, and the two BFM grid increments are similar. Both 

models produce better results for surface temperature than for dew point temperature, and both models had difficulties in 

producing good forecast results for wind vectors. The MM5 produced statistically better results. Both models showed better 

results for April-June than for January-March. 

15. SUBJECT TERMS

Mesoscale models, forecast of surface parameters, complex terrain 

16. SECURITY CLASSIFICATION OF: 

19a. NAME OF RESPONSIBLE PERSON

Teizi Henmi 

a. REPORT 

U 

b. ABSTRACT 

U 

c. THIS PAGE 

U 

17. LIMITATION OF 

 ABSTRACT 

SAR 

18. 

NUMB

ER 

 OF 

 PAGES 

88

19b. TELEPHONE NUMBER (Include area code) 

(505) 678 - 3519 

Standard Form 298 (Rev. 8/98) 

Prescribed by ANSI Std. Z39.18