Presented at the 6th Annual CMAS Conference, Chapel Hill, NC, October 1-3, 2007 

1 

REGIONAL AND LOCAL-SCALE EVALUATION OF 2002 MM5 METEOROLOGICAL FIELDS 

FOR VARIOUS AIR QUALITY MODELING APPLICATIONS 

Pat Dolwick* 

U.S. Environmental Protection Agency, Research Triangle Park, NC, USA 

Rob Gilliam 

National Oceanic and Atmospheric Administration, Research Triangle Park, NC, USA 

 Lara Reynolds and Allan Huffman 

Computer Sciences Corporation, Research Triangle Park, NC, USA 

1. INTRODUCTION 

Prognostic meteorological models are often 

used in a retrospective mode to provide inputs to 

air quality models. In turn, these air quality 

models are used for environmental planning. The 

meteorological inputs govern the advection, 

diffusion, chemical transformation, and eventual 

deposition of pollutants within regional air quality 

models such as the Community Multi-scale Air 

Quality (CMAQ) modeling system (Byun and 

Schere, 2006). The air quality models have 

traditionally been subjected to a rigorous 

performance assessment, but in many cases the 

meteorological inputs to these models are 

accepted as is, even though this component of the 

modeling arguably contains equal uncertainty. 

Before initiating the air quality simulations, it is 

important to identify the biases and errors 

associated with the meteorological modeling 

inputs. The goal of the meteorological evaluation 

should be to move toward an understanding of 

how the bias and error of the meteorological input 

data may impact the resultant AQ modeling. 

Typically, there are two specific objectives. First, 

determine if the meteorological model output fields 

represent a reasonable approximation of the 

actual meteorology that occurred during the 

modeling period (i.e., the "operational" evaluation). 

The second goal should be to identify how the 

existing biases and errors in the meteorological 

predictions may affect the air quality modeling 

results (i.e., the "phenomenological" evaluation). 

Once these two sets of information are generated, 

it is important to highlight the parts of the analysis 

expected to most influence the air quality model. 

 

*Corresponding author: Pat Dolwick, U.S. 

Environmental Protection Agency, MD C439-03, RTP, 

NC 27711; e-mail: dolwick.pat@epa.gov 

This analysis looks at the performance of the 

Penn State University / National Center for 

Atmospheric Research mesoscale model known 

as MM5 (Grell et al, 1994) for a specific year 

(2002) at two separate model resolutions. The 

model evaluation is summarized for the entire 

domain, individual subregions within the domain, 

and certain individual sites. The goal is to provide 

a snapshot of the types of analyses that can be 

completed to assess the utility of using this data to 

drive regional-scale, photochemical models (e.g., 

CMAQ) as well as local-scale dispersion models 

(e.g., AERMOD1

). 

2. MM5 MODEL CONFIGURATION 

Meteorological model input fields were 

prepared for two separate grids as shown in 

Figure 1. A 36 km grid covering the contiguous 

portion of the U.S. was modeled using MM5 

v.3.6.0 using land-surface modifications that were 

added in v3.6.3. Additionally, a 12 km grid that 

covered the eastern two-thirds of the U.S. was 

modeled with MM5 v3.7.2. Both domains 

contained 34 vertical layers with a ~38 m surface 

layer and a 100 mb top. Both sets of model runs 

were conducted in 5.5 day segments with 12 

hours of overlap for spin-up purposes. 

Table 1 lists the physics options used in the 

two sets of MM5 simulations. The initial state 

analyses were derived from the 3-hourly, Eta Data 

Assimilation System (EDAS) data. Analysis 

nudging was utilized outside of the boundary layer 

for temperatures and water vapor mixing ratios, 

and in all locations for wind components, using 

relatively weak nudging coefficients. 

 

1 AERMOD is a Gaussian dispersion model used to 

predict air quality in the vicinity of a single source, or 

group of sources. For more information on AERMOD: 

http://www.epa.gov/ttn/scram/dispersion_prefrec.htm#aermod 
Presented at the 6th Annual CMAS Conference, Chapel Hill, NC, October 1-3, 2007 

2 

Figure 1. Plot of 36 and 12 km MM5 modeling 

domains in the 2002 modeling. 

Table 1. List of MM5 model physics options used 

in the 2002 MM5 simulations. 

Physics Option 2002 MM5 

Modeling 

Planetary boundary layer model Pleim-Chang / ACM1 

Sub-grid convection scheme Kain-Fritsch 2 

Explicit microphysics scheme Reisner 1 

Land-surface model Pleim-Xiu 

Longwave radiation scheme RRTM 

3. OPERATIONAL EVALUATION 

The operational evaluation includes statistical 

comparisons of model/observed pairs (e.g., mean 

normalized bias, mean normalized error, index of 

agreement, root mean square errors, etc.) for 

multiple meteorological parameters. For this 

portion of the evaluation, four meteorological 

parameters were investigated: temperature, 

humidity, wind speed, and wind direction. The 

operational piece of the analyses focuses on 

surface parameters. The Atmospheric Model 

Evaluation Tool (AMET) was used to conduct the 

analyses as described in by Gilliam et al (2005). 

 

3.1 Eastern U.S. results (12 km grid) 

Tables 2a and 2b show the monthly results of 

the 12 km evaluation. One of the key findings of 

the operational evaluation is that MM5 surface 

temperatures have a cold bias. This bias is 

especially notable in the winter and early spring 

(e.g., Jan - Apr). This artifact in the meteorological 

modeling is expected to have a major impact on 

the eventual air quality modeling results for those 

pollutants that are temperature-dependent (e.g., 

nitrates). 

Otherwise, the model reproduces observed 

water vapor mixing ratios with considerable 

accuracy, though there is a seasonal pattern with 

highest error in the summer, due to overall higher 

moisture content. Wind speeds are generally 

underestimated across the year, but typically the 

underestimation bias is less than 0.5 m/s. The 

average error in wind direction ranges from 24-31 

degrees. Generally speaking, the operational 

statistics from the 36 km grid closely approximated 

the 12 km results. 

Table 2a. Mean absolute error by month in the 

2002 12 km MM5 simulation. 

Month Temp. 

(K) 

Mix Ratio 

(g/kg) 

W Spd 

(m/s) 

W Dir 

(deg) 

Jan 3.0 0.6 1.3 24.1 

Feb 2.5 0.6 1.3 23.6 

Mar 2.7 0.7 1.4 24.9 

Apr 2.3 0.9 1.3 24.9 

May 1.8 1.0 1.3 25.8 

Jun 1.5 1.2 1.2 29.2 

Jul 1.5 1.4 1.1 31.4 

Aug 1.5 1.3 1.1 30.5 

Sep 1.5 1.1 1.1 28.1 

Oct 1.6 0.8 1.1 26.7 

Nov 2.0 0.7 1.2 24.3 

Dev 2.5 0.6 1.3 24.5 

Table 2b. Mean bias by month in the 2002 12 km 

MM5 simulation. 

Month Temp. 

(K) 

Mix Ratio 

(g/kg) 

W Spd 

(m/s) 

W Dir 

(deg) 

Jan -2.4 -0.2 -0.3 7.4 

Feb -1.3 -0.2 -0.4 7.8 

Mar -1.5 -0.1 -0.7 7.5 

Apr -1.5 -0.1 -0.4 6.2 

May -0.5 -0.1 -0.3 6.8 

Jun -0.5 -0.2 -0.4 6.8 

Jul -0.1 -0.1 -0.4 7.8 

Aug -0.1 -0.1 -0.4 7.1 

Sep 0.0 -0.3 -0.3 7.8 

Oct 0.4 -0.2 -0.2 8.3 

Nov -0.2 0.0 -0.2 8.4 

Dev -0.6 0.1 -0.5 8.1 

 Model performance was also considered by 

region, by underlying land use type, and by 

proximity to coastline or mountainous regions. 

The tabular results are not shown here for space 
Presented at the 6th Annual CMAS Conference, Chapel Hill, NC, October 1-3, 2007 

3 

considerations, however, there was a clear spatial 

gradient in model performance. Model 

performance was more accurate in the eastern 

portions of the domain, than the more 

topographically-influenced western portion of the 

12 km EUS grid. For instance, mean annual 

temperature errors ranged from 1.4 to 1.6 degrees 

over subregions in the eastern U.S., while the 

errors ranged from 2.0 to 2.2 degrees over the 

western portion of the domain. Additionally, errors 

in wind direction were almost 20 degrees greater 

over the Rocky Mountain portions of the 12 km 

MM5 grid. Conversely, model performance at 

locations determined to be coastal tended to be 

similar to the performance at inland locations. 

There was no significant error tendencies that 

were a function of land use type. 

3.2 Local-scale results (12 km grid) 

Beyond the domainwide results, which are 

aggregations of numerous model - observed pairs, 

there was also a limited evaluation of 

meteorological model performance at a handful of 

specified sites. This was done to assess the utility 

of using gridded MM5 data to drive dispersion 

models such as AERMOD. For the 12 km grid, an 

local scale analysis was completed for two 

locations: Birmingham and Detroit. Figures 2a-2d 

show the bias and error of the four meteorological 

parameters, for each quarter, and at each site. 

The statistical output are compared against target 

values of bias and error in the "soccer plot" mode. 

Points within the brown boxes are thought to be 

representative of exceptional model performance. 

Points that are outside the green box indicate 

areas in which model performance may be in need 

of improvement. This analysis confirms the 

expected, that is, local performance can differ 

greatly from the national means. 

Figure 2a. Quarterly values of temperature bias 

and error for Birmingham (triangles) and Detroit 

(squares) within the 2002 12 km MM5 simulation. 

0.0

1.0

2.0

3.0

-2.5 -1.5 -0.5 0.5 1.5 2.5

Bias (K)

Mean Absolute Error (K)

DET 1Q

DET 2Q

DET 3Q

DET 4Q

BHM 1Q

BHM 2Q

BHM 3Q

BHM 4Q

Figure 2b. Quarterly values of water vapor mixing 

ratio bias and error for Birmingham (triangles) and 

Detroit (squares) within the 2002 12 km MM5 

simulation. 

0.0

0.5

1.0

1.5

2.0

2.5

3.0

-1.5 -1.0 -0.5 0.0 0.5 1.0 1.5

Bias (g/kg)

Mean Absolute Error (g/kg)

DET 1Q

DET 2Q

DET 3Q

DET 4Q

BHM 1Q

BHM 2Q

BHM 3Q

BHM 4Q

Figure 2c. Quarterly values of wind speed bias 

and error for Birmingham (triangles) and Detroit 

(squares) within the 2002 12 km MM5 simulation. 

0.0

1.0

2.0

3.0

-2.5 -1.5 -0.5 0.5 1.5 2.5

Bias (m/s)

Mean Absolute Error (m/s)

DET 1Q

DET 2Q

DET 3Q

DET 4Q

BHM 1Q

BHM 2Q

BHM 3Q

BHM 4Q

Figure 2d. Quarterly values of wind direction bias 

and error for Birmingham (triangles) and Detroit 

(squares) within the 2002 12 km MM5 simulation. 

0.0

10.0

20.0

30.0

40.0

-25.0 -15.0 -5.0 5.0 15.0 25.0

Bias (deg)

Mean Absolute Error (deg)

DET 1Q

DET 2Q

DET 3Q

DET 4Q

BHM 1Q

BHM 2Q

BHM 3Q

BHM 4Q

4. PHENOMENOLOGICAL EVALUATION 

The phenomenological evaluation ideally 

would be based on existing air quality conceptual 

models (i.e., how is air pollution generated, 

transported, and removed over the area of 

interest). At a local level, this type of evaluation 

would assess performance for varied phenomena 
Presented at the 6th Annual CMAS Conference, Chapel Hill, NC, October 1-3, 2007 

4 

such as trajectories, low-level jets, fronts, and air 

mass residence time. In these instances, a 

different universe of statistics such as false alarm 

rates and probabilities of detection are applicable. 

For an evaluation of any set of meteorological 

input data, it is necessary to expand the 

phenomenological evaluation to include all effects 

not captured in the operational evaluation. As 

such, for the 2002 MM5 evaluation additional 

parameters were considered, beyond the four 

discussed in Section 3 (e.g., precipitation, upper 

air analyses). Additionally, a more detailed 

examination of the cold bias identified earlier was 

undertaken. Lastly, a more detailed evaluation of 

an ozone episode was undertaken. 

4.1 Precipitation (12 km grid) 

There was a general tendency in the 2002 

MM5 12 km modeling to underestimate the 

monthly observed precipitation, when compared to 

the National Precipitation Analysis. The model 

under prediction was greatest in the fall and least 

in the spring months. Figures 3a and 3b show the 

comparison between observed and predicted 

precipitation for the months with the best and 

poorest performance. Qualitatively, the model 

appears to locate the precipitation in the proper 

location. 

Figure 3a. Comparison of observed (left) and MM5 

simulated (right) precipitation for May 2002. 

Figure 3b. Comparison of observed (left) and MM5 

simulated (right) precipitation for October 2002. 

4.2 Model performance aloft

Because of the importance of transport in 

many air pollution events, it is necessary to 

examine model performance above the surface 

layer. Figures 4a (spring) and 4b (fall) show the 

seasonally-averaged model and observational 

profiles of potential temperature, relative humidity, 

wind speed, and wind vectors over the lowest five 

km of the atmosphere at Greensboro NC (GSO). 

These analyses have been done for all available 

upper air sites. The GSO comparisons show a 

pattern seen elsewhere which is that lowertropospheric wind speeds appear to be too low in 

the MM5 simulations (~ 1 m/s). The profiles of 

potential temperatures and relative humidity are 

consistent between the model and the actual data. 

Figure 4a. Seasonally-averaged model (blue) and 

observed (red) values of four meteorological 

parameters at Greensboro NC for Apr - Jun 2002. 

Figure 4b. Seasonally-averaged model (blue) and 

observed (red) values of four meteorological 

parameters at Greensboro NC for Oct - Dec 2002. 

4.3 Assessment of the cold bias by 

time of day 

When the operational statistics are binned into 

hours of day and then aggregated nationally over 

the diurnal cycle, it becomes evident (see Figure 

5) that the overall cold bias is greatest during the 

nighttime period. During the daytime the cold bias 

still exists but averages only ~ 1 deg K as opposed 
Presented at the 6th Annual CMAS Conference, Chapel Hill, NC, October 1-3, 2007 

5 

to the ~2 deg K that is calculated near 0000 UTC. 

Thus, any impacts from the cold bias on the 

CMAQ simulations will be greatest at night. 

Figure 5. Plot of winter temperature bias (green), 

error (blue), and standard deviation (red) as a 

function of the hour of day in the 2002 12 km MM5 

simulation. The dark shading on the plot is 

intended to approximately represent nighttime 

conditions. 

4.4 Early-August Ozone Episode 

As discussed earlier, one of the most 

important meteorological evaluation steps is to 

incorporate the results into the air quality model 

evaluation. This can involve identifying 

regions/periods where the meteorological model 

predictions are most uncertain, and it can also 

involve reviewing the meteorology in 

regions/periods in which there are questions about 

the air quality model performance. A high ozone 

episode in early-August 2002 has been 

investigated in more detail because of questions 

about ozone performance in the CMAQ model. 

Figure 6 shows that there was an area of high 

ozone (e.g., maximum 8-hour ozone > 80 ppb) 

over the Ohio and Tennessee valleys and 

extending into the northeastern U.S. There is a 

sharp gradient between ozone values in this area 

and much lower values just to the north of this 

region. A stationary front marked the boundary 

between the relatively cleaner and more polluted 

airmasses. 

Fortunately, there happened to be a radar 

wind profiler in the vicinity of the front. Figure 7 

shows a comparison between the model and 

observed wind vectors on August 3rd at that site. 

Note that the model does not capture the strong 

NNE flow behind the front. Thus, in MM5, the 

front is likely weaker than observed, or possibly 

misplaced. Generally, the MM5 simulations were 

found to replicate synoptic patterns quite well, but 

for individual cases any differences in timing of 

frontal passages can be significant. 

Figure 6. Plot of observed 8-Hour ozone maxima 

over the eastern U.S. on August 3, 2002. 

Figure 7. Comparison of model wind vectors (red) 

and profiler-observed wind vectors (black) at a 

northern Indiana site on August 3, 2002. 
Presented at the 6th Annual CMAS Conference, Chapel Hill, NC, October 1-3, 2007 

6 

5. SUMMARY OF EVALUATION RESULTS 

Both sets of 2002 MM5 meteorological model 

output fields represent a reasonable 

approximation of the actual meteorology that 

occurred during the modeling period at a national 

level. It is expected that these sets of input 

meteorological data are appropriate for use in 

regional and national air quality modeling 

simulations. For local scale analyses, it is 

recommended that a detailed, area-specific 

evaluation be completed before using in those 

applications. 

The most troublesome aspect of 

meteorological model performance is the cold bias 

in surface temperatures during the winter of 2002, 

especially in January. Across the two MM5 

simulations, the January cold bias typically 

averaged around 2-3 deg C. The effect is largest 

overnight which results in a tendency to 

overestimate stability in the lowest layers. These 

artifacts from the meteorological modeling could 

have a significant impact on the air quality results. 

The underestimation of precipitation is almost 

certainly going to have an impact on model 

deposition calculations. 

Generally, the MM5 model bias and error do 

not appear to be a function of region. However, 

individual model / observation comparisons in 

space/time can show large deviations. Caution 

should be exercised when using these 

meteorological data west of 100-105 degrees 

longitude. Model errors/biases are much larger in 

this mountainous region than any other part of the 

domain. 

Caution will also have to be exercised when 

using these MM5 results on the local scale. When 

averaged regionally, there is little to no bias in 

wind directions, but as shown in section 3.2, local 

variances can be considerably higher. 

Additionally, the "key site" analyses shown in 

section 4.2 looked at MM5 performance over a 

specific ozone event in the Ohio Valley. These 

evaluations can be time-consuming but are 

important for identifying possible causes of air 

quality modeling biases in important periods. 

The 2002 MM5 model evaluation is not 

entirely complete. We would like to do more 

analysis on cloud coverage, planetary boundary 

layer heights, as well as try to assess model 

performance as a function of meteorological 

regime (clusters). This extended abstract 

represents only a small subset of the actual 

evaluation analyses completed. The figures and 

tables selected for inclusion were intended to 

provide a "snapshot" of the potential air quality 

modeling concerns expected to result from 

application of the 2002 MM5 input meteorological 

data. 

6. ACKNOWLEDGEMENTS 

The authors would like to acknowledge the 

contributions to this analysis from Jason Brewer 

who served as a 2006 summer EPA intern from 

North Carolina State University. Norm Possiel 

provided helpful comments on an initial draft of 

this extended abstract. Additionally, we would like 

to thank Gopal Sistla and the New York 

Department of Environmental Conservation for the 

use of Figure 6. 

7. REFERENCES 

Byun, D.W., and K. L. Schere, 2006: Review 

of the Governing Equations, Computational 

Algorithms, and Other Components of the Models3 Community Multiscale Air Quality (CMAQ) 

Modeling System. Applied Mechanics Reviews, 

Volume 59, Number 2 March 2006, pp. 51-77. 

Gilliam, R. C., W. Appel, and S. Phillips. The 

Atmospheric Model Evaluation Tool (AMET): 

Meteorology Module. Presented at 4th Annual 

CMAS Models-3 Users Conference, Chapel Hill, 

NC, September 26 - 28, 2005. 

Grell, G., J. Dudhia, and D. Stauffer, 1994: A 

Description of the Fifth-Generation Penn State / 

NCAR Mesoscale Model (MM5), NCAR/TN-398 

+STR., 138 pp, National Center for Atmospheric 

Research, Boulder CO.