Presented at the 6 th Annual CMAS Conference, Chapel Hill, NC, October 1-3, 2007

1 

### REGIONAL AND LOCAL-SCALE EVALUATION OF 2002 MM5 METEOROLOGICAL FIELDS
FOR VARIOUS AIR QUALITY MODELING APPLICATIONS

Pat Dolwick* U.S. Environmental Protection Agency, Research Triangle Park, NC, USA



### Rob Gilliam National Oceanic and Atmospheric Administration, Research Triangle Park, NC, USA



### Lara Reynolds and Allan Huffman Computer Sciences Corporation, Research Triangle Park, NC, USA



### 1. INTRODUCTION

Prognostic meteorological models are often used in a retrospective mode to provide inputs to air quality models. In turn, these air quality models are used for environmental planning. The meteorological inputs govern the advection, diffusion, chemical transformation, and eventual deposition of pollutants within regional air quality models such as the Community Multi-scale Air Quality (CMAQ) modeling system (Byun and Schere, 2006). The air quality models have traditionally been subjected to a rigorous performance assessment, but in many cases the meteorological inputs to these models are accepted as is, even though this component of the modeling arguably contains equal uncertainty.

Before initiating the air quality simulations, it is important to identify the biases and errors associated with the meteorological modeling inputs. The goal of the meteorological evaluation should be to move toward an understanding of how the bias and error of the meteorological input data may impact the resultant AQ modeling. Typically, there are two spec ific objectives. First, determine if the meteorological model output fields represent a reasonable approximation of the actual meteorology that occurred during the modeling period (i.e., the "operational" evaluation). The second goal should be to identify how the existing biases and errors in the meteorological predictions may affect the air quality modeling results (i.e., the "phenomenological" evaluation). Once these two sets of information are generated, it is important to highlight the parts of the analysis expected to most influence the air quality model.

* Corresponding author: Pat Dolwick, U.S. Environmental Protection Agency, MD C439-03, RTP, NC 27711; e-mail: dolwick.pat@epa.gov This analysis looks at the performance of the Penn State University / National Center for Atmospheric Research mesoscale model known as MM5 (Grell et al, 1994) for a specific year (2002) at two separate model resolutions. The

model evaluation is summarized for the entire domain, individual subregions within the domain, and certain individual sites. The goal is to provide a snapshot of the types of analyses that can be completed to assess the utility of using this data to drive regional-scale, photochemical models (e.g., CMAQ) as well as local-scale dispersion models (e.g., AERMOD 1 ).



### 2. MM5 MODEL CONFIGURATION

Meteorological model input fields were prepared for two separate grids as shown in Figure 1. A 36 km grid covering the contiguous portion of the U.S. was modeled using MM5 v.3.6.0 using land-surface modifications that were added in v3.6.3. Additionally, a 12 km grid that

covered the eastern two-th irds of the U.S. was

modeled with MM5 v3.7.2. Both domains contained 34 vertical layers with a ~38 m surface layer and a 100 mb top. Both sets of model runs were conducted in 5.5 day segments with 12 hours of overlap for spin-up purposes.

Table 1 lists the physics options used in the

two sets of MM5 simulations. The initial state analyses were derived from the 3-hourly, Eta Data Assimilation System (EDAS) data. Analysis nudging was utilized outside of the boundary layer for temperatures and water vapor mixing ratios, and in all locations for wind components, using relatively weak nudging coefficients.

1 AERMOD is a Gaussian dispersion model used to predict air quality in the vicinity of a single source, or group of sources. For more information on AERMOD:

http://www.epa.gov/ttn/scram/dispersion_prefrec.htm#aermodPresented at the 6 th Annual CMAS Conference, Chapel Hill, NC, October 1-3, 2007

2 Figure 1. Plot of 36 and 12 km MM5 modeling domains in the 2002 modeling.

Table 1. List of MM5 model physics options used

in the 2002 MM5 simulations.

Physics Option 2002 MM5 Modeling

Planetary boundary layer model Pleim-Chang / ACM1

Sub-grid convection scheme Kain-Fritsch 2

Explicit microphysics scheme Reisner 1

Land-surface model Pleim-Xiu

Longwave radiation scheme RRTM



### 3. OPERATIONAL EVALUATION

The operational evaluation includes statistical comparisons of model/observed pairs (e.g., mean normalized bias, mean normalized error, index of agreement, root mean square errors, etc.) for multiple meteorological parameters. For this portion of the evaluation, four meteorological parameters were investigated: temperature, humidity, wind speed, and wind direction. The operational piece of the analyses focuses on surface parameters. The Atmospheric Model Evaluation Tool (AMET) was used to conduct the analyses as described in by Gilliam et al (2005).



### 3.1 Eastern U.S. results (12 km grid)

Tables 2a and 2b show the monthly results of the 12 km evaluation. One of the key findings of the operational evaluation is that MM5 surface temperatures have a cold bias. This bias is especially notable in the winter and early spring (e.g., Jan - Apr). This artifact in the meteorological modeling is expected to have a major impact on the eventual air quality modeling results for those pollutants that are temperature-dependent (e.g., nitrates).

Otherwise, the model reproduces observed water vapor mixing ratios with considerable accuracy, though there is a seasonal pattern with highest error in the summer, due to overall higher moisture content. Wind speeds are generally underestimated across the y ear, but typically the
underestimation bias is less than 0.5 m/s. The average error in wind direction ranges from 24-31 degrees. Generally s peaking, the operational statistics from the 36 km grid closely approximated

the 12 km results.

Table 2a. Mean absolute error by month in the

2002 12 km MM5 simulation.

Month Temp. (K) Mix Ratio (g/kg) W Spd (m/s) W Dir (deg)

Jan 3.0 0.6 1.3 24.1

Feb 2.5 0.6 1.3 23.6

Mar 2.7 0.7 1.4 24.9 Apr 2.3 0.9 1.3 24.9 May 1.8 1.0 1.3 25.8

Jun 1.5 1.2 1.2 29.2

Jul 1.5 1.4 1.1 31.4 Aug 1.5 1.3 1.1 30.5 Sep 1.5 1.1 1.1 28.1

Oct 1.6 0.8 1.1 26.7

Nov 2.0 0.7 1.2 24.3

Dev 2.5 0.6 1.3 24.5

Table 2b. Mean bias by month in the 2002 12 km

MM5 simulation.

Month Temp. (K) Mix Ratio (g/kg) W Spd (m/s) W Dir (deg)

Jan -2.4 -0.2 -0.3 7.4

Feb -1.3 -0.2 -0.4 7.8

Mar -1.5 -0.1 -0.7 7.5 Apr -1.5 -0.1 -0.4 6.2 May -0.5 -0.1 -0.3 6.8

Jun -0.5 -0.2 -0.4 6.8

Jul -0.1 -0.1 -0.4 7.8 Aug -0.1 -0.1 -0.4 7.1 Sep 0.0 -0.3 -0.3 7.8

Oct 0.4 -0.2 -0.2 8.3

Nov -0.2 0.0 -0.2 8.4

Dev -0.6 0.1 -0.5 8.1

Model performance was also considered by region, by underlying land use type, and by proximity to coastline or mountainous regions. The tabular results are not shown here for spacePresented at the 6 th Annual CMAS Conference, Chapel Hill, NC, October 1-3, 2007

3 considerations, however, there was a clear spatial gradient in model performance. Model performance was more accurate in the eastern portions of the domain, than the more topographically-influenced western portion of the 12 km EUS grid. For instance, mean annual temperature errors ranged from 1.4 to 1.6 degrees over subregions in the eastern U.S., while the errors ranged from 2.0 to 2.2 degrees over the western portion of the domain. Additionally, errors in wind direction were almost 20 degrees greater over the Rocky Mountain portions of the 12 km MM5 grid. Conversely, model performance at

locations determined to be coastal tended to be similar to the performance at inland locations. There was no significant error tendencies that were a function of land use type.



### 3.2 Local-scale results (12 km grid)

Beyond the domainwide results, which are aggregations of numerous model - observed pairs,

there was also a limited evaluation of meteorological model performance at a handful of specified sites. This was done to assess the utility of using gridded MM5 data to drive dispersion models such as AERMOD. For the 12 km grid, an local scale analysis was completed for two locations: Birmingham and Detroit. Figures 2a-2d show the bias and error of the four meteorological parameters, for each quarter, and at each site. The statistical output are compared against target values of bias and error in the "soccer plot" mode. Points within the brown boxes are thought to be representative of exceptional model performance. Points that are outside the green box indicate areas in which model performance may be in need of improvement. This analysis confirms the expected, that is, local performance can differ greatly from the national means.

Figure 2a. Quarterly values of temperature bias and error for Birmingham (triangles) and Detroit (squares) within the 2002 12 km MM5 simulation.

0.0 1.0 2.0 3.0

-2.5 -1.5 -0.5 0.5 1.5 2.5

Bias (K)Mean Absolute Error (K) DET 1Q

DET 2Q

DET 3Q

DET 4Q

BHM 1Q

BHM 2Q

BHM 3Q

BHM 4Q Figure 2b. Quarterly values of water vapor mixing ratio bias and error for Birmingham (triangles) and Detroit (squares) within the 2002 12 km MM5

simulation.

0.0 0.5 1.0 1.5 2.0 2.5 3.0

-1.5 -1.0 -0.5 0.0 0.5 1.0 1.5

Bias (g/kg)Mean Absolute Error (g/kg) DET 1Q

DET 2Q

DET 3Q

DET 4Q

BHM 1Q

BHM 2Q

BHM 3Q

BHM 4Q

Figure 2c. Quarterly values of wind speed bias and error for Birmingham (triangles) and Detroit (squares) within the 2002 12 km MM5 simulation.

0.0 1.0 2.0 3.0

-2.5 -1.5 -0.5 0.5 1.5 2.5

Bias (m/s)Mean Absolute Error (m/s) DET 1Q

DET 2Q

DET 3Q

DET 4Q

BHM 1Q

BHM 2Q

BHM 3Q

BHM 4Q

Figure 2d. Quarterly values of wind direction bias and error for Birmingham (triangles) and Detroit (squares) within the 2002 12 km MM5 simulation.

0.0 10.0 20.0 30.0 40.0

-25.0 -15.0 -5.0 5.0 15.0 25.0

Bias (deg)Mean Absolute Error (deg) DET 1Q

DET 2Q

DET 3Q

DET 4Q

BHM 1Q

BHM 2Q

BHM 3Q

BHM 4Q



### 4. PHENOMENOLOGICAL EVALUATION

The phenomenological evaluation ideally would be based on existing air quality conceptual models (i.e., how is air pollution generated, transported, and removed over the area of interest). At a local level, this type of evaluation would assess performance for varied phenomenaPresented at the 6 th Annual CMAS Conference, Chapel Hill, NC, October 1-3, 2007

4 such as trajectories, low-level jets, fronts, and air mass residence time. In these instances, a

different universe of statistics such as false alarm rates and probabilities of detection are applicable.

For an evaluation of any set of meteorological input data, it is necessary to expand the phenomenological evaluation to include all effects not captured in the operational evaluation. As such, for the 2002 MM5 evaluation additional parameters were considered, beyond the four discussed in Section 3 (e.g., precipitation, upper air analyses). Additionally, a more detailed

examination of the cold bias identified earlier was undertaken. Lastly, a more detailed evaluation of an ozone episode was undertaken.



### 4.1 Precipitation (12 km grid)

There was a general tendency in the 2002 MM5 12 km modeling to underestimate the monthly observed precipit ation, when compared to the National Precipitation Analysis. The model under prediction was greatest in the fall and least in the spring months. Figures 3a and 3b show the comparison between observed and predicted precipitation for the months with the best and poorest performance. Qualitatively, the model appears to locate the precipitation in the proper

location.

Figure 3a. Comparison of observed (left) and MM5 simulated (right) precipitation for May 2002.

Figure 3b. Comparison of observed (left) and MM5 simulated (right) precipitation for October 2002. 

### 4.2 Model performance aloft

Because of the importance of transport in many air pollution events, it is necessary to examine model performance above the surface layer. Figures 4a (spring) and 4b (fall) show the seasonally-averaged model and observational profiles of potential temperature, relative humidity, wind speed, and wind vectors over the lowest five km of the atmosphere at Greensboro NC (GSO). These analyses have been done for all available upper air sites. The GSO comparisons show apattern seen elsewhere which is that lowertropospheric wind speeds appear to be too low in the MM5 simulations (~ 1 m/s). The profiles of potential temperatures and relative humidity are

consistent between the model and the actual data.

Figure 4a. Seasonally-averaged model (blue) and observed (red) values of four meteorological parameters at Greensboro NC for Apr - Jun 2002.

Figure 4b. Seasonally-averaged model (blue) and observed (red) values of four meteorological parameters at Greensboro NC for Oct - Dec 2002.



### 4.3 Assessment of the cold bias by time of day

When the operational statistics are binned into hours of day and then aggr egated nationally over the diurnal cycle, it becomes evident (see Figure 5) that the overall cold bias is greatest during the nighttime period. During the daytime the cold bias still exists but averages only ~ 1 deg K as opposedPresented at the 6 th Annual CMAS Conference, Chapel Hill, NC, October 1-3, 2007

5 to the ~2 deg K that is calculated near 0000 UTC. Thus, any impacts from the cold bias on the CMAQ simulations will be greatest at night.

Figure 5. Plot of winter temperature bias (green), error (blue), and standard deviation (red) as a function of the hour of day in the 2002 12 km MM5 simulation. The dark shading on the plot is intended to approximately represent nighttime

conditions.



### 4.4 Early-August Ozone Episode

As discussed earlier, one of the most important meteorological evaluation steps is to incorporate the results into the air quality model evaluation. This can involve identifying regions/periods where t he meteorological model predictions are most uncertain, and it can also involve reviewing the meteorology in regions/periods in which there are questions about the air quality model performance. A high ozone episode in early-August 2002 has been investigated in more detail because of questions about ozone performance in the CMAQ model.

Figure 6 shows that there was an area of high ozone (e.g., maximum 8-hour ozone > 80 ppb) over the Ohio and Tennessee valleys and extending into the northeastern U.S. There is a sharp gradient between ozone values in this area and much lower values just to the north of this region. A stationary front marked the boundary between the relatively cleaner and more polluted

airmasses.

Fortunately, there happened to be a radar wind profiler in the vicinity of the front. Figure 7 shows a comparison between the model and observed wind vectors on August 3rd at that site. Note that the model does not capture the strong NNE flow behind the front. Thus, in MM5, the front is likely weaker than observed, or possibly misplaced. Generally, the MM5 simulations were found to replicate synoptic patterns quite well, but for individual cases any differences in timing of frontal passages can be significant.

Figure 6. Plot of observed 8-Hour ozone maxima over the eastern U.S. on August 3, 2002.

Figure 7. Comparison of model wind vectors (red) and profiler-observed wind vectors (black) at a northern Indiana site on August 3, 2002.Presented at the 6 th Annual CMAS Conference, Chapel Hill, NC, October 1-3, 2007

6 

### 5. SUMMARY OF EVALUATION RESULTS

Both sets of 2002 MM5 meteorological model output fields represent a reasonable approximation of the actual meteorology that occurred during the modeling period at a national level. It is expected t hat these sets of input meteorological data are appropriate for use in regional and national air quality modeling simulations. For local scale analyses, it is recommended that a detailed, area-specific evaluation be completed before using in those applications.

The most troublesome aspect of meteorological model performance is the cold bias in surface temperatures during the winter of 2002, especially in January. Across the two MM5 simulations, the January cold bias typically averaged around 2-3 deg C. The effect is largest overnight which results in a tendency to overestimate stability in the lowest layers. These artifacts from the meteorological modeling could have a significant impact on the air quality results. The underestimation of precipitation is almost certainly going to have an impact on model deposition calculations.

Generally, the MM5 model bias and error do not appear to be a function of region. However, individual model / observation comparisons in space/time can show large deviations. Caution should be exercised when using these meteorological data west of 100-105 degrees longitude. Model errors/biases are much larger in this mountainous region than any other part of the

domain.

Caution will also have to be exercised when using these MM5 results on the local scale. When averaged regionally, there is little to no bias in wind directions, but as shown in section 3.2, local variances can be considerably higher. Additionally, the "key site" analyses shown in section 4.2 looked at MM5 performance over a specific ozone event in the Ohio Valley. These evaluations can be time-consuming but are important for identifying possible causes of air quality modeling biases in important periods.

The 2002 MM5 model evaluation is not entirely complete. We would like to do more analysis on cloud coverage, planetary boundary layer heights, as well as try to assess model performance as a function of meteorological regime (clusters). This extended abstract represents only a small subset of the actual evaluation analyses completed. The figures and

tables selected for inclusion were intended to provide a "snapshot" of the potential air quality modeling concerns expected to result from application of the 2002 MM5 input meteorological

data.



### 6. ACKNOWLEDGEMENTS

The authors would like to acknowledge the

contributions to this anal ysis from Jason Brewer

who served as a 2006 summer EPA intern from North Carolina State University. Norm Possiel provided helpful comments on an initial draft of this extended abstract. Additionally, we would like to thank Gopal Sistla and the New York Department of Environmental Conservation for the use of Figure 6.



### 7. REFERENCES

Byun, D.W., and K. L. Schere, 2006: Review of the Governing Equations, ComputationalAlgorithms, and Other Components of the Models3 Community Multiscale Air Quality (CMAQ) Modeling System. Applied Mechanics Reviews, Volume 59, Number 2 March 2006, pp. 51-77.

Gilliam, R. C., W. Appel, and S. Phillips. The Atmospheric Model Evaluation Tool (AMET): Meteorology Module. Presented at 4th Annual CMAS Models-3 Users Conference, Chapel Hill, NC, September 26 - 28, 2005.

Grell, G., J. Dudhia, and D. Stauffer, 1994: A Description of the Fifth-Generation Penn State / NCAR Mesoscale Model (MM5), NCAR/TN-398 +STR., 138 pp, National Center for Atmospheric Research, Boulder CO.