# Meteorological Modeling Performance

Evaluation for the Annual 2005 Western U.S. 12-km Domain Simulation

Kirk Baker Pat Dolwick U.S. Environmental Protection Agency Office of Air Quality and Pollution Standards

February 2, 2009

# Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2 Model Configuration and Application . . . . . . . . . . . . . . 2 2.1 Terrain . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2.2 Analysis Manipulation . . . . . . . . . . . . . . . . . . 3 2.3 Model Options and Execution . . . . . . . . . . . . . . 5 3 Model Performance Evaluation . . . . . . . . . . . . . . . . . 7 3.1 Performance Discussion . . . . . . . . . . . . . . . . . 9 3.2 Temperature . . . . . . . . . . . . . . . . . . . . . . . . 10 3.3 Wind Field . . . . . . . . . . . . . . . . . . . . . . . . 13 3.4 Moisture . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.5 Rainfall . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.6 Radiation . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.7 PBL Height . . . . . . . . . . . . . . . . . . . . . . . . 25 4 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 27 Appendices 31 1 Climatology Review . . . . . . . . . . . . . . . . . . . . . . . . 31

1February 2, 2009



## 1 IntroductionMeteorological inputs to support emissions, dispersion, and photochemical modeling are generated with the Fifth-Generation NCAR / Penn State Mesoscale Model (MM5) v3.7.4. MM5 is a limited area, nonhydrostatic,terrain following system that solves the full set of equations that govern atmospheric motion. MM5 consists of the Mesoscale model MM5 and a suite of pre-processors including PREGRID, REGRIDDER, RAWINS, LITTLE R, INTERPF, INTERPX, and TERRAIN [Dudhia, 1993] [Grell et al., 1994].The model parameterizations and physics options outlined in this document were chosen based on the results of a series of sensitivity runs. Theperformance of the sensitivity tests provided an indication of an optimal configuration based on temperature, mixing ratio, and wind field [Johnson, 2003] [McNally, 2002]. The MM5 model was applied to a continental scale 36 km domain and awestern United States 12 km domain over the entire year of 2005. Configuration, application, and model performance are covered in this document. This MM5 simulation was done by Alpine Geophysics, LLC under contractfrom the United States Environmental Protection Agency (contract EP-D07-102). 

## 2 Model Configuration and Application 2.1 TerrainThe TERRAIN processor defines the horizontal grid of the MM5 application. The continental scale 36 km domain has 165 cells in the X direction and 129 cells in the Y direction. The nested western United States 12 km domain has 229 cells in the X direction and 208 cells in the Y direction. The 12 km nested domain is offset from the 36 km domain by 13 cells in the Xdirection and 35 cells in the Y direction. All domains have a Lambert conformal projection centered at coordinates -97, 40 with first and second true latitudes at 33 and 45 degrees.
2February 2, 2009 Vegetative and landuse information is developed based on data released with the MM5 distribution. Terrain information is based on United States Geographic Survery (USGS) terrain databases distributed with MM5. The 36 km domain is based on 5 min (approx. 9 km) and the 12 km domain is based on 2 min (approx. 4 km) Geophysical Data Center global data.Additional options are set to allow generation of data to support the PleimXiu land surface module. Variables LSMDATA and IEXTRA are both set equal to TRUE.

Figure 1: 36 and 12 km model domains 

### 2.2 Analysis ManipulationThe PREGRID processor converts meteorological analyses data to an intermediate data format that the REGRIDDER processor can utilize. ETA/AWIP3D and surface analyses data (ds609.2) is used to initialize the model [National Centers for Environmental Prediction, 2008a]. The input analyses 3February 2, 2009

data is processed 3 hourly (10,800 seconds). The AWIP grib definition tables are used to map ETA data to MM5. Snow cover is estimated from water equivalent snow depth. Water surface temperature data is based on NCEP RTG global one-half degree analysis [National Centers for Environmental Prediction, 2008b]. The REGRIDDER processor takes the data extracted from analyses fields and interpolates the data to user specified pressure levels and to the user specified horizontal grid. The RAWINS and LITTLE R processors perform objective analysis on the output from REGRIDDER using surface and upper air observation data. Since these observations are incorporated into the ETA analysis fields this step is considered redundant but may further reduce error in the analysis data. LITTLE R is applied to enable surface nudging of soil moisture and temperature in the Pleim-Xiu land surface module. NCEP ADP surface (ds 464.0) and upper air (ds 353.1 and ds 353.4) data are the appropriate data to input into LITTLE R and/or RAWINS. The INTERPF processor takes the REGRIDDER/LITTLE R output that is at standard pressure levels and interpolates that data to the vertical grid defined by the user. The vertical grid is defined in terms of sigmas, where 1 is the surface and 0 is the top of the model atmosphere. The top of the model domain is 100 millibars, which is approximately 15 kilometers above ground level. The vertical atmosphere is resolved to 34 layers, with thinner layers in the planetary boundary layer (PBL). The surface layer is approximately 38 meters in height. The layer configuration is selected to capture the important diurnal variations in the boundary layer while also having layers in the upper troposphere to resolve deep cloud formation.

4February 2, 2009

Figure 2: Vertical layer structure and depth (m) 

### 2.3 Model Options and Execution When meteorological models are applied to replicate past events, the use of data assimilation helps to ”nudge” solutions so that they do not diverge greatly from the actual observed meteorological fields. This is one of themajor benefits of using dynamic meteorological models since they can pro5February 2, 2009

vide a way of consistently characterizing meteorological conditions at times and locations where observations do not exist while at the same time still accounting indirectly for actual observations.Three dimensional analysis nudging for temperature and moisture is applied above the boundary layer only. Analysis nudging for the wind field isapplied above and below the boundary layer. Analysis nudging is not performed on the rotational wind field. In addition, the observation nudging flag is turned off. This type of nudging is most appropriate when there isa very dense set of observation data from a field study, which this application lacked. The 36 km domain nudging weighting factors are 2.5 x 10
−4 for wind fields and temperatures and 1.0 x 10
−5 for moisture fields. The 12 km domain nudging weighting factors are 1.0 x 10
−4 for wind fields and temperatures and 1.0 x 10 −5 for moisture fields. Important physics options used are listed below.
• Pleim-Xiu PBL and land surface schemes
• Kain-Fritsh 2 cumulus parameterization
• Reisner 2 mixed phase moisture scheme
• RRTM longwave radiation scheme
• Dudhia shortwave radiation schemeAtmospheric radiation is calculated every 15 minutes in the model. Vertical moisture and temperature advection are set to use linear interpolation. Other important variables switched to ON include: moist vertical diffusion in clouds, temperature advection using potential temperature, diffusion using perturbation temperature, 3D coriolis force, and upper radiative boundary condition. Sea surface temperature and snow cover are set to vary with time. The Pleim-Xiu land surface module requires that additional variables be set in the MM5 deck: ISMRD and NUDGE. ISMRD is set to use soil moisture and soil temperature fields from the ETA analyses. NUDGE is set to nudge soil moisture data to the analyses fields.

6February 2, 2009 MM5 was executed in 5 day blocks (7200 minute simulation) initiated at 12Z with a 90 second time step. Model results are output every 60 minutes and the model output files are written out (i.e. split) every 24 hours to accommodate post-processing utilities. The 36 and 12 km domains were run for the entire calendar year of 2005. 

## 3 Model Performance EvaluationOne of the objectives of this evaluation is to determine if the meteorological model output fields represent a reasonable approximation of the actual meteorology that occurred during the modeling period. A second objectiveis to identify and quantify the existing biases and errors of the meteorological predictions in order to allow for a downstream assessment of how theair quality modeling results are affected by issues associated with the meteorological data. Performance results are presented to allow those using this data to determine the adequecy of the model simulation for their particular needs. The observation database for temperature, wind speed, wind direction, and mixing ratio is based on the Meteorological Assimilation Data IngestSystem (MADIS) available from the National Oceanic and Atmospheric Administration (NOAA) Earth System Research Laboratory (ESRL) Global Systems Division (GSD). The MADIS database is quality controlled and made available by file transfer protocol [National Oceanic and Atmospheric Administration, 2008]. Rainfall observation analysis data is available from the National WeatherService Climate Prediction Center (CPC) on an hourly basis for the Continental United States [National Weather Service, 2008]. The rainfall analysis resolution is 0.25 degree longitude by 0.25 degree latitude (approximately 40 km by 40 km) and extends from 140W to 60W and 20N to 60N. The CPC rainfall analysis data does not include any portion of Canada, Mexico, or anywhere off-shore of the United States. Shortwave downward radiation measurements are taken at SURFRAD and ISIS monitor locations [Earth System Research Laboratory, 2008b] [Earth
7February 2, 2009

System Research Laboratory, 2008a]. The SURFRAD network consists of 7 sites and the ISIS network consists of 8 sites across the United States.

Figure 3: SURFRAD (black dots) and ISIS (red dots) monitor locations Model performance is described using quantitative metrics: mean bias and mean (gross) error [Boylan et al., 2006]. These metrics are useful becausethey describe model performance in the measured units of the meteorological variable. Performance is best when these metrics approach 0. Rainfallperformance is examined qualitatively with side-by-side monthly total rainfall plots. The MM5 model outputs predictions approximately 15 metersabove the surface while observations are at 10 meters. MM5 outputs nearinstantaneous values (90 second time step) as opposed to the values with longer averaging times taken at monitor stations. This should be considered when interpreting model performance metrics.

8February 2, 2009

Figure 4: Model performance regions Performance metrics are aggregated over spatial areas to help determine how well the model is performing in different parts of the domain. This aggregation also aids the evaluation process since the examination of all individual monitor locations is resource intensive. Metrics are aggregated over 10 regions: northeast, mid-atlantic, Florida, south, great lakes, midwest, rockies-north, rockies-south, northwest, and California. 

### 3.1 Performance Discussion The upper Rockies region (defined as Montana, Idaho, and Wyoming) has a fairly minimal temperature and moisture bias. The area shows a slight cold bias in temperature during the winter and early spring months. Wind speeds tend to be over-predicted in this region during the winter and early spring months. The Northwest region (defined as Washington and Oregon) has minimal temperature bias in the cooler months and an under-prediction tendency in the summer months. This region does not have a strong pattern of over or 9February 2, 2009

under-prediction of wind speed by month. Wind speed error is low in the sprint and summer months and peaks slightly during the colder months. The lower Rockies (defined as Colorado, Utah, Nevada, Arizona, and NewMexico) tend to have model estimates of temperature under-predict observations. Wind speed bias is minimal in the lower Rockies. Moisture is overpredicted in the spring and summer months in this region. Wind direction error is also highest during the late spring and summer. This region also shows a pronounced over-prediction of moisture during the summer months. California has a high bias for wind speed during the winter months and very early spring. The model tends to under-predict temperature in this area. Moisture bias is fairly low all year in California and model estimates are well correlated with observations. The model simulation does well at estimating the spatial pattern of monthly total rainfall during the winter, fall, and early spring. However, duringthe late spring and summer the model seems to have large localized overpredictions of rainfall in the Rocky Mountain region. There may be someincommensurability between the observation network in this part of the country but the model prediction pattern is likelyover-stating summer convective rainfall in this part of the model domain. Shortwave radiation observations are minimal, but in general the model does not show strong systematic bias from month to month. A diurnaltrend in bias is noted in that shortwave downward radiation is usually underpredicted in the early morning and over-predicted in the late afternoon.Maximum model estimated PBL heights are plotted to determine if unrealistically high mixing might be expected anywhere in the modeling domain in a particular month. The highest estimated PBL heights are estimated from April to September. 

### 3.2 Temperature Average monthly bias, error, and coefficient of determination are shown by region. These plots illustrate the variability in model performance for 10February 2, 2009

temperature at monitors in defined regions for each month.

Figure 5: Monthly averaged temperature bias (C)

11February 2, 2009

Figure 6: Monthly averaged temperature error (C)

Figure 7: Monthly averaged coefficient of determination

12February 2, 2009



### 3.3 Wind Field Average monthly bias, error, and coefficient of determination are shown for wind speed by region. These plots illustrate the variability in model performance for wind speed at monitors in defined regions for each month. The error metric is shown for wind direction since bias and correlation have little meaning for a variable expressed in compass degrees.

Figure 8: Monthly averaged wind speed bias (m/s)

13February 2, 2009

Figure 9: Monthly averaged wind speed error (m/s)

Figure 10: Monthly averaged wind speed coefficient of determination

14February 2, 2009

Figure 11: Monthly averaged wind direction error (degrees)



### 3.4 Moisture Average monthly bias, error, and coefficient of determination are shown by region. These plots illustrate the variability in model performance for mixing ratio at monitors in defined regions for each month.

15February 2, 2009

Figure 12: Monthly averaged mixing ratio bias (g/kg)

Figure 13: Monthly averaged mixing ratio error (g/kg)

16February 2, 2009

Figure 14: Monthly averaged coefficient of determination



### 3.5 Rainfall Monthly total rainfall is plotted for each grid cell to assess how well themodel captures the spatial variability and magnitude of convective and nonconvective rainfall events.

17February 2, 2009

(a) Observation (b) Model Figure 15: January monthly total rainfall (inches)

(a) Observation (b) Model Figure 16: February monthly total rainfall (inches)

18February 2, 2009

(a) Observation (b) Model Figure 17: March monthly total rainfall (inches)

(a) Observation (b) Model Figure 18: April monthly total rainfall (inches)

19February 2, 2009

(a) Observation (b) Model Figure 19: May monthly total rainfall (inches)

(a) Observation (b) Model Figure 20: June monthly total rainfall (inches)

20February 2, 2009

(a) Observation (b) Model Figure 21: July monthly total rainfall (inches)

(a) Observation (b) Model Figure 22: August monthly total rainfall (inches)

21February 2, 2009

(a) Observation (b) Model Figure 23: September monthly total rainfall (inches)

(a) Observation (b) Model Figure 24: October monthly total rainfall (inches)

22February 2, 2009

(a) Observation (b) Model Figure 25: November monthly total rainfall (inches)

(a) Observation (b) Model Figure 26: December monthly total rainfall (inches)



### 3.6 Radiation Shortwave downward radiation estimates are compared to surface based measurements. Measurements of shortwave downward radiation are taken at 23February 2, 2009

SURFRAD and ISIS monitor locations. Photosynthetically activated radiation (PAR) is a fraction of shortwave downward radiation and is an importantinput for the biogenic emissions model for estimating isoprene. Isoprene emissions are important for regional ozone chemistry and play a role in secondary organic aerosol formation. Radiation performance evaluation also gives an indirect assessment of how well the model captures cloud formation during daylight hours. Outliers are not plotted on these boxplots to emphasize predominant features in model performance.

Figure 27: Shortwave downward radiation bias by month

24February 2, 2009

Figure 28: Shortwave downward radiation bias by hour 

### 3.7 PBL HeightMaximum PBL heigts are plotted for each grid cell to assess whether unrealistic stratospheric intrusion may occur in any of the simulated months.

25February 2, 2009

(a) January (b) February (c) March

(d) April (e) May (f) June

(g) July (h) August (i) September

(j) October (k) November (l) December Figure 29: Predicted monthly maximum PBL height (m)

26February 2, 2009



## 4 Acknowledgements This evaluation report would not be possible without the assistance of Lara Reynolds of CSC, Rob Gilliam of EPA/ORD, Mike Rizzo of EPA/OAQPS, and Adam Reff of EPA/OAQPS.

27

# Bibliography

J.W. Boylan, T.M. Odman, J.G. Wilkinson, and A.G. Russell. Integrated assessment modeling of atmospheric pollutants in the southern Appalachian Mountains: Part II. fine particulate matter and visibility. Journal of Air and Waste Management Association, 56:12–22, 2006. J. Dudhia. A Non-hydrostatic Version of the Penn State/NCAR Mesoscale Model: Validation Tests and Simulation of an Atlantic Cyclone and Cold Front. Mon. Wea. Rev. , 121:1493–1513, 1993. Earth System Research Laboratory. Integrated Surface Irradiance Study (ISIS) Network, 2008a. URL http://www.ncdc.noaa.gov/oa/ncdc.html
.Earth System Research Laboratory. SURFRAD (Surface Radiation) Network, 2008b. URL http://www.srrb.noaa.gov/surfrad/index.html
. G. Grell, J. Dudhia, and D. Stauffer. A Description of the Fifth-Generation Penn State NCAR Mesoscale Model. Technical Report TN-398-STR, 1994. M. Johnson. Meteorological Modeling Protocol: IDNR 2002 Annual MM5 Application. Technical report, 2003. D. McNally. Topical Report: A Comparison of MM5 Model Estimates forFebruary and July 2001 using Alternative Model Physics Options. Technical report, 2002. National Centers for Environmental Prediction. GCIP NCEP Eta 212 model output, 2008a. URL http://dss.ucar.edu/datasets/ds609.2
. National Centers for Environmental Prediction. Real-time, global, sea surface temperature analsis, 2008b. URL http://polar.ncep.noaa.gov/sst/oper/Welcome.html
. 28February 2, 2009

National Oceanic and Atmospheric Administration. Meteorological Assimilation Data Ingest System, 2008. URL http://madis.noaa.gov/
. National Weather Service. Climatic Prediction Center, 2008. URL http://www.cpc.noaa.gov/products/precip/realtime/
.

29

# Appendices

30February 2, 2009



## 1 Climatology Review Annual temperatures were slightly above normal for much of the United States in 2005. The region east of the Rocky Mountains to the northern plains experienced annual temperatures much higher than normal. The southeast had normal to slightly cooler temperatures than usual in 2005.

Figure 30: Annual temperature deviations from normal Annual precipiation patterns for 2005 are drier that normal in the central United States, from Texas north to Wisconsin. The northeast, north central plains, and most of the western United States were wetter than normal for annual precipitation.

31February 2, 2009

Figure 31: Annual rainfall deviations from normal

32