Meteorological Modeling Performance

Evaluation for the Annual 2005 Continental

U.S. 36-km Domain Simulation

Kirk Baker

Pat Dolwick

U.S. Environmental Protection Agency

Office of Air Quality and Pollution Standards

February 2, 2009
Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Model Configuration and Application . . . . . . . . . . . . . . 2

2.1 Terrain . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2.2 Analysis Manipulation . . . . . . . . . . . . . . . . . . 3

2.3 Model Options and Execution . . . . . . . . . . . . . . 5

3 Model Performance Evaluation . . . . . . . . . . . . . . . . . 7

3.1 Performance Discussion . . . . . . . . . . . . . . . . . 9

3.2 Temperature . . . . . . . . . . . . . . . . . . . . . . . . 11

3.3 Wind Field . . . . . . . . . . . . . . . . . . . . . . . . 13

3.4 Moisture . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.5 Rainfall . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3.6 Radiation . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.7 PBL Height . . . . . . . . . . . . . . . . . . . . . . . . 25

4 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 27

Appendices 31

1 Climatology Review . . . . . . . . . . . . . . . . . . . . . . . . 31

1
February 2, 2009

1 Introduction

Meteorological inputs to support emissions, dispersion, and photochemical modeling are generated with the Fifth-Generation NCAR / Penn State

Mesoscale Model (MM5) v3.7.4. MM5 is a limited area, nonhydrostatic,

terrain following system that solves the full set of equations that govern atmospheric motion. MM5 consists of the Mesoscale model MM5 and a suite of

pre-processors including PREGRID, REGRIDDER, RAWINS, LITTLE R,

INTERPF, INTERPX, and TERRAIN [Dudhia, 1993] [Grell et al., 1994].

The model parameterizations and physics options outlined in this document were chosen based on the results of a series of sensitivity runs. The

performance of the sensitivity tests provided an indication of an optimal configuration based on temperature, mixing ratio, and wind field [Johnson, 2003]

[McNally, 2002].

The MM5 model was applied to a continental United States 36 km domain for the entire year of 2005. Configuration, application, and model

performance are covered in this document.

This MM5 simulation was done by CSC under contract from the United

States Environmental Protection Agency.

2 Model Configuration and Application

2.1 Terrain

The TERRAIN processor defines the horizontal grid of the MM5 application. The eastern United States 36 km domain has 165 cells in the X

direction and 129 cells in the Y direction. The domain has a Lambert conformal projection centered at coordinates -97, 40 with first and second true

latitudes at 33 and 45 degrees.

Vegetative and landuse information is developed based on data released

with the MM5 distribution. Terrain information is based on United States

Geographic Survery (USGS) terrain databases distributed with MM5. The

2
February 2, 2009

12 km domain is based on 5 min Geophysical Data Center global data. Additional options are set to allow generation of data to support the Pleim-Xiu

land surface module. Variables LSMDATA and IEXTRA are both set equal

to TRUE.

Figure 1: 36 km model domain

2.2 Analysis Manipulation

The PREGRID processor converts meteorological analyses data to an intermediate data format that the REGRIDDER processor can utilize. ETA/AWIP

3D and surface analyses data (ds609.2) is used to initialize the model [National Centers for Environmental Prediction, 2008]. The input analyses data

is processed 3 hourly (10,800 seconds). The AWIP grib definition tables are

used to map ETA data to MM5. Snow cover is estimated from water equivalent snow depth. Water surface temperature data is based on ETA/EDAS

skin temperature data.

3
February 2, 2009

The REGRIDDER processor takes the data extracted from analyses fields

and interpolates the data to user specified pressure levels and to the user

specified horizontal grid.

The RAWINS and LITTLE R processors perform objective analysis on

the output from REGRIDDER using surface and upper air observation data.

Since these observations are incorporated into the ETA analysis fields this

step is considered redundant but may further reduce error in the analysis

field. RAWINS is applied to enable surface nudging of soil moisture and

temperature in the Pleim-Xiu land surface module. NCEP ADP surface (ds

464.0) and upper air (ds 353.1 and ds 353.4) data are the appropriate data

to input into LITTLE R and/or RAWINS to create SFCFDDA for surface

nudging.

The INTERPF processor takes the REGRIDDER/LITTLE R output that

is at standard pressure levels and interpolates that data to the vertical grid

defined by the user. The vertical grid is defined in terms of sigmas, where

1 is the surface and 0 is the top of the model atmosphere. The top of the

model domain is 100 millibars, which is approximately 15 kilometers above

ground level.

The vertical atmosphere is resolved to 34 layers, with thinner layers in

the planetary boundary layer (PBL). The surface layer is approximately 38

meters in height. The layer configuration is selected to capture the important

diurnal variations in the boundary layer while also having layers in the upper

troposphere to resolve deep cloud formation.

4
February 2, 2009

Figure 2: Vertical layer structure and depth (m)

2.3 Model Options and Execution

When meteorological models are applied to replicate past events, the use

of data assimilation helps to ”nudge” solutions so that they do not diverge

greatly from the actual observed meteorological fields. This is one of the

major benefits of using dynamic meteorological models since they can pro5
February 2, 2009

vide a way of consistently characterizing meteorological conditions at times

and locations where observations do not exist while at the same time still

accounting indirectly for actual observations.

Three dimensional analysis nudging for temperature and moisture is applied above the boundary layer only. Analysis nudging for the wind field is

applied above and below the boundary layer. Analysis nudging is not performed on the rotational wind field. In addition, the observation nudging

flag is turned off. This type of nudging is most appropriate when there is

a very dense set of observation data from a field study, which this application lacked. The 36 km domain nudging weighting factors are 3.0 x 10−4

for wind fields and temperatures and 1.0 x 10−5for moisture fields. The

12 km domain nudging weighting factors are 1.0 x 10−4for wind fields and

temperatures and 1.0 x 10−5for moisture fields. Important physics options

used are listed below.

• Pleim-Xiu PBL and land surface schemes

• Kain-Fritsh 2 cumulus parameterization

• Reisner 2 mixed phase moisture scheme

• RRTM longwave radiation scheme

• Dudhia shortwave radiation scheme

Atmospheric radiation is calculated every 15 minutes in the model. Vertical moisture and temperature advection are set to use linear interpolation.

Other important variables switched to ON include: moist vertical diffusion in

clouds, temperature advection using potential temperature, diffusion using

perturbation temperature, 3D coriolis force, and upper radiative boundary

condition. Sea surface temperature and snow cover are set to vary with time.

The Pleim-Xiu land surface module requires that additional variables be

set in the MM5 deck: ISMRD and NUDGE. ISMRD is set to use soil moisture

and soil temperature fields from the ETA analyses. NUDGE is set to nudge

soil moisture data to the analyses fields for the first 5 day MM5 simulation.

All subsequent simulations have soil temperature and moisture initialized

from the end of the previous 5 day simulation.

6
February 2, 2009

MM5 was executed in 5.5 day blocks (7920 minute simulation) initiated at

12Z with a 90 second time step. The 12 km domain was run for the entire

calendar year of 2005.

3 Model Performance Evaluation

One of the objectives of this evaluation is to determine if the meteorological model output fields represent a reasonable approximation of the actual

meteorology that occurred during the modeling period. A second objective

is to identify and quantify the existing biases and errors of the meteorological predictions in order to allow for a downstream assessment of how the

air quality modeling results are affected by issues associated with the meteorological data. Performance results are presented to allow those using this

data to determine the adequecy of the model simulation for their particular

needs.

The observation database for temperature, wind speed, wind direction,

and mixing ratio is based on the Meteorological Assimilation Data Ingest

System (MADIS) available from the National Oceanic and Atmospheric Administration (NOAA) Earth System Research Laboratory (ESRL) Global

Systems Division (GSD). The MADIS database is quality controlled and

made available by file transfer protocol [National Oceanic and Atmospheric

Administration, 2008].

Rainfall observation analysis data is available from the National Weather

Service Climate Prediction Center (CPC) on an hourly basis for the Continental United States [National Weather Service, 2008]. The rainfall analysis

resolution is 0.25 degree longitude by 0.25 degree latitude (approximately 40

km by 40 km) and extends from 140W to 60W and 20N to 60N. The CPC

rainfall analysis data does not include any portion of Canada, Mexico, or

anywhere off-shore of the United States.

Shortwave downward radiation measurements are taken at SURFRAD and

ISIS monitor locations [Earth System Research Laboratory, 2008b] [Earth

System Research Laboratory, 2008a]. The SURFRAD network consists of 7

sites and the ISIS network consists of 8 sites across the United States.

7
February 2, 2009

Figure 3: SURFRAD (black dots) and ISIS (red dots) monitor locations

Model performance is described using quantitative metrics: mean bias and

mean (gross) error [Boylan et al., 2006]. These metrics are useful because

they describe model performance in the measured units of the meteorological variable. Performance is best when these metrics approach 0. Rainfall

performance is examined qualitatively with side-by-side monthly total rainfall plots. The MM5 model outputs predictions approximately 15 meters

above the surface while observations are at 10 meters. MM5 outputs nearinstantaneous values (90 second time step) as opposed to the values with

longer averaging times taken at monitor stations. This should be considered

when interpreting model performance metrics.

8
February 2, 2009

Figure 4: Model performance regions

Performance metrics are aggregated over spatial areas to help determine

how well the model is performing in different parts of the domain. This

aggregation also aids the evaluation process since the examination of all

individual monitor locations is resource intensive. Metrics are aggregated

over 10 regions: northeast, mid-atlantic, Florida, south, great lakes, midwest,

rockies-north, rockies-south, northwest, and California.

3.1 Performance Discussion

The entire eastern United States has minimal temperature bias. An underprediction bias for temperature is seen in the northeast and Great Lakes

region during the winter months. The wind speed bias is also minimal for the

eastern United States with a very slight tendency toward under-prediction

in the Midwest in the winter and Mid-Atlantic in the summer. Moisture

is over-predicted in most regions of the eastern United States, in particular

during the summer months. The observed spatial pattern of monthly total

rainfall is matched well by model estimates. The magnitude of estimated

rainfall is over-predicted in much of the eastern United States during the

9
February 2, 2009

summer months.

The upper Rockies region (defined as Montana, Idaho, and Wyoming) has

a fairly minimal temperature and moisture bias. The area shows a slight

cold bias in temperature during the winter and early spring months. Wind

speeds tend to be over-predicted in this region during the winter and early

spring months.

The Northwest region (defined as Washington and Oregon) has minimal

temperature bias in the cooler months and an under-prediction tendency in

the summer months. This region does not have a strong pattern of over or

under-prediction of wind speed by month. Wind speed error is low in the

sprint and summer months and peaks slightly during the colder months.

The lower Rockies (defined as Colorado, Utah, Nevada, Arizona, and New

Mexico) tend to have model estimates of temperature under-predict observations. Wind speed bias is minimal in the lower Rockies. Moisture is overpredicted in the spring and summer months in this region. Wind direction

error is also highest during the late spring and summer. This region also

shows a pronounced over-prediction of moisture during the summer months.

California has a high bias for wind speed during the winter months and

very early spring. The model tends to under-predict temperature in this

area. Moisture bias is fairly low all year in California and model estimates

are well correlated with observations.

The model simulation does well at estimating the spatial pattern of monthly

total rainfall during the winter, fall, and early spring. However, during

the late spring and summer the model seems to have large localized overpredictions of rainfall in the Rocky Mountain region. There may be some

incommensurability between the observation network in this part of the country but the model prediction pattern is likelyover-stating summer convective

rainfall in this part of the model domain.

Shortwave radiation observations are minimal, but in general the model

does not show strong systematic bias from month to month. A diurnal

10
February 2, 2009

trend in bias is noted in that shortwave downward radiation is usually underpredicted in the early morning and over-predicted in the late afternoon.

Maximum model estimated PBL heights are plotted to determine if unrealistically high mixing might be expected anywhere in the modeling domain

in a particular month. The highest estimated PBL heights are estimated

from April to September.

3.2 Temperature

Average monthly bias, error, and coefficient of determination are shown

by region. These plots illustrate the variability in model performance for

temperature at monitors in defined regions for each month.

Figure 5: Monthly averaged temperature bias (C)

11
February 2, 2009

Figure 6: Monthly averaged temperature error (C)

Figure 7: Monthly averaged coefficient of determination

12
February 2, 2009

3.3 Wind Field

Average monthly bias, error, and coefficient of determination are shown

for wind speed by region. These plots illustrate the variability in model

performance for wind speed at monitors in defined regions for each month.

The error metric is shown for wind direction since bias and correlation have

little meaning for a variable expressed in compass degrees.

Figure 8: Monthly averaged wind speed bias (m/s)

13
February 2, 2009

Figure 9: Monthly averaged wind speed error (m/s)

Figure 10: Monthly averaged wind speed coefficient of determination

14
February 2, 2009

Figure 11: Monthly averaged wind direction error (degrees)

3.4 Moisture

Average monthly bias, error, and coefficient of determination are shown by

region. These plots illustrate the variability in model performance for mixing

ratio at monitors in defined regions for each month.

15
February 2, 2009

Figure 12: Monthly averaged mixing ratio bias (g/kg)

Figure 13: Monthly averaged mixing ratio error (g/kg)

16
February 2, 2009

Figure 14: Monthly averaged coefficient of determination

3.5 Rainfall

Monthly total rainfall is plotted for each grid cell to assess how well the

model captures the spatial variability and magnitude of convective and nonconvective rainfall events.

17
February 2, 2009

(a) Observation (b) Model

Figure 15: January monthly total rainfall (inches)

(a) Observation (b) Model

Figure 16: February monthly total rainfall (inches)

18
February 2, 2009

(a) Observation (b) Model

Figure 17: March monthly total rainfall (inches)

(a) Observation (b) Model

Figure 18: April monthly total rainfall (inches)

19
February 2, 2009

(a) Observation (b) Model

Figure 19: May monthly total rainfall (inches)

(a) Observation (b) Model

Figure 20: June monthly total rainfall (inches)

20
February 2, 2009

(a) Observation (b) Model

Figure 21: July monthly total rainfall (inches)

(a) Observation (b) Model

Figure 22: August monthly total rainfall (inches)

21
February 2, 2009

(a) Observation (b) Model

Figure 23: September monthly total rainfall (inches)

(a) Observation (b) Model

Figure 24: October monthly total rainfall (inches)

22
February 2, 2009

(a) Observation (b) Model

Figure 25: November monthly total rainfall (inches)

(a) Observation (b) Model

Figure 26: December monthly total rainfall (inches)

3.6 Radiation

Shortwave downward radiation estimates are compared to surface based

measurements. Measurements of shortwave downward radiation are taken at

23
February 2, 2009

SURFRAD and ISIS monitor locations. Photosynthetically activated radiation (PAR) is a fraction of shortwave downward radiation and is an important

input for the biogenic emissions model for estimating isoprene. Isoprene emissions are important for regional ozone chemistry and play a role in secondary

organic aerosol formation. Radiation performance evaluation also gives an

indirect assessment of how well the model captures cloud formation during

daylight hours. Outliers are not plotted on these boxplots to emphasize

predominant features in model performance.

Figure 27: Shortwave downward radiation bias by month

24
February 2, 2009

Figure 28: Shortwave downward radiation bias by hour

3.7 PBL Height

Maximum PBL heigts are plotted for each grid cell to assess whether unrealistic stratospheric intrusion may occur in any of the simulated months.

25
February 2, 2009

(a) January (b) February (c) March

(d) April (e) May (f) June

(g) July (h) August (i) September

(j) October (k) November (l) December

Figure 29: Predicted monthly maximum PBL height (m)

26
February 2, 2009

4 Acknowledgements

This evaluation report would not be possible without the assistance of Lara

Reynolds of CSC, Rob Gilliam of EPA/ORD, Mike Rizzo of EPA/OAQPS,

and Adam Reff of EPA/OAQPS.

27
Bibliography

J.W. Boylan, T.M. Odman, J.G. Wilkinson, and A.G. Russell. Integrated assessment modeling of atmospheric pollutants in the southern Appalachian

Mountains: Part II. fine particulate matter and visibility. Journal of Air

and Waste Management Association, 56:12–22, 2006.

J. Dudhia. A Non-hydrostatic Version of the Penn State/NCAR Mesoscale

Model: Validation Tests and Simulation of an Atlantic Cyclone and Cold

Front. Mon. Wea. Rev., 121:1493–1513, 1993.

Earth System Research Laboratory. Integrated Surface Irradiance Study

(ISIS) Network, 2008a. URL http://www.ncdc.noaa.gov/oa/ncdc.html.

Earth System Research Laboratory. SURFRAD (Surface Radiation) Network, 2008b. URL http://www.srrb.noaa.gov/surfrad/index.html.

G. Grell, J. Dudhia, and D. Stauffer. A Description of the Fifth-Generation

Penn State NCAR Mesoscale Model. Technical Report TN-398-STR, 1994.

M. Johnson. Meteorological Modeling Protocol: IDNR 2002 Annual MM5

Application. Technical report, 2003.

D. McNally. Topical Report: A Comparison of MM5 Model Estimates for

February and July 2001 using Alternative Model Physics Options. Technical report, 2002.

National Centers for Environmental Prediction. GCIP NCEP Eta 212 model

output, 2008. URL http://dss.ucar.edu/datasets/ds609.2.

National Oceanic and Atmospheric Administration. Meteorological Assimilation Data Ingest System, 2008. URL http://madis.noaa.gov/.

28
February 2, 2009

National Weather Service. Climatic Prediction Center, 2008. URL

http://www.cpc.noaa.gov/products/precip/realtime/.

29
Appendices

30
February 2, 2009

1 Climatology Review

Annual temperatures were slightly above normal for much of the United

States in 2005. The region east of the Rocky Mountains to the northern plains

experienced annual temperatures much higher than normal. The southeast

had normal to slightly cooler temperatures than usual in 2005.

Figure 30: Annual temperature deviations from normal

Annual precipiation patterns for 2005 are drier that normal in the central

United States, from Texas north to Wisconsin. The northeast, north central

plains, and most of the western United States were wetter than normal for

annual precipitation.

31
February 2, 2009

Figure 31: Annual rainfall deviations from normal

32