1 

Meteorological Model Performance for 

Annual 2011 Simulation WRF v3.4 
2 

1. INTRODUCTION 

The Weather Research and Forecasting model (WRF) was applied for the entire year of 2011 to 

generate meteorological data to support emissions and photochemical modeling applications 

for this year. The WRF meteorological fields will be converted to air quality modeling input data 

and used to support assessments of ozone, PM2.5, visibility, and a variety of toxics. 

The WRF model was applied to a 36 km continental United States scale domain (36US) and a 12 

km continental United States scale domain (12US2). Both model simulations were initialized 

directly from meteorological analysis data. Model parameterizations and options outlined in 

this document were chosen based on a series of sensitivity runs performed by U.S. 

Environmental Protection Agency (USEPA) Office of Research and Development that provided 

an optimal configuration based on temperature, mixing ratio, and wind field. All WRF 

simulations were done by Computer Sciences Corporation (CSC) under contract to the USEPA. 

2. MODEL CONFIGURATION 

2.1 Configuration of the 36US Domain 

Version 3.4 of the WRF model, Advanced Research WRF (ARW) core (Skamarock, 2008) was 

used for generating the 2011 simulations1

. Selected physics options include Pleim-Xiu land 

surface model, Asymmetric Convective Model version 2 planetary boundary layer scheme, KainFritsch cumulus parameterization utilizing the moisture-advection trigger (Ma and Tan, 2009), 

Morrison double moment microphysics, and RRTMG longwave and shortwave radiation 

schemes (Gilliam and Pleim, 2010). 

The WRF model was initialized using the 12km North American Model (12NAM) analysis 

product provided by National Climatic Data Center (NCDC). Where 12NAM data was 

unavailable, the 40km Eta Data Assimilation System (EDAS) analysis (ds609.2) from the National 

Center for Atmospheric Research (NCAR) was used. Analysis nudging for temperature, wind, 

and moisture was applied above the boundary layer only. The model simulations were 

conducted in 5.5 day blocks with soil moisture and temperature carried from one block to the 

 

1

 Version 3.4 was the most current version of WRF at the time the 2011 meteorological model simulations were 

performed. 
3 

next via the ipxwrf program (Gilliam and Pleim, 2010). Landuse and land cover data were based 

on the 1992-1993 U.S. Geological Survey (USGS) data. 

Figure 2.1 shows the 36US domain, which utilizes a Lambert conformal projection centered at (-

97,40) with true latitudes of 33 and 45 degrees north. The domain contains 164 cells in the X 

direction and 128 cells in the Y direction. All cells are 36 km2

. As shown in Table 2.1, the 

atmosphere is resolved with 35 vertical layers up to 50 mb, with the thinnest layers being 

nearest the surface to better resolve the planetary boundary layer (PBL). 

WRF 

Layer 

Height 

(m) 

Pressure 

(mb) 

Sigma 

35 17,556 5000 0.000 

34 14,780 9750 0.050 

33 12,822 14500 0.100 

32 11,282 19250 0.150 

31 10,002 24000 0.200 

30 8,901 28750 0.250 

29 7,932 33500 0.300 

28 7,064 38250 0.350 

27 6,275 43000 0.400 

26 5,553 47750 0.450 

25 4,885 52500 0.500 

24 4,264 57250 0.550 

23 3,683 62000 0.600 

22 3,136 66750 0.650 

21 2,619 71500 0.700 

20 2,226 75300 0.740 

19 1,941 78150 0.770 

18 1,665 81000 0.800 

17 1,485 82900 0.820 

16 1,308 84800 0.840 

15 1,134 86700 0.860 

14 964 88600 0.880 

13 797 90500 0.900 

12 714 91450 0.910 

11 632 92400 0.920 

10 551 93350 0.930 

9 470 94300 0.940 

8 390 95250 0.950 

7 311 96200 0.960 

6 232 97150 0.970 
4 

5 154 98100 0.980 

4 115 98575 0.985 

3 77 99050 0.990 

2 38 99525 0.995 

1 19 99763 0.9975 

Surface 0 100000 1.000 

Table 2.1 WRF layers and their approximate height above ground level. 

Figure 2.1 Map of WRF model domain: 36US 

2.2 Configuration of the 12US2 Domain 

The 12km configuration is the same as the 36km domain with two exceptions: the Group for 

High Resolution Sea Surface Temperatures (GHRSST) (Stammer et al., 2003) 1km SST data was 

used to provide more resolved information compared to the more coarse data in the NAM 

analysis. Additionally, landuse and land cover data were based on the National Land Cover 

Database 2006 (NLCD 2006). Analysis nudging for temperature, wind, and moisture was applied 

above the boundary layer only. The model simulations were conducted in 5.5 day blocks with 

soil moisture and temperature carried from one block to the next via the ipxwrf program 

(Gilliam and Pleim, 2010). 

Figure 2.2 shows the 12US2 domain, which utilizes a Lambert conformal projection centered at 

(-97,40) with true latitudes of 33 and 45 degrees north. The domain contains 396 cells in the X 
5 

direction and 246 cells in the Y direction. All cells are 12 km2. The atmosphere is resolved with 

35 vertical layers up to 50 mb (see table 2.1), with the thinnest layers being nearest the surface 

to better resolve the PBL. 

Figure 2.2 Map of WRF model domain: 12US2. 

3 MODEL PERFORMANCE DESCRIPTION

The WRF model simulations were evaluated to determine whether the output fields represent a 

reasonable approximation of the actual meteorology that occurred during the modeling period. 

Identifying and quantifying these output fields allows for a downstream assessment of how the 

air quality modeling results are impacted by the meteorological data. For the purposes of this 

assessment, 2-meter temperature and mixing ratio, 10-meter wind speed and direction, and 

shortwave radiation are quantitatively evaluated. A qualitative evaluation of precipitation is 

also provided. 

The observation database for surface-based temperature, wind speed and direction, and mixing 

ratio is based on measurements made at United States (i.e., National Weather Service) and 

Canadian (i.e., Environment Canada) airports. The observational dataset (ds472 network) is 

available from NCAR. Monitors used for evaluation are shown in Figure 3.1. 
6 

Figure 3.1 Stations used for model performance: ds472 network.

Shortwave downward radiation measurements are taken at Surface Radiation Budget Network 

(SURFRAD) (http://www.srrb.noaa.gov/surfrad) and Integrated Surface Irradiance Study (ISIS) 

(http://www.srrb.noaa.gov/isis/index.html) monitor locations. The SURFRAD network consists 

of 7 sites and the ISIS network consists of 9 sites across the United States (see Figure 3.2). Both 

networks are operated by the National Oceanic and Atmospheric Administration (NOAA), with 

SURFRAD sites existing as a subset of ISIS monitors that provide higher level radiation 

information not used in this evaluation. 
7 

Figure 3.2. Location of ISIS and SURFRAD radiation monitors. 

Rainfall amounts are estimated by the Parameter-elevation Relationships on Independent 

Slopes Model (PRISM) model, which uses an elevation-based regression model to analyze 

precipitation. PRISM’s horizontal resolution is approximately 2 to 4 km and is re-projected to 

the WRF modeling domain for direct qualitative comparison to model estimates. The rainfall 

analysis is limited to the contiguous United States as the model utilizes elevation and measured 

precipitation data at automated weather stations. 

Model performance (i.e., temperature, wind speed, and mixing ratio) is described using 

quantitative metrics: mean bias, mean (gross) error, fractional bias, and fractional error (Boylan 

and Russell, 2006). These metrics are useful because they describe model performance in the 

measured units of the meteorological variable and as a normalized percentage. Since wind 

direction is reported in compass degrees, estimating performance metrics for wind direction is 

problematic as modeled and observed northerly winds may be similar but differences would 

result in a very large artificial bias. For example, the absolute difference in a northerly wind 

direction measured in compass degrees of 1° and 359° is 358° when the actual difference is only 

2°. To address this issue, wind field displacement, or the difference in the U and V vectors 

between modeled (M) and observed (O) values, is used to assess wind vector performance 

(Equation 1). Performance is best when these metrics approach 0. 

(1) Wind displacement (km) = (UM – UO + VM – VO)*(1 km/1000 m)*(3600 s/hr)*(1 hr) 
8 

Rainfall performance is examined spatially using side-by-side comparisons of monthly total 

rainfall plots. The WRF model outputs predictions approximately 15 meters above the surface 

while observations are at 10 meters. WRF generates output at near instantaneous values (90 

second time step) as opposed to longer averaging times taken at monitor stations. This should 

be considered when interpreting model performance metrics. 

3.1 Model Performance for Winds 

WRF-predicted wind speed estimates are compared to surface-based measurements made in 

the ds472 network described earlier. The results for the 36US (Figure 3.1.1) and 12US2 (Figure 

3.1.2) domains are shown below. 

At 36km, wind speeds are generally overpredicted across most hours of the day for all seasons, 

in terms of mean bias. In general, performance improves at 12km, with less overprediction. 

However, at 12km WRF tends to slightly overpredict wind speeds in the early morning and 

afternoon hours, while slightly underpredicting wind speeds in the late evening and overnight 

hours. There is no significant seasonal variability at either resolution in terms of wind speed. 

The monthly spatial distributions of the wind speed biases (m/s) for all hours (Figures 3.1.3-

3.1.6) and daytime hours2 (Figures 3.1.7-3.1.10) are also presented. No appreciable difference is 

observed in the biases for daytime hours versus all hours. However, WRF tends to slightly 

overpredict wind speeds for areas in the eastern US and underpredicts wind speeds in the 

western US, particularly southern California. As noted above, these biases persist regardless of 

changes in season. 

 

2

 12UTC to 00UTC 
9 

Figure 3.1.1. Distribution of hourly bias by hour and hourly bias, error, fractional bias, and 

fractional error for wind speed by month for 36US domain. 
10 

Figure 3.1.2. Distribution of hourly bias by hour and hourly bias, error, fractional bias, and 

fractional error for wind speed by month for 12US2 domain. 
11 

Figure 3.1.3. Spatial distribution of wind speed bias (m/s) across all hours for the months of 

January, February, and March (top to bottom) for the 12US2 domain. 
12 

Figure 3.1.4. Spatial distribution of wind speed bias (m/s) across all hours for the months of 

April, May, and June (top to bottom) for the 12US2 domain. 
13 

Figure 3.1.5. Spatial distribution of wind speed bias (m/s) across all hours for the months of 

July, August, and September (top to bottom) for the 12US2 domain. 
14 

Figure 3.1.6. Spatial distribution of wind speed bias (m/s) across all hours for the months of 

October, November, and December (top to bottom) for the 12US2 domain. 
15 

Figure 3.1.7. Spatial distribution of wind speed bias (m/s) across daytime hours for the months 

of January, February, and March (top to bottom) for the 12US2 domain. 
16 

Figure 3.1.8. Spatial distribution of wind speed bias (m/s) across daytime hours for the months 

of April, May, and June (top to bottom) for the 12US2 domain. 
17 

Figure 3.1.9. Spatial distribution of wind speed bias (m/s) across daytime hours for the months 

of July, August, and September (top to bottom) for the 12US2 domain. 
18 

Figure 3.1.10. Spatial distribution of wind speed bias (m/s) across daytime hours for the months 

of October, November, and December (top to bottom) for the 12US2 domain. 
19 

Wind vector displacement (km) is presented below for the 36US (Figure 3.1.11) and 12US2 

(Figure 3.1.12) domains utilizing the ds472 observation network described earlier. These plots 

show the entire distribution of hourly wind displacement by month and by hour of the day. 

Overall, model performance is adequate in terms of wind vector differences. Both the 36- and 

12-km simulations have a mean wind displacement of around 5km. Since this difference is less 

than the horizontal resolution, negligible impacts due to wind displacement are expected. 
20 

Figure 3.1.11. Distribution of hourly wind displacement by hour and month for the 36US 

domain. 
21 

Figure 3.1.12. Distribution of hourly wind displacement by hour and month for the 12US2 

domain. 
22 

3.2 Temperature 

Temperature estimates are compared to the ds472 observation network described earlier and 

are presented below for the 36US (Figure 3.2.1) and 12US2 (Figure 3.2.2) domains. 

Overall, WRF slightly underpredicts surface temperature at both 36- and 12-km for most hours, 

with a slight overprediction in the early morning hours. In the summer months (June, July, and 

August), there appears to be less variability in both simulations, with the inner quartile range 

(IQR) more closely centered around zero in both simulations. Overall, with an average IQR of +/- 

2 degrees, this is considered good model performance. 

In Figures 3.2.3-3.2.6 and 3.2.7-3.2.10 the monthly spatial distributions of the temperature bias 

for the 12km simulation is presented for all hours and daytime only, respectively. Overall, a 

persistent slight underprediction of temperature is noted for most months. During daytime 

hours, a more significant underprediction of temperature is noted across much of the central 

and eastern US when compared to all hours. In areas of the western US, there is a persistent 

slight overprediction of temperature, regardless of season. 
23 

Figure 3.2.1. Distribution of hourly bias by hour and hourly bias, error, fractional bias, and 

fractional error for temperature by month for the 36US domain. 
24 

Figure 3.2.2. Distribution of hourly bias by hour and hourly bias, error, fractional bias, and 

fractional error for temperature by month for the 12US2 domain. 
25 

Figure 3.2.3. Spatial distribution of temperature bias (C) across all hours for the months of 

January, February, and March (top to bottom) for the 12US2 domain. 
26 

Figure 3.2.4. Spatial distribution of temperature bias (C) across all hours for the months of April, 

May, and June (top to bottom) for the 12US2 domain.
27 

Figure 3.2.5. Spatial distribution of temperature bias (C) across all hours for the months of July, 

August, and September (top to bottom) for the 12US2 domain. 
28 

Figure 3.2.6. Spatial distribution of temperature bias (C) across all hours for the months of 

October, November, and December (top to bottom) for the 12US2 domain. 
29 

Figure 3.2.7. Spatial distribution of temperature bias (C) across daytime hours for the months of 

January, February, and March (top to bottom) for the 12US2 domain. 
30 

Figure 3.2.8. Spatial distribution of temperature bias (C) across daytime hours for the months of 

April, May, and June (top to bottom) for the 12US2 domain. 
31 

Figure 3.2.9. Spatial distribution of temperature bias (C) across daytime hours for the months of 

July, August, and September (top to bottom) for the 12US2 domain. 
32 

Figure 3.2.10. Spatial distribution of temperature bias (C) across daytime hours for the months 

of October, November, and December (top to bottom) for the 12US2 domain. 
33 

3.3 Mixing Ratio 

Water mixing ratio estimates are compared to the ds472 observation network described earlier 

and are presented below for the 36US (Figure 3.3.1) and 12US2 (Figure 3.3.2) domains. 

In either simulation, no significant positive or negative bias is observed. However, WRF tends to 

be slightly drier in the early afternoon hours relative to the rest of the day. Additionally, there is

more uncertainty in model predictions during the spring and summer months. This increase in 

error is explained by the increased convective activity and influx of moist air masses that are 

typical of that time of year. In general, WRF performance was adequate for water vapor mixing 

ratio. 

The monthly spatial distributions of the mixing ratio bias for the 12km simulation are shown in 

Figures 3.3.3-3.3.6 (all hours) and 3.3.7-3.3.10 (daytime). Little appreciable difference is 

observed in the biases either across all hours or just daytime. This is to be expected since water 

vapor mixing ratio has less temporal variability when compared to other variables (i.e., 

temperature). In the central and western US, mixing ratio is generally underpredicted across all 

months of the year, whereas a slight overprediction is observed in the eastern states for most 

months. In October and November, WRF exhibits a general underprediction for most locations 

across the country. 
34 

Figure 3.3.1. Distribution of hourly bias by hour and hourly bias, error, fractional bias, and 

fractional error for water vapor mixing ratio by month for the 36US domain. 
35 

Figure 3.3.2. Distribution of hourly bias by hour and hourly bias, error, fractional bias, and 

fractional error for water vapor mixing ratio by month for the 12US2 domain. 
36 

Figure 3.3.3. Spatial distribution of water vapor mixing ratio bias (g/kg) across all hours for the 

months of January, February, and March (top to bottom) for the 12US2 domain. 
37 

Figure 3.3.4. Spatial distribution of water vapor mixing ratio bias (g/kg) across all hours for the 

months of April, May, and June (top to bottom) for the 12US2 domain. 
38 

Figure 3.3.5. Spatial distribution of water vapor mixing ratio bias (g/kg) across all hours for the 

months of July, August, and September (top to bottom) for the 12US2 domain. 
39 

Figure 3.3.6. Spatial distribution of water vapor mixing ratio bias (g/kg) across all hours for the 

months of October, November, and December (top to bottom) for the 12US2 domain. 
40 

Figure 3.3.7. Spatial distribution of water vapor mixing ratio bias (g/kg) across daytime hours 

for the months of January, February, and March (top to bottom) for the 12US2 domain. 
41 

Figure 3.3.8. Spatial distribution of water vapor mixing ratio bias (g/kg) across daytime hours 

for the months of April, May, and June (top to bottom) for the 12US2 domain. 
42 

Figure 3.3.9. Spatial distribution of water vapor mixing ratio bias (g/kg) across daytime hours 

for the months of July, August, and September (top to bottom) for the 12US2 domain. 
43 

Figure 3.3.10. Spatial distribution of water vapor mixing ratio bias (g/kg) across daytime hours 

for the months of October, November, and December (top to bottom) for the 12US2 domain. 
44 

3.4 Precipitation 

Monthly total rainfall is plotted for each grid cell to assess how well the model captures the 

spatial variability and magnitude of convective and non-convective rainfall. As described earlier, 

the PRISM estimations for rainfall are only within the continental United States. WRF rainfall 

estimates by month are shown for all grid cells in the domain. Monthly total estimates are 

shown for the 36US domain (Figures 3.4.1 through 3.4.4) and 12US2 domain (Figures 3.4.5 

through 3.4.8). 

In general, WRF performs adequately in terms of the spatial patterns and magnitude of 

precipitation across the US throughout the year. Both simulations, however, tend to 

overestimate precipitation in elevated terrain (e.g., northern CA and the Pacific Northwest). The 

12km simulation tends to generate slightly higher precipitation amounts compared to the 36km 

simulation, and at times generates amounts higher than observed values (e.g., the southeast in 

July). Isolated amounts of overpredicted or underpredicted precipitation in the summer months 

relative to the observations is likely due to uncertainty in the convective parameterization 

scheme utilized. 
45 

Figure 3.4.1. PRISM analysis (left) and WRF (right) estimated monthly total rainfall (in) for 

January, February, and March. 
46 

3.4.2. PRISM analysis (left) and WRF (right) estimated monthly total rainfall (in) for April, May, 

and June. 
47 

Figure 3.4.3. PRISM analysis (left) and WRF (right) estimated monthly total rainfall (in) for July, 

August, and September. 
48 

Figure 3.4.4. PRISM analysis (left) and WRF (right) estimated monthly total rainfall (in) for 

October, November, and December. 
49 

Figure 3.4.5. PRISM analysis (left) and WRF (right) estimated monthly total rainfall (in) for 

January, February, and March. 
50 

Figure 3.4.6. PRISM analysis (left) and WRF (right) estimated monthly total rainfall (in) for April, 

May, and June. 
51 

Figure 3.4.7. PRISM analysis (left) and WRF (right) estimated monthly total rainfall (in) for July, 

August, and September. 
52 

Figure 3.4.8. PRISM analysis (left) and WRF (right) estimated monthly total rainfall (in) for 

October, November, and December. 

3.5 Solar Radiation 

Photosynthetically activated radiation (PAR) is a fraction of shortwave downward radiation and 

is an important input for the biogenic emissions model for estimating isoprene (Carlton and 

Baker, 2011). Isoprene emissions are important for regional ozone chemistry and play a role in 

secondary organic aerosol formation. Radiation performance evaluation also gives an indirect 

assessment of how well the model captures cloud formation during daylight hours. 
53 

Shortwave downward radiation estimates are compared to surface based measurements made 

at SURFRAD and ISIS network monitors for the 36US (Figure 3.6.1) 12US2 (Figure 3.6.2) 

domains. 

Overall, both the 36- and 12km simulations show WRF has little bias in shortwave radiation 

predictions during the fall and winter months. Biases tend to grow during the spring and peak in 

the summer, though the spread in overpredictions tends to be less than 100 W/m2

 on average, 

with a median bias close to zero. 

More variability is noted on an hourly basis. WRF tends to overpredict early morning to early 

afternoon shortwave radiation, while underpredicting the late afternoon and early evening 

values. The median overprediction at the time of greatest incoming solar radiation is near 100 

W/m2. In the late afternoon and evening hours, the median bias is close to -50 W/m2. These 

errors are likely attributable to the model being unable to accurately simulate cloud features at 

subgrid (<12km) scales. This assumption is based on the slight improvement in predictions at 

12km versus 36km. 
54 

Figure 3.5.1. Distribution of hourly bias for shortwave radiation (W/m2) by month (top) and by 

hour of the day (bottom) for the 36US domain. 
55 

Figure 3.5.2. Distribution of hourly bias for shortwave radiation (W/m2

) by month (top) and by 

hour of the day (bottom) for the 12US2 domain. 

4 CLIMATE REPRESENTATIVENESS OF 2011 

Figures 4.1 and 4.2 show the divisional rankings for observed temperatures across the US for 

2011. A climatic representation of the precipitation for 2011 is shown in Figures 4.3 and 4.4. 

These plots are useful in determining the representativeness of 2011 in terms of certain 

climatological variables compared to historical averages. 

Temperatures in 2011 were average to above average for most of the central and eastern US 

throughout the spring and summer months. Some areas in the southern and southeastern 

portions of the country exhibited near record warmth from June through August. Conversely, 

February through July tended to be cooler than average for the western US with near-record 

cold exhibited in the Pacific Northwest. 
56 

The spring and summer months experienced below average precipitation for much of the 

southern and southeastern US, whereas wetter conditions than average were experienced 

across the northern tier states. During the fall and winter months, most of the eastern US 

experienced average to above average precipitation, while the western and southern US 

remained generally below average. 
57 

Figure 4.1 Climatic temperature rankings by climate division: January to June 2011. 

http://www.ncdc.noaa.gov/temp-and-precip/maps.php
58 

Figure 4.2 Climatic temperature rankings by climate division: July to December 2011. 

http://www.ncdc.noaa.gov/temp-and-precip/maps.php
59 

Figure 4.3 Climatic rainfall rankings by climate division: January to June 2011. 

http://www.ncdc.noaa.gov/temp-and-precip/maps.php
60 

Figure 4.4 Climatic rainfall rankings by climate division: July to December 2011. 

http://www.ncdc.noaa.gov/temp-and-precip/maps.php
61 

5 REFERENCES 

Boylan, J.W., Russell, A.G., 2006. PM and light extinction model performance metrics, goals, and 

criteria for three-dimensional air quality models. Atmospheric Environment 40, 4946-4959. 

Carlton, A.G., Baker, K.R., 2011. Photochemical Modeling of the Ozark Isoprene Volcano: 

MEGAN, BEIS, and Their Impacts on Air Quality Predictions. Environmental Science & 

Technology 45, 4438-4445. 

Cooper, O.R., Stohl, A., Hubler, G., Hsie, E.Y., Parrish, D.D., Tuck, A.F., Kiladis, G.N., Oltmans, S.J., 

Johnson, B.J., Shapiro, M., Moody, J.L., Lefohn, A.S., 2005. Direct Transport of Midlatitude 

Stratospheric Ozone into the Lower Troposphere and Marine Boundary Layer of the Pacific 

Ocean. Journal of Geophysical Research – Atmospheres 110, D23310, 

doi:10.1029/2005JD005783. 

ENVIRON, 2008. User's Guide Comprehensive Air Quality Model with Extensions. ENVIRON 

International Corporation, Novato. 

Gilliam, R.C., Pleim, J.E., 2010. Performance Assessment of New Land Surface and Planetary 

Boundary Layer Physics in the WRF-ARW. Journal of Applied Meteorology and Climatology 49, 

760-774. 

Langford, A.O., Reid, S.J., 1998. Dissipation and Mixing of a Small-Scale Stratospheric Intrusion 

in the Upper Troposphere. Journal of Geophysical Research 103, 31265-31276. 

Otte, T.L., Pleim, J.E., 2010. The Meteorology-Chemistry Interface Processor (MCIP) for the 

CMAQ modeling system: updates through MCIPv3.4.1. Geoscientific Model Development 3, 

243-256. 

Skamarock, W.C., Klemp, J.B., Dudhia, J., Gill, D.O., Barker, D.M., Duda, M.G., Huang, X., Wang, 

W., Powers, J.G., 2008. A Description of the Advanced Research WRF Version 3. 

Stammer, D., F.J. Wentz, and C.L. Gentemann, 2003, Validation of Microwave Sea Surface 

Temperature Measurements for Climate Purposes, J. Climate, 16, 73-87.