Revised Report 

2002 ANNUAL MM5 SIMULATION 

TO SUPPORT WRAP CMAQ VISIBILITY MODELING 

FOR THE SECTION 308 SIP/TIP: 

MM5 Sensitivity Simulations to Identify a More Optimal 

MM5 Configuration for Simulating Meteorology 

in the Western United States 

Prepared for 

The Western Regional Air Partnership 

www.wrapair.org 

Prepared by 

Sue Kemball-Cook 

Yiqin Jia 

Chris Emery 

Ralph Morris 

ENVIRON International Corporation 

101 Rowland Way, Suite 220 

Novato, CA 94945-5010 

Zion Wang 

Gail Tonnesen 

University of California at Riverside 

College of Engineering 

Center for Environmental Research and Technology 

Riverside, CA 92507 

December 10, 2004 

 101 Rowland Way, Suite 220, Novato, CA 94945 415.899.0700 
December 2004 

i 

TABLE OF CONTENTS 

EXECUTIVE SUMMARY .................................................................................................... ES-1 

1. INTRODUCTION.......................................................................................................... 1-1 

 Background...................................................................................................................... 1-1 

 Overview of Approach..................................................................................................... 1-3 

2. ADDITIONAL MM5 SENSITIVTY TESTS .............................................................. 2-1 

Analysis Procedures......................................................................................................... 2-1

MM5 Sensitivity Test Results.......................................................................................... 2-3 

3. SUMMARY OF 36-KM RESULTS ............................................................................. 3-1 

Comparison to Previous Simulations............................................................................... 3-1 

Evaluation of Other Subdomains..................................................................................... 3-6 

Summary.......................................................................................................................... 3-8 

4. SUMMARY OF 12-KM SENSITIVITY TESTS ........................................................ 4-1 

Cumulus Parameterization Tests...................................................................................... 4-1 

Soil Moisture Nudging Tests ........................................................................................... 4-6 

REFERENCES.......................................................................................................................... R-1 

TABLES 

Table 1-1. MM5 Configurations from original (Run 0) and modified 

 (Run 5) MM5 runs. .............................................................................................. 1-2 

Table 2-1. Summary of additional MM5 sensitivity tests..................................................... 2-2 

Table 2-2. METSTAT subdomain abbreviations.................................................................. 2-3 

Table 3-1. Surface model performance evaluation statistics summary................................. 3-6 

Table 4-1. MM5 configuration for the 12-km grid for cumulus sensitivity tests.................. 4-2 

Table 4-2. Summary of cumulus sensitivity configurations.................................................. 4-2 

Table 4-3. Physics options common to all 12-km domain soil moisture sensitivity tests..... 4-6 

Table 4-4. Physics options varied for 12-km domain sensitivity tests.................................. 4-6 

FIGURES 

Figure 2-1. Definition of Metstat subdomains........................................................................ 2-3 

Figure 2-2a. Wind scatterplot for cumulus sensitivity test....................................................... 2-5 

Figure 2-2b. Temperature scatterplot for cumulus sensitivity test. .......................................... 2-5 

Figure 2-2c. Humidity scatterplot for cumulus sensitivity test. ............................................... 2-6 
December 2004 

ii 

Figure 2-3. Precipitation comparison for the cumulus parameterization 

 sensitivity test. Run 2a: Betts-Miller, Run 2b:Grell, 

 Run 5: Kain-Fritsch II.......................................................................................... 2-6 

Figure 2-4a. Wind scatterplot for LSM/PBL sensitivity tests .................................................. 2-8 

Figure 2-4b. Temperature scatterplot for LSM/PBL sensitivity tests....................................... 2-8 

Figure 2-4c. Humidity scatterplot for LSM/PBL sensitivity tests............................................ 2-9 

Figure 2-5. Precipitation comparison for the LSM/PBL parameterization 

 sensitivity tests. Run 2a: PX/ACM, 

 Run 3b:5-layer/M-MRF, Run 1ba: NOAH/M-MRF. ........................................ 2-10 

Figure 2-6a. Wind speed scatterplot for FDDA sensitivity test. ............................................ 2-11 

Figure 2-6b. Wind speed/direction scatterplot for FDDA sensitivity test.............................. 2-12 

Figure 2-6c. Temperature scatterplot for FDDA sensitivity test............................................ 2-12 

Figure 2-6d. Humidity scatterplot for FDDA sensitivity test................................................. 2-13 

Figure 2-7a. Wind scatterplot for NOAH/M-MRF nudging sensitivity test. ......................... 2-13 

Figure 2-7b. Temperature scatterplot for NOAH/M-MRF nudging 

 sensitivity test..................................................................................................... 2-14 

Figure 2-7c. Humidity scatterplot for NOAH/M-MRF nudging sensitivity test.................... 2-14 

Figure 2-8. Precipitation comparison for the FDDA sensitivity tests. 

 Observed (top right) Run 2a: PX/ACM (top left), Run 3b: 

 5-layer/M-MRF (bottom left) and, Run 1ba: NOAH/M-MRF (bottom right)... 2-15 

Figure 2-9a. Scatterplots for seasonal wind comparisons for Run 2ae .................................. 2-17 

Figure 2-9b. Scatterplots for seasonal wind comparison for Run 1bb ................................... 2-17 

Figure 2-9c. Temperature scatterplot for Run 2ae seasonal test. ........................................... 2-18 

Figure 2-9d. Temperature scatterplot for Run 1bb seasonal test............................................ 2-18 

Figure 2-9e. Humidity scatterplot for Run 2ae seasonal test.................................................. 2-19 

Figure 2-9f. Humidity scatterplot for Run 1bb seasonal test. ................................................ 2-19 

Figure 2-10a. CPC observed precipitation for the 5-day modeled 

 segment in each season. ..................................................................................... 2-20 

Figure 2-10b. Seasonal precipitation test for Run 2ae (PX)..................................................... 2-21 

Figure 2-10c. Seasonal precipitation test for Run 1bb (M-MRF). ........................................... 2-22 

Figure 3-1a. Wind speed scatterplot for comparison of original, interim, 

 and final MM5 runs ............................................................................................. 3-2 

Figure 3-1b. Wind direction scatterplot for comparison of original, interim, 

 and final MM5 runs. ............................................................................................ 3-2 

Figure 3-1c. Temperature scatterplot for comparison of original, interim, 

 and final MM5 runs. ............................................................................................ 3-3 

Figure 3-1d. Humidity scatterplot for comparison of original, interim, and 

 final MM5 runs. ................................................................................................... 3-3 

Figure 3-2a. July 1-5 MM5 predicted versus observed surface temperature 

 time series for Run 0............................................................................................ 3-4 

Figure 3-2b. July 1-5 MM5 predicted versus observed surface temperature 

 time series for Run 2ae. ....................................................................................... 3-4 

Figure 3-3. Precipitation comparison for July for original, interim, and 

 final MM5 runs. ................................................................................................... 3-5 

Figure 3-4a. Wind scatterplot for comparison of original, interim, and final 

 MM5 runs for subdomains outside WRAP.......................................................... 3-7 

Figure 3-4b. Temperature scatterplot for comparison of original, interim, and 

 final MM5 runs for subdomains outside WRAP. ................................................ 3-7 
December 2004 

iii 

Figure 3-4c. Humidity scatterplot for comparison of original, interim, 

 and final MM5 runs for subdomains outside WRAP........................................... 3-8 

Table 4-1a. Wind speed scatterplot for comparison of 12-km cumulus test runs................... 4-3 

Table 4-1b. Temperature scatterplot for comparison of 12-km cumulus test runs ................. 4-3 

Table 4-1c. Humidity scatterplot for comparison of 12-km cumulus test runs ...................... 4-4 

Figure 4-2. Precipitation comparison for the cumulus parameterization sensitivity test ....... 4-5 

Figure 4-3a. Time series of observed and MM5 surface wind, temperature 

 and humidity for the SW domain......................................................................... 4-8 

Figure 4-3b. Time series of observed and MM5 surface wind, temperature 

 and humidity for the DesertSW domain. ............................................................. 4-9

Figure 4-3c. Time series of observed and MM5 surface wind, temperature 

 and humidity for the PacificNW domain........................................................... 4-10 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\EXECUTIVE SUMMARY_revsd.doc ES-1 

EXECUTIVE SUMMARY 

The WRAP Regional Modeling Center (RMC) is in the process of performing the analyses 

needed for the Section 308 SIPs/TIPs due in 2007/2008. The first Section 308 SIP will need to 

project visibility from 2000-2004 to 2018 to demonstrate progress toward natural conditions to 

be achieved by 2064. Visibility is projected using mathematical models to simulate emissions, 

meteorology and air quality. The RMC is utilizing the EPA’s Models-3 Community Multi-scale 

Air Quality (CMAQ) modeling system to simulate particulate matter (PM) from which visibility 

degradation is obtained. CMAQ will be applied for the entire year of 2002 on two grids: a 

continental-scale domain with 36 km grid spacing, and a regional-scale domain with 12 km grid 

spacing covering the western United States. The Fifth Generation Mesoscale Model (MM5), 

developed and maintained by the Pennsylvania State University and National Center for 

Atmospheric Research (PSU/NCAR), is being used to develop hourly meteorological fields on 

these two grids. The CEP Sparse Matrix Operator Kernel Emissions (SMOKE) system is being 

used to develop the anthropogenic and biogenic gas and PM emission rate estimates for CMAQ. 

An MM5 Modeling Protocol has been prepared (ENVIRON and UCR, 2004), which describes in 

detail the MM5 model and the setup and evaluation methods used in the 2002 modeling effort. 

The model performance evaluation for an initial 2002 MM5 run on a single continental-scale 36-

km grid, and results from preliminary MM5 sensitivity tests, have been discussed by Morris et al. 

(2004a), Kemball-Cook et al. (2004), and Emery et al. (2004). This document describes 

additional MM5 sensitivity simulations performed to identify a more optimal MM5 

configuration for simulating meteorology on a 36-km grid in the western United States. Once a 

particular configuration for the 36-km simulation has been selected, further tests are described to 

assess whether the new configuration is appropriate for use at 12 km resolution. 

The initial 2002 36-km WRAP simulation results showed that MM5 performs better in the 

Central and Eastern U.S. than in the West, and performs generally better in winter than in 

summer. Specifically, in the western U.S., the amplitude of the diurnal surface temperature 

cycle was persistently underestimated during the summer, especially in the southwest. In the 

desert southwest, the surface humidity was greatly overestimated during the summer, and there 

was a pronounced cold bias. The excessive humidity and weak diurnal temperature cycle may be 

linked to the overprediction of summer precipitation by MM5, especially in the southwest. This 

can have serious repercussions for CMAQ modeling since too much rain can "wash out" 

pollutants, while the too cool, humid and cloudy environment may lead to incorrect pollutant 

chemistry and aerosol thermodynamics. 

A series of sensitivity tests was carried out with MM5 to converge upon a set of physics options 

which would ameliorate the above MM5 performance problems in the west. The more optimal 

model configuration for the 36-km application was: 

Analysis FDDA 

LSM PBL Cumulus Microphysics 3D Surface Obs FDDA

Pleim-Xiu ACM Betts-Miller Reisner II W/T/H W/T/H Wind 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\EXECUTIVE SUMMARY_revsd.doc ES-2 

MM5 runs for the test period of July 1-5, 2002 made with these physics options selected showed: 

• A dramatic reduction in the summertime cold, wet bias in the desert southwest; 

• Surface temperature and humidity performance now within benchmarks for all WRAP 

subdomains except desert southwest for temperature; 

• More accurate representation of the diurnal temperature cycle in the desert southwest; 

• Improvements in the index of agreement for temperature, humidity and wind speed for all 

WRAP subdomains; 

• A more realistic precipitation pattern over the western U.S.; 

• Better model performance in the eastern U.S.; and 

• Runs made with these physics options generally represented an improvement in 

performance throughout the year, making it unnecessary to select different physics 

schemes for different seasons. 

In summary, the new configuration of MM5 represents a significant improvement in model 

performance relative to the original 2002 36-km MM5 run. 

 

We then conducted a series of tests to determine the optimal MM5 configuration for the nested 

12-km grid of the WRAP modeling domain. In these tests, we varied only the physics options 

that we expected to ameliorate the biases in the WRAP region seen in the 36-km simulation. The 

over prediction of summer rainfall and associated surface humidity bias suggested that the 

cumulus parameterization might need to be changed. Another possible reason for the wet bias 

might be the soil moisture initialization. Therefore, we focused on sensitivity tests of the soil 

moisture specification and the cumulus scheme. 

The main results from the 12-km runs over the test period of July 1-5, 2002 are as follows: 

• Use of soil moisture nudging degrades temperature and humidity model performance. 

• Initializing the soil moisture with the 36-km domain MM5 output degrades temperature 

and humidity model performance. 

• Better temperature and humidity performance was obtained by initializing the soil model 

with the EDAS analysis fields. 

Based upon our evaluation of surface variable and rainfall performance, we selected a 

configuration with the Betts-Miller cumulus scheme running on the 36-km grid and no cumulus 

parameterization on the 12-km grid. This was among the best performers in terms of surface 

wind, temperature, and humidity, and performed the best for rainfall. We have also chosen to 

remove surface analysis nudging of temperature and humidity from both grids, based on 

guidance from reviewers that nudging to these variables can degrade performance aloft. 

This final 36/12-km configuration is now being used for the annual 2002 MM5 simulation to 

support WRAP’s regional haze modeling. 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec1_revsd.doc 1-1 

1. INTRODUCTION 

The U.S. Environmental Protection Agency (EPA) Regional Haze Rule (RHR) calls for state and 

federal agencies to work together to improve visibility in 156 Federally mandated Class I areas 

that include national parks and wilderness areas. The objective of the RHR is to achieve natural 

visibility conditions in these federally protected lands by the year 2064. The rule requires the 

states to develop and implement air quality protection plans to reduce the pollution that causes 

visibility impairment. The Western Regional Air Partnership (WRAP) is a consortium of 

federal, state, and tribal agencies charged with implementing regional planning processes to 

improve visibility in western US Class I areas. To meet this goal, WRAP is providing the 

necessary technical and policy tools to help states and tribes implement the RHR. 

The WRAP has formed the Regional Modeling Center (RMC), consisting of the University of 

California at Riverside (UCR), ENVIRON International Corporation, and the University of 

North Carolina (UNC). The RMC performed the modeling and analysis necessary to develop the 

Section 309 SIPs/TIPs and is in the process of performing the analyses needed for the Section 

308 SIPs/TIPs due in 2007/2008. The first Section 308 SIP will need to project visibility from 

2000-2004 to 2018 to demonstrate progress toward natural conditions to be achieved by 2064. 

Visibility is projected using mathematical models to simulate emissions, meteorology and air 

quality. The RMC is utilizing the EPA’s Models-3 Community Multi-scale Air Quality 

(CMAQ) modeling system to simulate particulate matter (PM) from which visibility degradation 

is obtained. CMAQ will be applied for the entire year of 2002 on two grids: a continental-scale 

domain with 36 km grid spacing, and a regional-scale domain with 12 km grid spacing covering 

the western United States. The Fifth Generation Mesoscale Model (MM5), developed by the 

Pennsylvania State University and maintained by National Center for Atmospheric Research 

(PSU/NCAR), is being used to develop hourly meteorological fields on these two grids. The 

CEP Sparse Matrix Operator Kernel Emissions (SMOKE) system is being used to develop the 

anthropogenic and biogenic gas and PM emission rate estimates for CMAQ. 

An MM5 Modeling Protocol has been prepared (ENVIRON and UCR, 2004), which describes in 

detail the MM5 model and the setup and evaluation methods used in the 2002 modeling effort. 

The model performance evaluation for an initial 2002 MM5 run on a single continental-scale 36-

km grid, and results from preliminary MM5 sensitivity tests, have been discussed by Morris et al. 

(2004a), Kemball-Cook et al. (2004), and Emery et al. (2004). This document describes 

additional MM5 sensitivity simulations performed to identify a more optimal MM5 

configuration for simulating meteorology on a 36-km grid in the western United States. Once a 

particular configuration for the 36-km simulation has been selected, further tests are described to 

assess whether the new configuration is appropriate for use at 12 km resolution. 

BACKGROUND 

MM5 simulations are being carried out by the WRAP Regional Modeling Center (RMC) for the 

entirety of 2002 to support CMAQ visibility modeling for the Section 308 SIP/TIP. During the 

fall of 2003, the RMC made an initial MM5 run for the year 2002 on the single unified National 

RPO 36-km grid. The model configuration for this run was based upon a prior 2002 MM5 

application undertaken by the Iowa Department of Natural Resources (IDNR; Johnson, 2003 

Personal Communication), which was in turn set according to the optimal MM5 physics options 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec1_revsd.doc 1-2 

that resulted from an in-depth sensitivity project carried out by IDNR and the Lake Michigan Air 

Directors Consortium (LADCo). While the IDNR simulations used MM5 version 3.5, the 

applications carried out for WRAP used the latest version of the model (v3.6.1) available at the 

end of 2003. Additional modifications to the physics configuration and application methodology 

were made for the WRAP simulation based on the latest information from EPA, IDNR, LADCo, 

and others. First, the RMC opted to use the Reisner II mixed-phase cloud microphysics package 

according to suggestions from EPA/ORD. Second, the INTERPPX option, which allows for 

continuous soil moisture initialization from one run segment to the next, was not used based on 

poor performance reported by IDNR and LADCo. 

The initial 2002 36-km WRAP simulation (referred to hereafter as Run 0) results showed that 

MM5 performs better in the Central and Eastern U.S. than in the West, and performs generally 

better in winter than in summer (Morris et al., 2004a; Kemball-Cook et al., 2004). In the western 

U.S., the amplitude of diurnal temperature cycle was persistently underestimated during the 

summer, especially in the southwest. In the desert southwest, the humidity was greatly 

overestimated during the summer as well, and there was a pronounced cold bias. Some of these 

problems appear to be linked to the excessive simulated precipitation generated by MM5 during 

the summer, especially in the southwest. This can have serious repercussions for CMAQ 

modeling since too much rain can “wash out” pollutants, while the too cool, humid and cloudy 

environment may lead to incorrect pollutant chemistry and aerosol thermodynamics. 

Temperature and humidity problems overshadowed the surface wind performance, which is not 

particularly good, but is likely affected by smaller-scale topographic influences that are not well 

represented even at the finer 12 km resolution. Wind performance improved quickly with height 

above the surface, suggesting that regional transport speeds/directions are reasonably 

represented. 

Initial sensitivity tests were undertaken for a 5-day period in July 2002 for the 36-km National 

domain in an attempt to find alternative MM5 configurations that would improve the poor 

summertime performance at 36 km. Furthermore, in-depth analyses were conducted to compare 

our performance against 36-km MM5 runs made for VISTAS, and against operational EDAS 

fields. From these analyses, we found that the use of the Kain-Fritsch II scheme (as used in the 

VISTAS 2002 MM5 modeling) improved, but did not entirely solve, the precipitation over 

prediction problem. We also found that removal of soil moisture nudging improved temperature 

and humidity performance for the short summertime tests. The evaluation of the EDAS fields 

used in the MM5 four-dimensional-data-assimilation (FDDA) revealed that EDAS did not 

exhibit the summer time cold wet bias in the southwest so the bias was not introduced by the 

FDDA. A new model configuration (Run 5) was identified from these initial tests for new 

annual 2002 simulations, and the WRAP MM5 modeling protocol was updated to reflect this, as 

shown in the Table 1-1. 

Table 1-1. MM5 Configurations from original (Run 0) and modified (Run 5) MM5 runs. 

LSM PBL Cumulus Microphysics 

Analysis FDDA 

3D Surface 

Obs 

FDDA 

Soil Moisture 

Nudging 

Run 0 PX ACM KF I Reisner II W/T/H W/T/H None Yes 

Run 5 PX ACM KF II Reisner II W/T/H W/T/H None No 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec1_revsd.doc 1-3 

Although the configuration identified in the new MM5 modeling protocol did improve humidity, 

temperature, and precipitation performance over the short July test period (especially where 

improvement was needed most), the desert southwest continued to exhibit unsatisfactory levels 

of precipitation and humidity. As stated above, these issues will likely play crucial roles in 

CMAQ performance. 

These initial MM5 sensitivity simulations attempted to identify a better performing MM5 

simulation that can be used to simulate all of 2002 but retain the Pleim-Xiu (PX) LSM and ACM 

PBL options. When results of the Run5 MM5 model performance were presented at the May 24-

25, 2004 National RPO modeling meeting in Denver, Colorado (Emery et al, 2004), there were 

still concerns about the 2002 MM5 model performance in the western US. Thus, WRAP 

requested that the Regional Modeling Center (RMC) perform further MM5 sensitivity test to 

identify a better performing configuration, including investigating alternative LSM/PBL 

configurations to PX/ACM and include the possibility of using different MM5 configurations for 

different times of year. 

OVERVIEW OF APPROACH 

We carried out additional 36-km MM5 test simulations in an attempt to further improve MM5 

performance. The latest available version of MM5 (v3.6.2) was used. Since the most severe 

problems for the initial WRAP 2002 MM5 annual simulation are the wet bias and precipitation 

overestimation in summer, we first investigated these issues for the 5-day July test period (July 

1-5, 2002) on the 36-km grid to maximize computational efficiency. We tested four physics 

options: (1) the cumulus parameterization; (2) the land surface models (LSM); (3) the planetary 

boundary layer (PBL) models; and (4) the four-dimensional data assimilation (FDDA). 

The first group of tests focused on the cumulus parameterization, the major MM5 component 

responsible for the overstated summertime rainfall. Most of the summer rainfall in the U.S. is 

due to convective clouds; stratiform rain produced by mid-latitude cyclones is a smaller fraction 

of the total rainfall in summer. The performance of the Kain-Fritsch II, Betts-Miller, and Grell 

cumulus schemes was examined. 

Next, we conducted a series of tests using different combinations of PBL and LSM options. In 

other MM5 applications around the country, including in California, we have achieved good 

model performance using the historically standard 5-layer slab model as well as the NOAH 

LSM. Our choices for PBL models include the ETA and our "modified" MRF (M-MRF). The 

standard MRF available with MM5 is known to contain deficiencies that lead to overestimates of 

mixing depth and diurnal phase lags in wind speed due to over estimates of surface momentum 

flux – our simple modifications have been shown to fix these problems (ATMET, 2003). 

In the past, we have avoided performing MM5 sensitivity tests that altered the Pleim-Xiu/ACM 

LSM/PBL configuration because it is recommended by EPA and it has been shown by other 

RPOs (e.g., VISTAS) to provide the best overall meteorological model performance and best 

CMAQ performance (Olerud and Sims, 2003; Morris et al, 2004b). Also, it allows the use of the 

Pleim-Xiu dry deposition scheme in CMAQ, which is technically superior and performs better 

than the alternative (Wesely scheme). However, most of the MM5 Pleim-Xiu tests to date have 

focused on the eastern U.S. and, given the importance of the MM5 LSM/PBL scheme to its 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec1_revsd.doc 1-4 

performance, we tested alternative LSM/PBL schemes to see if they would provide better 

performance in the western US. 

Once the cumulus and LSM/PBL schemes had been selected, we performed a series of 

experiments to determine the best FDDA configuration. Finally, because the above sensitivity 

tests were done for July only, we ran 5-day segments in winter, spring, and fall to assess whether 

the performance showed improvement in other seasons besides summer. Once we had an 

optimal configuration, we went back and tested it against the original MM5 run from the initial 

WRAP 2002 36-km modeling (Table1-1, Run 0), and also our interim Run 5 configuration from 

Table 1-1. Then, we examined the non-WRAP subdomains to determine how the new set of 

physics options affected MM5’s performance in the eastern U.S. 

We then undertook a series of tests to determine the optimal MM5 configuration for the nested 

12-km grid of the WRAP modeling domain. In these tests, we varied only the physics options 

that we expected to ameliorate the biases in the WRAP region seen in the 36-km simulation. The 

over prediction of summer rainfall and associated surface humidity bias suggested that the 

cumulus parameterization might need to be changed. Another possible reason for the wet bias 

might be the soil moisture initialization. Therefore, we focused on sensitivity tests of the soil 

moisture specification and the cumulus scheme. 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-1 

2. ADDITIONAL MM5 SENSITIVTY TESTS 

Table 2-1 lists the additional MM5 sensitivity tests beyond those reported earlier (Morris et al, 

2004a; Kemball-Cook et al, 2004; Emery et al; 2004) to identify a more optimal MM5 

configuration for simulating meteorology in the western U.S. These sensitivity tests differed in 

the choice of Land Surface Model (LSM), Planetary Boundary Layer (PBL) scheme, cumulus 

parameterization, explicit cloud microphysics and type and level of four-dimensional data 

assimilation (FDDA). Below is listing of the different schemes tested in this study and an 

explanation of the abbreviations used in Table 2-1. 

Land Surface Models (LSM) 

• NOAH LSM 

• Pleim-Xiu LSM (PX) 

• 5-layer LSM 

Planetary Boundary Layer (PBL) Schemes 

• Eta PBL 

• Modified-MRF PBL 

• Asymmetric Convective Mixing Model (ACM) PBL 

Four Dimensional Data Assimilation (FDDA)

• W: Wind 

• T: Temperature 

• H: Water vapor mixing ratio 

Cumulus Schemes

• Betts-Miller (BM) 

• Grell 

• Kain-Fritsch II (KFII) 

Cloud Microphysics Schemes

• Simple Ice Microphysics 

• Reisner II Microphysics 

ANALYSIS PROCEDURES 

The MM5 evaluation was performed by examining surface performance statistics using the 

METSTAT evaluation software, and by comparing the output MM5 total (convective + largescale) precipitation fields with gridded observed precipitation amounts. 

Observed precipitation data was derived from the CPC (Climate Prediction Center) gridded 

precipitation amount dataset. This data set is available from the National Weather Service's 

Climate Prediction Center website at: 

http://www.cpc.ncep.noaa.gov/products/precip/realtime/retro.html. 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-2 

Table 2-1. Summary of additional MM5 sensitivity tests.

Analysis FDDA 

LSM PBL Cumulus Microphysics 3D Surface Obs FDDA

Run 0 PX ACM KF I Reisner2 W/T/H W/T/H 

Run 5 PX ACM KF II Reisner2 W/T/H W/T/H 

Run2a PX ACM BM Reisner2 W/T/H W/T/H 

Run2aa PX ACM BM Reisner2 W/T/H W 

Run2ac PX ACM BM Reisner2 W/T/H W/T/H W/T/H 

Run2ad PX ACM BM Reisner2 W/T/H W W 

Run2ae PX ACM BM Reisner2 W/T/H W/T/H W 

Run2b PX ACM Grell Reisner2 W/T/H W/T/H 

Run1ba NOAH M-MRF BM Reisner2 W/T/H W/T/H 

Run1bb NOAH M-MRF BM Reisner2 W/T/H W/T/H W 

Run3b 5-Layer M-MRF BM Reisner2 W/T/H W/T/H 

 No soil nudging was performed for the above runs 

The CPC daily precipitation amounts are provided on a grid that covers the U.S. mainland at a 

resolution of 0.25° x 0.25°, and are ramped down to zero immediately offshore. The advantage 

of this field is that it has a reasonably high resolution, which is especially important when it 

comes to resolving the effects of orography on precipitation over the western U.S. 

The performance statistics for surface wind, temperature, and humidity were calculated using 

METSTAT with the NCAR ds472 surface observations data set. METSTAT is comprised of a 

single Fortran program, which generates observation-prediction pairings and calculates the 

statistics, and a Microsoft Excel macro that plots the results. This program is described in detail 

in the WRAP MM5 Modeling Protocol (ENVIRON and UCR, 2004). 

For the purpose of organizing the analysis, “Soccer Plots” are used that display average 

performance statistics for each subdomain for the modeled time period. “Soccer Plots” present 

average bias versus error for a particular meteorological variable. Model performance 

benchmarks or goals are plotted on the Soccer Plot to see whether the performance of the 

variable meets the goal. Subdomain definitions are shown in Figure 2-1, and Table 2-2 lists the 

numbers of the METSTAT domains shown in Figure 2-1 and the corresponding abbreviations 

used in the discussion below. 

Soccer Plots are shown for wind speed bias versus wind speed RMSE, temperature bias versus 

temperature error, and humidity bias versus humidity error. In each plot, a solid line indicates 

the benchmark. The benchmarks were developed by Emery and Tai. (2001), and were based 

upon the evaluation of more than 30 MM5 and RAMS meteorological simulations in support of 

air quality applications performed in the last few years, as reported by Tesche et al. (2002). A 

data point that falls inside the box represents a model run that meets the performance benchmark. 

Perfect model performance is indicated by a data point at (0,0). The closer a data point is to the 

origin, the better the model’s performance. It should be emphasized that the benchmarks are not 

used as an acceptance/rejection criteria of the MM5 model simulation. Rather they put the MM5 

model performance into perspective and allow the identification of potential problems in the 

MM5 fields. 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-3 

Figure 2-1. Definition of Metstat subdomains.

Table 2-2. METSTAT subdomain abbreviations.

METSTAT Subdomain Name Abbreviation 

1 PacificNW 

2 SW 

3 North 

4 DesertSW 

5 CenrapN 

6 CenrapS 

7 Great Lakes 

8 Ohio Valley 

9 SE 

10 NE 

11 MidAtlantic 

MM5 SENSITIVITY TEST RESULTS

Cumulus Parameterization Sensitivity Test 

We began by testing MM5’s sensitivity to the choice of cumulus parameterization. The original 

2002 MM5 run (Run 0) used the Kain-Fritsch I scheme. Results from VISTAS 36-km MM5 

modeling showed improved surface fields relative to Run 0 using Kain-Fritsch II, so KainFritsch II (KFII) was the first scheme tested here (Run 5). Next, we compared runs that used the 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-4 

Grell (Run 2b) and Betts-Miller (Run2a) parameterizations to Run 5. All of these runs were 

made using the PX/ACM LSM/PBL schemes and W/T/H analysis nudging at the surface and 

aloft and no nudging to observations (obs nudging). 

Figure 2-2a summarizes the wind performance for the July 2002 test period. All three cumulus 

schemes show a low wind speed bias. For the SW, PacificNW, and North subdomains, the Grell 

scheme has a smaller error than the other schemes, and is similar in bias. In the DesertSW 

subdomain, the Betts-Miller scheme slightly outperforms the others in both bias and error, but all 

three schemes are far from the benchmark for DesertSW. 

Figure 2-2b shows the results for temperature. All three schemes show a general cold bias. The 

Betts-Miller scheme has the smallest overall bias, and does slightly better than the Grell scheme 

for error. The KFII scheme has the smallest error, except in the PacificNW subdomain, in which 

it is an outlier. The Betts-Miller scheme has the best overall temperature performance. It has the 

smallest bias, and, although it has a slightly larger error than the KFII scheme for DesertSW, 

SW, and North, it does significantly better than KFII in the Pacific Northwest. 

The humidity performance is shown in Figure 2-2c. Here, again, the Betts-Miller scheme shows 

the best overall performance. For the SW and PacificNW subdomains, the performance of the 

three schemes is comparable, but the Betts-Miller scheme is clearly better for the North and 

DesertSW subdomains. 

Figure 2-3 is a plot of observed versus MM5 precipitation for the modeled time period. All three 

runs are too wet across the northern U.S. Kain-Fritsch II has a strong wet bias, particularly in the 

SW and DesertSW subdomains, and also underpredicts the precipitation maximum over Texas. 

Both the Grell and Betts-Miller schemes produce a maximum over Texas, although they translate 

it to the southwest of the observed location. Over the southwestern U.S., the Betts-Miller 

precipitation field is closer to the observations than that of Grell, which generates too much rain 

in this region. Based on the improvement in performance over the southwest and Texas, we 

conclude that the Betts-Miller scheme produced the best simulation of precipitation for this time 

period. 

Of the three cumulus packages tested in the WRAP region, Betts-Miller is the best-performing 

scheme for temperature, humidity, and precipitation. We therefore selected the Betts-Miller 

scheme and use it in all subsequent runs as we continue with further sensitivity tests. 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-5 

Figure 2-2a. Wind scatterplot for cumulus sensitivity test. 

Figure 2-2b. Temperature scatterplot for cumulus sensitivity test. 

WRAP 36 km July Wind Performance Comparison

0

0.5

1

1.5

2

2.5

3

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

Wind Speed Bias (m/s)

WindSpeed RMSE (m/s)

Benchmark Run 2a: Betts-Miller Run 2b: Grell Run 5: Kain-Fritsch II

Cumulus Parameterization Sensitivity Test

DesertSW 

PacNW

SW

North

WRAP 36 km July Temperature Performance Comparison

0

0.5

1

1.5

2

2.5

3

-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5

Temperature Bias (K)

Temeprature Error (K)

Benchmark Run 2a: Betts-Miller Run 2b: Grell Run 5: Kain-Fritsch II

Cumulus Parameterization Sensitivity Test

DesertSW 

PacNW

SW

North
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-6 

Figure 2-2c. Humidity scatterplot for cumulus sensitivity test. 

Figure 2-3. Precipitation comparison for the cumulus parameterization sensitivity test. 

Run 2a: Betts-Miller (top left), Run 2b:Grell (bottom left) and, Run 5: Kain-Fritsch II (bottom 

right). Observed (top right).

WRAP 36 km Humidity Performance Comparison

0

1

2

3

4

5

-3.5 -2.5 -1.5 -0.5 0.5 1.5 2.5 3.5

Humidity Bias (g/kg)

Humidity Error (g/kg)

Benchmark Run 2a:Betts-Miller Run 2b: Grell Run 5: Kain-Fritsch II

Cumulus Parameterization Sensitivity Test

DesertSW 

PacNW

SW

North
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-7 

LSM Sensitivity Test 

Next, we tested the land surface and planetary boundary layer (PBL) parameterizations. All of 

the runs described in this section used the Betts-Miller cumulus parameterization, W/T/H 

Analysis nudging at the surface and aloft and no obs nudging. We begin by comparing the 

NOAH LSM (Run 1ba) with the 5-layer model (Run 3b). In both of these runs, the modified 

MRF (M-MRF) PBL scheme was used. 

Figure 2-4a shows the wind performance for the Run 1ba and 3b LSM sensitivity tests. Both the 

5-layer and NOAH runs show a low wind speed bias, with the bias more pronounced in the 5-

layer case. For both runs, the SW and PacificNW subdomains lie within the benchmark, while 

North and DesertSW do not. The NOAH run performs better for all subdomains except for the 

DesertSW. For DesertSW, the NOAH run has a slightly smaller bias, but a larger error. 

The LSM temperature comparison is shown in Figure 2-4b. Both runs have a cold bias, but the 

bias is remarkably pronounced for the NOAH run in the PacificNW and North subdomains. The 

error is smaller for all subdomains in the 5-layer run, as is the bias in PacificNW and North. For 

the SW and DesertSW domains, the NOAH run has a smaller bias, but larger error. Overall, the 

5-layer model temperature performance was an improvement over that of the NOAH run. 

Figure 2-4c shows the humidity performance for these runs. There is an overall wet bias, except 

in the SW subdomain. For humidity, the NOAH run meets the performance goal, while the 5-

layer run lies entirely outside the benchmark. The NOAH run performs better in the North and 

PacificNW than in the SW and DesertSW domains. 

The precipitation performance is shown in Figure 2-5. Run 1ba does slightly better here, 

producing less rain over the desert southwest states than does run 3b; Run 1ba is therefore closer 

to the observed rainfall field than Run 3b. 

Overall, the NOAH model outperformed the 5-layer model for this case. The 5-layer model 

introduces a large positive humidity bias in the DesertSW and SW, which is one of the main 

problems from the original Run 0 we are trying to ameliorate in this study. Having eliminated 

the 5-layer LSM model from further consideration, we now test the NOAH/M-MRF combination 

and compare it to the run which used the PX/ACM configuration. The results of these two runs 

are also shown in Figures 2-4 and 2-5. 

 

Figure 2-4a displays the wind comparison for the NOAH/M-MRF and PX/ACM runs. The two 

runs show similar performance for error, and the NOAH/M-MRF run has a smaller overall bias. 

For temperature (Figure 2-4b), neither run meets the performance benchmark, but the PX/ACM 

run is a clear improvement in both bias and error over the NOAH/M-MRF run. The cold bias is 

far more pronounced in the NOAH/M-MRF run for the PacificNW and North subdomains. 

For humidity (Figure 2-4c) there is not much difference between the two runs, except in 

DesertSW, where PX/ACM has a higher bias and error. For precipitation, the PX/ACM run 

showed some slight improvement over the NOAH/M-MRF case over the southwestern U.S. The 

intensity of the maximum over Texas is less well simulated in the PX/ACM run, however. 

Performance was roughly equivalent elsewhere. 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-8 

Figure 2-4a. Wind scatterplot for LSM/PBL sensitivity tests. 

Figure 2-4b. Temperature scatterplot for LSM/PBL sensitivity tests. 

WRAP 36 km July Wind Performance Comparison

0

0.5

1

1.5

2

2.5

3

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

Wind Speed Bias (m/s)

WindSpeed RMSE (m/s)

Benchmark Run 1ba: NOAH-MRF Run 3b: 5 lyr-MRF Run 2a: PX-ACM Run 1bb: NOAH-MRF, OBS Nudged

LSM/PBL Sensitivity Test

DesertSW 

PacNW

SW

North

WRAP 36 km July Temperature Performance Comparison

0

0.5

1

1.5

2

2.5

3

-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5

Temperature Bias (K)

Temeprature Error (K)

Benchmark Run 1ba: NOAH-MRF Run 3b: 5 lyr-MRF Run 2a: PX-ACM 1bb: NOAH-MRF, OBS Nudged

LSM/PBL Sensitivity Test

DesertSW 

PacNW

SW

North
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-9 

Figure 2-4c. Humidity scatterplot for LSM/PBL sensitivity tests.

WRAP 36 km Humidity Performance Comparison

0

1

2

3

4

5

-3.5 -2.5 -1.5 -0.5 0.5 1.5 2.5 3.5

Humidity Bias (g/kg)

Humidity Error (g/kg)

Benchmark Run 1ba: NOAH-MRF Run 3b: 5 lyr-MRF

Run 2a: PX-ACM Run 1bb: NOAH-MRF, OBS Nudged

LSM/PBL Sensitivity Test

DesertSW 

PacNW

SW

North
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-10 

Figure 2-5. Precipitation comparison for the LSM/PBL parameterization sensitivity tests. 

Observed (top right) Run 2a: PX/ACM (top left), Run 3b:5-layer/M-MRF (bottom left) and, Run 

1ba: NOAH/M-MRF (bottom right). 

Overall, there was no single factor that made it easy to select one LSM/PBL scheme over the 

other. Because there was no significant improvement in going to the NOAH/M-MRF 

configuration, we propose to use the PX/ACM configuration because it allows the use of the 

Pleim dry deposition algorithm in subsequent CMAQ modeling, is endorsed by EPA, and allows 

consistency with the other RPO’s. 

FDDA Sensitivity Test 

Having selected a cumulus parameterization and an LSM/PBL combination, we turn now to the 

FDDA configuration. The runs discussed in this section differ only in their FDDA, and all use 

the PX/ACM, Betts-Miller, and Reisner2 schemes (see Table 2-1 for run definitions). 

Figure 2-6a is a comparison of the wind performance for the runs with different FDDA 

configurations. For all runs, all domains except PacificNW (2a and 2aa) and SW (2a) lie outside 

the benchmark. There is low wind speed bias for all runs and all subdomains. Run 2a shows the 

smallest bias, while 2ac had the lowest overall error. It is difficult to say which runs exhibit the 

best wind performance based on this figure. Figure 2-6b shows the wind direction error versus 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-11 

the wind speed RMSE. In this figure, the effects of obs nudging on the surface wind field are 

apparent. Runs which had obs nudging lie together closer to the benchmark than the runs which 

were not obs nudged. 

For temperature (Figure 2-6c) Run 2ae clearly shows the best performance. All subdomains but 

one are within the benchmark; Run 2ae also has the smallest bias and error. For humidity 

(Figure 2-6d), performance is comparable among the runs, and all subdomains except DesertSW 

lie within the benchmark. Differences in the precipitation fields for all the FDDA sensitivity 

runs were small (not shown) and did not help to distinguish among the runs in terms of 

performance. 

On the basis of its superior temperature performance and wind performance, which was similar 

to the best-performing runs, Run 2ae was determined to have the best FDDA configuration. 

Next, to allow for the possibility that nudging the NOAH/M-MRF case might cause significant 

improvement, we made another run (Run 1bb), which was identical to Run 2ae except that the 

NOAH/M-MRF LSM/PBL scheme was used instead of PX/ACM. The results are shown in 

Figure 2-7, with the more weakly nudged Run 1ba included to show the effects of nudging. For 

wind speed bias and error, Run 1bb (NOAH) is a slight improvement, except for the SW 

subdomain, for which it shows marked improvement over Run 2ae (PX). For temperature, Run 

2ae (PX) had a smaller bias and error than the other runs (NOAH). For humidity and 

precipitation, there is no run that clearly performed better than the others. We selected Run 2ae 

(PX) as the best-performing run overall, because the temperature performance is deemed to be 

more important than the wind performance, given that the terrain in the western U.S. may 

prevent good agreement between modeled and observed surface winds at 36 km resolution. 

Figure 2-6a. Wind speed scatterplot for FDDA sensitivity test. 

WRAP 36 km July Wind Performance Comparison

0

0.5

1

1.5

2

2.5

3

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

Wind Speed Bias (m/s)

WindSpeed RMSE (m/s)

Benchmark Run 2a Run 2aa Run 2ac Run 2ad Run 2ae

FDDA Sensitivity Test

DesertSW 

PacNW

SW

North
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-12 

Figure 2-6b. Wind speed/direction scatterplot for FDDA sensitivity test. 

Figure 2-6c. Temperature scatterplot for FDDA sensitivity test.

WRAP 36 km July Wind Performance Comparison

0

0.5

1

1.5

2

2.5

3

0 20 40 60

Wind Direction Error (Degrees)

WindSpeed RMSE (m/s)

Benchmark Run 2a Run 2aa Run 2ac Run 2ad Run 2ae

DesertSW 

PacNW

SW

North

WRAP 36 km July Temperature Performance Comparison

0

0.5

1

1.5

2

2.5

3

-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5

Temperature Bias (K)

Temeprature Error (K)

Benchmark Run 2a Run 2aa Run 2ac Run 2ad Run 2ae

FDDA Sensitivity Test

DesertSW 

PacNW

SW

North
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-13 

Figure 2-6d. Humidity scatterplot for FDDA sensitivity test. 

Figure 2-7a. Wind scatterplot for NOAH/M-MRF nudging sensitivity test. 

WRAP 36 km Humidity Performance Comparison

0

1

2

3

4

5

-3.5 -2.5 -1.5 -0.5 0.5 1.5 2.5 3.5

Humidity Bias (g/kg)

Humidity Error (g/kg)

Benchmark Run 2a Run 2aa Run 2ac Run 2ad Run 2ae

FDDA Sensitivity Test

DesertSW 

PacNW

SW

North

WRAP 36 km July Wind Performance Comparison

0

0.5

1

1.5

2

2.5

3

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

Wind Speed Bias (m/s)

WindSpeed RMSE (m/s)

Benchmark Run 1ba Run 1bb Run 2ae

DesertSW 

PacNW

SW

North
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-14 

Figure 2-7b. Temperature scatterplot for NOAH/M-MRF nudging sensitivity test. 

Figure 2-7c. Humidity scatterplot for NOAH/M-MRF nudging sensitivity test. 

WRAP 36 km July Temperature Performance Comparison

0

0.5

1

1.5

2

2.5

3

3.5

-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5

Temperature Bias (K)

Temeprature Error (K)

Benchmark Run 1ba 1bb Run 2ae

DesertSW 

PacNW

SW

North

WRAP 36 km Humidity Performance Comparison

0

1

2

3

4

5

-3.5 -2.5 -1.5 -0.5 0.5 1.5 2.5 3.5

Humidity Bias (g/kg)

Humidity Error (g/kg)

Benchmark Run 1ba Run 1bb Run 2ae

DesertSW 

PacNW

SW

North
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-15 

Figure 2-8. Precipitation comparison for the FDDA sensitivity tests. Observed (top right) Run 

2a: PX/ACM (top left), Run 3b:5-layer/M-MRF (bottom left) and, Run 1ba: NOAH/M-MRF 

(bottom right). 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-16 

Tests in Other Seasons 

Having determined a more optimal MM5 configuration based on the July sensitivity tests, we 

move now to other parts of the year, leaving open the possibility that we may use a different 

MM5 configuration for different times of year for the final year-long 2002 simulation. The two 

best-performing runs overall (Runs 2ae and 1bb) were run for 5-day periods in January, April, 

and October, representing the winter, spring, and fall seasons. 

Figures 2-9a-f show the performance of Runs 2ae and 1bb during the four 5-day runs 

representing the four seasons. We begin by comparing the wind performance of the Runs 2ae 

and Run 1bb for all four seasons by examining Figures 2-9a and 2-9b. For January, shown in 

blue, performance in the PacificNW, SW, and DesertSW is similar, while Run 1bb shows 

significant improvement in the North subdomain. In both April (light blue) and October (red), 

Run 1bb has generally lower bias and error, with the 4 subdomains clustered closer to the 

benchmark than in Run 2ae. In July (magenta), Run 1bb has a smaller bias, while Run 2ae has a 

smaller error. This is true for all subdomains except PacificNW, for which Run 2ae has a 

smaller bias than Run 1bb. Overall, then Run 1bb performs better for wind throughout the year, 

with the points clustered closer to the benchmark. Note however, that all points lie outside the 

benchmark for both simulations except for January PacificNW and North for Run 1bb. 

For temperature (Figures 2-9c and 2-9d) in January, Run 2ae has a smaller bias and error for 

North and DesertSW. For SW, performance is similar, and for PacificNW, Run 1bb has a 

smaller bias and error. In April, Run 1bb does better; in Run 2ae, PacificNW and North have a 

strong cold bias and lie far outside the benchmark. In July and October, Run 2ae performs 

better, with most of the subdomains situated within the benchmark, while in Run 1bb all 

subdomains lie in a halo around the benchmark. 

For humidity (Figures 2-9e and 2-9f), performance between the two runs is comparable. The 

precipitation performance (not shown) was similar enough that differences in the precipitation 

fields were not useful in choosing between the two runs. 

Figure 2-10 compares the observed versus estimated precipitation for the four seasonal tasks and 

Runs 2ae and 1bb. There is good agreement between the observed and estimated precipitation in 

January. Note that the observed precipitation is just for the U.S. Both MM5 runs overstate the 

extent of the precipitation in April and July, but agree with the observations fairly well in 

October. The precipitation performance for Runs 2ae and 1bb are nearly identical. 

Because the regional haze events critical for WRAP CMAQ modeling tend to occur during the 

summer, we select Run 2ae, which gives better temperature performance during the summer, as 

the best-performing run overall. In the July simulation, the winds and humidity were 

comparable. The performance improvement in Run 1bb in January and April was insufficient 

reason to switch the LSM/PBL scheme in winter/spring because the improvement brought only 

one additional subdomain point into the benchmark, and there was no improvement in 

temperature or humidity. We therefore stay with PX/ACM throughout the year to get the best 

summer simulation and to allow the use of the Pleim dry deposition algorithm in CMAQ 

throughout the year. 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-17 

Figures 2-9a. Scatterplots for seasonal wind comparisons for Run 2ae. 

Figure 2-9b. Scatterplots for seasonal wind comparison for Run 1bb. 

WRAP 36 km Run 2ae Wind Performance Comparison

0

0.5

1

1.5

2

2.5

3

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

Wind Speed Bias (m/s)

WindSpeed RMSE (m/s)

Benchmark Jan Apr Oct Jul

DesertSW 

PacNW

SW

North

WRAP 36 km Run 1bb Seasonal Wind Performance Comparison

0

0.5

1

1.5

2

2.5

3

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

Wind Speed Bias (m/s)

WindSpeed RMSE (m/s)

Benchmark Jul Jan Apr Oct

DesertSW 

PacNW

SW

North
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-18 

Figure 2-9c. Temperature scatterplot for Run 2ae seasonal test. 

Figure 2-9d. Temperature scatterplot for Run 1bb seasonal test. 

WRAP 36 km Run 2ae Temperature Performance Comparison

0

0.5

1

1.5

2

2.5

3

-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5

Temperature Bias (K)

Temeprature Error (K)

Benchmark Jul Jan Apr Oct

DesertSW 

PacNW

SW

North

WRAP 36 km Run 1bb Seasonal Temperature Performance 

Comparison

0

0.5

1

1.5

2

2.5

3

-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5

Temperature Bias (K)

Temeprature Error (K)

Benchmark Jul Jan Apr Oct

DesertSW 

PacNW

SW

North
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-19 

Figure 2-9e. Humidity scatterplot for Run 2ae seasonal test. 

Figure 2-9f. Humidity scatterplot for Run 1bb seasonal test. 

WRAP 36 km Run 2ae Humidity Performance Comparison

0

1

2

3

4

5

-3.5 -2.5 -1.5 -0.5 0.5 1.5 2.5 3.5

Humidity Bias (g/kg)

Humidity Error (g/kg)

Benchmark Jul Jan Apr Oct

DesertSW 

PacNW

SW

North

WRAP 36 km Run 1bb Seasonal Humidity Performance Comparison

0

1

2

3

4

5

-3.5 -2.5 -1.5 -0.5 0.5 1.5 2.5 3.5

Humidity Bias (g/kg)

Humidity Error (g/kg)

Benchmark Jul Jan Apr Oct

DesertSW 

PacNW

SW

North
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-20 

Figure 2-10a. CPC observed precipitation for the 5-day modeled segment in each season. 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-21 

Figure 2-10b. Seasonal precipitation test for Run 2ae (PX). 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec2_revsd.doc 2-22 

Figure 2-10c. Seasonal precipitation test for Run 1bb (M-MRF). 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec3_revsd.doc 3-1 

3. SUMMARY OF 36-KM RESULTS 

COMPARISON TO PREVIOUS SIMULATIONS 

A graphical representation of the differences between the initial (Run 0) (Morris et al., 2004a; 

Kemball-Cook et al., 2004), interim (Run 5) (Emery et al., 2004), and final (Run 2ae) MM5 runs 

is provided in Figures 3-1 through 3-3. Figure 3-1a shows the wind bias and error for all three 

runs. For the DesertSW subdomain, there is improvement in both bias and error. For the other 

three subdomains, as we progress from Run 0 to Run 5, there is a reduction in error and an 

increase in the low wind speed bias. All points but one (Run 0, PacificNW) lie outside the 

benchmark. Examination of Figure 3-1b, on which is plotted wind direction bias and gross error, 

shows that Run 2ae also produces improvements in the wind direction field relative to Run 0 and 

5. 

For temperature (Figure 3-1c), all subdomains but DesertSW now lie within the benchmarks for 

Run 2ae, and Run 2ae is clearly the best-performing run for both error and bias in all 

subdomains. The most noticeable change in going from Run 0 to Run 2ae is that DesertSW is no 

longer an outlier, but now lies close to the benchmark for error, and within it for bias. 

Figures 3-2a and 3-2b are time series of predicted (MM5) and observed temperature for Run 2ae 

and Run 0. In Run 0, the amplitude of the predicted diurnal temperature cycle is smaller than 

observed. This was attributed to the presence of excess water at the surface, indicated by the wet 

bias in DesertSW (Figure 3-2b) and the excessive precipitation seen in Run 0 in Figure 3-3. 

Since water has a higher specific heat than soil, the presence of excess water at the surface will 

tend to dampen the amplitude of daily excursions in temperature. There is a marked 

improvement in Run 2ae. The amplitude of the diurnal cycle in Run 2ae more nearly matches 

that of the observations, and the substantial cold bias has been replaced by a slight warm bias. 

For humidity (Figure 3-1d), all subdomains now lie within the benchmark. Again, DesertSW has 

gone from being an outlier, with wet bias far outside of the benchmark, to falling within the 

benchmark. For North, going from Run 0 to Run 5 and from Run 5 to Run 2ae brings the 

subdomain inside the benchmark. For SW, Run 2ae is an improvement in error but not bias. 

The precipitation field for Run 2ae is notable for its improvement over Runs 0 and 5 (Figure 3-

3). The general overprediction of precipitation is less severe in Run 2ae than in the other two 

runs. The WRAP subdomains in particular are now seeing rainfall amounts closer to observed; 

this is reflected in the humidity scatterplot (Figure 3-1c) that shows a reduction of the wet bias in 

the DesertSW seen in Runs 0 and 5. The observed precipitation maximum over Texas is 

simulated reasonably well in Run 2ae, but is underpredicted in Run 0 and 5. 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec3_revsd.doc 3-2 

Figure 3-1a. Wind speed scatterplot for comparison of original, interim, and final MM5 

runs. 

Figure 3-1b. Wind direction scatterplot for comparison of original, interim, and final MM5 

runs. 

WRAP 36 km Run 0, Run 5, and Run 2ae Wind Performance 

Comparison

0

0.5

1

1.5

2

2.5

3

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

Wind Speed Bias (m/s)

WindSpeed RMSE (m/s)

Benchmark Run 0 Run 5 Run 2ae

DesertSW 

PacNW

SW

North

WRAP 36 km Run 0, Run 5, and Run 2ae Wind Direction 

Performance Comparison

0

10

20

30

40

50

-15 -10 -5 0 5 10 15

Wind Direction Bias (m/s)

Wind Direction Gross Error (m/s)

Benchmark Run 0 Run 5 Run 2ae

DesertSW 

PacNW

SW

North
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec3_revsd.doc 3-3 

Figure 3-1c. Temperature scatterplot for comparison of original, interim, and final MM5 

runs. 

Figure 3-1d. Humidity scatterplot for comparison of original, interim, and final MM5 

runs.

WRAP 36 km Run 0, Run 5, and Run 2ae Temperature Performance 

Comparison

0

0.5

1

1.5

2

2.5

3

3.5

4

-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5

Temperature Bias (K)

Temeprature Error (K)

Benchmark Run 0 Run 5 Run 2ae

DesertSW 

PacNW

SW

North

WRAP 36 km Run 0, Run 5, and Run 2ae Humidity Performance 

Comparison

0

1

2

3

4

5

-3.5 -2.5 -1.5 -0.5 0.5 1.5 2.5 3.5

Humidity Bias (g/kg)

Humidity Error (g/kg)

Benchmark Run 0 Run 5 Run 2ae

DesertSW 

PacNW

SW

North
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec3_revsd.doc 3-4 

Figure 3-2a. July 1-5 MM5 predicted versus observed surface temperature time series 

for Run 0. 

Figure 3-2b. July 1-5 MM5 predicted versus observed surface temperature time series 

for Run 2ae. 

Observed/Predicted Temperature

285

290

295

300

305

310

315

 7/ 1 7/ 2 7/ 3 7/ 4 7/ 5

K

ObsTemp PrdTemp 

Observed/Predicted Temperature

285

290

295

300

305

310

315

 7/ 1 7/ 2 7/ 3 7/ 4 7/ 5

K

ObsTemp PrdTemp 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec3_revsd.doc 3-5 

Figure 3-3. Precipitation comparison for July for original, interim, and final MM5 runs. 

Table 3-1 is an overview of the change in surface MPE statistics going from the original WRAP 

2002 Annual run (Run 0) through the improved version (Run 5) to the final version (Run 2ae) in 

the western subdomains. Statistics of particular interest are highlighted. There is significant 

improvement in going from Run 0 to Run 2ae. Note the increases in IOA in SW and DesertSW. 

These are the subdomains that had a substantial deficiency in the original run, with a large wet 

bias. Note that the wet bias in the DesertSW subdomain has been more than halved in going 

from Run 0 to Run 2ae. The gross humidity error is also greatly reduced in these two 

subdomains. PacificNW and SW also show improvement in humidity gross error, but show a 

slight degradation in bias performance. Overall, the new Run 2ae model configuration has 

resulted in a greatly improved simulation of humidity over the western U.S. 

There is also a reduction in the cold bias apparent in Run 0. In DesertSW, the cold bias was 

originally –2.07°C, and drops to 0.2°C in Run 2ae. Reductions in the cold bias are also seen in 

the other subdomains. Temperature error is also reduced in all subdomains, although not enough 

to bring it entirely within the performance benchmark. Corresponding improvements are seen in 

the temperature index of agreement. 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec3_revsd.doc 3-6 

Table 3-1. Surface model performance evaluation statistics summary. 

DesertSW North PacificNW SW 

Benchmark Run0 Run5 Run2ae Run0 Run5 Run2ae Run0 Run5 Run2ae Run0 Run5 Run2ae 

Wind Spd 

Bias 

<±0.5 -1.47 -1.30 -1.22 -0.90 -0.85 -0.94 -0.34 -0.35 -0.59 -0.82 -0.77 -0.94 

Wind Spd 

RMSE 

<2.0 2.69 2.51 2.21 2.32 2.2 1.88 1.97 2.05 1.67 2.16 1.98 1.85 

Wind Spd 

IOA 

≥0.6 0.55 0.57 0.70 0.72 0.73 0.83 0.67 0.65 0.77 0.68 0.70 0.77 

Wind Dir 

Bias 

<±10 0.05 4.79 3.13 -3.22 6.23 2.19 -3.75 9.23 5.18 -

11.85 

-2.28 -2.23 

Wind Dir 

Gross E 

<30 50.5 50.3 32.8 42.2 43.4 28.3 45.9 46.6 34.2 49.3 47.3 34.2 

Temp Bias <±0.5 -2.07 -0.4 0.2 -0.8 -0.26 0.22 -0.56 -1.28 -0.10 -0.64 -0.39 -0.26 

Temp Gross 

Error 

<2 3.42 2.19 2.07 2.32 2.09 1.77 2.14 2.8 1.86 2.37 2.72 2.02 

Temp IOA ≥0.8 0.89 0.96 0.96 0.94 0.95 0.97 0.94 0.89 0.96 0.95 0.99 0.97 

Humidity 

Bias 

<±1 2.73 1.90 1.00 1.08 0.75 0.40 -0.23 -0.41 -0.30 -0.28 -0.24 -0.47 

Humidity 

Gross E 

<2 3.14 2.40 1.91 1.80 1.60 1.30 0.92 0.96 0.88 1.72 1.41 1.42 

Humidity 

IOA 

≥0.6 0.65 0.75 0.80 0.73 0.79 0.83 0.70 0.69 0.72 0.62 0.73 0.77 

EVALUATION OF OTHER SUBDOMAINS 

The tests described above focused on the western U.S., and the final configuration was selected 

on the basis of those results. This leaves open the possibility that we have optimized the MM5 

configuration for the western U.S., causing degradation of performance in other parts of the 

country. To investigate this, we compared model performance in the non-WRAP subdomains 

for Run 0, Run 5, and Run 2ae. 

Figure 3-4a displays the wind performance comparison. Run 2ae is clearly the best of the three 

runs here, with a lower error than Run 0 or Run 5 for all subdomains. For midAtl, cenrapN, and 

NE, Run 2ae also has the smallest bias (note that the point for cenrapN coincides with the point 

for midAtl for Run 2ae). For cenrapS subdomain, Run 2ae has a smaller bias than Run 0, but a 

slightly larger bias than Run 5. For OhioVal Subdomain, the bias for Run 0 and Run 2ae is 

identical, and is slightly larger than that of Run 5. 

For temperature (Figure 3-4b), Run 2ae has a lower or equal error for all subdomains, and a 

smaller bias for midAtl, and cenrapS. For the other subdomains, the bias in 2ae is slightly larger 

than in Run 5, but still falls well within the benchmark. All points show a warm bias in Run 2ae, 

which is a shift from Run 0, for which all subdomains have a cold bias. Run 5 shows the 

subdomains equally split between warm and cold biases. 

For humidity (Figure 3-4c), Run 2ae shows improvement over Runs 0 and 5 for all subdomains, 

lowering or eliminating the wet bias seen in Runs 0 and 5. Changes in error are less pronounced, 

but where they occur, Run 2ae has a smaller error. 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec3_revsd.doc 3-7 

July 1-5, 2002 36 km: Run 0, Run 5, and Run 2ae Wind Performance 

Comparison

0

0.5

1

1.5

2

2.5

3

-2 -1.5 -1 -0.5 0 0.5 1 1.5

Wind Speed Bias (m/s)

WindSpeed RMSE (m/s)

Benchmark Run 0 Run 5 Run 2ae

cenrapN 

cenrapS

SE

midAtl

NE

OhioVal

 x

 o

Figure 3-4a. Wind scatterplot for comparison of original, interim, and final MM5 runs for 

subdomains outside WRAP. 

Figure 3-4b. Temperature scatterplot for comparison of original, interim, and final MM5 

runs for subdomains outside WRAP. 

July 1-5, 2002 36 km: Run 0, Run 5, and Run 2ae Temperature 

Performance Comparison

0

0.5

1

1.5

2

2.5

-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5

Temperature Bias (K)

Temeprature Error (K)

Benchmark Run 0 Run 5 Run 2ae

cenrapN 

cenrapS

SE

midAtl

NE

OhioVal

 x

 o
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec3_revsd.doc 3-8 

Figure 3-4c. Humidity scatterplot for comparison of original, interim, and final MM5 runs 

for subdomains outside WRAP.

SUMMARY 

In this study, we made a series of sensitivity tests and arrived at a revised 36-km MM5 

configuration that resulted in significant improvements in the precipitation, temperature, 

humidity and wind fields, all of which are likely to be helpful in producing more accurate 

CMAQ results. 

MM5 runs made with the new more optimal configuration showed: 

• A dramatic reduction in the summertime cold, wet bias in the desert southwest; 

• Surface temperature and humidity performance now within benchmarks for all WRAP 

subdomains except desert southwest for temperature; 

• More accurate representation of the diurnal temperature cycle in the desert southwest; 

• Increases in the Index of Agreement (IOA) for temperature, humidity and wind speed for 

all WRAP subdomains; 

• A more realistic precipitation pattern over the western U.S.; 

• Improved model performance in the eastern U.S.; and 

• Runs made with these physics options generally represented an improvement in 

performance throughout the year, making it unnecessary to select different physics 

schemes for different seasons. 

Some performance issues with the final Run 2ae remain. For example, Run 2ae still lies outside 

the benchmarks for wind for all WRAP subdomains, and the temperature error borders on or 

July 1-5, 2002, 36 km: Run 0, Run 5, and Run 2ae Humidity

Performance Comparison

0

1

2

3

4

5

-3.5 -2.5 -1.5 -0.5 0.5 1.5 2.5 3.5

Humidity Bias (g/kg)

Humidity Error (g/kg)

Benchmark Run 0 Run 5 Run 2ae

cenrapN 

cenrapS

SE

midAtl

NE

OhioVal

 x

 o
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec3_revsd.doc 3-9 

exceeds the benchmark value for 3 of the four WRAP subdomains. MM5 is still producing too 

much rainfall in the northern half of the U.S. during the July period. Although use of the BettsMiller scheme improved the rainfall field, the DesertSW and CenrapN and CenrapS areas, in 

particular, are still too wet. 

However, the sensitivity tests suggest that getting perfect MM5 performance for all variables in 

the western U.S. is probably beyond the skill of the MM5 model. The existing run 2ae 

configuration is probably as close to optimal as can be achieved within the time constraints of 

this project. 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec4.doc 4-1 

4. SUMMARY OF 12-KM SENSITIVITY TESTS 

We now describe a series of tests undertaken to determine the optimal MM5 configuration for 

the nested 12-km grid of the WRAP modeling domain. In these tests, we varied only the physics 

options that we expected to ameliorate the biases in the WRAP region seen in the 36-km 

simulation. The over prediction of summer rainfall and associated surface humidity bias 

suggested that the cumulus parameterization might need to be changed. Another possible reason 

for the wet bias might be the soil moisture initialization. Therefore, we focused on sensitivity 

tests of the soil moisture specification and the cumulus scheme. 

CUMULUS PARAMETERIZATION TESTS 

At the 12 km scale, the problem of cumulus parameterization is not well-posed, as there is no 

clear spectral gap between the resolved grid-scale process and the scale of the parameterized 

process (Arakawa and Chen 1987). Molinari (1993) suggests that parameterization of cumulus 

convection for grid spacings of 2-20 km cannot be addressed with either the fully explicit (i.e. no 

cumulus parameterization) method or a hybrid parameterization approach. However, our 

ultimate goal of performing CMAQ visibility modeling requires us to be pragmatic and 

configure MM5 in a physically reasonable way that produces the most accurate representation of 

the 2002 meteorology. Our plan for the 12-km grid cumulus scheme, therefore, was to conduct 

several sensitivity tests to determine which cumulus scheme (or no cumulus scheme at all) gives 

the best performance in terms of rainfall and surface temperature, humidity, and wind. 

We tested the Betts-Miller scheme, which gave the best performance on the 36-km grid over the 

WRAP region, as well as the Kain-Fritsch I and II schemes. The Kain-Fritsch schemes include a 

parameterization of convective downdrafts. The Betts-Miller scheme, which was originally 

designed for use in general circulation (i.e. large-scale) models, does not. We might then expect 

that the Kain-Fritsch schemes should do a better job of handling convection on the 12-km grid, 

particularly in cases where convection is severe and vertical transport of energy and moisture in 

downdrafts is critical. 

A third alternative was to use no cumulus parameterization, and let the cloud microphysics 

scheme and core model dynamics explicitly simulate convection. This approach has some 

limitations. Molinari and Dudek (1992) have shown that the explicit simulation of convection in 

mesoscale models can cause an unphysical phase delay in the onset of convective rainfall 

because condensation cannot occur until the model becomes saturated at the resolved grid scale. 

In the real world, saturation occurs on a scale that is smaller than mesoscale model grid 

resolution; in a parameterization, the cloud model must saturate the column, but the grid need not 

be saturated. Molinari and Dudek also show that explicit convection in mesoscale models, which 

cannot contain an updraft and a downdraft in the same grid cell, can cause difficulty simulating 

the moist static energy profile in a convecting grid cell. This can create an unrealistic 

representation of the removal of the convective instability, leading to inaccuracies in the timing 

and amount of predicted convective rainfall. 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec4.doc 4-2 

For the purposes of the sensitivity tests, we configured MM5 so that the 12-km setup was 

identical to that of the 36-km run, and varied only the cumulus parameterization on both grids 

(Table 4-1). 

Table 4-1. MM5 configuration for the 12-km grid for cumulus sensitivity tests. 

Radiation Microphysics Cumulus LSM/PBL 

RRTM Reisner II Kain Fritsch I PX/ACM 

For example, the simulation denoted by KFII ran with KFII as the cumulus parameterization on 

the 12-km grid; the 12-km run was made using boundary conditions from a 36-km run which 

also used the KFII scheme. Likewise, boundary conditions for the KF and BM 12-km runs were 

driven from 36-km runs that used the KF and BM schemes, respectively. The BM0 run had the 

Betts-Miller scheme running on the 36-km grid, and no cumulus parameterization on the 12-km 

grid. The BM0a run is identical in its configuration to BM0, except that no surface analysis 

nudging of temperature or humidity was performed in BM0a. A summary of the different 

configurations is shown in Table 4-2. 

Table 4-2. Summary of cumulus sensitivity configurations.

Run Name 36-km Cumulus Scheme 12-km Cumulus Scheme 

KF I Kain-Fritsch I Kain-Fritsch I 

KF II Kain-Fritsch II Kain-Fritsch II 

BM Betts-Miller Betts-Miller 

BM0 Betts-Miller None 

BM0a Betts-Miller None 

Figure 4-1a is a summary of the surface wind speed performance. For the DesertSW subdomain, 

the RMSE is comparable among all the simulations, but the BM and BM0 runs have a smaller 

bias. For the other three WRAP domains, there is no significant difference in performance, 

although the BM0 has a slightly larger RMSE and bias than the other runs. It is difficult to say 

whether one scheme outperforms the others for wind speed. 

For temperature (Figure 4-1b), the KF run is a clear outlier for the DesertSW. For the North 

subdomain, the BM, BM0, and KFII runs are comparable, and KF and BM0a have larger bias 

and RMSE. For PacificNW and SW, performance is comparable, with BM0a having a slightly 

larger RMSE than the other schemes. Overall, BM0 and KFII are the best-performing 

simulations for temperature. 

The humidity performance is shown in Figure 4-1c. For DesertSW, KF is again an outlier, along 

with KFII. For DesertSW and North, the three BM configurations (BM, BM0, and BM0a) 

perform better, with BM0a having the smallest RMSE and bias of all schemes. For the Pacific 

NW and SW subdomains, performance is comparable among all the schemes. Overall, then, the 

BM0a configuration performs best for humidity. 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec4.doc 4-3 

Figure 4-1a. Wind speed scatterplot for comparison of 12-km cumulus test runs. 

Figure 4-1b. Temperature scatterplot for comparison of 12-km cumulus test runs. 

WRAP 12 km July Wind Performance Comparison

0

0.5

1

1.5

2

2.5

3

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

Wind Speed Bias (m/s)

WindSpeed RMSE (m/s)

Benchmark BM BM0 KF KFII BM0a

Cumulus SensitivityTest

WRAP 12 km July Temperature Performance Comparison

0

0.5

1

1.5

2

2.5

3

3.5

-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5

Temperature Bias (K)

Temeprature Error (K)

Benchmark BM BM0 KF KFII BM0a

Cumulus Sensitivity Test
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec4.doc 4-4 

Figure 4-1c. Humidity scatterplot for comparison of 12-km cumulus test runs. 

We turn now to an evaluation of the precipitation performance over the July 1-5 time period on 

the 12-km grid. Figure 4-2 shows the total (convective+resolved) rainfall for the period for each 

MM5 run as well as the CPC precipitation observations. The KF and KFII runs overestimate the 

rainfall over the SW and DesertSW, and North subdomains. They predict light rainfall over 

large areas of the domain where the observations show no rainfall at all. In the runs where BM is 

used in the 36-km simulations, there is a smaller overprediction of rainfall over these three 

subdomains. Of the three configurations with BM running at 36-km, the BM0a scheme shows 

the best performance. It has the smallest overprediction and provides the best simulation of all 

runs of the feature located over Montana. The KF I and KF II schemes, however, do a better job 

with the feature over northern Washington. This is under predicted in the Betts-Miller scheme 

runs. 

Based on the surface variable and rainfall performance, then, we select the BM0a configuration. 

The configuration with Betts-Miller running on the 36-km grid and no cumulus parameterization 

on the 12-km grid was among the best performers in terms of surface wind, temperature, and 

humidity, and performed the best for rainfall. We choose the BM0a configuration over BM0 

based on guidance from reviewers that surface analysis nudging of temperature and humidity 

degrades performance aloft. 

It is interesting to note that the Betts-Miller scheme, which is not recommended for use in MM5 

at resolutions less than 30 km, performs better than the KF and KFII schemes, which were 

designed with higher resolution simulations in mind. It is also noteworthy that the explicit 

WRAP 12 km Humidity Performance Comparison

0

1

2

3

4

5

-3.5 -2.5 -1.5 -0.5 0.5 1.5 2.5 3.5

Humidity Bias (g/kg)

Humidity Error (g/kg)

Benchmark BM BM0 KF KFII BM0a

Cumulus Sensitivity Test
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec4.doc 4-5 

 

 

Figure 4-2. Precipitation comparison for the cumulus parameterization sensitivity test. 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec4.doc 4-6 

12-km simulation also outperformed KF and KFII. We must bear in mind that these tests were 

performed over a short time period in July. This is a time period when MM5 is producing too 

much convective rainfall. The BM0a configuration may not perform as well as the others for 

other seasons. In winter, for example, overprediction of convective rainfall is not a performance 

issue. Most of the rainfall is stratiform precipitation produced by mid-latitude cyclones, and 

MM5 does a reasonably good job of simulating the winter rainfall in 2002. It is unclear whether 

BM0a would still be the best-performing simulation for this season. Further testing, however, is 

limited by time constraints, and so we select the BM0a configuration for the 12-km domain 

simulation. 

SOIL MOISTURE NUDGING TESTS 

The poor 36-km MM5 model performance in the western U.S. relative to the eastern U.S. may be 

due in part to the different (drier) western soil characteristics. Therefore, we performed a series 

of 12-km MM5 simulations for the period July 1-5 2002 in which the soil moisture nudging and 

the method of soil moisture specification were varied. All of these runs used the same physics 

options shown in Table 4-3 and the KFI cumulus scheme. In Run 1, the 36-km MM5 simulation 

output was used to generate initial and boundary conditions for the 12-km run. In Run 2, the 

Pleim-Xiu LSM was initialized from the default land use lookup table (LANDUSE.TBL), and 

the soil moisture was not nudged. In Run 3, the PX LSM was initialized with the 36-km MM5 

output, while in Run 4, the PX LSM was initialized using the EDAS analysis fields. No soil 

moisture nudging was performed in Runs 3 and 4. These are listed in Table 4-4. 

Table 4-3. Physics options common to all 12-km domain soil moisture sensitivity tests. 

Cloud Microphysics Cumulus PBL/LSM Radiation 

Reisner II Kain-Fritsch I ACM/PX RRTM 

Table 4-4. Physics options varied for 12-km domain sensitivity tests. 

Soil Moisture 

Nudging 

Soil Moisture 

Initialization 

12km_run1 Yes 36km output 

12km_run2 No LANDUSE.TBL 

12km_run3 No 36km output 

12km_run4 No EDAS analysis 

The time series of hourly statistics for wind, temperature and humidity over the WRAP 

subdomains SW, PacificNW, and DesertSW are displayed in Figure 4-3. Figure 4-3a shows time 

series of surface wind, temperature, and humidity for the SW subdomain. For winds, there is not 

much difference among the four runs. For temperature, Run 4 performs the best, and Run 1 

underestimates the amplitude of the diurnal cycle, giving the worst performance. For humidity, 

all runs show a spurious diurnal cycle, and Run 4 is the best performing run. 

For DesertSW (Figure 4-3b), it is hard to say whether one run is better than the others for wind. 

For temperature, however, Run 2 has the smallest bias. For humidity, all 4 runs have a wet bias, 

which is most pronounced in Run 1. The wet bias is less pronounced in Runs 2 and 4. 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec4.doc 4-7 

For PacificNW (Figure 4-3c), the temperature and wind time series are inconclusive, but for 

humidity, it is clear than Run 4 has the smallest bias. We conclude then, that Run 4 is the best 

performing run overall. These tests show that turning off the soil moisture nudging has a 

positive impact on performance. 

The main results from the 12-km Runs 1-4 for the western subdomains over the period July 1-5, 

2002 are as follows: 

• Use of soil moisture nudging degrades temperature and humidity model performance. 

• Initializing the soil moisture with the 36-km domain MM5 output degrades temperature 

and humidity model performance. 

• Better temperature and humidity performance was obtained by initializing the soil model 

with the EDAS analysis fields. 
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec4.doc 4-8 

 

Figure 4-3a. Time series of observed and MM5 surface wind, temperature and humidity for the 

SW domain. 

Observed/Predicted Windspeed

0

2

4

6

8

 7/ 1 7/ 2 7/ 3 7/ 4 7/ 5

m/s

ObsWndSpd run1 run2 run3 run4 run5

Observed/Predicted Wind Direction

0

60

120

180

240

300

360

 7/ 1 7/ 2 7/ 3 7/ 4 7/ 5

deg

ObsWndDir run1 run2 run3 run4 run5

Observed/Predicted Temperature

275

280

285

290

295

300

305

310

 7/ 1 7/ 2 7/ 3 7/ 4 7/ 5

K

ObsTemp run1 run2 run3 run4 run5

Predicted/Observed Humidity

0

2

4

6

8

10

12

 7/ 1 7/ 2 7/ 3 7/ 4 7/ 5

g/kg

ObsHum run1 run2 run3 run4 run5
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec4.doc 4-9 

Figure 4-3b. Time series of observed and MM5 surface wind, temperature and humidity for the 

DesertSW domain. 

Observed/Predicted Windspeed

0

2

4

6

8

 7/ 1 7/ 2 7/ 3 7/ 4 7/ 5

m/s

ObsWndSpd run1 run2 run3 run4 run5

Observed/Predicted Wind Direction

0

60

120

180

240

300

360

 7/ 1 7/ 2 7/ 3 7/ 4 7/ 5

deg

ObsWndDir run1 run2 run3 run4 run5

Observed/Predicted Temperature

275

280

285

290

295

300

305

310

 7/ 1 7/ 2 7/ 3 7/ 4 7/ 5

K

ObsTemp run1 run2 run3 run4 run5

Predicted/Observed Humidity

0

2

4

6

8

10

12

14

 7/ 1 7/ 2 7/ 3 7/ 4 7/ 5

g/kg

ObsHum run1 run2 run3 run4 run5
December 2004 

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\Sec4.doc 4-10 

Figure 4-3c. Time series of observed and MM5 surface wind, temperature and humidity for the 

PacificNW domain. 

Observed/Predicted Windspeed

0

2

4

6

8

 7/ 1 7/ 2 7/ 3 7/ 4 7/ 5

m/s

ObsWndSpd run1 run2 run3 run4 run5

Observed/Predicted Wind Direction

0

60

120

180

240

300

360

 7/ 1 7/ 2 7/ 3 7/ 4 7/ 5

deg

ObsWndDir run1 run2 run3 run4 run5

Observed/Predicted Temperature

270

275

280

285

290

295

300

 7/ 1 7/ 2 7/ 3 7/ 4 7/ 5

K

ObsTemp run1 run2 run3 run4 run5

Predicted/Observed Humidity

0

1

2

3

4

5

6

7

8

 7/ 1 7/ 2 7/ 3 7/ 4 7/ 5

g/kg

ObsHum run1 run2 run3 run4 run5
December 2004

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\References_revsd.doc R-1 

REFERENCES 

Arakawa, A. and J.-M. Chen, 1987: Closure assumptions in the cumulus parameterization 

problem. Short and Medium Range Numerical Prediction, Collection of Papers 

Presented at the WMO/IUGG NWP Symposium, Tokyo. T. Matsuno, Ed. Meteor. 

Soc. Japan. 107-131. 

ATMET. 2003. "MM5 Simulations for TexAQS 2000 Episode, Task 3: Sensitivities to 

modifications of the MRF PBL scheme, Draft Final Report." Prepared for the Houston 

Advanced Research Center, The Woodlands, TX, and the Texas Commission on 

Environmental Quality, Austin, TX, by ATMET, LLC, Boulder, CO (30 September, 

2003). 

Emery, C.A., Tai, 2001: Enhanced meteorological modeling and performance evaluation for two 

Texas ozone episodes. Prepared for the Texas Natural Resource Conservation 

Commission, by ENVIRON International Corporation. 

Emery, C.A., S. Kemball-Cook, Y. Jia, Z. Wang and R. Morris. 2004 “2002 MM5 Model 

Evaluation: 12 vs 36 km Results.” Presented at May 24-25, 2004 National RPO 

Modeling Meeting, Denver, CO. http://www.cleanairinfo.com/rpomodelingdenver . 

ENVIRON and UCR. 2004. “2002 Annual MM5 Simulations to Support WRAP CMAQ 

Visibility Modeling for the Section 308 SIP/TIP. Draft Protocol. ENVIRON 

International Corporation and the University of California at Riverside. April. Available 

at: http://pah.cert.ucr.edu/aqm/308/mm5.shtml 

Kemball-Cook, S., Y. Jia, C. Emery, R. Morris, Z. Wang and G. Tonnesen. 2004. “2002 Annual 

MM5 36 km simulation to support WRAP CMAQ Visibility Modeling for the Section 

308 SIP/TIP: Preliminary Report on the Initial 2002 36 km MM5 Simulation Performed 

during 2003. Interim Report. ENVIRON International Corporation and UC Riverside. 

March 2004. Available at: http://pah.cert.ucr.edu/aqm/308/mm5.shtml 

Molinari, J. and M. Dudek, 1992: Parameterization of convective precipitation in mesoscale 

numerical models. Mon. Wea. Rev., 120, 326-344. 

Molinari, J., 1993. An overview of cumulus parameterization in mesoscale models. The 

Representation of Cumulus Convection in Numerical Models. K. Emanuel and D. 

Raymond, Ed. American Meteorological Society, p.155-159. 

Morris, R.E., S. Kemball-Cook, Y. Jia, C. Emery and Z. Wan. 2004a. “2002 MM5 36 km 

Evaluation.” Presented at WRAP Regional Modeling Center Workshop, Tempe Arizona. 

January 28-29, 2004. Available at: 

http://pah.cert.ucr.edu/aqm/308/meetings/Jan_2004_ppt/040128Modeling_Workshop_A

genda.htm

Morris, R.E., B. Koo, S. Lau, T.W. Tesche, D. McNally, C. Loomis, G. Stella, G. Tonnesen and 

Z. Wang. 2004b. “VISTAS Emissions and Air Quality Modeling – Task 4cd Report: 
December 2004

G:\WRAP RMC\2002MM5\MM5_Sensitivity_Report\Revised\References_revsd.doc R-2 

Model Performance Evaluation and Model Sensitivity Tests for Three Phase I Episodes. 

ENVIRON International Corporation, Novato, California. September. Available at: 

http://pah.cert.ucr.edu/vistas/docs.shtml . 

Olerud, D. and A. Sims, 2003: MM5 Sensitivity modeling in support of VISTAS (Visibility 

Improvement-State and Tribal Association) (DRAFT). Prepared for Mike Abraczinskas 

of the VISTAS Technical Analysis Workgroup by Baron Advanced Meteorological 

Systems, LLC. 

Tesche, T.W. et al. 2002. "Operational Evaluation of the MM5 Meteorological Model over the 

Continental United States: Protocol for Annual and Episodic Evaluation." Prepared for 

the U.S. Environmental Protection Agency, Office of Air Quality Planning and 

Standards, prepared by Alpine Geophysics, LLC, Ft. Wright, KY.