Draft Final Report

Annual 2002 MM5 Meteorological Modeling to Support Regional Haze Modeling of the Western United States

Prepared for

The Western Regional Ai r Partnership (WRAP) 1515 Cleveland Place, Suite 200 Denver, CO 80202

Prepared by

Sue Kemball-Cook Yiqin Jia Chris Emery Ralph Morris ENVIRON International Corporation 101 Rowland Way, Suite 220 Novato, CA 94945-5010

Zion Wang

Gail Tonnesen University of California at Riverside College of Engineering Center for Environmental Research and Technology Riverside, CA 92507

March 2005

101 Rowland Way, Suite 220, Novato, CA 94945 415.899.0700March 2005

i TABLE OF CONTENTS

Page

1.0 INTRODUCTION............................................................................................................... . 1-1

Background.................................................................................................................... ........ 1-1 Overview of WRAP MM5 Modeling .................................................................................... 1-1

2.0 MM5 SETUP FOR 36 KM CENRAP , VISTAS, and WRAP RUNS .............................. 2-1

MM5 Configuration.............................................................................................................. . 2-1 Four Dimensional Data Assimilation (FDDA)...................................................................... 2-7

3.0 EVALUATION METHODOLOGY FOR THE 2002 ANNUAL MM5 RUNS.............. 3-1

Graphical Evaluation ........................................................................................................... .. 3-1

Statistical Evaluation ......................................................................................................... .... 3-2 The METSTAT Analysis Package......................................................................................... 3-5

Statistical Performance Benchmarks ..................................................................................... 3-6 Surface Statistical Analyses for WRAP................................................................................. 3-6 Precipitation Evaluation....................................................................................................... .. 3-8

4.0 SURFACE EVALUATION OF THE 2002 CE NRAP, VISTAS, and WRAP RUNS.... 4-1

Subdomain Surface Field METSTAT Evaluation ................................................................. 4-1 Evaluation of Precipitation in the 36 km 2002 CENRAP, VISTAS, and WRAP Runs............................................................................................ 4-17 Upper-Air Meteorological Evaluation ................................................................................. 4-26

5.0 EVALUATION OF THE 2002 12 KM WRAP MM5 SIMULATION............................ 5-1

Evaluation of Precipitation in the 2002 12 km WRAP Run .................................................. 5-6

6.0 CONCLUSIONS ................................................................................................................ .. 6-1

REFERENCES..................................................................................................................... ..... R-1

TABLES

Table 1-1. MM5 Configuration from original (WRAP_0) 2002 MM5 run .......................... 1-3 Table 1-2. MM5 Configuration for the final WRAP 2002 36 km MM5 run........................ 1-3 Table 2-1. Physics options common to all four MM5 runs................................................... 2-3 Table 2-2. Physics/FDDA options which differed among the four MM5 runs..................... 2-4 Table 2-3. Final MM5 configur ation for the 12-km grid ...................................................... 2-6

Table 3-1. Statistical benchmarks for evaluating meteorological model performance. ............................................................................................. 3-6

Table 4-1. METSTAT subdomain abbreviations.................................................................. 4-2March 2005

ii FIGURES

Figure 2-1. Spatial coverage of the RPO National Grid with 36-km grid point spacing. .................................................................................... 2-2 Figure 2-2. Coverage of the WR AP western regional domain with 12-km grid point spacing. .................................................................................... 2-3 Figure 3-1. METSTAT subdomains for the WRAP 2002 MM5 performance evaluation........................................................................................ 3-7 Figure 4-1. 36 km surface wind comparison soccerplot for January

over western U.S. for VIST AS, CENRAP, WRAP_0 (old WRAP) and final WRAP (WRAP) 2002 36 km simulation.............................................. 4-3 Figure 4-2. 36 km surface wind comparison soccerplot for January

over central and eastern U.S. for VISTAS, CENRAP, WRAP_0 (old WRAP) and final WRAP (WRAP) 2002 36 km simulation........ 4-3 Figure 4-3. WRAP_0 run surface wind speed and bias for January over

desertSW subdomain ........................................................................................... 4-4 Figure 4-4. Final WRAP run surface wind speed and bias for January

over desertSW subdomain. .................................................................................. 4-4 Figure 4-5. WRAP_0 run surface wind direction and bias for January

over desertSW subdomain. .................................................................................. 4-5 Figure 4-6. Final WRAP run surface wind direction and bias for January

over desertSW subdomain ................................................................................... 4-5 Figure 4-7. 36 km temperature co mparison soccerplot for January

over western U.S. for VIST AS, CENRAP, WRAP_0 (old WRAP) and final WRAP (WRAP) 2002 36 km simulation.............................................. 4-6 Figure 4-8. 36 km temperature comp arison for January over central and eastern U.S. for VISTAS, CENRAP, WRAP_0 (old WRAP) and final WRAP (WRAP) 2002 36 km simulation.............................................. 4-6 Figure 4-9. WRAP_0 run January temperature time series for desertSW subdomain. .......... 4-7 Figure 4-10. Final WRAP run January temperature time series for

desertSW subdomain ........................................................................................... 4-7 Figure 4-11. 36 km humidity comparison soccerplot for January

over western U.S. for VIST AS, CENRAP, WRAP_0 (old WRAP) and final WRAP (WRAP) 2002 36 km simulation.............................................. 4-8 Figure 4-12. 36 km humidity comparison soccerplot for January over

central and eastern U.S. for VISTAS, CENRAP, WRAP_0 (old WRAP) and final WRAP (WRAP) 2002 36 km simulation ........................ 4-8 Figure 4-13. 36 km wind comparison soccerplot for July over western U.S. for VISTAS, CENRAP, WR AP_0 (old WRAP) and final WRAP (WRAP) 2002 36 km simulation........................................................... 4-11 Figure 4-14. 36 km wind comparison soccerplot for July over central and eastern U.S. for VISTAS, CENRAP, WRAP_0 (old WRAP) and final WRAP (WRAP) 2002 36 km simulation............................................ 4-11 Figure 4-15. 36 km temperature comparison soccerplot for July over western U.S. for VISTAS, CENRAP, WRAP_0 (old WRAP) and final WRAP (WRAP) 2002 36 km simulation............................................ 4-12 Figure 4-16. 36 km temperature comparison soccerplot for July over

central and eastern U. S for VISTAS, CENRAP, WRAP_0 (old WRAP) and final WRAP (WRAP) 2002 36 km simulation.. .................... 4-12March 2005

iii Figure 4-17. WRAP_0 run July temperatur e time series for SW subdomain. ....................... 4-13 Figure 4-18. Final WRAP run July temperat ure time series for SW subdomain. .................. 4-13 Figure 4-19. WRAP_0 run July temperature time series for North subdomain. .................... 4-14 Figure 4-20. Final WRAP run July temperatur e time series for North subdomain................ 4-14 Figure 4-21. 36 km humidity comparison soccer plot for July over western U.S................... 4-15 Figure 4-22. 36 km humidity comparison soccerplot for July over eastern

and central U.S................................................................................................... 4-15 Figure 4-23. January observed (top le ft) and modeled precipitation for CENRAP (top right), VISTAS (middle left), final WRAP (middle right) and WRAP_0 (bottom right) 2002 36 km simulations

and differences between WRAP-0 and final WRAP (bottom left).................... 4-19 Figure 4-24. Partitioning of January tota l rainfall into its convective and stratiform components for the WRAP_0 (top) and final WRAP (bottom) 36 km MM5 runs.................................................................... 4-20 Figure 4-25. March observed (top le ft) and modeled precipitation for CENRAP (top right), VISTAS (middle left), final WRAP (middle right) and WRAP_0 (bottom right) 2002 36 km

simulations and differen ces between WRAP-0 and final WRAP (bottom left).......................................................................................... .4-21 Figure 4-26. July observed (top left ) and modeled precipitation for CENRAP (top right), VISTAS (middle left), final WRAP (middle right) and WRAP_0 (bottom right) 2002 36 km

simulations and differen ces between WRAP-0 and final WRAP (bottom left)........................................................................................... 4-22 Figure 4-27. Partitioning of July to tal rainfall into its convective and stratiform components for the WRAP_0 (top) and final WRAP (bottom) 36 km MM5 runs.................................................................... 4-23 Figure 4-28. October observed (top le ft) and modeled precipitation for CENRAP (top right), VISTAS (middle left), final WRAP (middle right) and WRAP_0 (bottom right) 2002 36 km simulations

and differences between WRAP-0 and final WRAP (bottom left).................... 4-24 Figure 4-29. December observed (top left) and modeled precipitation for CENRAP (top right), VISTAS (middle left), final WRAP (middle right) and WRAP_0 (bottom right) 2002 36 km

simulations and differen ces between WRAP-0 and final WRAP (bottom left)........................................................................................... 4-25 Figure 4-30. January 7, 2002, 12Z Sounding for Midland, TX. Upper left hand panel: CENRAP, Upper right hand panel: VISTAS, Lower left hand panel: WRAP_0, Lower right hand panel: Final WRAP ............................................................... 4-29 Figure 4-31. July 2, 2002, 12Z Sounding for Spokane, WA. Upper left hand panel: CENRAP, Upper right hand panel: VISTAS, Lower left hand panel: WRAP_0, Lower right hand panel: Final WRAP. .............................................................. 4-30 Figure 4-32. July 16, 2002, 0Z Sounding for Midland, TX. Upper left hand panel: CENRAP, Upper right hand panel: VISTAS, Lower left hand panel: WRAP_0, Lower right hand panel: Final WRAP........................................................................... 4-31March 2005

iv Figure 4-33. January 4, 2002, 12Z Sounding for Flagstaff, AZ. Upper left hand panel: CENRAP, Upper right hand panel: VISTAS, Lower left hand panel: WRAP_0, Lower right hand panel: Final WRAP........................................................................... 4-32 Figure 4-34. July 4,6, 2002, 0Z Soundings for Flagstaff, AZ. Upper left hand panel: WRAP_0 July 4, Upper right hand panel: Final WRAP July 4, Lower left hand panel: WRAP_0 July 6, Lower right hand pane l: Final WRAP Jul 6....................................................... 4-33 Figure 5-1. 36km/12 km surface temperatur e comparison soccerplot for January. ............... 5-3 Figure 5-2. 36km/12 km surface wind comparison soccerplot for January ........................... 5-3 Figure 5-3. 36km/12 km surface humidity comparison soccerplot for January..................... 5-4 Figure 5-4. 36km/12 km surface temperat ure comparison soccerplot for July ...................... 5-4 Figure 5-5. 36km/12 km surface wind comparison soccerplot for July ................................. 5-5 Figure 5-6. 36km/12 km humidity wind comparison soccerplot for July .............................. 5-5 Figure 5-7A-J. Annual cycle in 12 km MM5 precipitation.......................................................... 5-7March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newW RAP_eval\DraftFinal\sec1.doc 1-1 1.0 INTRODUCTION

BACKGROUND

In 1999, the U.S. Environmental Protection Agency (EPA) announced a major effort to improve visibility in national parks and wilderness areas. Its Regional Haze Rule (RHR) calls for state and federal agencies to work together to impr ove visibility in 156 Federally mandated Class I areas that includes national parks, wilderness areas and wildlife refuges (ref erred to as “Class I” areas). The objective of the RHR is to achieve natural visibility conditions in these federally protected lands by the year 2064. The rule requi res the states to develop and implement air quality protection plans to reduce the pollution th at causes visibility impairment. The Western Regional Air Partnership (WRAP) is a consortium of federal, state, and tribal agencies charged with implementing regional planning processes to improve visibility in western Class I areas. To meet this goal, WRAP is providing the necessary technical and policy tools to help states and tribes implement the RHR.

The WRAP has formed the Regional Modeling Cent er (RMC), consisting of the University of California at Riverside (UCR), E NVIRON International Corporation and the University of North Carolina, Carolina Environmental Program (UNC /CEP). The RMC performed the modeling and analysis necessary to develop the Section 309 St ate and Tribal Implemen tation Plans (SIPs/TIPs) and are performing the analysis for the Section 308 SIPs/TIPs. This project is focused on seasonal-annual regional visibility modeling us ing a deterministic phot ochemical grid model capable of simulating the formation, transport, an d fate of tropospheric particulate matter (PM) and photochemical oxidants on time scales of on e hour. The RMC is utilizing EPA’s Models-3 Community Multiscale Air Qual ity (CMAQ) modeling system and the Comprehensive Airquality Model with extensions (CAMx) for this purpose. The models will be applied for the entire year of 2002 on two grids: a continental-scale domain w ith 36-km grid spacing, and a western U.S. regional-scale domain with 12-km gr id spacing covering the western U.S. Class I

areas. The Fifth Generation Me soscale Model (MM5), devel oped and maintained by the Pennsylvania State University and National Cent er for Atmospheric Research (PSU/NCAR), is being used to supply hourly meteorological fiel ds on these two grids. The Sparse Matrix Operating Kernel Emissions (SMOKE) system is being used to develop the anthropogenic and biogenic gas and PM emission rate estimates for CMAQ and CAMx.

OVERVIEW OF WRAP MM5 MODELING

Initial MM5 Modeling

MM5 simulations were carried out by the WR AP Regional Modeling Center (RMC) for the entirety of 2002 to support vi sibility modeling for the Sectio n 308 SIP/TIP that are due in December 2007. During the fall of 2003, the RMC made an initial MM5 run for the year 2002 on the single unified National RPO 36-km gri d. An MM5 Modeling Protocol was prepared (ENVIRON and UCR, 2004), whic h describes in detail the MM5 model and the setup and evaluation methods used in the 2002 modeling effo rt. The model performance evaluation for an initial 2002 MM5 run on a single continental-scale 36-km grid and results from preliminary MM5 sensitivity tests, have been discussed by Mo rris et al. (2004a), Ke mball-Cook et al. (2004a and 2004b), and Emery et al. (2004).March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newW RAP_eval\DraftFinal\sec1.doc 1-2 The model configuration for this initial run was based upon a prior 2002 MM5 application undertaken by the Iowa Department of Na tural Resources (IDNR; Johnson, 2003 Personal Communication), which was in turn set accordin g to the optimal MM5 physics options that resulted from an in-depth sensitivity project carried out by IDNR and the Lake Michigan Air Directors Consortium (LADCo). While the I DNR simulations used MM5 version 3.5, the applications carried out for WRAP used the late st version of the model (v3.6.1) available at the

end of 2003. Additional modifications to the physics configuration and application methodology

were made for the WRAP simulation based on th e latest information from EPA, IDNR, LADCo, and others. First, the RMC opted to use the Reisner II mixed-phase cloud microphysics package according to suggestions from EPA/ORD. Second, the INTERPPX option, which allows for

continuous soil moisture initialization from one run segment to the next, was not used based on poor performance reported by IDNR and LADCo.

The initial 2002 36-km WRAP simu lation (referred to hereafter as WRAP_0) results showed that MM5 performs better in the Central and Eastern U. S. than in the West, and performs generally

better in winter than in summ er (Morris et al., 2004a; Kemba ll-Cook et al., 2004a). In the western U.S., the amplitude of diurnal temperat ure cycle was persistently underestimated during the summer, especially in the southwest. In the desert southwest, the humidity was greatly overestimated during the summer as well, and ther e was a pronounced cold bias. Some of these problems appeared to be linked to the exces sive simulated precipi tation generated by MM5 during the summer, especially in the southwest. This can have serious repercussions for CMAQ modeling since too much rain can “wash out” po llutants, while the too cool, humid and cloudy environment may lead to incorrect polluta nt chemistry and aerosol thermodynamics. Temperature and humidity problems overshadowed the surface wind performance, which was not particularly good, but was likel y affected by smaller-scale topogr aphic influences that are not well represented even at the finer 12 km resolu tion. Wind performance improved quickly with height above the surface, suggesting that regi onal transport speeds/dire ctions were reasonably represented.

Preliminary Sensitivity Tests

Initial sensitivity tests were undertaken for a 5-day period in July 2002 for the 36-km National domain in an attempt to find alternative MM5 configurations that would improve the poor summertime performance in the western U.S. Fu rther analyses were conducted to compare the WRAP_0 run’s performance against that of a 36-km MM5 run made for VISTAS, and against operational Eta Data Assimilation System (EDAS) fields used as input to MM5. From these analyses, we found that the use of the Kain-Fri tsch II scheme (as used in the VISTAS 2002 MM5 modeling compared to the Kain-F ritsch I scheme used in the initial WRAP modeling) improved, but did not entirely solve, the precipitation ove r prediction problem in the WRAP_0 run. We

also found that removal of soil moisture nudging improved temperature and humidity performance for the short summertime tests. Ev aluation of the EDAS fields used in the MM5

four-dimensional-data-assimilation (FDDA) revealed that EDAS did not exhibit the summer time

cold wet bias in the southwest. This indicated that the bias was not in troduced by the FDDA. A new model configuration (Run 5) was identified from these initial tests for new annual 2002 simulations, and the WRAP MM5 modeling protocol was updated to reflect this, as shown in the

Table 1-1.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newW RAP_eval\DraftFinal\sec1.doc 1-3 Table 1-1 . MM5 Configuration from original (WRAP_0) 2002 MM5 run.

LSM PBL Cumulus Microphysics Analysis FDDA
3D Surface Obs

FDDA Soil

Moisture Nudging WRAP_0 PX ACM KF I Reisner II W/T/H W/T/H None Yes WRAP_5 PX ACM KFII Reisner II W/T/H None No

Although the WRAP_5 configuratio n did improve humidity, temperature, and precipitation performance over the short July test period (esp ecially where improvement was needed most),

the desert southwest continued to exhibit unsatisfactor y levels of precipita tion and humidity. As stated above, these issues will li kely play crucial roles in the air quality model performance.

When results of the interim MM5 model run were presented at the May 24-25, 2004 National RPO modeling meeting in Denver, Colorado (Emery et al, 2004), there were still concerns about the 2002 MM5 model performance in the western U. S. Thus, WRAP requested that the Regional Modeling Center (RMC) perform fu rther MM5 sensitivity tests to identify a better performing configuration, including invest igating alternative Land Surface Model and Planetary Boundary Layer (LSM/PBL) configurations to Plein- Xiu/Asymmetric Convec tive Mixing (PX/ACM) schemes used in the initial WRAP MM5 modeling and include the possibility of using different MM5 configurations for different times of year.

Revised Sensitivity Tests

We carried out additional 36-km MM5 test simula tions in an attempt to further improve MM5 performance. The latest available version of MM5 (v3.6.2) was used. Since the most severe problems for the initial WRAP 200 2 MM5 annual simulation were the wet bias and precipitation overestimation in summer, we first investigated th ese issues for the 5-day July test period (July 1-5, 2002) on the 36-km grid to maximize computat ional efficiency. We tested four physics options: (1) the cumulus parameterization; (2) th e land surface models (LSM); (3) the planetary boundary layer (PBL) models; and (4) the fou r-dimensional data assimilation (FDDA).

We then undertook a series of tests to determin e the optimal MM5 configuration for the nested 12-km grid of the WRAP modeling domain. In these tests, we varied only the physics options that we expected to ameliorate the biases in the WRAP region seen in the 36-km simulation. The over prediction of summer rainfall and associat ed surface humidity bias suggested that the cumulus parameterization might need to be chan ged. Another possible reason for the wet bias was the soil moisture initialization. Therefore, we focused on sensitivity tests of the soil moisture specification and the cumulus scheme . The results of these sensitivity tests are

discussed in Kemball-Cook et al . (2004b). The more optimal model configuration for the 36-km application is given in Table 1-2.

Table 1-2 . MM5 Configuration for the final WRAP 2002 36 km MM5 run.

A nalysis FDDA
LSM PBL Cumulus Microphysics 3D Surface Obs FDDA

Pleim-Xiu ACM Betts-Miller Reisner II W/T/H W/T/H WindMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newW RAP_eval\DraftFinal\sec1.doc 1-4 MM5 runs for the test period of July 1-5, 2002 made with these physics opti ons selected showed:

• A dramatic reduction in the summertime co ld, wet bias in the desert southwest;

• Surface temperature and humidity performance within benchmarks for all WRAP regions except desert southwest for temperature;

• More accurate representation of the diurnal temperature cy cle in the desert southwest;

• Improvements in the index of agreement for te mperature, humidity and wind speed for all WRAP subdomains;

• A more realistic precipitation pattern over the western U.S.;

• Better model performance in the eastern U.S.; and

• Runs made with these phys ics options generally repres ented an improvement in performance throughout the year, making it unn ecessary to select different physics

schemes for different seasons.

In summary, the new configuration of MM5 re presented a significant improvement in model performance relative to the WRAP_0 and WRAP_5 2002 36-km MM5 runs for the 5 day test period.

We then conducted a series of tests to determin e the optimal MM5 configuration for the nested 12-km grid of the WRAP modeling domain. In th ese tests, we again focused on sensitivity tests of the soil moisture specification and the cumulus scheme.

The main results from the 12-km runs over the test period of July 1-5, 2002 are as follows:

• Use of soil moisture nudging degraded temp erature and humidity model performance.

• Initializing the soil moisture with the 36-km domain MM5 output degraded temperature and humidity model performance.

• Better temperature and humidity performance was obtained by initializing the soil model with the EDAS analysis fields.

Final Model Configurations

Based upon our evaluation of surface variable an d rainfall performance, we selected a configuration with the Betts-Mi ller cumulus scheme running on the 36-km grid and no cumulus parameterization on the 12-km gri d. This was among the best performers in terms of surface wind, temperature, and humidity, and performed the best for rainfall. We also chose to remove surface analysis nudging of temperature and humid ity from both grids, based on guidance from reviewers that nudging to these variables can de grade performance alof t. Other MM5 model options remained as given in Table 1-2. The fi nal 36/12-km configuration was then used for the annual 2002 MM5 simulation to support WR AP’s regional haze modeling. The WRAP Modeling Protocol (ENVIRON and UCR 2004) was updated to reflect the new model configuration.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newW RAP_eval\DraftFinal\sec1.doc 1-5 Report Organization

In this document, we evaluate the final contin ental-scale 36-km grid MM5 run performed by the WRAP RMC during 2004 and compare its performan ce with the initial WRAP_0 run as well as with two additional 36 km continental-scale annual runs performed by the CENRAP and VISTAS RPOs. First, we present brief descrip tions of the MM5 physical configuration used in each MM5 run. Then, we evaluate the perfor mance of each model r un in replicating the evolution of observed winds, temperature, humidity, and boundary layer morphology to the

extent that resources and data ava ilability allow; this serves as an assessment of the reliability of

the final WRAP run’s 36-km meteor ological fields in adequately characterizing the state of the atmosphere and for serving as boundary conditions for the 12-km regional-scale WRAP domain MM5 run and the meteorological driver for the CM AQ 36-km continental-scal e run. Finally, we

evaluate the final 12 km WRAP run and discuss its suitability for provi ding meteorological conditions for the proposed 12 km CMAQ application.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\sec2.doc 2-1 2.0 MM5 SETUP FOR 36 KM CE NRAP, VISTAS, and WRAP RUNS

In this section, we define the configurations for the CE NRAP, VISTAS, WRAP_0 (initial WRAP) and final WRAP 2002 36 km and 12 km MM5 simulations.

MM5 CONFIGURATION

The CENRAP modeling configurat ion is described in Johnson (2004), and the VISTAS run is

described in Olerud and Sims (2003). The WRAP MM5 modeling system configuration for the

2002 36 km and 12 km annual runs is describe d in the WRAP RMC MM5 Modeling Protocol (ENVIRON and UCR 2004). Additional modifi cations to the physics configuration and application methodology were made for the WRAP MM5 simulation based on the results of a sensitivity study aimed at improving the performa nce problems seen in the first WRAP run (Kemball-Cook et al. 2004).

Modeling Domain

In all four 36 km runs (CEN RAP, VISTAS, WRAP_0 and final WRAP), MM5 was configured to run on the standard continental-scale Regi onal Planning Organizati on (RPO) National Grid with 36-km grid point spacing (Figure 2-1). The RPO National Grid is defined on a Lambert conformal projection, with true latitudes at 33 ° N and 45 ° N, and the central la titude and longitude

at 40 ° N and 97 ° W, respectively. The grid point spacing is 36 km. The con tinental expanse of this domain results in a grid of 165 (east-west ) by 129 (north-south) dot points, and 164 (eastwest) by 128 (north-south ) cross points (Figure 2-1). Overall, the domain covers 5904 km by 4608 km. The 12 km WRAP domain is shown in Figure 2-2.

Vertical Grid Structure

The vertical layer structure of the WRAP_0 and final WRAP runs is detailed in the Modeling

Protocol. The vertical structure for the CENRAP and VISTAS runs was similar to that of the

two WRAP runs. Both CENRAP a nd VISTAS runs used 34 layers in the vertical, but the exact layer definitions varied slightly.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\sec2.doc 2-2 Figure 2-1 . Spatial coverage of the RPO National Grid with 36-km grid point spacing.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\sec2.doc 2-3 Figure 2-2 . Coverage of the WRAP western regional domain with 12-km grid point spacing.

Physics Parameterizations and FDDA

In this section, we describe the major similari ties and differences between the three model runs

. Table 2-1 shows the physics options common to all four runs, whereas Table 2-2 displays these

differences.

Table 2-1. Physics options common to all four MM5 runs.

Land Surface Model Pleim-Xiu LSM (Used without interppx option in all four runs) Planetary Boundary Layer ACM Planetary Boundary Layer Radiation RRTM longwave radiation scheme, Dudhia

shortwave radiation scheme

Shallow Convection No shallow convectionMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\sec2.doc 2-4 Table 2-2. Physics/FDDA options which differed among the four MM5 runs.

CENRAP VISTAS WRAP_0 WRAP_Final

Cumulus

convection Kain-Fritsch II Kain-Fritsch II Kain-Fritsch I Betts-Miller

Moist Physics Reisner I Reisner I Reisner II Reisner II Analysis Nudging
at the Surface U/V U/V U/V/T/Q U/V

Analysis Nudging
Aloft U/V/T/Q U/V/T/Q U/V/T/Q U/V/T/Q

Surface Obs Nudging W

Land Surface Model/Planetary Boundary Layer Model

The Pleim-Xiu (PX) parameterization (Xiu a nd Pleim 2000) is a pred ictive/intera ctive soil temperature and moisture budget model that responds to atmospheric proce sses that affect the thermodynamics of the surface (e.g., rainfall) wh ile in turn dictati ng the surface fluxes of momentum, heat and moisture into the boundary laye r to further affect atmospheric processes. The PX approach maintains a historical “memory” of the soil conditions over the course of a

continuous simulation.

Through selection of the Pleim-Xiu land surface mode l (LSM) all four runs were required to use the Asymmetric Convective Mixing (ACM) Planet ary Boundary Layer (PBL) option (Pleim and Chang 1992), as these two are directly couple d. The INTERPX option, which may be employed in multi-segment runs to re-initialize the soil temperature and moisture fields for the next model run segment according to the conditions at the end of the previous run, was not used in any of the four runs. This was in response to sensitivity studies by the IDNR and VISTAS (Olerud and Sims, 2003), which showed that use of INTERPPX can lead to surface temp erature and moisture biases that the model physics cannot overcome if the model is initialized during a period of extreme cold. Instead, the PX scheme was run fo r each simulation segment with the initial soil moisture interpolated from the input EDAS objective analyses fields.

Radiation / Shallow Convection

All four model runs used the RRTM (Rapid Ra diative Transfer Model) package for longwave

radiation and the Dudhia shortw ave radiation option. Shallow convection was turned off.

Cumulus Convection

The parameterization of cumulus convection was a critical factor in model performance in the 2002 annual application. Most of the summer rain fall in the U.S. is due to convective clouds; stratiform rain produced by mid-la titude cyclones is a smaller fraction of the total rainfall in summer, as the mid-latitude storm track sh ifts northward of most of the U.S.

The WRAP_0 run used the Kain-Fritsch I cumu lus parameterization (Kain and Fritsch 1990). Unlike many cumulus schemes that were designed for use in handling de ep tropical convectionMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\sec2.doc 2-5 in general circulation models (i.e. Arakawa- Schubert, Kuo, Betts-Mi ller), the KF I and II schemes were designed with mid- latitude convection in mind. The Kain-Fritsch I scheme is a mass flux scheme whose closure assumption is th at the convective mass flux is constrained to remove grid-scale instability in a specified time period, which is often taken to be the advective time scale (about an hour). The KF I scheme has a complex cloud model that treats both updrafts and downdrafts and can entrain or detrai n throughout the depth of the convecting layer. (The Fritsch-Chappell scheme, on which KF I wa s based, detrains only at the cloud top).

Entrainment and detrainment in KF I are ha ndled through a buoyancy-sorting algorithm, in which parcels of cloudy and environmental are mi xed, and their buoyancy evaluated. Positively buoyant parcels are entrained, while negatively buoyant parcels are detrained. The KF I cloud model includes ice phase physics. The Kain-Fr itsch II (KF II) scheme extends KF I by adding a

treatment of shallow convection. The CENRAP and VISTAS 36 km runs both used the KFII cumulus scheme, whereas the WRAP_0 run used KFI.

The final WRAP 36 km run used the Betts-Miller cumulus scheme (Betts and Miller 1986). The Betts-Miller scheme was originally designed to parameterize the effects of deep tropical convection on grid scale variables in general circulation models w ith grid sizes that could be

hundreds of kilometers across. The scheme is ba sed on observations of tr opical convection that

showed that radiative and c onvective processes drive the atmosphere toward a vertical temperature and moisture structure that is close to a moist adiabat passing through the equivalent potential temperature of the subcloud layer. Th e Betts-Miller scheme was designed to reproduce the observed quasi-equilibrium between large scal e forcing and convection without resorting to a complicated cloud model, as cloud models are co mplex and computationally intensive and have closure parameters which may not be well constr ained by observations. In stead, the Betts-Miller scheme relaxes the convecting grid column to ward reference profile s of temperature and moisture. Betts-Miller includes separate schemes for shallow (i.e. non-precipitating) and deep (precipitating) convection.

The deep convection scheme computes a referenc e profile that includes the virtual effects of liquid water loading up to the fr eezing level and adjusts the colu mn toward it with a specified time constant. The subcloud layer is adjusted to ward an evaporatively driven downdraft profile with its own timescale derived from an evaporat ion efficiency. The temperature and moisture reference profiles are different over land and wate r, and the reference profiles can be further suited to a particular convective environmen t through a choice of profile based on a cloud efficiency factor. Precipitation from the deep co nvective scheme is calculated as the integrated

residual water between the la rge-scale moisture profile and the reference profile.

When it comes to using a cumulus parameteri zation in MM5 at mid-latitudes at ~36 km resolution, the standard choices are the Ka in-Fritsch or Grell schemes (MM5 on-linedocumentation). Betts-Miller, designed for use in the tropics, has some difficulties in the midlatitudes, and is a less common choice. For exampl e, Hart (2000) notes:

“Betts-Miller (which has been used in th e ETA and mesoETA mode ls) is an undesirable parameterization for mid-latitude mesoscale convective systems. The scheme is too slow to respond to convective instab ility, often producing rainfall ra tes which are an order of magnitude less than observed and several hours too late. Further, the scheme cannot respond to elevated instability above the boundary layer. During such cases, the scheme responds too strongly to the low-level unstable air in the warm sector, which then robs

moisture from the areas of elevated instability, typically to the north”.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\sec2.doc 2-6 In response to the initial WRAP run’s performance difficulties in simulating the summer rainfall over the western U.S., we tested MM5’s sensitivity to the choice of cumulus parameterization in the WRAP_0 configuration. The original 2002 MM5 run (WRAP_0) used the Kain-Fritsch I scheme. Results from VISTAS 36 km MM5 modeling (Olerud and Sims 2003) showed improved surface fields relative to the WRAP_0 r un using Kain-Fritsch II, so Kain-Fritsch II(KFII) was the first scheme tested. We also made similar runs that used the Grell and BettsMiller parameterizations. All of these runs were made usi ng the PX/ACM LSM/PBL schemes and W/T/Q analysis nudging at the surface and aloft and no nudging to observations (obs nudging). We found that for July, the Betts-Miller scheme gave the best performance in terms of rainfall and surface temperature, humidity and wi nds, and selected this scheme for use in the final WRAP run, despite the fact that KF is more traditionally used for mid-latitudes.

As part of the WRAP sensitivity modeling, we ra n additional tests to evaluate the effect of different cumulus parameterizations on the 12-km WRAP domain. At the 12 km scale, the problem of cumulus-parameterization is not well- posed, as there is no clear spectral gap between the resolved grid-scale process and the scale of the parameterized process (Arakawa and Chen 1987). Molinari (1993) suggests th at parameterization of cumulus convection for grid sizes of 2-

20 km cannot be addressed w ith either the fully explicit method (i.e. no cumulus parameterization) or the hybrid parameterization approach (e.g., KFI, KFII, and Betts-Miller). However, our ultimate goal of performing CMAQ vi sibility modeling require s us to be pragmatic and configure MM5 in a physically reasonabl e way that produces the most accurate representation of the 2002 meteorol ogy. Our plan for the selection of the 12 km grid cumulus scheme, therefore, was to conduct several sensit ivity tests to determine which cumulus scheme (or no cumulus scheme at all) gives the best performance in terms of rainfall, surface temperature, humidity and wind. The best MM5 performance was obtained using no cumulus

scheme at all on the 12-km. We recognize that this resu lt could be serendipitous (i.e., leading to a better answer for the wrong reason), however th e alternative (poorer performance) is obviously worse and the overstated precipita tion would affect the air quality model. The other physics options for the 12-km grid were identical to t hose used on the 36-km grid. Table 2-3 shows the selected final MM5 configuration for the 12-km application.

Table 2-3. Final MM5 configuration for the 12-km grid.

A nalysis FDDA
LSM PBL Cumulus Microphysics 3D Surface Obs FDDA

Pleim-Xiu ACM None Reisner II W/T/H W Wind

Explicit Moisture Physics

The VISTAS and CENRAP runs used the Reisner I explicit moisture physics scheme (Reisner 1998). The Reisner I scheme employs a bulk cl oud microphysics parame terization to predict cloud water, cloud ice, rain, and snow fields. It treats supercooled water and allows for the time evolution of melting snow. The Reisner II sche me builds on the Reisner I scheme by predicting the graupel mixing ratio field, and by adding i ce number concentration prediction equations. Although the Reisner I and II schemes are more e xpensive than the “sim ple ice” option, the EPA recommends that a mixed phase ice scheme be employed in MM5 to drive aqueous chemistry and wet scavenging in CMAQ. This is because the simple ice approach treats all condensed water forms as a single liquid variable, which, wh en passed to CMAQ, over states the quantity ofMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\sec2.doc 2-7 liquid cloud and precipitation wate r available for chemistry and removal. Furthermore, the new versions of CMAQ (v4.4) and MCIP (v2.3) now allow the graupel mixing ratio to be passed from MM5 to CMAQ. Thus, the Reisner II was chosen for the final WRAP 36/12 km runs in order to take advantage of this new ca pability in future visibility modeling.

FOUR DIMENSIONAL DATA ASSIMILATION (FDDA)

In all four 36 km runs, MM5 was configured to use its FDDA capabilities to nudge the model toward wind, temperature, and moisture analys is fields throughout th e 2002 annual simulation. Analysis (or grid) nudging was performed in all model runs at 3-hourly intervals both for the surface wind fields and for the wind, moisture, and temperature fields aloft, excluding the boundary layer. Exclusion of the boundary layer in the FDDA process removed the potential for damping resolved mesoscale forcings in th e model that are important to boundary layer development and thus the vertical fluxes of momentum, heat, and moisture between the free atmosphere and the surface. The WRAP_0 run used additional surface analysis nudging of temperature and humidity at the surface, and th e final WRAP run used observational nudging of winds (to ds472) surface meteorol ogical observational database at the surface. The final WRAP run did not use surface analysis nudging of temp erature and humidity at the surface, based on reviewer comments on the Sensitivity Study Re port (Kemball-Cook et al. 2004b). Reviewer

comments are available at: http://pah.cert.ucr.edu/ aqm/308/mm5_reports.shtml

.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRA P_eval\DraftFinal\Sec3.doc 3-1 3.0 EVALUATION METHODOLOGY FOR THE 2002 ANNUAL MM5 RUNS

The goal of the MM5 evaluation is to determine whether the meteorological fields generated by

the four 36 km 2002 MM5 simulations and the 12 km WRAP simulation are sufficiently accurate to properly characterize the transport, chemis try, and removal processes for regional haze modeling in the western U.S. If e rrors in the meteorological fields are too large, the ability of the air quality model to repli cate regional pollutant leve ls over the entire base year will be severely hampered and the predicted air quality and visi bility impacts from future year growth and controls will be highly questionabl e. To provide a reasonable me teorological characterization to the photochemical/visibility model, MM5 must represent with some fidelity the:

• Large-scale weather patterns (i.e., synoptic patterns depicted in the 850-300 mb height fields), as these are key forci ngs for mesoscale circulations;

• Mesoscale and regional wind, temperature, PB L height, humidity, and cloud/precipitation

patterns;

• Mesoscale circulations such as sea br eezes and mountain/drainage circulations;

• Diurnal cycles in PBL depth, temperature, and humidity.

For visibility applications, the moisture and conden sate fields are particularly important as they significantly impact PM chemical formation, rem oval, and light scattering efficiency. In addition, cloud and precipitation fi elds are a good measure of the integrated performance of the

model since these are model-deri ved quantities and are not nudged to observations. Because of the model's coarse resolution of 36 km/12 km, the present runs cannot be expected to faithfully simulate the high frequency pattern or variab ility of the convective precipitation, but should reproduce the synoptic precip itation and cloud patterns.

In this study, the basis for the operational perf ormance assessment entailed a comparison of the predicted meteorological fields to available surface and aloft meteorol ogical data that are collected, analyzed, and disseminated by the Na tional Weather Service. The performance evaluation was carried out both gr aphically and statistically to evaluate model performance for winds, temperatures, humidity, and the placemen t, intensity, and evolution of key weather phenomena.

GRAPHICAL EVALUATION

Because of the regional nature of the haze problem, and the fact that most pollutant transport of interest occurs above the surface, evaluati on of the MM5 upper air meteorological fields assumed particular importance in this study. Analysis of MM5 tropospheric output fields is made more difficult by the spatial and tempor al resolution of the upper air sounding network, which is sparse in time and space compared with the surface observing network. Radiosonde measurements are usually taken twice per day by an observing network of approximately 120

stations in the continental U.S. Hourly radar profiler measurements are available from NOAA, but are lacking in spatial cove rage. Additional plotting capabili ties have been developed by theMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRA P_eval\DraftFinal\Sec3.doc 3-2 IDNR (RAOBPLOT) to compare sounding data at individual sites and times to predicted soundings. This allows for a site-by-site comparison of wind, temperature, and humidity profiles, and also provides the user the capability to diagnose and evaluate the heights, strengths, and depths of key stability regimes (e.g., boundary layer depths, nocturnal inversion heights) from observational soundings and MM5 predictions . The observed soundings were derived from the Forecast Systems Laborator y (FSL)/ National Climatic Data Center (NCDC) Dataset.

STATISTICAL EVALUATION

Johnson (2003) provides a synopsis of the challe nges associated with undertaking an objective and meaningful performance evaluation. Clim atic variability, complex mesoscale phenomenon, stochastic variability, and scientific limitations contribute to unique perf ormance issues in each meteorological modeling exercise, thereby forcing modelers to take a subjective approach to model performance evaluation. Ob jective statistics th at offer a quantitative model assessment exist, but implementation of the metrics is subjec tive to a degree. For example, defining the area over which domain-averaged metrics are calcul ated is a subjective decision, buffered only through guidelines. In general, metrics averag ed over large modeling domains are avoided, as

error cancellation dilutes rele vance. Conversely, splitting th e modeling domain into small analysis sub-domains can render sample sizes unre presentative. The logical approach falls well

within the bounds of the ex tremes, leaving optimum sub- domain definitions open to interpretation.

Within the statistical degrees of freedom availabl e to the meteorological modeler, a subset of standard statistical measures has emerged, as de scribed below. These metrics are calculated on hourly and daily time frames for wind speed, wind direction, temperature, and humidity at the

surface. Below we list and describe the various statistical measures that will be employed. While no strict criteria establishing acceptable m odel performance exist, general guidelines have been developed by Emery et al. ( 2001) and are described later.

Mean Observation (M

o ): calculated from all sites with valid data within a given analysis region and for a given time period (hourly or daily):

where O i j is the individual observed qua ntity at site i and time j, and the summations are over all sites (I) and over time periods (J).

Mean Prediction (M p ): calculated from simulation results that are interpolated to each observation point used to calculate the mean observation (hourly or daily):

where P i j is the individual predicted quantity at site i and time j. Note that mean observed and predicted winds are vector-avera ged (for east-west component u and north-south component v), from which the mean wind speed and m ean resultant direction are derived. 

## ∑∑

= = = J

j I

i i j o O IJ M

1 1 1



# ∑∑

= = = J

j I

i i j p P IJ M

1 1 1March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRA P_eval\DraftFinal\Sec3.doc 3-3 Least Square Regression: performed to fit the prediction set to a linear model that describes the

observation set for all sites with valid data with in a given analysis region and for a given time period (daily or episode). The y-intercept a and slope b of the resulting straight line fit are calculated to describe the regres sed prediction for each observation:

The goal is for a 1:1 slope and a "0" y-inter cept (no net bias over the entire range of observations), and a regression coefficient of 1 (a perfect regression). The slope and intercept

facilitate the calculation of several erro r and skill statistics described below.

Bias Error (B): calculated as the mean difference in pred iction-observation pairings with valid data within a given analysis region and fo r a given time period (hourly or daily):

Gross Error (E): calculated as the mean absolute difference in predictio n-observation pairings

with valid data within a given analysis regi on and for a given time period (hourly or daily): Note that the bias and gross error for winds are calculated from the predicted-observed residuals in speed and direction (not from vector compone nts u and v). The direction error for a given prediction-observation pairing is limited to range from 0 to ±180°.

Root Mean Square Error (RMSE): calculated as the square root of the mean squared difference in prediction-observation pairings with valid data within a give n analysis region and for a given time period (hourly or daily):

The RMSE, as with the gross er ror, is a good overall measure of model performance. However, since large errors are weighted heavily (due to squari ng), large errors in a small subregion may produce a large RMSE even though the errors may be small and quite acceptable elsewhere.

Systematic Root Mean Square Error (RMSE S ): calculated as the square root of the mean squared difference in regressed prediction-observation pair ings within a given analysis region and for a given time period (hourly or daily): i j i j bO a P + = ˆ



# ( ) ∑∑

= = − = J

j I

i i j i j O P IJ B

1 1 1



# ∑∑

= = − = J

j I

i i j i j O P IJ E

1 1 1



# ( ) 2 / 1

1 1 2 1 ⎥ ⎦ ⎤ ⎢ ⎣ ⎡

− = 

## ∑∑

= = J

j I

i i j i j O P IJ RMSE



# ( ) 2 / 1

1 1 2 ˆ 1 ⎥ ⎦ ⎤ ⎢ ⎣ ⎡

− = 

## ∑∑

= = J

j I

i i j i j S O P IJ RMSEMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRA P_eval\DraftFinal\Sec3.doc 3-4 where the regressed prediction is estimated fo r each observation from the least square fit

described above. The RMSE S estimates the model's linear (or sy stematic) error; hence, the better the regression between predictions and observations, the smaller the systematic error.

Unsystematic Root Mean Square Error (RMSE U ): calculated as the square root of the mean squared difference in prediction-regressed predicti on pairings within a gi ven analysis region and for a given time period (hourly or daily):

The unsystematic difference is a measure of how much of the discrepancy between estimates and observations is due to random pro cesses or influences outside the legitimate range of the model.

A "good" model will provide low values of the RMSE, explaining most of the variation in the observations. The systematic error should a pproach zero and the unsystematic error should approach RMSE since:

RMSE 2 = RMSE S 2 + RMSE U 2

It is important that RMSE, RMSE S , and RMSE U are all analyzed. For example, if only RMSE is estimated (and it appears acceptable) it could cons ist largely of the systematic component. This error might be removed through improvements in the model inputs or use of more appropriate options, thereby reducing the error transferred to the photochemical mo del. On the other hand, if the RMSE consists largely of the unsystematic component, this indicate s that further error reduction may require model refinement (new algor ithms, higher resolution grids, etc.), or that the phenomena to be replicated can not be fully addressed by the model. It also provides error bars that may be used with the input s in subsequent sensitivity analyses.

Index of Agreement (IOA): calculated following the approach of Willmont (1981). This metric

condenses all the differences between model estimates and observations within a given

analysis region and for a given time period (hourly a nd daily) into one statistical quantity. It is

the ratio of the total RMSE to the sum of tw o differences - between each prediction and the observed mean, and each observation and the observed mean:

Viewed from another perspective, the index of agreement is a measure of the match between the departure of each prediction from the observed m ean and the departure of each observation from the observed mean. Thus, the correspondence betw een predicted and obser ved values across the domain at a given time may be qua ntified in a single metric and di splayed as a time series. The 

# ( ) 2 / 1

1 1 2 ˆ 1 ⎥ ⎦ ⎤ ⎢ ⎣ ⎡

− = 

# ∑∑

= = J

j I

i i j i j U P P IJ RMSE

⎥ ⎥ ⎥ ⎥

⎦ ⎤

⎢ ⎢ ⎢ ⎢

⎣ ⎡

− +

−

⋅

− = 

## ∑∑

= = J

j I

i o i j o i j M O M P RMSE

IJ IOA

1 1 2 1March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRA P_eval\DraftFinal\Sec3.doc 3-5 index of agreement has a theoretical range of 0 to 1, the latter sc ore suggesting perfect

agreement.

Most of statistics used to evaluate meteorologica l model performance are gi ven in absolute terms (e.g., wind speed error in m/s), rather than in relative terms (percent error) as is commonly shown for air quality assessments. The major reason for this is that a very different significance is associated with a given relative error for diffe rent meteorological parameters. For example, a 10% error for wind speed measured at 10 m/s is an absolute error of 1 m/s, a minor error. Yet a 10% error for temperature at 300 ° K is an absolute error of 30 ° K, a ridiculously large error. On the other hand, pollutant concentration errors of 10% at 1 ppb or 10 ppm carry similar significance.

THE METSTAT ANALYSIS PACKAGE

ENVIRON has developed a statistic al analysis software package to calculate and graphically present the statistics described above. The p ackage is comprised of a single Fortran program (METSTAT) to generate observati on-prediction pairings and to calculate the statistics, and a Microsoft Excel macro (METSTAT.XL S) that plots the results.

The Fortran program reads MM5 output prediction files and surface observational data files. The program reads either MM5 observation FDDA input files directly, or obs ervation data in an ASCII format. The program then spatially and temporally pairs MM5 predictions with observations for a user-defined time and space window. Since the surface layer in MM5 is usually rather thick relative to the heights at which the observational data were recorded, the METSTAT program includes a micro-meteorologi cal module that scales mid-layer predicted winds to 10 m heights, and mid-layer predicte d temperatures to 2 m heights, using common stability-dependent similarity relationships. Th e horizontal analysis range can be given for an entire MM5 grid, by an LCP coordi nate box, or as a list of speci fic site identifiers (such as WBAN or AIRS numbers), as labele d on the observational file. This allows for an evaluation at a single site, a subset of specific sites (e.g., those along a coastline that would be difficult to select by defining an LCP box) or over an entire regional domain.

The program then proceeds to calculate the stat istics described above for each hour and for each day of the time window. The follo wing parameters are considered:

Wind Speed, Temperature, Humidity:

- Mean Observed

- Mean Predicted

- Bias

- Gross Error

- RMSE

- RMSES

- RMSEU

- IOAMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRA P_eval\DraftFinal\Sec3.doc 3-6 Wind Direction:

- Mean Observed

- Mean Predicted

- Bias

- Gross Error

The RMSE and IOA have not been typically used to quantify error for wind direction, and thus are not calculated by the program.

Separate ASCII files containing the hourly a nd daily statistics are generated, formatted specifically to facilitate import into the Excel macr o. The Excel macro is used to plot the data. The hourly statistics are plotted as time seri es, to show the diurna l variation of model performance. The daily statisti cs are plotted as bar charts to show daily performance over an episode. The macro also allows the daily results from multiple MM5 runs to be plotted together to ease the inter-comparison of performance.

STATISTICAL PERFORMANCE BENCHMARKS

Emery and co-workers (2001) have derived and proposed a set of daily performance "benchmarks" for typical meteorological model performance. These standards were based upon

the evaluation of about 30 MM5 and RAMS meteor ological simulations in support of air quality applications performed in the la st few years, as reported by Tesc he et al. (2001). The purpose of these benchmarks was not necessarily to give a passing or failing grade to any one particular meteorological model application, but rather to put its results in to the proper context. For example, expectations for meteorological model performance for the U.S. west coast might not be as high as a simpler domain located over the Midwest. The key to the benchmarks is to understand how poor or good the results are relative to the universe of other model applications

run for various areas of the U.S.

The statistical performance benchmarks are given in Table 3-1.

Table 3-1. Statistical benchmarks for evaluating meteorological model performance. Wind Speed Wind Direction Temperature Humidity

RMSE ≤ 2 m/s

Mean Bias ≤ ± 0.5 m/s ≤ ± 10 ° ≤ ± 0.5 K ≤ ± 1 g/kg

Gross Error ≤ 30 ° ≤ 2 K ≤ 2 g/kg Index of Agreement ≤ 0.6 ≤ 0.8 ≤ 0.6

SURFACE STATISTICAL ANALYSES FOR WRAP

The statistical evaluation of MM5 surface fiel ds was performed using NCAR dataset ds472, which contains hourly observations of the comm only measured variables from airports in the

U.S. and Canada. Dataset variables include temperature, dew point, wind speed/direction and gusts, cloud cover fraction and cloud base for mul tiple cloud layers, visu al range, precipitation rates and snow cover, and a descri ptive weather code. The key data of interest were extracted for the various sub-domains, and processed into the appropriate formats for METSTAT.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRA P_eval\DraftFinal\Sec3.doc 3-7 As discussed earlier, some care must be taken in selecting an area for averaging. The problem with evaluating statistics is that the more data pa irings that are summarized in a given metric, the better the statistics generally look, and so calculating a si ngle set of st atistics for a very large area (e.g., the entire 36-km domain) w ould not yield significant insight in to performance. Therefore, a balance must be struck between taking a larg e enough area to create a representative sample and choosing such a large area as to smear out the signal of inte rest. Johnson (2003) suggests the use of the subdomains shown in Figure 3-1, and those subdomains were adopted for use in this analysis.

For each subdomain, METSTAT was used to calcul ate hourly and daily statistical measures, and these were compared against benchmarks for acceptable model performance set forth by Emery et al. (2001) as summari zed in Table 3-1.

Figure 3-1. METSTAT subdomains for the WRAP 2002 MM5 performance evaluation.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRA P_eval\DraftFinal\Sec3.doc 3-8 PRECIPITATION EVALUATION

For each month of 2002, we developed fields of observed total rainfall over the 36 km National RPO grid and the 12 km WRAP grid. The obser ved precipitation amounts were generated with the CPC (Climate Prediction Cente r) gridded precipitation amount dataset, which is available

from the National Weather Service's Climate Prediction Center at:

http://www.cpc.ncep.noaa.gov/produc ts/precip/realtime/retro.html

The CPC daily precipitation amounts are on a latit ude-longitude grid that covers the U.S. mainland at a resolution of 0.25°x 0.25°, and ar e ramped down to zero immediately offshore. The CPC dataset was interpolated to the 36 km and 12 km MM5 Lambert conformal grids. The advantage of the CPC precipitation field is that it has a reasonably hi gh resolution, which is especially important when it co mes to resolving the effects of orography on the precipitation over the western U.S. However, this CPC produ ct does not include the entire 36 km or 12 km modeled domain. For example, the CPC dataset cannot be used to assess MM5’s performance in simulating stratiform summer rain off the coast of California because the data effectively ends at

the coastline.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-1 4.0 SURFACE EVALUATION OF THE 2002 CENRAP, VISTAS, and WRAP RUNS

In this section, we evaluate the performance of the four 2002 36 km MM5 runs. The evaluation of the MM5 surface meteorological variables usin g METSTAT is presented first, followed by the evaluation of the MM5 precip itation patterns and MM5 upper-ai r meteorological variables.

SUBDOMAIN SURFACE FIEL D METSTAT EVALUATION

The starting point for the analysis of the 2002 36 km MM5 runs was an as sessment of the surface statistics for wind, temperature, and humidity us ing METSTAT. As described in Section 3, the

continental U.S. was divided into subdomains (Figure 3-1), and each subdomain was evaluated month-by-month to isolate differences in how the model performed region by region over the course of the year. For the purpose of organizing the analysis, we will show soccerplots on which are displayed average performance stat istics for each run over each subdomain for a particular month. Soccerplots are shown for wind speed RMSE versus wind direction error, temperature bias versus temperature error, and humi dity bias versus humidity error. In each plot,

a solid blue line indicates the be nchmark. A data point that fa lls inside the box represents a model run that meets the performance benchmark. Perfect model performance is indicated by a data point at (0,0). The closer a da ta point is to the origin, the be tter the model’s performance. It should be re-emphasized that the benchmarks are not used as an acceptance/rejection criteria of the MM5 model simulation. Rather they put th e MM5 model performance into perspective and allow the identification of potential problems in the MM5 fields.

Soccerplots were generated and evaluated for ea ch month of the year 2002. For brevity, we include only January and July in this report. MM5's performa nce was qualitatively different during summer and winter, with the spring and fall seasons serving as transitional periods between winter and summer. January and July were found to be representative of MM5’s performance during the winter a nd summer seasons. The model’s spring statistical performance

was similar to that of fall. The main st rengths and weaknesses of the 2002 MM5 runs are captured in the January and July plots. Fo r both months, we show a soccerplot for the subdomains in the western U.S. (the WRAP regi on) and a second soccerplot for the central and

eastern U.S. This is done to reduce the amount of data shown on one plot to a manageable level, and also because MM5’s performance was qualitativel y different in the western U.S. and the rest of the country.

Table 4-1 lists the METSTAT domains shown in Figure 3-1 and corresponding abbreviations

used in the discussion below.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-2 Table 4-1. METSTAT subdomain abbreviations (see Figure 3-1).

METSTAT Subdomain Name Abbreviation

1 PacificNW

2 SW

3 North

4 DesertSW 5 CenrapN 6 CenrapS

7 Great Lakes 8 Ohio Valley

9 SE

10 NE

11 MidAtlantic

January MM5 Surface Performance

Comparison of the wind soccerplots for January (F igures 4-1 and 4-2) shows a marked division by geographic region, with the western subdomains PacificNW, SW, DesertSW, North, and NE falling outside the benchmarks for the CENRAP, VISTAS, and WRAP_0 runs. In the eastern and central U.S., wind performance is within the benchmark for all four runs for all subdomains except NE. In the final WRAP run, the western subdomains are within or close to (SW) the

benchmark. Performance in all central a nd eastern subdomains improved, and the NE subdomain is brought closer to the benchmark. Th e final WRAP run is the best performing run overall for surface wind in January.

Figures 4-3 and 4-4 show January time series fo r the initial and final WRAP runs for wind speed and wind speed bias and wind dire ction and wind direction bias for the desertSW subdomain. The WRAP_0 run reproduced the overall shape of the observed wind speed time series, but had a significant low wind speed bias. In the final WRAP run, this bias is reduced to the point where the desertSW subdomain falls within the perfor mance benchmark for wind speed. There is also an improvement in simulation of wind direction. The WRAP_0 run has a positive wind direction bias throughout much of January, and this bias is reduced in the final WRAP run (Figure 4-4). The final WRAP run also did a better job of reproducing the large wind direction excursions on January 12 and January 30. In the final WRAP ru n, there is an overall reduction in wind speed RMSE relative to the WRAP_0 run (time series not shown) over the entire month. The improvement in the simulation of surface winds in the final WRAP run likely occurs because of nudging of surface winds to observations. Surface obs nudging of winds is not done in the WRAP_0 runs, nor in the CE NRAP or VISTAS runs.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-3 Figure 4-1 . 36 km surface wind comparison soccerplot for January over western U.S. for VISTAS, CENRAP, WRAP_0 (old WRAP) and final WRAP (WRAP) 2002 36 km simulation.

Figure 4-2 . 36 km surface wind comparison soccerplot for January over central and eastern U.S. for VISTAS, CENRAP, WRAP_0 (old WRAP) and final WRAP (WRAP) 2002 36 km

simulation. CENRAP / VISTAS / WRAP January Wind Performance Comparison Over the Western U.S.

0 10 20 30 40 50 60 70 80

0 0.5 1 1.5 2 2.5 3 Wind Speed RMSE (m/s)Wind Direction Error (deg)

Benchmark VISTAS OldWRAP CENRAP WRAP DesertSW

PacNW

SW

North

CENRAP / VISTAS / WRAP January Wind Performance Comparison Over the Central and Eastern U.S.

0 10 20 30 40 50 60 70 80

0 0.5 1 1.5 2 2.5 3 Wind Speed RMSE (m/s)Wind Direction Error (deg)

Benchmark VISTAS OldWRAP CENRAP WRAP CENRAP N

CENRAP S

Ohio Val

SE

NE

MidAtlMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-4 Figure 4

- 3 . WRAP_0 run surface wind speed and bias for January over desertSW subdomain.

Figure 4

- 4 . Final WRAP run surface wind speed and bias for January over desertSW

subdomain. Observed/Predicted Windspeed

0 2 4 6 8 10

1/ 1 1/

2

1/ 3 1/ 4 1/ 5 1/ 6 1/ 7 1/

8

1/ 9 1/

10

1/

11

1/

12

1/

13

1/

14

1/

15

1/

16

1/

17

1/

18

1/

19

1/

20

1/

21

1/

22

1/

23

1/

24

1/

25

1/

26

1/

27

1/

28

1/

29

1/

30

1/

31m/s ObsWndSpd PrdWndSpd

Bias Windspeed

-3 -2 -1 0 1

1/ 1 1/

2

1/ 3 1/ 4 1/ 5 1/ 6 1/ 7 1/

8

1/ 9 1/10 1/11 1/12 1/13 1/14 1/15 1/16 1/17 1/18 1/19 1/20 1/21 1/22 1/23 1/24 1/25 1/26 1/27 1/28 1/29 1/30 1/31m/s BiasWndSpd

Observed/Predicted Windspeed

0 2 4 6 8 10

1/ 1 1/ 2 1/ 3 1/ 4 1/ 5 1/ 6 1/ 7 1/ 8 1/ 9 1/

10

1/

11

1/

12

1/

13

1/

14

1/

15

1/

16

1/

17

1/

18

1/

19

1/

20

1/

21

1/

22

1/23 1/

24

1/

25

1/

26

1/

27

1/

28

1/

29

1/

30

1/

31m/s ObsWndSpd PrdWndSpd

Bias Windspeed

-3 -2 -1 0 1 2

1/ 1 1/ 2 1/ 3 1/ 4 1/ 5 1/ 6 1/ 7 1/ 8 1/ 9 1/10 1/11 1/12 1/13 1/14 1/15 1/16 1/17 1/18 1/19 1/20 1/21 1/22 1/

23

1/24 1/25 1/26 1/27 1/28 1/29 1/30 1/31m/s BiasWndSpdMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-5 Figure 4

- 5 . WRAP_0 run surface wind direction and bias for January over desertSW

subdomain.

Figure 4

- 6 . Final WRAP run surface wind direction and bias for January over desertSW

subdomain. Observed/Predicted Wind Direction

0 60 120 180 240 300 360

1/ 1 1/ 2 1/ 3 1/ 4 1/ 5 1/ 6 1/ 7 1/ 8 1/ 9 1/

10

1/

11

1/

12

1/

13

1/

14

1/

15

1/

16

1/

17

1/

18

1/

19

1/

20

1/

21

1/

22

1/23 1/

24

1/

25

1/

26

1/

27

1/

28

1/

29

1/

30

1/

31deg ObsWndDir PrdWndDir

Bias Wind Direction

-90 -60 -30

0 30 60 90

1/ 1 1/ 2 1/ 3 1/ 4 1/ 5 1/ 6 1/ 7 1/ 8 1/ 9 1/10 1/11 1/12 1/13 1/14 1/15 1/16 1/17 1/18 1/19 1/20 1/21 1/22 1/

23

1/24 1/25 1/26 1/27 1/28 1/29 1/30 1/31deg BiasWndDir

Observed/Predicted Wind Direction

0 60 120 180 240 300 360

1/ 1 1/

2

1/

3

1/ 4 1/ 5 1/ 6 1/ 7 1/

8

1/

9 1/10 1/11 1/12 1/13 1/14 1/15 1/16 1/17 1/18 1/19 1/20 1/21 1/22 1/23 1/24 1/25 1/26 1/27 1/28 1/29 1/30 1/31deg ObsWndDir PrdWndDir

Bias Wind Direction

-90 -60 -30

0 30 60 90

1/ 1 1/

2

1/

3

1/ 4 1/ 5 1/ 6 1/ 7 1/

8

1/

9 1/10 1/11 1/12 1/13 1/14 1/15 1/16 1/17 1/18 1/19 1/20 1/21 1/22 1/23 1/24 1/25 1/26 1/27 1/28 1/29 1/30 1/31deg BiasWndDirMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-6 Figure 4-7 . 36 km temperature comparison soccerplot for January over western U.S. for VISTAS, CENRAP, WRAP_0 (old WRAP) and final WRAP (WRAP) 2002 36 km simulation.

Figure 4-8 . 36 km temperature comparison for January over central and eastern U.S. for VISTAS, CENRAP, WRAP_0 (old WRAP) and final WRAP (WRAP) 2002 36 km simulation. CENRAP / VISTAS / WRAP January Temperature

Performance Comparison Over Western U.S.

0 0.5

1 1.5

2 2.5

3 3.5

-2 -1.5 -1 -0.5 0 0.5 1

Temperature Bias (K)Temperature Error (K)

Benchmark VISTAS CENRAP WRAP OldWRAP DesertSW

PacNW

SW

North

CENRAP / VISTAS / WRAP January Temperature Performance Comparison over Central and Eastern U.S.

0 0.5

1 1.5

2 2.5

3 3.5

-2 -1.5 -1 -0.5 0 0.5 1

Temperature Bias (K)Temperature Error (K)

Benchmark VISTAS OldWRAP CENRAP WRAP CENRAP N

CENRAP S

Ohio Val

SE

NE

MidAtlMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-7 Figure 4-9 . WRAP_0 run January temperature time series for desertSW subdomain

.

Figure 4-10 . Final WRAP run January temperature time series for desertSW subdomain. Observed/Predicted Temperature

245 250 255 260 265 270 275 280 285 290

1/

1 1

/ 2

1/ 3

1/ 4 1/

5 1

/ 6

1/ 7 1/

8

1/ 9 1/10

1/11
1 /1 2 1 /1 3 1/14

1/15

1 /1 6 1 /1 7 1 / 18

1/1

9

1 /2 0 1 /2 1 1 / 22

1 /2 3 1 /2 4 1 /2 5 1 / 26

1 /2 7 1 /2 8 1 / 29 1/30

1 /3 1K ObsTemp PrdTemp

Bias Temperature

-4 -3 -2 -1 0 1 2 3

1/

1 1

/ 2

1/ 3

1/ 4 1/

5 1

/ 6

1/ 7 1/

8

1/ 9 1/10

1/11
1 /1 2 1 /1 3 1/14

1/15

1 /1 6 1 /1 7 1 / 18

1/1

9

1 /2 0 1 /2 1 1 / 22

1 /2 3 1 /2 4 1 /2 5 1 / 26

1 /2 7 1 /2 8 1 / 29 1/30

1 /3 1K BiasTemp

RMSE Temperature

0 0.5

1 1.5

2 2.5

3 3.5

4 4.5

5

1/ 1

1 / 2

1/

3

1/

4 1/ 5

1 / 6

1/

7 1/ 8

1/

9

1 /1 0

1 /1 1 1/12 1/13

1 /1 4

1 /1 5 1/16 1/17

1/ 1 8

1 /19

1/20 1/21

1/ 2 2 1/23 1/24

1/25

1/ 2 6 1/27 1/28

1/ 2 9

1 /3 0 1/31K RMSETemp RMSESTemp RMSEUTemp

Observed/Predicted Temperature

245 250 255 260 265 270 275 280 285 290

1/ 1 1 / 2

1 / 3

1 / 4 1/ 5

1 / 6 1/ 7

1/ 8 1 / 9

1 / 1 0

1 / 1 1
1 / 1 2

1 / 1 3

1 / 1 4 1/15

1/16

1 / 1 7 1/18

1 / 1 9

1 / 2 0

1 / 2 1 1/22

1 / 2 3 1/24

1/25

1 / 2 6

1 / 2 7

1 / 2 8

1 / 2 9

1 / 3 0

1 / 3 1K ObsTemp PrdTemp

Bias Temperature

-5 -4 -3 -2 -1 0 1 2 3

1/ 1 1 / 2

1 / 3

1 / 4 1/ 5

1 / 6 1/ 7

1/ 8 1 / 9 1/10

1/11

1/12

1/13

1/14

1 / 1 5

1 / 1 6 1/17

1 / 1 8 1/19

1/20

1/21
1 / 2 2 1/23

1 / 2 4

1 / 2 5 1/26

1/27

1/28

1/29

1/30

1/31K BiasTemp

RMSE Temperature

0 1 2 3 4 5 6 7

1 / 1 1/

2 1/

3 1/ 4 1 / 5 1/ 6 1 / 7

1 / 8 1/

9 1/10

1/11

1/12

1/13

1/14

1 / 1 5

1 / 1 6 1/17

1 / 1 8 1/19

1/20

1/21
1 / 2 2 1/23

1 / 2 4

1 / 2 5 1/26

1/27

1/28

1/29

1/30

1/31K RMSETemp RMSESTemp RMSEUTempMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-8 Figure 4-11 . 36 km humidity comparison soccerplot for January over western U.S. for VISTAS, CENRAP, WRAP_0 (old WRAP) and final WRAP (WRAP) 2002 36 km simulation.

Figure 4-12 . 36 km humidity comparison soccerplot for January over central and eastern U.S. for VISTAS, CENRAP, WRAP_0 (old WRAP) and final WRAP (WRAP) 2002 36 km simulation. CENRAP / VISTAS / WRAPJanuary Humidity Performance Comparison Over the Western U.S.

0 0.5

1 1.5

2 2.5

3

-1.5 -1 -0.5 0 0.5 1 1.5 2 2.5

Humidity Bias (g/kg)Humidity Error (g/kg)

Benchmark VISTAS WRAP CENRAP OldWRAP DesertSW

PacNW

SW

North

CENRAP / VISTAS / WRAPJanuary Humidity Performance Comparison Over the Central and Eastern U.S.

0 0.5

1 1.5

2 2.5

3

-1.5 -1 -0.5 0 0.5 1 1.5 2 2.5

Humidity Bias (g/kg)Humidity Error (g/kg)

Benchmark VISTAS OldWRAP CENRAP WRAP CENRAP N

CENRAP S

Ohio Val

SE

NE

MidAtlMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-9 For temperature in January (Figures 4-7 and 4- 8), the WRAP_0 run outperformed the other runs,

with three of four western subdomains and all ea stern and central subdomain s in the goal. This was, of course, because surface analysis nudging of temperature was performed in the WRAP_0 run, but not the other three runs. With the anal ysis nudging turned off, the performance of the final WRAP run becomes comparable to the CENRAP run in the western subdomains. Of the

three runs with no surface anal ysis nudging of temperature (V ISTAS, CENRAP, final WRAP), VISTAS is the best-performing run. Figure 4- 9 and Figure 4-10 show the change in surface temperature statistic time se ries from WRAP_0 to the final WRAP run for the desertSW subdomain. Over the month of January, the bias and the RMSE both increase in the final WRAP run (note that the RMSE is on a different scale in the old and final WRAP runs). The final WRAP run has a larger cold bias than the WRAP_0 run for the desertSW subdomain, and this

increased cold bias was common to all subdomai ns across the U.S. in going from the WRAP_0

run to the final WRAP run

In the eastern and central subdomains, the temp erature error does not change significantly
between the WRAP_0 and final WRAP runs (except for NE), but th e cold bias increases (Figure 4-8). The cause of this is unclear. Overall, mo st subdomains show a cold bias. In the central and eastern subdomains, all four runs tended to be wi thin or near the benchm ark, but in the west, all runs were outside the benchmark except for the analysis nudged WRAP_0 run. Olerud and Sims (2003) have suggested that s purious temperature differences are introduced by mismatches between observing station elevation and modeled terrain heights that is caused in part by coarse model grid resolution used.

For January humidity (Figures 4-11 and 4-12), ther e is a general wet bias except in the SW and PacNW subdomains. Differences among the runs ar e small, except that VI STAS does better in the Ohio Valley, SE and MidAtlantic subdomains. A ll four runs lie within the benchmark for all

subdomains.

Overall for January, all four model runs tended to be too wet, too cold, and had a low wind speed bias (not shown). The surface analysis nudged WRAP_0 run performed best for temperature, with the VISTAS run the next best performer. A ll four runs performed be tter in the central and

eastern subdomains for wind and te mperature, and this is likely due to the effects of modeling the topography of the west at 36 km resoluti on. There was not much difference among the runs in humidity performance in the west, but VISTAS was clearly the best performing run in the

east.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-10 July MM5 Surface Model Performance

In July, the wind direction error and the wind sp eed RMSE (Figure 4-13 and Figure 4-14) show a general increase relative to January (Figures 4-1 and Figure 4-2) in the western and central and eastern subdomains. In the western subdomains, there is not much difference in performance among the VISTAS, CENRAP, and WRAP_0 runs, al though VISTAS does s lightly better than the others. The final WRAP run shows a slight decrease (improvement) in wind speed RMSE and a larger decrease in wind di rection error relative to the othe r runs. Again, this is due to nudging to surface wind observations, which is done in the final WRAP run, but not in the VISTAS, CENRAP, or WRAP_0 runs. As in Janua ry, MM5 does better in simulating winds in

the central and eastern U.S. than it does in the west. In the ce ntral and eastern subdomains, there is not much difference among the WRAP_0, CENRAP , and VISTAS runs in terms of July wind performance. The final WRAP run, which is nudged to surface wind observations, displays improved wind direction error relative to the ot her runs, as well as a smaller wind speed RMSE.

For temperature (Figures 4-15 and 4-16), July perf ormance is better in the east than the west, as seen for wind. The temperature RMSE is gene rally smaller in the east, and the western subdomains have a much stronger cold bias. As me ntioned earlier, part of this cold bias in the west may be due to terrain reso lution effects. In the eastern and central U.S. (Figure 4-16), temperature performance among the WRAP_0, CENRAP, and VISTAS runs is roughly equivalent, while the final WRAP run has an ou tlier in the cenrapS subdomain. In the west, CENRAP is the best performing run, as it is the on ly one of the four runs with no outlier. The

final WRAP run has three of four subdomains within the benchmark for bias, but its SW subdomain has a strong warm bias, and is the only western subdomain in any of the four runs to have a warm bias. All runs excep t the final WRAP run showed larg e cold biases in the desertSW subdomain. This is likely due to excessive si mulated convective rainfall, which affects the partitioning of surface energy, divert ing an excess of energy to latent rather than sensible heating. This lowers the daily surface temperature maximum (K emball-Cook et al., 2004a). For example, Figures 4-17 and 4-18 show the time series of temperature and temperature bias for the
SW subdomain. The WRAP_0 run has a cold bias caused by a consistent underestimation of the daily maximum surface temperature. Precipitatio n was overestimated in the SW subdomain in the WRAP_0 run. In the final WRAP run, the rain fall is much closer to observations than in WRAP_0, and the cold bias has shifted to a warm bias. The North subdomain (Figures 4-19 and 4-20) exhibits similar behavior.

For humidity, (Figure 4-21), the erro r is higher in July than in January for all subdomains. Over the central and eastern U.S., performance am ong the WRAP_0 CENRAP, and VISTAS runs is roughly comparable. The final WRAP run has a cold bias for most subdomains, and outliers cenrapN and cenrapS. In the western U.S. th e WRAP_0 run has outliers desertSW and North, which have a strong wet bias rela ted to excess precipitation over these subdomains. This strong wet bias over the WRAP region was one of the main motivations for redoing the WRAP run, and the final WRAP run does show improved humid ity performance in Ju ly, with all western subdomains now lying within the benchmark. The cost of this improvement in WRAP humidity performance in the west is degraded humidity performance in the east.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-11 Figure 4-13 . 36 km wind comparison soccerplot for July over western U.S. for VISTAS, CENRAP, WRAP_0 (old WRAP) and final WRAP (WRAP) 2002 36 km simulation.

Figure 4-14 . 36 km wind comparison soccerplot for July over central and eastern U.S. for VISTAS, CENRAP, WRAP_0 (old WRAP) and final WRAP (WRAP) 2002 36 km simulation. CENRAP / VISTAS / WRAP July Wind Performance Comparison Over the Western U.S.

0 10 20 30 40 50 60 70 80

0 0.5 1 1.5 2 2.5 3 Wind Speed RMSE (m/s)Wind Direction Error (deg)

Benchmark VISTAS OldWRAP CENRAP WRAP DesertSW

PacNW

SW

North

CENRAP / VISTAS / WRAP July Wind Performance Comparison Over the Central and Eastern U.S.

0 10 20 30 40 50 60 70 80

0 0.5 1 1.5 2 2.5 3 Wind Speed RMSE (m/s)Wind Direction Error (deg)

Benchmark VISTAS OldWRAP CENRAP WRAP CENRAP N

CENRAP S

Ohio Val

SE

NE

MidAtlMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-12 Figure 4-15 . 36 km temperature comparison soccerplot for July over western U.S. for VISTAS, CENRAP, WRAP_0 (old WRAP) and final WRAP (WRAP) 2002 36 km simulation.

Figure 4-16 . 36 km temperature comparison soccerplot for July over central and eastern U.S. for VISTAS, CENRAP, WRAP_0 (old WRAP) and final WRAP (WRAP) 2002 36 km simulation. CENRAP / VISTAS / WRAP July Temperature Performance Comparison

Over the Western U.S.

0 0.5

1 1.5

2 2.5

3 3.5

-2 -1.5 -1 -0.5 0 0.5 1

Temperature Bias (K)Temeprature Error (K)

Benchmark VISTAS WRAP CENRAP OldWRAP DesertSW

PacNW

SW

North

CENRAP / VISTAS / WRAP July Temperature Performance Comparison

Over the Central and Eastern U.S.

0 0.5

1 1.5

2 2.5

3 3.5

-2 -1.5 -1 -0.5 0 0.5 1

Temperature Bias (K)Temeprature Error (K)

Benchmark VISTAS WRAP CENRAP OldWRAP CENRAP N

CENRAP S

Ohio Val

SE

NE

MidAtlMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-13 Figure 4-17 . WRAP_0 run July temperature time series for SW subdomain.

Figure 4-18 . Final WRAP run July temperature time series for SW subdomain. Observed/Predicted Temperature

275 280 285 290 295 300 305 310

7/ 1 7/ 2 7/ 3 7/ 4 7/ 5 7/ 6 7/ 7 7/ 8 7/ 9

7/10 7/11 7/12 7/13 7/14 7/15 7/16 7/17 7/18 7/19 7/20 7/21 7/22 7/

23

7/24 7/25 7/26 7/27 7/28 7/29 7/30 7/31K ObsTemp PrdTemp

Bias Temperature

-5 -4 -3 -2 -1 0 1 2 3

7/ 1 7/ 2 7/ 3 7/ 4 7/ 5 7/ 6 7/ 7 7/ 8 7/ 9 7/10 7/11 7/12 7/13 7/14 7/15 7/16 7/17 7/18 7/19 7/20 7/21 7/22 7/

23

7/24 7/25 7/26 7/27 7/28 7/29 7/30 7/31K BiasTemp

Observed/Predicted Temperature

275 280 285 290 295 300 305 310 315

7/ 2 7/ 3 7/ 4 7/ 5 7/ 6 7/ 7 7/ 8 7/ 9 7/10 7/11 7/12 7/13 7/14 7/15 7/16 7/17 7/18 7/19 7/20 7/21 7/22 7/23 7/

24

7/25 7/26 7/27 7/28 7/29 7/30 7/31K ObsTemp PrdTemp

Bias Temperature

-2 -1 0 1 2 3 4

7/

2

7/

3

7/

4

7/

5

7/

6

7/

7

7/

8

7/

9 7/10 7/11 7/12 7/13 7/14 7/15 7/16 7/17 7/18 7/19 7/20 7/21 7/22 7/23 7/

24

7/25 7/26 7/27 7/28 7/29 7/30 7/31K BiasTempMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-14 Figure 4-19. WRAP_0 run July temperature time series for North subdomain.

Figure 4-20. Final WRAP run July temperature time series for North subdomain. Observed/Predicted Temperature

270 275 280 285 290 295 300 305 310 315

7/ 1 7/ 2 7/ 3 7/ 4 7/ 5 7/ 6 7/ 7 7/ 8 7/ 9 7/

10

7/

11

7/

12

7/

13

7/

14

7/

15

7/

16

7/

17

7/

18

7/

19

7/

20

7/

21

7/

22

7/23 7/

2 4 7/

25

7/

26

7/

27

7/

28

7/

29

7/

30

7/

31K ObsTemp PrdTemp

Bias Temperature

-7 -6 -5 -4 -3 -2 -1 0 1 2 3

7/ 1 7/ 2 7/ 3 7/ 4 7/ 5 7/ 6 7/ 7 7/ 8 7/ 9 7/10 7/11 7/12 7/13 7/14 7/15 7/16 7/17 7/18 7/19 7/20 7/21 7/22 7/

23

7/2 4 7/25 7/26 7/27 7/28 7/29 7/30 7/31K BiasTemp

Observed/Predicted Temperature

270 275 280 285 290 295 300 305 310 315

7/ 2 7/ 3 7/ 4 7/ 5 7/ 6 7/ 7 7/ 8 7/ 9 7/1 0 7/11 7/12 7/13 7/14 7/15 7/16 7/17 7/1 8 7/19 7/20 7/21 7/22 7/23 7/

24

7/2 5 7/26 7/27 7/28 7/29 7/30 7/31K ObsTemp PrdTemp

Bias Temperature

-5 -4 -3 -2 -1 0 1 2 3 4

7/ 2 7/ 3 7/ 4 7/ 5 7/ 6 7/ 7 7/ 8 7/ 9

7/1 0 7/11 7/12 7/13 7/14 7/15 7/16 7/17 7/1 8 7/19 7/20 7/21 7/22 7/23 7/

24

7/2 5 7/26 7/27 7/28 7/29 7/30 7/31K BiasTempMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-15 Figure 4-21 . 36 km humidity comparison soccerplot for July over western U.S.

Figure 4-22 . 36 km humidity comparison soccerplot for July over eastern and central U.S. CENRAP / VISTAS / WRAP July Humidity Performance Comparison Over the Western U.S.

0 0.5

1 1.5

2 2.5

3

-1.5 -1 -0.5 0 0.5 1 1.5 2 2.5

Humidity Bias (g/kg)Humidity Error (g/kg)

Benchmark VISTAS WRAP CENRAP OldWRAP DesertSW

PacNW

SW

North

CENRAP / VISTAS / WRAP Jul y Humidit y Performance Comparison

Over the Central and Eastern U.S.

0 0.5

1 1.5

2 2.5

3

-1.5 -1 -0.5 0 0.5 1 1.5 2 2.5

Humidity Bias (g/kg)Humidity Error (g/kg)

Benchmark VISTAS OldWRAP CENRAP WRAP CENRAP N

CENRAP S

Ohio Val

SE

NE

MidAtlMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-16 Summary of the Annual MM5 Surface Performance

• For temperature and humidity, the best performing run overall was VISTAS because it had no outliers and has no serious performan ce problems at any particular time or

subdomain.

• For winds, the best performing run was th e final WRAP run. There was not much difference in wind performance among the other three runs.

• For wind, there is not much va riation in performance in th e east or the west over the course of the annual cycle, al though the wind direction error is smaller in winter than in

summer. The wind direction RMSE does not vary significantly over the course of the year for any of the four runs.

• For temperature in the west, all runs ex cept the nudged WRAP_0 run lie outside the benchmark (except for the SW and PacificN W subdomains, which lie barely within the benchmark in December). Performance is mu ch better in the east than in the west.

• The final WRAP run had a less accurate su rface temperature simulation than WRAP_0, which used analysis nudging of surface temperature

• For temperature in the east, all or nearly a ll of the subdomains lie within the benchmark during most months.

• Humidity performance was generally within th e benchmarks for all regions for all runs except the WRAP_0 run.

• For humidity, in the west, performance is be st in winter and de teriorates as summer approaches. The culprit is the humidity erro r, not the bias, except for the WRAP_0 run outliers in July in north and desertSW.

• Except for SW, all western subdomains have a wet bias in July. This is likely related to excess simulated precipitation.

• For eastern subdomains, humid ity performance is comparable to western subdomains, as

for wind.

• The final WRAP run showed significant improvement in July humidity performance in

the west relative to the WRAP _0 run, but does worse in the east. We can attribute both of these changes in performance to reduction in rainfall and therefore surface moisture that we get in going from the Kain-Fritsch cumulus scheme to the Betts-Miller scheme.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-17 EVALUATION OF PRECIPITATION IN THE 36 KM 2002 CENRAP, VISTAS, AND

WRAP RUNS

In this section, we evaluate the precipitation fields in the VISTAS, CENRAP, WRAP_0 and final

WRAP 2002 36 km MM5 runs. We examine the annua l cycle in the monthly rainfall totals in the MM5 runs and compare them to observed monthly rainfall totals.

In January (Figure 4-23), the rainfall amount is predicted with good skill in all four runs. All four runs overestimate rainfall in the cenrapN and North subdomains. The final WRAP run produces more rainfall than the other three runs in the mountains along the west coast; this brings the final WRAP run into closer agreement with observations than the other three runs in this region. This rainfall increase is due to the stra tiform rainfall component (Figure 4-24), and is associated with mid-latitude cyclones comi ng in off the Pacific Ocean. The convective component of the rainfall decreases slightly over the western coas tal regions in the final WRAP run (Figures 4-24). The surf ace humidity for the PacificNW s ubdomain for January (Figure 4- 11) shows an improvement in the surface humid ity in going from the WRAP_0 to the final WRAP run, which is reasonable, given th e more realistic rainfall simulation.

In March, rainfall is predicted with good skill in all four runs (Figure 4-25). All four runs underpredict the maximum over Arkansas and translat e it to the east. All four runs overpredict rainfall over the desertSW subdomain. Compari ng the new and old WRAP runs (Figure 4-25), we see that there is a signal of increased rain ov er the storm track areas, but it is smaller than the increase noted above for January. The area of increased intensity over the southeastern US (Kentucky) in the final WRAP r un is an improvement over WRAP_0.

In July (Figure 4-26), rainfall is overpredicted in all four runs over most of the U.S. This is a result of excessive convective rainfall (e.g., Fi gure 4-27). Results for CENRAP and VISTAS runs are similar (but are not s hown). The problem is most se vere in the WRAP_0 run, and is

somewhat ameliorated in the final WRAP run. Th e overprediction of rain fall (especially in the North and desertSW subdomains) is associated w ith the wet humidity bi as seen in the surface soccer plots (Figure 4-21). The overprediction of ra infall in the southwestern U.S. is smallest in the final WRAP run, which has a different type of cumulus scheme than the other three runs. Overall, the level of skill could be improved in all four runs.

Comparing July precipitation in the new and old WRAP runs, we see a large reduction in the number of grid cells experienci ng convective rainfall in the fina l WRAP run. This is the main reason for the improvement in the surface humidity in the desertSW and North subdomains. However, in regions where convection did occur, it tended to be overly strong in the final WRAP run. Note, for instance, the increase in wet bias in the SE subdomain in Figure 4-22. Inspection of the July precipitation plot fo r the final WRAP run shows that the precipitation field is very granular in regions with light precipitation. It is not uncommon to have a grid cell with no rain bordering a grid cell where heavy ra in has fallen. This behavior is not observed in any of the other three runs, which use Kain-Fritsch schemes. Th is suggests that it is more difficult to trigger

convection in the Betts-Miller scheme than in the Kain-Fritsch scheme, which shows a larger number of convecting grid cells which rain weakly.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-18 Our findings in this study are similar to those of Gochis et al. (2002), w ho studied the sensitivity of the simulation of North American Monsoon preci pitation in MM5 to the choice of convective scheme. Gochis et al. (2002) noted:

“The trigger function in the Be tts-Miller-Janjic sche me also appears to inhibit convective activity in the northern North American Monsoon regions, which gives reduced precipitation and a relatively c ooler and drier mid level atmo sphere compared with the

Kain-Fritsch formulation...Where Betts-Miller-J anjic is activated... there appears to be an overestimate of precipitation. This suggests that the profile adjustment procedure used in the Betts-Miller-Janjic scheme is either yielding too much column water during the relaxation of the large-scale profile toward the reference profile, or that too much of the residual moisture is being c onverted into precipitation.”

In their study, the MM5 runs usi ng the Kain-Fritsch scheme tended to produce rainfall that was more widespread than did MM5 running with th e Betts-Miller scheme, but when the Betts-Miller scheme did switch on, it tended to rain more heavily than Kain-Fritsch. Their hypothesis is that the Kain-Fritsch scheme is more easily triggere d than the Betts-Miller scheme, and they support this hypothesis by showing that the Kain-Fritsch scheme consistently turned on earlier in the day

than the Betts-Miller scheme.

In October (Figure 4-28), all four runs underpre dict precipitation in the cenrapN, cenrapS and NE subdomains, and overpredict in the Ohio Va lley subdomain. All three runs have excessive precipitation over Florida, and underpredict a nd/or misplace the maximum over Louisiana. Comparison of the new and old WRAP runs show s a reduction in the area undergoing convective

rainfall in the Gulf of Mexico and Atlantic O cean in regions where the water is still warm. Where convection occurs, however, it tends to be more intense in the final WRAP run than in the
WRAP_0 run.

The December precipitation fields (Figure 4-29) are similar to those of January (Figure 4-23). In December, all four models predict the precipit ation field with good skill, although the maximum

over Arkansas/Louisiana is underestimated and th e intensity of the preci pitation over the Oregon

coast is too low.

In summary, differences among the four runs in terms of precipitation performance were relatively small. All four runs were able to reproduce many of the major stratiform (i.e. resolved at grid-scale) precipitation feat ures, and all four had difficult y with convectiv e precipitation. This is to be expected, given that co nvection is a subgrid-scale phenomenon whose parameterization is poorly understood (e.g. Randa ll et al. 2003). The biggest difference among the four simulations came in July; in the west ern U.S., the overestimation of precipitation was smallest in the final WRAP run, but the final WR AP run also overpredicts convective rainfall in the southeastern U.S. These changes are clearly re lated to the use of a different type of cumulus

scheme in the final WRAP run than in the other three runs. For the purpo ses of WRAP visibility modeling, the second WRAP run is clearly an improvement over the WRAP_0 and other runs, and has more accurate precipitation fields in the western U.S. in both summer and winter.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-19 Figure 4-23. January observed (top left) and modeled precipitation for CENRAP (top right), VISTAS (middle left), final WRAP (middle right) and WRAP_0 (bottom right) 2002 36 km simulations and differences between WRAP-0 and final WRAP (bottom left).March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-20 Figure 4-24. Partitioning of January total rainfall into its convective and stratiform components for the WRAP_0 (top) and final WRAP (bottom) 36 km MM5 runs.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-21 Figure 4-25. March observed (top left) and modeled precipitation for CENRAP (top right), VISTAS (middle left), final WRAP (middle right) and WRAP_0 (bottom right) 2002 36 km simulations and differences between WRAP-0 and final WRAP (bottom left).March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-22 Figure 4-26. July observed (top left) and modeled precipitation for CENRAP (top right), VISTAS (middle left), final WRAP (middle right) and WRAP_0 (bottom right) 2002 36 km simulations and differences between WRAP-0 and final WRAP (bottom left).March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-23 Figure 4-27. Partitioning of July total rainfall into its convective and stratiform components for the WRAP_0 (top) and final WRAP (bottom) 36 km MM5 runs.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-24 Figure 4-28. October observed (top left) and modeled precipitation for CENRAP (top right), VISTAS (middle left), final WRAP (middle right) and WRAP_0 (bottom right) 2002 36 km simulations and differences between WRAP-0 and final WRAP (bottom left).March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-25 Figure 4-29. December observed (top left) and modeled precipitation for CENRAP (top right), VISTAS (middle left), final WRAP (middle right) and WRAP_0 (bottom right) 2002 36 km simulations and differences between WRAP-0 and final WRAP (bottom left).March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-26 UPPER-AIR METEOROL OGICAL EVALUATION

To assess whether MM5 is simulating the vertical structure of the atmo sphere with reasonable accuracy in the four 36 km runs, we compared model temperature and dew point soundings with

those from a limited number of radiosonde sta tions. Based on the analysis of surface meteorology and precipitation model performance, it seemed reasonable to focus on the months of January and July. These two months were at extremes of good and poor model performance

at the surface.

For the months of January and July, a radiosonde station was selected from each subdomain, and the 00Z and 12Z observed and modeled soundings fr om each day of the January and July were compared for the CENRAP, VISTAS, WRAP_0, a nd final WRAP runs. 12Z and 00Z soundings were analyzed in order to examine differences between stable (nocturnal ) and unstable (daytime) conditions (00Z and 12Z are 4pm PST/7pm EST a nd 4am PST/7am PST, respectively). In the following discussion, we will summarize the genera l features of the upper air soundings found in the extended analysis of January and July. By looking at two months for a limited number of stations, we had a manageable amount of sounding data to analyze, but we caution that it would be unwise to place too much confidence in genera lizing these results from two months of data

from 11 stations to the entire MM5 domain and the entire year.

Summary of Upper Air Sounding Comparison

In this section, we discuss some common features of the four simulations and some important differences among them. Overall, in all four ru ns, MM5 simulates the temperature profile more accurately than the dew point temperature profile . Because of its limited resolution, the model tends to have a smoother dew point temperat ure than the observations, and may miss sudden excursions in the dew point temperature related to the vertical distribution of humidity. In general, MM5 tended to be more accurate in reproducing the 00Z sounding than the 12Z sounding for a given station. This seems to be because MM5 has greater skill in simulating the fully developed afternoon/early evening convective boundary layer than the early morning nocturnal boundary layer, whose physical processe s are more complex and less well understood.

In both January and July, the CENRAP and VIST AS runs were better able to simulate PBL temperature inversions than were the two WR AP runs. Figure 4-30 shows the 12Z sounding for Midland, TX (MAF) for January 7, 2002. The CE NRAP and VISTAS runs both underestimate the strength of the inversion, but the problem is worse in the WRAP_0 and final WRAP runs. In both WRAP runs, the temperature is colder in the 900-800 mb laye r than it is in CENRAP or VISTAS. The dew point temperature sounding in both WRAP runs is also too warm in the 900- 825 mb layer; CENRAP and VISTAS also have this problem, but to a lesser extent. Figure 4-31 shows the 12Z sounding for Spokane, WA (OTX) for July 2, 2002. Here, again, MM5 underestimates the inversion strengt h in all four runs, but the probl em is more severe in the two

WRAP runs. Note that the WRAP _0 run simulation of the dew point profile is too cold in the 950-750 mb layer, and that CENRAP and VISTAS both do a better job with this profile. The final WRAP run is a significant improvement over WRAP_0 in its handling of the low level dew point profile.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-27 CENRAP and VISTAS dew point temperature profiles were gene rally closer than that of WRAP_0 to observed profile, and we re better able to handle extr eme excursions in temperature and dew point profiles in the lo wer troposphere. The final WRAP run tended to have a more accurate dew point profile than the WRAP_0 run at 12Z, and a temperature profile that was less accurate near the surface, which is reasonable gi ven the lack of surface temperature analysis nudging the final WRAP run. Because of the an alysis nudging of surface temperature, the WRAP_0 surface temperature was generally closer than CENRAP or VISTAS to the observed surface temperature, but profiles above surface we re less accurate in WRAP_0 than in CENRAP or VISTAS indicating that although the surface temperature nudging improves model performance at the surface it does not for temperatures aloft.

For July soundings with a deep, convecting boun dary layer, CENRAP and VISTAS frequently better reproduced the observed temperature inve rsion at the top of the PBL, while the WRAP_0 run showed a smoother decrease of temperatur e with increasing altitude. This problem is

somewhat ameliorated in the final WRAP run. Figure 4-32 shows the observed and simulated soundings for July 16, 0Z at Midland, TX. Th e observed sounding shows a deep, dry adiabatic convecting layer with an inversi on at 700 mb. Both the CENRAP and VISTAS runs simulate an inversion just above 700 mb, a lthough it is weaker than the ob served inversion. The WRAP_0 temperature sounding shows temperature decreasi ng smoothly with increasing altitude, with no

inversion. The final WRAP run still does not mana ge to produce an inversion, but does a better job of approximating the temperature profile immediately above the observed inversion.

Vertical wind profiles in all f our runs were similar, particul arly above the boundary layer (this reasonable since all runs were analysis nudged ab ove the PBL). Even in cases where the PBL temperature profiles were different , wind structure was similar. This suggests that local thermal processes are driving the temperature differences, rather than advection.

One issue for all four runs is a discrepancy be tween the observed and modeled surface pressure. Figure 4-33 shows the 12Z sounding for Flagstaff, AZ (FGZ) for January 4, 2002 for all four runs. The modeled soundings for the CENRAP and VISTAS runs are quite similar. The WRAP_0 sounding is less accurate than either CENRAP or VISTAS, completely missing the inversion near the 700 mb level, and is too wa rm and too dry in the 800-700 mb layer. The excessive warmth and dryness also occur in th e CENRAP and VISTAS runs, but to a lesser degree. The final WRAP run has a more accurate surface pressure than any of the other three runs. This improvement in the surface pressure in the final WRAP run seems to be a general feature of this run, and was not ed in both summer and winter soundings. It is unclear what caused this improvement. Like the WRAP_0 r un, the final WRAP run does not simulate an inversion near 700 mb, but is closer to the observed temperature and dew point profiles.

In general, the differences between the initial an d final WRAP runs varied with season. The two runs differ in their parameterization of cumulu s convection and surface observation and analysis nudging. During winter, there was not much di fference between the WRAP runs. What

differences there were occurred near the surf ace. The WRAP_0 run tended to have a more accurate temperature profile near the surface, due to its surface anal ysis nudging. Because cumulus convection is not as active during th e winter, the surface temperature differences predominated in winter. There was generally little change in the dew point prof ile in winter.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-28 In summer, there were larger differences stem ming from the difference in parameterization of cumulus convection. Although there was a deterior ation in the temperature profile near the surface in the final WRAP run (due to turni ng off surface temperature nudging), the dew point profile was often more accura te in the final WRAP run (s ee for example, Figure 4-31)

Also, the use of the Betts-Miller cumulus scheme tended to produce profiles of temperature and dew point indicating that the atmosphere was close to or at saturation above the PBL top inversion. Figure 4-34 shows soundi ngs for Flagstaff, AZ on July 4 and July 6 at 0Z. These soundings are fairly typical for July at this s ite. The final WRAP run, which uses the BettsMiller cumulus scheme, is clearly being driven harder toward a saturated profile than the WRAP_0 run. This is likely due to the way the Betts-Miller scheme relaxes toward a particular reference profile as cumulus convection acts to remove grid-scale instability. This type of profile was not observed in the CENRAP or VISTAS runs, which, like WRAP_0, use a KainFritsch scheme.

Overall, the upper air profiles in the CENRAP and VISTAS runs were very similar to one another, and both of these runs had some mark ed differences with the two WRAP runs (i.e. success at simulating low-level no cturnal inversions or the PBL top inversion). Because the WRAP_0 run used a Kain-Fritsch scheme, as did the CENRAP and VISTAS runs, it seems likely that the differences between the two WRAP runs and the other two runs may be attributed to the different explicit moisture schemes. CENRAP and VISTAS use Reisner I and the WRAP

runs used Reisner II. Because simulation of invers ions is so critical fo r visibility modeling, the issue of explicit scheme selection and its rela tion to the upper air atmospheric structure in MM5 bears further investigation.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-29 Figure 4-30 . January 7, 2002, 12Z Sounding for Midland, TX. Upper left hand panel: CENRAP, Upper right hand panel: VISTAS, Lower left hand panel: WRAP_0, Lower right hand panel: Final WRAP.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-30 Figure 4-31. July 2, 2002, 12Z Sounding for Spokane, WA. Upper left hand panel: CENRAP, Upper right hand panel: VISTAS, Lower left hand panel: WRAP_0, Lower right hand panel:

Final WRAP.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-31 Figure 4-32 . July 16, 2002, 0Z Sounding for Midland, TX. Upper left hand panel: CENRAP, Upper right hand panel: VISTAS, Lower left hand panel: WRAP_0, Lower right hand panel:

Final WRAP.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-32 Figure 4-33 . January 4, 2002, 12Z Sounding for Flagstaff, AZ. Upper left hand panel: CENRAP, Upper right hand panel: VISTAS, Lower left hand panel: WRAP_0, Lower right hand panel: Final WRAP.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec4.doc 4-33 Figure 4-34 . July 4,6, 2002, 0Z Soundings for Flagstaff, AZ. Upper left hand panel: WRAP_0 July 4, Upper right hand panel: Final WRAP July 4, Lower left hand panel: WRAP_0 July 6, Lower right hand panel: Final WRAP Jul 6.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec5.doc 5-1 5.0 EVALUATION OF THE 2002 12 KM WRAP MM5 SIMULATION

In Section 5, we assess the performance of the 2002 12 km MM5 run. The evaluation of the MM5 surface meteorological variables using METSTAT is presented first. This is followed by a comparison of the CPC observed and MM5 predicted precipitation patterns.

As in the analysis of the 36 km run in Secti on 4, soccerplots were generated and evaluated for each month of the year 2002 for the 12 km grid. On these soccerplots, we also include for reference the data from the 36 km run, so that we can see the effects on performance of running at higher resolution and with diffe rent cumulus parameterization ( none, in the case of the 12 km run). As in the 36 km run, MM5's performance was qualitatively different during summer and winter, with the spring and fall seasons servi ng as transitional periods between winter and summer. January and July were found to be representative of MM5’s performance during the winter and summer seasons. For brevity, we incl ude only January and July in this report. The main strengths and weaknesses of the 2002 12 km MM5 run are captured in the January and July plots. For both months, we show a soccerplot for the subdomains in the western U.S. (the 12 km WRAP region). The subdomain de finitions are the same as those used in the 36 km analysis (shown in Figure 3-1).

Surface MM5 Meteorological Performance

Figure 5-1 displays the January soccerplot for te mperature for the 12 km and 36 km final WRAP runs. For both 36 km and 12 km runs, all subdoma ins fall outside the benchmark. For the North, SW, and desertSW subdomains, the 12 k run offers a small improvement in performance, but not enough to move the subdomains signifi cantly closer to the benchmark.

The wind performance for January is shown in Fi gure 5-2. For all four subdomains, the 12 km wind performance falls within the benchmark a nd is a slight improvement over the 36 km performance. For humidity (Figure 5-3), bot h the 36 km and 12 km runs are within the benchmark, with the 12 km run again showi ng a slight improvement over the 36 km run.

Figure 5-4 shows the July soccerplot for temperatur e for the 12 km and 36 km final WRAP runs. The overall pattern, with the PacificNW, North, and desertSW having a cold bias and SW having a warm bias is the same as in January. The co ld bias of the PacificNW, North, and desertSW subdomains has decreased relative to January, wh ile the warm bias of the SW subdomain has increased. The bias for the PacificNW, Nort h, and desertSW subdomains is now within the benchmark. Although the temperature error is not within the benchmark for these three subdomains, it has been reduced relative to Ja nuary. This is a significant improvement in performance relative to the WRAP_0 36 km and 12 km runs, and is a result of the final WRAP run configuration’s optimization for improveme nt of the July humidity and temperature performance. The reduction in cold bias in thes e three subdomains is very likely a result of the improved precipitation field relative to the WR AP_0 run. The final WRAP run does not have the same overprediction of rainfall, so that surf ace energy is no longer spuriously diverted into evaporating the excess moisture from the surface, and is more properly used in heating the

surface. This reduces the cold bias in the surf ace temperature. The July 12 km run is less skillful overall than the 36 km run for temper ature. It is unclear why this is so.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec5.doc 5-2 In Figure 5-5, we show the wind performance soccerplot for July. The wind performance is slightly degraded relative to January, but is re latively close to the benchmark, and is improved relative to the 12 km WRAP_0 run (not shown). The 12 km run shows an improvement in wind speed RMSE, but not wind direction error relative to the 36 km run.

For July humidity (Figure 5-6), all subdomains fall within the be nchmark. This is in marked contrast to the WRAP_0 run ( not shown), where strong wet bias es in the occurred North and

desertSW subdomains due to excessive convective rain fall in those regions. The large wet biases of these two subdomains, which placed them we ll outside the range of results for previous meteorological databases used in air quality applications, was a major motivation for performing the final WRAP run with a different configura tion than the WRAP_0 run. Relative to January, the 12 km results for July humidity show that the humidity error has increased for all four subdomains with the onset of summer, and that the wet bias has increased for the PacificNW, desertSW, and North subdomains. This is consiste nt with the overestimate of July rainfall (true even in the final WRAP run; see Figures 5-7E and 5-7F) in these thr ee subdomains. The SW subdomain does not receive much rainfall in July, and its humidity bias does not change much relative to January, although its humidity error increases.

In summary, the METSTAT surface analysis shows that the 12 km run is within or near performance benchmarks for wind and humid ity over the annual cycle of 2002, but the temperature results fall outside the bias benchmark. It is possible that this is due to terrain

resolution effects. The final 12 km WRAP run ha s been significantly improved in terms of its surface performance relative to the WRAP_0 12 km run.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec5.doc 5-3 Figure 5-1. 36km/12 km surface temperature comparison soccerplot for January.

Figure 5-2. 36km/12 km surface wind comparison soccerplot for January. WRAP 12/36 km January Temperature Performance Comparison

0 0.5

1 1.5

2 2.5

3 3.5

-2 -1.5 -1 -0.5 0 0.5 1

Temperature Bias (K)Temeprature Error (K)

Benchmark WRAP 12 km WRAP 36 km DesertSW

PacNW

SW

North

WRAP 12/36 km January Wind Performance Comparison

0 10 20 30 40 50 60 70 80

0 0.5 1 1.5 2 2.5 3

Wind Speed RMSE (m/s)Wind Direction Error (m/s)

Benchmark WRAP 12 km WRAP 36 km DesertSW

PacNW

SW

NorthMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec5.doc 5-4 Figure 5-3. 36km/12 km surface humidity comparison soccerplot for January

.

Figure 5-4. 36km/12 km surface temperature comparison soccerplot for July. WRAP 12/36 km January Humidity Performance Comparison

0 1 2 3

-1.5 -1 -0.5 0 0.5 1 1.5 2 2.5

Humidity Bias (g/kg)Humidity Error (g/kg)

Benchmark WRAP 12 km WRAP 36 km DesertSW

PacNW

SW

North

WRAP 12/36 km July Temperature Performance Comparison

0 0.5

1 1.5

2 2.5

3 3.5

-2 -1.5 -1 -0.5 0 0.5 1
Temperature Bias (K)Temeprature Error (K)

Benchmark WRAP 12 km WRAP 36 km DesertSW

PacNW

SW

NorthMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec5.doc 5-5 Figure 5-5. 36km/12 km surface wind comparison soccerplot for July.

Figure 5-6. 36km/12 km humidity wind comparison soccerplot for July. WRAP 12/36 km July Wind Performance Comparison

0 10 20 30 40 50 60 70 80

0 0.5 1 1.5 2 2.5 3

Wind Speed RMSE (m/s)Wind Direction Error (m/s)

Benchmark WRAP 12 km WRAP 36 km DesertSW

PacNW

SW

North

WRAP 12/36 km July Humidity Performance Comparison

0 1 2 3

-1.5 -1 -0.5 0 0.5 1 1.5 2 2.5

Humidity Bias (g/kg)Humidity Error (g/kg)

Benchmark WRAP12 km WRAP 36 km DesertSW

PacNW

SW

NorthMarch 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec5.doc 5-6 EVALUATION OF PRECIPITATION IN THE 2002 12 KM WRAP RUN

In this section, we evaluate the precipitation pe rformance of the 2002 12 km MM5 run. The original WRAP_0 run had a strong positive precipitation bias over the WRAP region on both the 36 km and 12 km grids. In tandem with the 36 km sensitivity tests described in Kemball-Cook et al. (2004b), a second series of sensitivity tests was performed on the 12 km grid to identify a more optimal configuration for that domain. Th e best choice for cumulus scheme turned out to be no cumulus parameterization, as discussed in Section 2. Figures 5-7A through 5-7J show a comparison of the observed CPC precipitation an d the MM5-predicted precipitation over the

course of the final WRAP 2002 12 km simulation.

In January (Figures 5-7A and 5-7B), the agr eement between the overall predicted and observed precipitation pattern is reas onably good. MM5 picks up the precipitation maxima over the mountain ranges in the Pacific Northwest, although rainfall amounts are too high over both ranges. There is excessive precipitation in the cenrapN and North regions, and MM5 under predicts the rainfall over the central California coast. In March (Figure 5-7C and 5-7D), the model again over predicts the rainfall over the mountain ranges of the Pacific Northwest. Aside from this, however, MM5 shows impressive skill.

As we move from winter to summer and conv ective rainfall becomes more important, MM5’s forecast skill deteriorates. Figure 5-7E a nd Figure 5-7F show the observed and modeled precipitation for July. Although MM5 does a reasonable job with the SW and PacNW subdomains, where little or no rain falls, there is a gene ral overprediction of rainfall. This is

consistent with the wet bias seen in the surface hu midity soccerplot for Ju ly (Figure 5-6) for all subdomains except SW. In general, the model does a good job with th e overall precipitation pattern, but individual maxima are over predicted. The model is running with no convective parameterization. This means that in order for c onvection to occur, the en tire 12 km grid column

must saturate and be unstable. In the real wo rld, convective updrafts te nd to be smaller than 12 km across. This may mean that it is relatively difficult for mode led convection to be initiated,

and it becomes unrealistic ally intense when it does occur because an unphysical amount of instability has been allowed to build up in the convecting grid cell.

As fall arrives, and the partitioning of rainfall moves toward an increase in the stratiform component, MM5’s performance improves. Figu res 5-7G and 5-7H show the observed and modeled precipitation for October. In October, the model unde restimates precipitation in the banded features over the cenrapS region, but otherwise agrees reas onably well with observations.

MM5’s December performance (Figures 5-7I and 5- 7J) is similar to that of January. The model simulates the overall pattern of precipitation over the Pacific Nort hwest, but overestimates the intensity of the maxima. As in January, precipitation over the No rth and cenrapN subdomains is overestimated. Otherwise, the model does a good job of simulating the December precipitation

field.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec5.doc 5-7 (A) (B)

(C) (D)

(E) (F) Figure 5-7. Annual cycle in 12 km MM5 precipitation. (A) January CPC Observed Precipitation. (B) January MM5 Predicted Total Precipitation. (C) March CPC Observed Precipitation. (D) March MM5 Predicted Total Precipitation. (E) July CPC Observed Precipitation. (F) July MM5 Predicted Total Precipitation.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec5.doc 5-8 (G) (H)

(I) (J) Figure 5-7. Annual cycle in 12 km MM5 precipitation. (Cont.) (G) October CPC Observed Precipitation (H) October MM5 Predicted Total Precipitation. (I) December CPC Observed Precipitation. (J) December MM5 Predicted Total Precipitation.

In summary, MM5 predicts the precipitation on the 12 km grid with reasonable skill over most of the annual cycle. The performance is better in winter than in summer. Throughout the year, the model tends to overpredict prec ipitation maxima, but does a good job in simulating the overall precipitation pattern. The final WRAP run e xhibits better skill than the WRAP_0 run, particularly in July. The modele d rainfall is still excessive in Ju ly, but the severity of the over prediction and the corresponding biases in July su rface temperature and humidity have lessened.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec6.doc 6-1 6.0 CONCLUSIONS

At both 12 km and 36 km resolutions, the fina l WRAP 2002 MM5 simulation produced results that are generally within the ra nge of meteorological model results that have been used in the past for air quality applications. The final 36 km and 12 km runs represent a significant improvement in performance ove r the original WRAP_0 run.

In the 36 km WRAP_0 simulation, surface humidity and temperature fields fell within the benchmarks for much of the year for most s ubdomains, but the model had a marked cold, wet bias especially during the summer in the wester n subdomains. This was unfortunate since the 20% worst visibility days at we stern class I areas that are of high interest to WRAP generally occur during the summer. The surface wind field failed to meet the performance benchmark for the entire year in subdomains PacificNW, SW, North, and DesertSW. Gi ven that the purpose of this run was to provide a database for a 12 km nested MM5 run and air quality modeling studies centered on the western U.S., this was a serious problem.

The final 36 km and 12 km MM5 runs show im provement in the modeled precipitation fields, particularly in the summer in the southwest. With a reduction in the ove rprediction of summer convective rainfall in both 36 and 12 km runs, wet bi ases in the surface humidity and cold biases in the surface temperature ar e now smaller in the North and DesertSW subdomains in the summer months. Although the final WRAP run wa s optimized for summer performance over the WRAP region, the winter performance in the west did not deteriorate si gnificantly. There was a small loss of accuracy, particularly in temperature and humidity in the east, but this region is not

the focus of WRAP. To summari ze, the final WRAP 36 km run:

• Saw its surface wind performance improve sign ificantly throughout the year due to observational nudging of surface winds

• Showed significant improvement in summer ra infall and surface humidity performance in the WRAP region

• Did worse than the original WRAP run for humidity performance in the east. We can attribute both of these changes in performan ce to reduction in the areal coverage in

rainfall and increase in convective rainfall in active cells (and therefore surface moisture) that we get in going from the KF scheme to the Betts-Miller scheme.

• Showed improved temperature performance in summer in the west, and slightly worse performance in winter

• Showed a small overall degrada tion in performance in the ea st. Some of this was the results of eliminating surface analysis nudging of temperature and moisture that was done
in WRAP_0March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Sec6.doc 6-2 Our comparison of the two 36 km WRAP runs (WRAP_0 and final WRAP) to the CENRAP and

VISTAS 36 km continental-scale annual runs showed:

• Overall, VISTAS performed best in the simu lation of surface temperature and humidity. It had no outliers and has no serious performan ce problems at any particular time or

subdomain. It is unclear whether this is due to the explicit moist physics or convection

schemes or to the interaction between them , or to effects of differences in FDDA.

• The final WRAP run performed best for surface winds throughout the year.

• For precipitation, the four runs were similar in terms of performan ce if the whole 36 km domain and the entire year are considere d. Over the WRAP region, however, the final WRAP run performs best, with the smallest ove rprediction of convective rain of all runs. The overprediction of convective precipitat ion was most severe in the WRAP_0 run.

• For upper air structure, CENRAP and VISTAS performed best and were similar to one

another. We attribute this to the use of the Reisner I scheme, as the simulation of, for example, the PBL inversion was relatively insens itive to the change of convection scheme in

the WRAP runs.

• In all four runs, MM5 performed be tter in winter than in summer.

• Although WRAP_0 had the best surf ace statistics for temperature, its upper air performance was worst of all four runs, which suggests th at analysis nudging of surface temperature and humidity is counterproductive. The final WRAP run, which did not have analysis nudging of surface temperature and humidity, had larger errors in its temperature st ructure in the lowest levels of the atmosphere, but had a more re alistic dew point profile and a smaller surface pressure bias than any of the four runs.

• Based on the upper air soundings, the most seri ous problem is the difficulty MM5 has is establishing the observed PBL structure. MM5 has trouble getting the PBL depth right, particularly in the stable noc turnal case. Also, the model’s difficulty in simulating the observed fine structure of the dew point te mperature profile and the overall level of saturation in the lower troposphere is cause for concern. It is important that model produce cloud decks at the correct height. Errors in humidity and cloud prediction will have a negative impact on the accuracy of downwe lling solar radiation, cau se errors in the temperature profile and the surface fluxes, aff ect chemistry, and make it difficult for the PM model to perform properly.

We conclude, based on the results of this study, that the final 36 km and 12 km WRAP MM5 runs exhibit reasonably good performance and are certainly within the bounds of other meteorological databases used for prior air quality modeling efforts. It is therefore reasonable to proceed with their use as inputs for visibility modeling.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Ref.doc R-1 REFERENCES

Arakawa, A. and J.M. Chen. 1987. “Closure assumptions in the cumulus parameterization problem.” Short and Medium Range Nume rical Prediction, Collection of Papers. Presented at the WMO/IUGG NWP Symposium, Tokyo. T. Matsuno, Ed. Meteor. Soc. Japan. 107-131.

ATMET. 2003. "MM5 Simulations for TexAQS 2000 Episode, Task 3: Sensitivities to

modifications of the MRF PBL scheme." Pr epared for the Houston Advanced Research Center, The Woodlands, TX, and the Texa s Commission on Environmental Quality, Austin, TX, by ATMET, LLC, Boulder, CO September.

Betts, A. and M. Miller. 1986. A new convective ad justment scheme. Part I: Observational and

theoretical basis. Quart. J. Roy. Meteor. Soc. 99 , 178-196.

Emery, C.A. and E. Tai. 2001. “Enhanced meteorological modeling and performance evaluation for two Texas ozone episodes.” Prepared for the Texas Natural Resource Conservation Commission, by ENVIR ON International Corporation.

Emery, C.A., S. Kemball-Cook, Y. Jia, Z. Wang and R. Morris. 2004. “2002 MM5 Model

Evaluation: 12 vs 36 km Results.” Presented at May 24-25, 2004 National RPO Modeling Meeting, Denver, CO. http://www.cleanairinfo.com/rpomodelingdenver

.

ENVIRON and UCR. 2004. “2002 Annual MM5 Simulations to Support WRAP CMAQ Visibility Modeling for the Section 308 SIP/TIP”. Draft Protocol. ENVIRON International Corporation and the University of California at Riverside. April. Available

at: http://pah.cert.ucr.edu/aqm/308/mm5.shtml

.

Gochis, D., W.J. Shuttleworth, and Z.L. Ya ng. 2002. “Sensitivity of the modeled North American Monsoon regional climate to convective parameterization.” Mon. Wea. Rev

., 130 , 1282-1298.

Hart, J. 2000. “Forecasting studies using hourly model-generated data.” Final Report to the COMET Outreach Program.

Kemball-Cook, S., Y. Jia, C. Emery, R. Morris, Z. Wang and G. Tonnesen. 2004a. “2002 Annual MM5 36 km simulation to support WR AP CMAQ Visibility Modeling for the Section 308 SIP/TIP: Preliminary Report on the Initial 2002 36 km MM5 Simulation Performed during 2003.” ENVIRON Internati onal Corporation and UC Riverside.

Available at: http://pah.cert.ucr.edu/aqm/308/mm5.shtml . March.

Kemball-Cook, S., Y. Jia, C. Emery, R. Morris, Z. Wang and G. Tonnesen. 2004. “MM5 Configuration Sensitivity Testing” ENVI RON International Corporation and UC

Riverside. Available at: http://pah.cert.ucr.edu/aqm/308/mm5.shtml . December.

Kemball-Cook, S., Y. Jia, C. Emery, R. Morris. 2004. “Response to Comments on Sensitivity Report” Available at: http://pah.cert.ucr.edu/aqm/308/mm5.shtml . December.March 2005

H:\WRAP RMC\2002MM5\CEN_VIS_newWRAP_eval\DraftFinal\Ref.doc R-2 Molinari, J. and M. Dudek. 1992. “Parameteriza tion of convective precip itation in mesoscale

numerical models.” Mon. Wea. Rev

., 120 , 326-344.

Molinari, J.. 1993. “An overview of cumulus parameterization in mesoscale models. The Representation of Cumulus Convection in Numerical Models. K. Emanuel and D. Raymond, Ed. American Meteorological Society , p.155-159.

Morris, R.E., S. Kemball-Cook, Y. Jia, C. Emery and Z. Wan. 2004a. “2002 MM5 36 km

Evaluation.” Presented at WRAP Re gional Modeling Center Workshop, Tempe

Arizona.” Available at: http://pah.cert.ucr.edu/aqm/308/mee tings/Jan_2004_ppt/040128Modeling_Workshop_A genda.htm . January.

Morris, R.E., B. Koo, S. Lau, T.W. Tesche, D. Mc Nally, C. Loomis, G. Stella, G. Tonnesen and Z. Wang. 2004b. “VISTAS Emissions and Ai r Quality Modeling – Task 4cd Report:

Model Performance Evaluation and Model Sensi tivity Tests for Three Phase I Episodes. ENVIRON International Corporat ion, Novato, California. Se ptember. Available at: http://pah.cert.ucr.edu/vistas/docs.shtml

.

Olerud, D. and A. Sims. 2003. “MM5 Sensitivity modeling in support of VISTAS (Visibility Improvement-State and Tribal Association).” Prepared fo r Mike Abraczinskas of the VISTAS Technical Analysis Workgroup by Ba ron Advanced Meteorological Systems,

LLC.

Pleim, J.E. and J.S. Chang. 1992: A non-loca l closure model for vertical mixing in the convective boundary layer. Atmos. Env

., 26A , pp. 965-981.

Randall, D.A., M. Khairoutdinov, A. Arakawa, a nd W. Grabowski. 2003. Breaking the cloud parameterization deadlock. Bull. Am. Met Soc. 84 , 1547-1564.

Reisner, J.R., R.M. Rasmussen, and R.T. Bruintje s. 1998. Explicit forcing of supercooled liquid water in winter storms using the MM5 mesoscal e model. Quart. J. Roy. Met. Soc., 124B, pp. 1071-1107.

Tesche, T.W. et al. 2002. "Operational Eval uation of the MM5 Meteor ological Model over the

Continental United States: Protocol for Annua l and Episodic Evaluation." Prepared for the U.S. Environmental Protection Agency, Office of Air Quality Planning and Standards, prepared by Alpine Ge ophysics, LLC, Ft. Wright, KY.

Xiu, A. and J.E. Pleim. 2000. Development of a land surface model. Part I: application in a mesoscale meteorology model. J. App. Met.

, 40 , pp. 192-209.