## CMAQ Model Performance Evaluation for 2000 in

Central California

Planning and Technical Support Division

Air Resources Board California Environmental Protection Agency 1001 I Street, Sacramento, CA 94814

March 12, 2008ii

DISCLAIMER

The conclusions and recommendations contained in th is document are those of the authors and do not necessarily reflect the view s and policies of the California

Air Resources Board. The mention of trade names or commercial products does

not constitute endorsement or recommendation for th eir use.iii

TABLE OF CONTENTS

DISCLAIMER.........................................

................................................... ........................................ii

TABLE OF CONTENTS ..................................

................................................... ............................. iii

TABLE OF FIGURES ...................................

................................................... ................................iv

1. Introduction: ......................................

................................................... ................................... 1

2. Regional Air Quality Methodology ...................

................................................... .................... 1

3. Chemical Mechanism.................................

................................................... .......................... 2

4. Model Inputs and Setup.............................

................................................... .......................... 2

4.1. Domain Structure ...................................

................................................... ..................... 2

4.2. Initial and Boundary Conditions ....................

................................................... .............. 3

4.3. Emissions ..........................................

................................................... .......................... 3

4.4. Meteorological Inputs ..............................

................................................... .................... 4

4.5. MM5 Simulation.....................................

................................................... ...................... 4

5. Model Performance Evaluation.......................

................................................... ..................... 6

5.1. Spatial Patterns for PM mass and components........

................................................... .. 6

5.2. Statistics for Mean Fractional Error (MFE) and Bias (MFB) ........................................ 12

5.3. Time-History Plots:................................

................................................... .................... 14

6. Conclusion: ........................................

................................................... ................................ 14iv

TABLE OF FIGURES

Figure 1: The CCAQS modeling domain with the SJV mo deling domain inset. ......................... 3 Figure 2. The location of the three nested grids ad opted for numerical modeling of the SJV

using MM5..........................................

................................................... ....................... 5 Figure 3: The annual average concentrations for PM 2.5 and its components ............................. 7 Figure 4: The first quarter average concentrations for PM 2.5 and its components...................... 8 Figure 5: The second quarter average concentrations for PM 2.5 and its components................ 9 Figure 6: The third quarter average concentrations for PM 2.5 and its components ................. 10 Figure 7: The fourth quarter average concentrations for PM 2.5 and its components ................ 11 Figure 8: The annual average mean fractional error (MFE) and bias (MFB) for PM 2.5 and its

major components. The red line is the criteria and the blue line the goal. ............... 13 Figure 9: The time-history plots of PM 2.5 and its major components at the Bakersfield

monitoring station.................................

................................................... ................... 15 Figure 10: The time-history plots of PM 2.5 and its major components at the Fresno monitoring

station............................................

................................................... .......................... 161

1. Introduction:

The San Joaquin Valley (SJV) of California is curre ntly designated as

nonattainment for PM 2.5 . Addressing this issue properly requires rigorous longterm chemical transport modeling to support the dev elopment of a plan to attain the National Ambient Air Quality Standard (NAAQS) f or PM 2.5 . These chemical transport modeling exercises serve to determine fut ure year attainment status for the SJV given projected emissions scenarios and als o the most effective emissions reduction pathways to control PM concentr ations for different seasons and regions of the domain.

This document describes the performance evaluation of the United States Environmental Protection Agency (USEPA) Community M ultiscale Air Quality (CMAQ) model (USEPA, 1999; CMAS, 2007) annual simul ation conducted to assist the preparation of the State Implementation Plan for PM 2.5 in the SJV. The base year for this simulation is 2000, for which a rich database of observations was collected as part of the California Regional PM 10 /PM 2.5 Air Quality Study.

This evaluation includes PM 2.5 mass and its major components such as nitrate, sulfate, ammonium, organic carbon, element al carbon, and dust. Since there are no universally accepted criteria for the evaluation of PM mass or components, we selected those proposed by Boylan an d Russell (2006). Here
we use the traditional statistical metrics, correla tion diagrams, spatial plots, and time-history plots.

2. Regional Air Quality Methodology:

As stipulated in USEPA modeling guidance, a grid-ba sed photochemical model is necessary to perform the modeled attainmen t test for PM 2.5 (USEPA, 2007). Such models offer the best available repres entation of important atmospheric processes and are an essential tool in analyzing the impacts of proposed emissions controls on pollutant concentrat ions. The USEPA recommends guidelines for choosing a model for use in the attainment test. For example, the model source code should be free or lo w cost, modeling elements should have undergone rigorous scientific peer-revi ew, and the model should have been shown to perform well in the past for sim ilar applications.

The CMAQ model has been selected for use in the PM 2.5 modeled attainment demonstration for the San Joaquin Valley Air Pollution Control District. CMAQ is a state-of-the-science “one-atmosphere” sys tem that treats major atmospheric and land processes (e.g., advection, di ffusion, gas phase chemistry, gas-particle mass transfer, nucleation, coagulation , wet and dry deposition, aqueous phase chemistry, etc.) and a range of speci es (e.g., anthropogenic and2

biogenic, primary and secondary, gaseous and partic ulate) in a comprehensive framework (USEPA, 1999; CMAS, 2007).

CMAQ has been extensively peer-reviewed, is well-do cumented, and is regularly updated to reflect the latest changes in scientific understanding. CMAQ has been applied successfully in a range of environ ments and on many spatial and temporal scales. Given that CMAQ has also been applied successfully to episodic modeling in Central California, the CMAQ m odeling system version 4.6 with California-specific updates, as described in L iang and Kaduwela (2005), was selected for use in support of the PM 2.5 modeled attainment demonstration.

3. Chemical Mechanism:

There are a number of gas-phase chemical mechanisms readily available for application in CMAQ (e.g., CB-IV, CB-V, SAPRC-9 9). The user has the additional option of whether to couple the chosen g as phase mechanism with aerosol and/or aqueous phase chemical processes. I n order to simulate the complex mixture of PM 2.5 species in the San Joaquin Valley, SAPRC99 coupled with CMAQ aerosol code version 4 and aqueous phase chemistry has been chosen for this application. SAPRC-99, a complete update of SAPRC-90, is a detailed mechanism describing the gas-phase reactio ns of volatile organic compounds (VOCs) and oxides of nitrogen (NOx) (Cart er, 2000). AE4-AQ, the fourth-generation CMAQ aerosol code with aqueous ph ase chemistry, when coupled with a gas phase mechanism, represents such phenomena as gas– aerosol/aqueous phase mass transfer, chemical trans formation of particulate species and their gas phase precursors, and the evo lution of the aerosol size

distribution.

4. Model Inputs and Setup:



### 4.1. Domain Structure :

Two modeling domains were used for this work. The first modeling domain, named CCAQS, covered the Central Valley and its surroundings with

63x63 lateral 12x12 km 2 grid cells for each vertical layer. The CCAQS dom ain

extends from the Pacific Ocean in the west to the M ojave Desert and western

Nevada in the east and runs from the northern Sacra mento Valley to the Tehachapi Mountains in the south. The second domai n, named SJV, is fully

nested within the CCAQS domain and covers the San J oaquin Valley with 80x89

lateral 4x4 km 2 grid cells for each vertical layer (Figure 1). Th e vertical structure for both domains is composed of 15 layers of varyin g thickness up to the top of the meteorological domain (100 mb). The finest res olution belongs to those layers closest to the surface and is determined lar gely by the vertical structure of the meteorological inputs. The surface layer is ap proximately 30 meters thick.3

Figure 1: The CCAQS modeling domain with the SJV modeling domain inset.



### 4.2. Initial and Boundary Conditions :

Boundary conditions for the CCAQS domain were taken from the global chemical transport Model for Ozone And Related chem ical Tracers (MOZART). Model boundary conditions for major species were ex tracted for the Central California modeling domain from MOZART results repr esentative of the year 2000. In addition to VOCs and inorganic gases, bou ndary conditions were extracted for ammonium, nitrate, sulfate, and organ ic and elemental carbon. Initial conditions were estimated as an average of the extracted boundary conditions for each species. The CCAQS domain prov ided the initial and boundary conditions for the nested SJV domain. All species reported in the three-dimensional output fields for the simulation over the CCAQS domain were included in the initial and boundary conditions for the SJV domain. While boundary conditions for the CCAQS domain were held constant for each month, boundary conditions for the SJV domain varied for e ach hour. The impact of initial conditions was minimized for each domain by simulating 8-day spin up periods prior to the simulation of each month of th e year.



### 4.3. Emissions :

A spatially, temporally, and chemically resolved em issions inventory of combined area, mobile, and point sources was genera ted using the California Emissions Forecasting System (CEFS) version 1.06 wi th offline adjustments. The inventory includes emissions estimates for gase ous and particulate species4

of anthropogenic and biogenic origin. Gridded hour ly emissions were developed for the CMAQ modeling domain for the years 2000, 20 05, and 2014. Quality assurance checks of domain emissions totals and spa tial distribution were performed at various steps in the emissions process ing in order to ensure that the CMAQ emissions input files were sound. Additio nal information on emissions is provided separately.



### 4.4. Meteorological Inputs :

The meteorological input fields to CMAQ were genera ted with the Meteorology-Chemistry Interface Processor (MCIP) ve rsion 3.0. MCIP serves as a link between meteorological models like MM5 or WR F with CMAQ, and generates model-ready meteorological inputs like th e wind and temperature fields necessary to drive the transport and chemist ry calculations in CMAQ. Inputs to MCIP were generated using the PSU/NCAR Me soscale Model (MM5) (version 3.6) (Grell et al, 1995). Planetary bound ary layer and radiation characteristics were calculated in MCIP, and the Mo dels-3 dry deposition routine (Pleim - with chlorine and mercury species) was cho sen to represent dry deposition.



### 4.5. MM5 Simulation :

MM5 is a limited area, terrain-following sigma coor dinate model on Lambert Conformal projection that was developed by Penn State and NCAR as a community mesoscale model. The model is based on n on-hydrostatic, fully compressible motions that allow users to study the atmospheric motions at small scales by explicitly treating the effects of convec tive motions on atmospheric circulations. The MM5 model has been improved over more than two decades by contributions from a broad scientific community.

MM5 was set up for a 14 month simulation (December 1999 – January 2001) with three nested grids using (70 x 70), (133 x 133), and (94 x 85) grid points in the (x, y) or (south-north, west-east) di rections with 36, 12, and 4 km horizontal resolution in each grid, respectively (F igure 2). The vertical structure of the domain was defined with 30 layers extending to 100 mb at the top of the domain. The first two coarse grids defined the atm ospheric initial and boundary conditions for the area at large scale, while the i nnermost grid with 4 km

horizontal resolution resolved the fine details of atmospheric motions within the SJV modeling domain. MM5 has several options to ca lculate the components of internal and external forces acting on a volume of air, such as those for radiation, convection, cloud microphysics, soil fluxes, and bo undary layer physics. While many sensitivity studies were conducted using vario us model options to find the best agreement with observations, an effort was als o made to use the same model options from one simulation to the next. The Grell (1995) cumulus parameterization scheme for coarse grids was used a long with the Blackadar boundary layer scheme for calculation of fluxes (Bl ackadar, 1979, Grell, 1995).5

The Dudhia simple ice scheme was used for the treat ment of excess moisture (Dudhia, 1989) and the Dudhia cloud radiation schem e (Dudhia, 1993) was used for radiational heating and cooling of the atmosphe re. The Blackadar multi-layer, force-restore method soil model (Blackadar, 1976) w as used for soil physics in all grids.

Initial and boundary conditions were prepared using the analyses of observations prepared by the National Center for En vironmental Prediction (NCEP) and archived at NCAR. The 14-month period ( December 1999 through January 2001) was first simulated using initial and boundary conditions (IC/BCs) with the analysis nudging option on the two coarse grids. Then, IC/BCs were prepared from the 12 km grid output for the initial ization of the 4 km grid. The three-dimensional (3-D) wind and temperature values simulated by the model were compared against surface observations obtained from four SJV stations (Fresno, Bakersfield, Arvin, and Parlier) to study the temporal and 3-D spatial structure of atmospheric motions as well as to eval uate the model performance within the SJV. Figure 2 shows the MM5 domains emp loyed to generate meteorological fields for the CMAQ simulation. A d etailed performance evaluation of the MM5 simulation is provided separa tely.

Figure 2. The location of the three nested grids adopted for numerical modeling of the SJV using

MM5.6

5. Model Performance Evaluation:

Methods for evaluating the performance of episodic ozone simulations are reasonably well developed and universally accepted. However, model evaluation for particulate matter (PM) simulations is still in an infant stage. Methods for performance evaluation of long-term simulations for both ozone and PM are neither reasonably developed nor universally accept ed.

Thus, we employ the following approach to evaluate the annual PM simulation for 2000. First, we will examine the sp atial patterns of PM 2.5 mass and its components on an annual and quarterly basis. S econd we compute fractional statistics (mean fractional error and mean fraction al bias) for each measurement location. We have also computed other traditional statistics such as mean normalized error and mean normalized bias, but we p resent only fractional statistics here. Third, we examine time-history pl ots for PM 2.5 mass and its
components.

5.1. Spatial Patterns for PM mass and components :

Figure 3 shows the annual average spatial patterns for PM mass, nitrate, sulfate, ammonium, organic carbon, elemental carbon and dust. As expected, PM mass and components are restricted to the valley floor. The maximum mass loading for this domain is near Visalia. The major contributors to the mass are dust and ammonium nitrate, and they also peak near Visalia. The second highest mass loading is in the Bakersfield area. T he sulfate concentrations are very low in the majority of the domain (<5 μg/m 3 ) except in the vicinity of Taft, Bakersfield, and Stockton. The concentrations of b oth organic and elemental carbon are relatively small on an annual basis comp ared to those of dust and

ammonium nitrate.

Figures 4-7 show the quarterly average for PM 2.5 mass and its major components. It is evident from these figures that nitrate formation follows the seasonal pattern expected based on seasonal tempera tures and relative humidity. However, the nitrate concentrations for the third quarter appear to be excessive. The (12x12) km 2 simulation that provided the initial and boundary conditions for this (4x4) km 2 simulation does not show this behavior. Thus, the

cause of this excessive nitrate is not clear at thi s time. It is important to note that the Federal Reference Monitor (FRM) does not retain significant amounts of nitrates for both the second and third quarters. T hus, the excess nitrate concentrations simulated here do not have a signifi cant impact on the future year design value determination. As expected, sulfate c oncentrations remain relatively low for all quarters. The organic and e lemental carbon concentrations are higher in the first and fourth quarters, perhap s due to enhanced wood burning which was not controlled in the 2000 base y ear for this simulation. Dust becomes more wide-spread in the third and fourth qu arters.7Figure 3: The annualaverage concentrationsfor PM2.5and itscomponents8Figure 4: The firstquarter averageconcentrations for PM2.5and its components9Figure 5: The secondquarter averageconcentrations for PM2.5and its components10Figure 6: The thirdquarter averageconcentrations for PM2.5and its components11Figure 7: The fourthquarter averageconcentrations for PM2.5and its components12

5.2. Statistics for Mean Fractional Error (MFE) and Bias (MFB) :

There are several statistical measures that have be en used in the evaluation of episodic ozone simulations. However, their utility in the evaluation of PM models is not universally agreed upon. The g eneral consensus among practitioners of photochemical air quality simulati ons is that novel concepts are

needed to evaluate annual PM simulations. To that end, Boylan and Russell (2006) proposed a two-tier approach for model perfo rmance evaluation of PM.

These two tiers are based on fractional statistics known as mean fractional error (MFE) and the mean fractional bias (MFB). The firs t tier is the model performance criteria. This is defined as “the leve l of accuracy that is considered to be acceptable for modeling applications.” The m odel performance criteria is

met when both MFE ≤ +75% and MFB ≤ ±60%. The second, and more restrictive, tier is the model performance goal. T his is defined as the “the level of accuracy that is considered to be close to the best a model can be expected to achieve.” The model performance goal is met when b oth MFE ≤ +50% and MFB

≤ ±30%. This method also allows for less stringent criteria/goals at lower

concentrations.

The mean fractional error and mean fractional bias are defined as



## ∑

= + −

= N

i o

i m

i o

i m

i

C C C C

N MFE

1 2

,

and



## ∑

= + −

= N

i o

i m

i o

i m

i C C C C

N MFE

1 2

,

where m

i C and o

i C are the modeled and observed concentration at a giv en

monitor. The index i

indicates an individual member of a set of N match ing pairs

of simulated and observed values at that monitor. These are centered statistics that give equal weights to simulated and measured v alues. Note here that we calculate these statistics, as shown in figure 8, f or each monitoring site and each specie.

Figure 8 shows both MFE and MFB for PM 2.5 and its major components. Each point in these graphs represents an individual measurement location. We have used approximately 30 locations within the (4x 4) km 2 domain that reported

24-hour PM 2.5 filter measurements. Chemical speciation informat ion was used directly when available for that location. For loc ations where speciation information was not available, information from a n earby speciation site was used. The general procedure used to assign un-spec iated monitors to speciation

monitors is described elsewhere.13Figure 8: The annualaverage mean fractionalerror (MFE) and bias(MFB) for PM2.5and itsmajor components. Thered line is the criteria andthe blue line the goal.14

In comparing the observations with simulations, we selected the best matched simulated value within nine grid cells of t he monitor. The results, not shown here, are very comparable when a nine-cell av erage concentration is used

instead.

It is clear from figure 8 that the 2000 base simula tion meets the performance criteria for both MFE and MFB for all s pecies except for dust. The

cause of overestimation of dust concentrations is n ot known at this time and further investigations of emissions inventories may be needed to better understand this discrepancy. It is interesting to note that the apparent overestimation of ammonium nitrate during the third quarter does not affect the model performance appreciably (despite the fact tha t the third quarter MFE and MFB, which are not shown here, do not meet model pe rformance criteria).

5.3. Time-History Plots:

Figures 9-10 show time-history plots for PM 2.5 and its major components at the Bakersfield and Fresno monitoring stations. These two stations measured high concentrations of particulate matter. It is c lear from these two figures that the air quality model is able to reproduce the seas onality of particulate matter production observed in the Central Valley of Califo rnia. The overestimation of ammonium nitrate in the third quarter is obvious fr om these figures. The cause

of this overestimation is not known at this time. But, we suspect that the simulated meteorology may be responsible for this o verestimation. It is also shown that dust is overestimated throughout the yea r. This may be due to overestimation of dust in the emission inventory. Organic carbon, elemental carbon, and sulfate show satisfactory model perform ance throughout the year.

6. Conclusion :

We have conducted photochemical particulate matter modeling for the San Joaquin Valley of central California for the ba se year of 2000. The model performance is satisfactory based on criteria propo sed by Boylan and Russell (2006). These criteria were based on an analysis o f various particulate-matter modeling conducted in several areas of the United S tates. Our simulation

overestimates ammonium nitrate concentrations for t he third quarter. However, this has no significant impact on the future-year d esign value determination since the Federal Reference Monitor does not report signi ficant amounts of ammonium nitrate for the third quarter. The concentration o f dust is also consistently overestimated for the entire year. This may not po se a significant problem since

the model results are used in a relative sense and the dust component in the measurements is not very significant. Organic carb on, elemental carbon, and sulfate show satisfactory model performance.15Figure 9: The time-history plotsof PM2.5and its majorcomponents at the Bakersfieldmonitoring station.16Figure 10: The time-history plotsof PM2.5and its majorcomponents at the Fresnomonitoring station.17

References:

Blackadar, A. K. (1976) Modeling the Nocturnal Boun dary Layer. Preprints of Third Symposium on Atmospheric Turbulence and Air Q uality, Raleigh, NC. 19-22, October 1976, Amer. Meteor. Soc ., Boston, 46-49.

Blackadar, A. K. (1979) Advances in Environmental S cience and Engineering, 1. No. 1. Pfaflin and Ziegler, Eds. Gordon and Breach Publishers, 50-85.

Boylan, J.W. and A.G. Russell (2006) PM and light extinction model performance metrics, goals, and criteria for three- dimensional air quality

models. Atmospheric Environment , 40:4946-4959. doi:10.1016/j.atmosenv.2005.09.087

Carter, W. (2000) Documentation of the SAPRC-99 ch emical mechanism for VOC reactivity assessment. Final Report to Califo rnia Air Resources Board. Contract 92-329 and Contract 95-308, http://pah.cert.ucr.edu/ftp/pub/carter/pubs/s99txt. pdf

Chow, J.C., Chen, L.W.A., Watson, J.G., Lowenthal, D.G., Magliano, K.A., Turkiewicz, K., and D.E. Lehrman (2006) PM 2.5 chemical composition and spatiotemporal variability during the California Re gional PM10/PM2.5 Air Quality Study (CRPAQS). J. Geophys. Res ., 111, D10S04,

doi:10.1029/2005JD006457.

Chow, J.C., Watson, J.G., Lowenthal, D.H., Magliano , K. (2005) Loss of PM 2.5 Nitrate from Filter Samples in Central California. J. Air & Waste Manage.

Assoc . 55:1158-1168.

Community Modeling and Analysis System (2007) CMAQ v4.6 Operational

Guidance Document. http://www.cmaqmodel.org/op_guidance_4.6/manual.pdf

Dudhia, J. (1989) Numerical Study of Convection Obs erved During the Winter Monsoon Experiment Using a Two-Dimensional Model. J. Atmos. Sci ., 46,

3077-3107.

Dudhia, J. (1993) Radiation Studies with a High-Res olution Mesoscale Model. Proceedings of the Third Atmospheric Radiation Meas urement (ARM) Science Team Meeting. March 1-4, 1993, Norman, OK. pp. 363-366.

Grell, G. A., J. Dudhia and D. R. Stauffer. (1995) A Description of the FifthGeneration Penn State/NCAR Mesoscale Model (MM5). NCAR Technical

Note , NCAR/TN-398+STR.18

Hering, S.V. and Cass, G.R. (1999) The Magnitude of Bias in the Measurement

of PM 2.5 Arising from Volatilization of Particulate Nitrate from Teflon Filters. J. Air & Waste Manage. Assoc . 49:725-733.

Liang, J. and A. Kaduwela, 2005: Microdevelopment o f CMAQ for California Regional Particulate-Matter Air Quality Study. Pro ceedings of the 4th Annual CMAS Models-3 Users’ Conference; September 2 6-28, 2005, Chapel-Hill, NC.

U.S. Environmental Protection Agency (1999) Science Algorithms of the EPA Models-3 Community Multiscale Air Quality (CMAQ) Mo deling System

. EPA/600/R-99/030. http://www.epa.gov/asmdnerl/CMAQ/CMAQscienceDoc.htm l

U.S. Environmental Protection Agency. (2007) Guid ance on the Use of Models and Other Analyses for Demonstrating Attainment of Air Quality Goals for Ozone, PM 2.5 , and Regional Haze. EPA-454/B-07-002