APPENDIX B

Air Quality Model Performance Evaluation: Regional Model Evaluation across the VISTAS RegionB-2 B.1 OVERVIEW

The discussion of model performance evaluation in this Appendix focuses on the evaluation of the 2002 Base G2 Actual base case modeling resu lts from the Community Multiscale Air Quality (CMAQ) modeling system Versi on 4.5 with SOAmods enhancement across the VISTAS states and VISTAS 12 km grid. The CMAQ results are compared with observational data from the

I nteragency M onitoring of PRO tected V isual E nvironments (IMPROVE), S peciated T rends

N etwork (STN), C lean A ir S tatus T rends Net work (CASTNet), F ederal R eference M ethod (FRM) PM 2.5 mass, N ational A cid D eposition P rogram (NADP) and S outh E ast A erosol

R esearch and CH aracterization (SEARCH) study monitori ng networks. The evaluation will primarily focus on the air quality model’s perfor mance with respect to individual components of fine particulate matter (PM 2.5 ), as good model performance of the PM component species will dictate good model performance of total or reconstitu ted fine particulate matter (i.e., visibility). The FRM networks only collects total PM 2.5 mass and these monitors are almost exclusively

located in urban areas so are not as relevant for judging how well the model is predicting the components of light extinction at the more rural Class I areas. However, model performance of the total fine particulate matter and the resulting to tal light extinction will also be provided as a means to discuss the overall model performance.

B.2 CMAQ Evaluation Methodology

EPA’s integrated ozone, PM 2.5 and regional haze modeling guida nce calls for a comprehensive, multi-layered approach to model performance testi ng, consisting of the four major components: operational, diagnostic, mechanistic (or sc ientific) and probabilistic (EPA, 2007a).

B.2.1 Evaluation Approach

The CMAQ model performance evaluation effort fo cused on the first two components, namely:

• Operational Evaluation : Tests the ability of the model to estimate PM concentrations (both fine and coarse) and the components at PM 10 and PM 2.5 including the quantities used to characterize visibility (i.e., sulfat e, nitrate, ammonium, organic carbon, elemental carbon, other PM 2.5 , and coarse matter (PM 2.5-10 ). This evaluation examines whether the measurements are properly represented by th e model predictions but does not necessarily ensure that the model is getting “the right answer for the right reason”; and

• Diagnostic Evaluation: Tests the ability of the model to predict visibility and extinction, PM chemical composition including PM precursors (e.g., SOx, NOx, and NH 3 ) and associated oxidants (e.g., ozone and nitric acid ); PM size distribution; temporal variation; spatial variation; mass fluxes; and components of light extinction (i .e., scattering and absorption).B-3 The diagnostic evaluation also includes the perf ormance of diagnostic tests to better understand model performance and identify potential flaws in the modeling system that can be corrected. The diagnostic evaluation may al so includes the use of “probing tools” to understand why the model obtains a given prediction; probing tools in clude Process Analysis (PA), decoupled direct method (DDM) and source apportionment (SA).

In this final model performa nce evaluation for the VISTAS 2002 Actual Base G2 CMAQ 36/12 km base case simulation, the operational evaluati on has been given the gr eatest attention since this is the primarily thrust of EPA’s modeling guidance. However, we have also examined certain diagnostic features dealing with the model’s ability to simulate sub-regional and monthly/diurnal gas phase and aero sol concentration distributions. In the course of the VISTAS study numerous diagnostic sensitiv ity tests were performed to investigate and improve model performance. Key diagnostic tests performed ar e discussed and the results for the rest are available on the VISTAS modeling website: http://pah.cert.ucr.edu/vistas/vistas2/.

B.2.2 Particulate Matter and Component Species Regional haze is calculated by estimating light scattering and absorption by components of

PM 2.5 . Regional haze is measured by an extinction coefficient (b ext ), which represents the light attenuation resulting from scatte ring and absorption of light from ambient particulate matter plus scattering of light due to gas molecules in the air (i.e., Raylei gh scattering).

Some scattering does occur from air molecules; ho wever, the bulk of light extinction is caused by the presence of suspended particles, partic ularly fine particle s (aerodynamic mass <2.5 μ m). Fine particulate matter can be composed of varying amounts of several different species, including:

• Sulfate (SO4)

• Nitrate (NO3)

• Ammonium (NH4)

• Organic Matter Carbon (OMC)

• Elemental Carbon (EC)

• Coarse particulate matter or Coarse Mass (i.e., PM 10 - PM 2.5 or CM).

• Soil (also known as crustal material, fine soil , major metal oxides, inorganic particulates, or other PM)

• Sea Salt (NaCl)

B.2.3 Ambient Air Quality Data for VISTAS Model Evaluation

A ground-level model evaluation database for 2002 was compiled by the modeling team using several routine and research-grade databases. The focus of the regional evaluation of the CMAQ model for use in simulating visibility in the sout heastern is on the PM components that can cause visibility impairment. The prim ary monitoring networks available to evaluate this component of the CMAQ are: (a) Interagency Monitoring of Protected Visual Environments (IMPROVE); (b)B-4 Clean Air Status and Trends Network (CASTNET); (c) Southeastern Aerosol Research and Characterization (SEARCH); (d) EPA Federal Reference Method PM 2.5 and PM 10 Mass Networks (EPA-FRM); (e) EPA Speci ation Trends Network (STN) of PM 2.5 species; and (f) National Acid Deposition Network (NADP). Thes e PM monitoring networks may also provide ozone and other gas phase precursor s and product species, and visi bility measurements at some sites. There are also gas-phase criteria polluta nt measures from the EPA’s Air Quality System (AQS) that are available for use. However, thes e sites are urban-oriented and not as useful for the regional evaluation of the CMAQ modeli ng system. The study team has performed a detailed evaluation of the CMAQ 2002 base case si mulation using the AQS that is available on the modeling website: http://pah.cert.ucr.edu/vistas/vistas2/mpe2.shtml . Table B-1 and Figure B-1 summarizes the species collected and loca tions of the monitoring sites for the IMPROVE, STN, CASTNet, NADP and SEARCH monitori ng networks use in the VISTAS model

evaluation.

Table B-1. Ambient monitoring data available in the VISTAS region during 2002.

Monitoring

Network Chemical Species Measured Sampling Frequency;

Duration

IMPROVE Speciated PM 2.5 and PM 10 1 in 3 days; 24 hr

CASTNET Speciated PM 2.5 , Ozone Hourly, Weekly; 1 hr, Week

SEARCH 24-hr PM 25 (FRM Mass, OC, BC, SO 4 , NO 3 , NH 4 , Elem.); 24-hr PM coarse (SO 4 , NO 3 , NH 4 , elements); Hourly PM 2.5 (Mass, SO 4 , NO 3 , NH 4 , EC, TC); and Hourly gases (O 3 , NO, NO 2 , NO y , HNO 3 , SO 2 , CO) Daily, Hourly;

NADP WSO 4 , WNO 3 , WNH 4 Weekly

EPA-FRM Only total fine mass (PM 2.5 ) 1 in 3 days; 24 hr

EPA-STN Speciated PM 2.5 Varies; Varies
AIRS/AQS CO, NO, NO 2 , NO

x , O 3 Hourly; HourlyB-5 

### 250 500 750 1000 1250 1500 1750 2000 -1500 -1250 -1000

-750 -500 -250 0 250 500

IMPROVE

CASTNet

SEARCH

STN

NADP

AQS

Figure B-1. IMPROVE, STN, CASTNet, NADP and SEARCH monitoring sites located in the VISTAS region and the VISTAS 12 km grid.

B.3 Model Performance Statistics

To quantify model performance, several statistical measures we re calculated and evaluated for all the monitors within the VISTAS region and within each VISTAS state. The statistical

measures selected were based on the reco mmendations outlined in section 18.4.1 of the

USEPA’s Guidance On The Use Of Models And Ot her Analyses for Demonstrating Attainment of Air Quality Goals for Ozone, PM2.5, and Regional Haze (EPA, 2007a). In 2004, VISTAS established model performance goals and criteria for components of fine particle mass based on previous model performance for ozone and fine particles. EPA modeling guidance for fine particulate matter at the time noted that PM models might not be able to achieve the same level of performance as ozone models. VISTAS ’ evaluation considered several statistical performance measures and displays.B-6 B.4 Modeling Performance Across the VISTAS Region

In the following discussions we use selected mont hly scatter plots, time series plots and model performance statistical measures from the U CR Analysis Tool appli cation to the 2002 CMAQ Base G2 Actual base case simulation in an opera tional evaluation of the model for PM species. We focus on the six main components of PM that are used to project visibility.

B.4.1 Sulfate (SO4) Monthly Model P erformance in the VISTAS Region

B.4.1.1 SO4 in January 2002

Figure B-2a displays scatter pl ots of predicted and observed SO4 concentrations or wet depositions for January 2002, sites in the VI STAS region using observations from the IMPROVE, STN, CASTNet and NADP monitoring networks and the CMAQ Base G2 Actual base case 36 km (red) and 12 km (blue) m odeling results. The IMPROVE and STN SO4 concentrations are 24-hour averages whereas the CASTNet SO4 concen trations and NADP SO4 wet deposition are weekly averages. The January SO4 performance at the IMPROVE and STN networks in the VISTAS region is quite good w ith low fractional bias (-14%) and gross error (34%-38%) that almost achieves the most string ent model performance ev aluation goals (Table C-1XXX). There is a larger SO4 concentration underestima tion bias across the CASTNet network (-21% to -23%) with less scatter so there is lower gross error (30% to 32%). The lower scatter and error is likely due to the longer sampling period for CASTNet (weekly) versus IMPROVE/STN networks (24-hour) that reduces the variability in both the predicted and observed SO4 concentrations. Surprisingly the model exhibits low SO4 wet deposition bias across the VISTAS region (-7% to -8%) but lots of scatter so pr oduces a high bias than for the concentrations (67% to 70%). This is surprising because obtaining good SO4 model performance requires the simultaneous occurrence of correct depiction of SO4 concentrations and precipitation model predictions. As noted by Olerud (2003c,d), the MM5 simulation does a respectable job in reproducing pr ecipitation during synoptic rain events as o ccur in the winter, however it exhibited less skill in predicting prec ipitation during the summer convective events.

Figure B-2b present quantile-qua ntile (Q-Q) plots for January 2002 SO4 concentration and wet deposition model performance. Q-Q plots co mpare the frequency distribution between the predicted and observed SO4 concentrations/d epositions. Across the IMPROVE network the model slightly underestimates the frequency dist ribution of the lower SO4 below approximately 5 ug/m3 and overpredictions the values above this value. Across the more urban STN network the model exhibits a slight underprediction of the observed frequency distribution across the VISTAS region during January 2002. Across th e mixed urban/non-urban SEARCH network the model reproduces the observed frequency distri bution of SO4 concentra tions quite well, except for overestimating the highest values above ap proximately 4 ug/m3. Finally, the observed wet SO4 deposition in the VISTAS region and Janu ary 2002 is matched exactly by the model except for the very highest events that are underpre dicted. It is unclear whether the model overestimation of the highest observed concentrations and underestimation of the highest observed wet deposition events are connected.B-7 Figure B-2a. Scatter plots of predicted and observed sulfate (SO4) concentrations for January 2002 and sites in the VISTAS region using IMPROVE (top left), STN (top right), CASTNet (bottom left) and NADP monitoring networks and the CMAQ 2002 36 and 12 km Base G2 Actual base case simulation.B-8 Figure B-2b. Q-Q plots of predicted and observed sulfate (SO4) concentrations or depositions for January 2002 and sites in the VISTAS region using IMPROVE (top left), CASTNet (top right), SEARCH (bottom left) and NADP monitoring networks and the CMAQ 2002 12 km Base G2 Actual

base case simulationB-9 B.4.1.2 SO4 in April 2002

In April CMAQ still exhibits reasonable good SO4 model performance although with an

underestimation bias of -20% to -22% across th e IMPROVE and STN networks. The scatter in the SO4 performance across these two networks is very tight with low fractions error values of 30% to 35% (Figure B-3a). Across the CASTNe t network in April 2002, both the 36 km and 12 km versions of the model exhibits a higher undere stimations bias (-25% to -26%), but low error (27% to 28%), which is due to the less variabi lity in the weekly samples. Wet SO4 deposition performance in April across the VISTAS region is much worse than seen for January with an overprediction bias of 48% (12 km) and 50% ( 36 km) and fractions errors of approximately 100%. The Q-Q plots of SO4 con centrations and deposition mode l performance in April shown in Figure C-3b indicates that the model does a reasonable job in re producing the observed frequency distribution, although with a slight underprediction bias except for the highest values that are either reproduced fairly well (IMPR OVE and STN) or underpredicted (SEARCH and NADP)..B-10 Figure B-3a. Scatter plots of predicted and observed sulfate (SO4) concentrations for April 2002 and sites in the VISTAS region using IMPROVE (top left), STN (top right), CASTNet (bottom left) and NADP monitoring networks and the CMAQ 2002 36 and 12 km Base G2 Actual base case

simulation.B-11 Figure B-3b. Q-Q plots of predicted and observed sulfate (SO4) concentrations for April 2002 and sites in the VISTAS region using IMPROVE (top left), CASTNet (top right), SEARCH (bottom left) and NADP monitoring networks and the CMAQ 2002 36 and 12 km Base G2 Actual base case

simulation.B-12 B.4.1.3 SO4 in July 2002

The SO4 model performance in July 2002 across the VISTAS regi on is also fairly good. The CMAQ Base G2 12 km model simulation produces slight lower fractions bias than the 36 km model for the IMPROVE (-18% vs. -20%), ST N (-13% vs. -16%) and CASTNet (-17% vs. - 23%) networks (Figure B-4a). The fractional error for July SO4 concentrations across these same networks ranges from 27% to 48% with the 12 km model exhibiting slight lower error than the 36 km model. The July SO4 wet deposition e xhibits an overestimation bias of 39% to 45%

and fractions errors of ~90%. The overestima tion of the July 2002 SO4 wet deposition in the VISTAS region is consistent with the MM5 me teorological model evaluation that found that MM5 overstated the amount of summer pr ecipitation due to convective activity.

The model reproduces the observed frequency di stribution of SO4 con centrations across the

IMPROVE monitors in the VISTAS region during July 2002 very well (Figure B-4b). The Q-Q plots of July SO4 performance for the STN and CASTNet monitoring network across the VISTAS region also exhibits fairly good mode l performance, although with an underestimation

bias for STN and overestimation bias for SEA RCH at the high end. The observed frequency distribution of SO4 wet deposi tion across the VISTAS region is greatly overestimated by the model. This summer wet SO4 overestimation bias undoubtedly contributes to the underprediction of the SO4 concentrations and extinction for the Worst 20 percent days as discussed in Appendix C.B-13 Figure B-4a. Scatter plots of predicted and observed sulfate (SO4) concentrations for July 2002 and sites in the VISTAS region using IMPROVE (top left), STN (top right), CASTNet (bottom left) and NADP monitoring networks and the CMAQ 2002 36 and 12 km Base G2 Actual base case

simulation.B-14 Figure B-4b. Q-Q plots of predicted and observed sulfate (SO4) concentrations for July 2002 and sites in the VISTAS region using IMPROVE (top left), CASTNet (top right), SEARCH (bottom left) and NADP monitoring networks and the CMAQ 2002 36 and 12 km Base G2 Actual base case

simulation.B-15 B.4.1.4 SO4 in October 2002

In October 2002, CMAQ is doing a better job of reproducing the observed SO4 concentrations

with much lower fractional bias values acro ss the IMPROVE (0% to 4%) and STN (-6% to - 15%) and fractional errors of 33% to 48% (Fi gure C-5a). Good SO4 concentrations model performance for October 2002 and the VISTAS regi on is also seen across the CASTNet monitors with zero bias and fractions error of approximate ly 30% (i.e., meeting the most stringent ozone model performance goals). Alt hough the SO4 wet deposition model performance bias is fairly low (~20%) there is lots of sca tter producing large errors (~85%).

The Q-Q plot comparisons (Figure C-5b) indi cate that the model overstates both the SO4 concentrations and deposition distribution in Oc tober 2002 with the overstatement increasing at the higher values.B-16 Figure B-5a. Scatter plots of predicted and observed sulfate (SO4) concentrations for October 2002 and sites in the VISTAS region using IMPROVE (top left), STN (top right), CASTNet (bottom left) and NADP monitoring networks and the CMAQ 2002 36 and 12 km Base G2 Actual base case

simulation.B-17 Figure B-5b. Q-Q plots of predicted and observed sulfate (SO4) concentrations for October 2002 and sites in the VISTAS region using IMPROVE (top left), CASTNet (top right), SEARCH (bottom left) and NADP monitoring networks and the CMAQ 2002 36 and 12 km Base G2 Actual base case

simulation.B-18 B.4.1.5 SO4 Monthly Bias and Error

Figure B-6a compares the monthly SO4 concen tration fractional bias and error across the VISTAS region for the IMPROVE, STN, CAST Net, SEARCH Daily and SEARCH Hourly monitoring networks. Note that these monthly model performance statistics were calculated across the VISTAS 12 km grid versus just th e VISTAS states for used in the monthly performance evaluation in Figures B-2 thr ough B-5. The underprediction bias in SO4 concentrations is clearly evid ent across most of the year a nd monitoring networks with the lowest SO4 bias in October. However, this unde rprediction bias is not severe usually not going lower than -20% and never going lower than -25%. The exception to this is the SEARCH Hourly network that exhibits a positive ove rprediction SO4 bas for several months.

The SO4 monthly fractional error values are usually under 40% a nd, with one exception, always under 50% (Figure B-6a). The exception is th e SEARCH Hourly network with error values ranging from 50% to 70%.

Figure B-6b displays Bugle Plots of monthly SO4 fractional bias a nd error. Bugle Plots compare the monthly bias and error values with a perf ormance goal and performa nce criteria and accounts for the concept that better performance is needed (i.e., more stringent model performance goals and criteria) for species with hi gher concentrations/ext inctions and that performance goals and criteria can be relaxed as the species concentra tions become low and are not longer an important component of the PM or extinction budget (see Ch apter 3 and Appendix C for more discussion). The Bugle Plots in Figure B-6b confirm the previ ous analysis that SO4 model performance tends to be good across all months and, with one ex ception, always achieving the model performance goal for both bias and error. The one excepti on is for fractional error across the SEARCH Hourly network that is above the performance goal, but still achieves the performance criteria (Figure B-6b, bottom).B-19 Figure B-6a. Monthly SO4 fractional bias (top) and fractional gross error (bottom) statistical measures for CASTNet, IMPROVE, STN and SEA RCH (daily and hourly) monitoring sites in the VISTAS 12 km grid.B-20 Figure B-6b . Bugle Plots of monthly fractional bias (top) and gross error (bottom) for SO4 in the VISTAS 12 km grid and comparison to model performance goals (blue) and criteria (yellow).B-21 B.4.2 Nitrate (NO3) Monthly Model P erformance in the VISTAS Region

The regional model performance evaluation for nitr ate (NO3) follows the same format as given

above for SO4.

B.4.2.1 NO3 in January 2002

The January NO3 performance (Figure B-7a) is not as good as seen for SO4. The January NO3

fractional bias is in the -20% to -25% and ve ry similar for the 36 km and 12 km CMAQ Base G2

simulations for the STN and CASTNet monitori ng networks across the VISTAS region, with fractional errors that range from 30% to 37%. However, across the IMPROVE network in the VISTAS region the CMAQ 12 km run exhibits a higher underprediction bi as (-39%) than the 36 km run (-23%), although it has lower error (90%) than the 36 km run (96%). The January NO3 wet deposition performance for the 36 km and 12 km simulations are very similar with an approximate -7% bias and 36% gross error.

The Q-Q plots comparing the predicted and obs erved January NO3 fre quency distributions indicate that although th ere is a net underestimation bias in NO3 when averaged across the monitoring networks as shown IN Figure B-7a, the model overestimates the highest observed

NO3 concentrations. This overest imation of the highest observed NO3 concentrations in January occurs for the IMPROVE, CASTNet and NADP netw orks, but not the more urban-oriented STN

network.B-22 Figure B-7a. Scatter plots of predicted and observed nitrate (NO3) concentrations for January 2002 and sites in the VISTAS region using IMPROVE (top left), STN (top right), CASTNet (bottom left) and NADP monitoring networks and the CMAQ 2002 36 and 12 km Base G2 Actual base case

simulation.B-23 Figure B-7b. Q-Q plots of predicted and observed nitrate (NO3) concentrations for January 2002 and sites in the VISTAS region using IMPROVE (top left), CASTNet (top right), SEARCH (bottom left) and NADP monitoring networks and the CMAQ 2002 36 and 12 km Base G2 Actual base case

simulation.B-24 B.4.2.2 NO3 in April 2002

The monthly NO3 performance degrades in April 2002 with the model exhibiting an underprediction bias of approximately -60% and -160% across the IMPROVE and STN monitoring in the VISTAS region and errors of over 100% and 160%, respectively. Better April NO3 performance is seen across the CASTNet mon itoring network with bias values of -9% and - 18% for the CMAQ 36 km and 12 km base case simu lations and errors of approximately 80%. April NO3 wet deposition is overestimated by 36% to 31% with errors of approximately 95%.

Despite the -60% underestimation bias for Ap ril monthly NO3 across the IMPEOVE network, the Q-Q plot indicates an overestimation bias for the highest NO3 values (Figure B-8b). This is because most of the modeled NO3 values are near zero so show up as an underprediction bias for the very lowest observed NO3 values in the Q-Q bias, but there are some modeled values higher than observed that are accentuated in the Q-Q plot.B-25 Figure B-8a. Scatter plots of predicted and observed nitrate (NO3) concentrations for April 2002 and sites in the VISTAS region using IMPROVE (top left), STN (top right), CASTNet (bottom left) and NADP monitoring networks and the CMAQ 2002 36 and 12 km Base G2 Actual base case

simulation.B-26 Figure B-8b. Q-Q plots of predicted and observed nitrate (NO3) concentrations for April 2002 and sites in the VISTAS region using IMPROVE (top left), CASTNet (top right), SEARCH (bottom left) and NADP monitoring networks and the CMAQ 2002 36 and 12 km Base G2 Actual base case

simulation.B-27 B.4.2.3 NO3 in July 2002

NO3 monthly model performance in July is char acterized by a large underprediction bias with

fractional bias values of - 150% to -160% across the IMPROV E and STN networks and of approximately -100% across the CASTNet monitoring network (Figure B-8a), This is due to the model most of the time predicting near zero NO3 concentrations in July as the hot temperatures in the southeastern U.S. will volatilize the partic ulate NO3 in the model to gaseous nitric acid. Even so, there are occurrences of overestimated model NO3 concentrations in July and for the IMPROVE network the highest modeled NO3 values exceed the measured values as shown in the Q-Q plot in Figure B-8b.

Figure B-9a. Scatter plots of predicted and observed nitrate (NO3) concentrations for July 2002 and sites in the VISTAS region using IMPROVE (top left), STN (top right), CASTNet (bottom left) and NADP monitoring networks and the CMAQ 2002 36 and 12 km Base G2 Actual base case

simulation.B-28 Figure B-9b. Q-Q plots of predicted and observed nitrate (NO3) concentrations for July 2002 and sites in the VISTAS region using IMPROVE (top left), CASTNet (top right), SEARCH (bottom left) and NADP monitoring networks and the CMAQ 2002 36 and 12 km Base G2 Actual base case

simulation.B-29 B.4.2.4 NO3 in October 2002

The NO3 scatter plots for October 2002 suggest that the model is overestimating the observed NO3 but there is a net negative fraction bias of approximately -30% to 0-40% for the IMPROVE and STN networks and the VISTAS 12 km grid (Fi gure B-10a). This negative bias is due to the model predicting very low near zero NO3 concen trations when the observed NO3 is less than approximately a 0.5 ug/m3. However, when the observed value is greater than approximately 0.5 ug/m3 at the IMPROVE and STN networks it a ppears that the model mostly overstates the

observed value. Across the CASTNet network a positive bias is seen (32% to 42%) which may be due in part to volatilization of the particul ate NO3 in the weekly sample. The magnitude of the NO3 wet deposition observed values is reprodu ced reasonable well with low bias (6% to 8%)

but with lots of scatter around the 1:1 line producing errors of approximately 85%.

The overprediction of the highest observed NO3 concentrations and depo sition in October is clearly evident in the Q-Q plots (Figure B-10b).B-30 Figure B-10a. Scatter plots of predicted and observed nitrate (NO3) concentrations for October 2002 and sites in the VISTAS region using IMPROVE (top left), STN (top right), CASTNet (bottom left) and NADP monitoring networks and the CMAQ 2002 36 and 12 km Base G2 Actual base case

simulation.B-31 Figure B-10b. Q-Q plots of predicted and observed nitrate (NO3) concentrations for October 2002 and sites in the VISTAS region using IMPROVE (top left), CASTNet (top right), SEARCH (bottom left) and NADP monitoring networks and the CMAQ 2002 36 and 12 km Base G2 Actual base case

simulation.B-32 B.4.2.5 NO3 Monthly Bias and Error

The monthly fractional bias time series plot clearly shows the seasonal dependence of this performance measure with severe summer underp rediction bias of - 100% to -150% in the summer and bias values generally within ±50% in the winter (Figure B-11a). The time series of

fraction error values also exhibi t a seasonal dependence with an 80% error in the winter peaking

to a 160% error in the summer with a bell-like distribution. At fi rst glance it appears that the summer underprediction bias would a dversely affect the regional haze modeling as that is when a vast majority of the worst 20 percent days o ccur at Class I areas in the VISTAS region. However, as shown in the Bugle Plots in Fi gure B-11b, these large summer underprediction bias

values occur when NO3 concentrations are lo w (< 1.0 ugm3) and in fact when the average observed NO3 concentration exceeds 1 ug/m3 the model mostly achieves the model performance goal and almost always achieves the model performan ce criteria for bias (Figure B-11a top). The model performance goal and criteria is achieved less often for fractional error and NO3 (Figure B-11b, bottom), but the goal is always achieved across the CASTNet and usually achieved across the IMPROVE monitoring networks.B-33 Figure B-11a. Monthly NO3 fractional bias (top) and fractional gross error (bottom) statistical measures for CASTNet, IMPROVE, STN and SEA RCH daily and hourly monitoring sites in the VISTAS region.B-34 Figure B-11b . Bugle Plots of monthly fractional bias (top) and gross error (bottom) for NO3 in the VISTAS region and comparison to model performance goals (blue) and criteria (yellow).B-35 B.4.3 Organic Matter Carbon (O MC) Monthly Model Performanc e in the VISTAS Region

The CMAQ model simulates organic matter carbo n (PMC) that includes both primary and secondary organic aerosol (OA) that incl udes carbon, oxygen and other compounds. However, the monitoring networks measure just the orga nic carbon (OC) portion of the OMC. Thus, the

OC measurements need to be converted to OM C for the model evaluation. OMC/OC ratios tend to vary between 1.2 to 2.2 depending on the organic compounds and the amount of photochemistry the compounds have undergone, with newer OMC tending to have lower OMC/OC ratios than aged OMC. The origin al IMPROVE equation uses a 1.4 OMC/OC ratio, whereas the new IMPROVE equation uses a 1.8 OM C/OC ratio. In this model performance

evaluation we used a 1.4 OMC/OC ratio to conver t the measured OC to OMC. There is also

concern with the STN OC measurements because they are not black corrected. In fact the differences in OC sampling met hodologies results in differences in measurements that exceeds our performance goals (Solomon et al., 1994). Thus, these uncertainties in the OMC

measurements need to be accounted fo r in the model performance evaluation.

B.4.3.1 OMC in January 2002

In January the CMAQ 12 km and 36 km Base G2 2002 Actual base case simulation exhibit a 11% and 18% overprediction bias across the ru ral IMPROVE monitors and a -40% and -46% underprediction bias across the urban STN monitors in the VI STAS region with errors of approximately 50% (Figure B-12a). For the mi xed urban/rural SEARCH monitoring network an OMC underprediction bias of -24% and -16% occurs using the 12 km and 36 km modeling

results with errors of 40% to 50%. The SEARCH monitors also collected hourly measurements of total carbon mass (TCM=EC+OC) which wa s overestimated by the 12 km and 36 km modeling results by 20% and 12% with errors of approximately 70%. The Q-Q plots indicate the model is reproducing the low end of the observed OMC frequency distributions fairly well across

the IMPROVE and STN networks but overestimat es the high end, whereas the high end is

underestimated across the SEARCH network.B-36 Figure B-12a. Scatter plots of predicted and observed organic matter carbon (OMC) concentrations for January 2002 and sites in the VISTAS region using IMPROVE (top left), STN (top right), and SEARCH Daily (bottom left) monitoring networks and for total carbon mass (TCM) for the SEARCH Hourly network (bottom right) using the CMAQ 2002 36 and 12 km Base G2 Actual base case

simulation.B-37 Figure B-12b. Q-Q plots for January 2002 and the VISTAS 12 km grid of predicted and observed organic matter carbon (OMC) and elemental carbon (EC) across the IMPROVE network (top left and right) and SEARCH Daily OCM (bottom left) and SEARCH Hourly total carbon mass (TCM) (bottom right) using the CMAQ 2002 12 km Base G2 Actual base case simulation.B-38 B.4.3.2 OMC in April 2002

In April 2002, the CMAQ Base B2 modeling results has a near zero bias across the IMPROVE monitors with a 39% error, and a -59% (12 km ) and -54% (36 km) underprediction bias across the STNB monitors. There is also very low OMC bias across the SEARCH network in April, although the errors are higher (~60%). The SE ACH hourly TCM also shows low OMC bias in April (11%-16%) and errors of 60%. The April OMC Q-Q plots are similar to January with the model overstating the observed OMC frequency distribution at the high end for IMPRVE and STN, but understating it for SEARCH.

Figure B-13a. Scatter plots of predicted and observed organic matter carbon (OMC) concentrations for April 2002 and sites in the VISTAS region using IMPROVE (top left), STN (top right), and SEARCH Daily (bottom left) monitoring networks and for total carbon mass (TCM) for the SEARCH Hourly network (bottom right) usi ng the CMAQ 2002 36 and 12 km Base G2 Actual

base case simulation.B-39 Figure B-13b. Q-Q plots for January April and the VISTAS 12 km grid of predicted and observed organic matter carbon (OMC) and elemental carbon (EC) across the IMPROVE network (top left and right) and SEARCH Daily OCM (bottom left) and SEARCH Hourly total carbon mass (TCM) (bottom right) using the CMAQ 2002 12 km Base G2 Actual base case simulation.B-40 B.4.3.3 OMC in July 2002

In July the model underestimates OMC across the IMPOREVE (-13%), STN (-54% to -59%) and SEARCH networks (-34% to -40%) monitori ng networks in the VISTAS states (Figure B14a). The scatter plots show that the model is exhibiting some skill in predicting the observed OMC, albeit with an underpredic tion bias. Unlike January and Apr il, the observed high end of

the OMC measurements is underestimated by the model across all three networks.

Figure B-14a. Scatter plots of predicted and observed organic matter carbon (OMC) concentrations for July 2002 and sites in the VISTAS region using IMPROVE (top left), STN (top right), and SEARCH Daily (bottom left) monitoring networks and for total carbon mass (TCM) for the SEARCH Hourly network (bottom right) using the CMAQ 2002 36 and 12 km Base G2 Actual base case

simulation.B-41 Figure B-14b. Q-Q plots for July 2002 and the VISTAS 12 km grid of predicted and observed organic matter carbon (OMC) and elemental carbon (EC) across the IMPROVE network (top left and right) and SEARCH Daily OCM (bottom left) and SEARCH Hourly total carbon mass (TCM) (bottom right) using the CMAQ 2002 12 km Base G2 Actual base case simulation.B-42 B.4.3.4 OMC in October 2002

In October the model exhibits near zero bias across the IMPROVE, large underprediction bias across STN and smaller underprediction bias acro ss the SEARCH networks (Figure B-15a). The highest observed OMC frequency distributions are overstated at the IMPROVE and STN but matched across the SEARCH monitors (Figure B-15b)

Figure B-15a. Scatter plots of predicted and observed organic matter carbon (OMC) concentrations for October 2002 and sites in the VISTAS region using IMPROVE (top left), STN (top right), and SEARCH Daily (bottom left) monitoring networks and for total carbon mass (TCM) for the SEARCH Hourly network (bottom right) using the CMAQ 2002 36 and 12 km Base G2 Actual base case

simulation.B-43 Figure B-15b. Q-Q plots for October 2002 and the VISTAS 12 km grid of predicted and observed organic matter carbon (OMC) and elemental carbon (EC) across the IMPROVE network (top left and right) and SEARCH Daily OCM (bottom left) and SEARCH Hourly total carbon mass (TCM) (bottom right) using the CMAQ 2002 12 km Base G2 Actual base case simulation.B-44 B.4.3.5 OMC Monthly Bias and Error

The monthly fractional bias and error plot fo r the 12 km CMAQ base case simulation in Figure B-16a displays fairly good performance across th e IMPROVE monitors with bias usually within ±15% and errors between 40% and 50%. However, at the urban STN sites the model exhibits a large underprediction bias of -35% to -60%. Pa rt of the underprediction bias may be due to measurement uncertainties (e.g., blank corrections) and part may be due to the model over diluting the urban OMC emissions through the co arse 36/12 km grid. The underprediction of urban OMC is a common problem in PM modeling and likely also points to uncertainties in the OMC and SOA precursor emission inventories.

The Bugle Plots of monthly fractional bias and gross error (Figure B-16b) concurs that the OMC performance across the IMPROVE monitoring sites is fairly good achieving the model performance goals. But the OMC underprediction bias for the STN monitors is significant enough so that always falls out of the model performance goals and is right at the model performance criteria. The OMC performance across the SEARCH network is mixed usually achieving the model performance goal and always achieving the model performance criteria.B-45 Figure B-16a. Monthly OMC fractional bias (top) and fractional gross error (bottom) statistical measures for IMPROVE, STN and SEARCH monitoring sites in the VISTAS region.B-46 Figure B-16b . Bugle Plots of monthly fractional bias (top) and gross error (bottom) for OMC in the VISTAS region and comparison to model performance goals (blue) and criteria (yellow).B-47 B.4.4 Elemental Carbon (EC) Monthly Mode l Performance in the VISTAS Region

Elemental carbon is measured at the IMPR OVE, STN and SEARCH monitoring networks. SEARCH included 24-hur and hourly TCM measurements. In the OMC performance plots in Figures B-12 through B-15 the SEARCH hourly TCM results were plotted. In the EC performance plots in Figures B-17 through B-20 we display the SEARCH daily average TCM performance.

B.4.4.1 EC in January 2002

Elemental Caron (EC) in January 2002 is fairlygood with fractions bi as < ±15% across the

IMPROVE and STN networks and errors of ~45% (Figure B-17). A larger underprediction bias and higher errors are seen across th e SEARCH network for OMC and TCM.B-48 Figure B-17. Scatter plots of predicted and observed elemental carbon (EC) concentrations for January 2002 and sites in the VISTAS region using IMPROVE (top left), STN (top right), and SEARCH Daily (bottom left) monitoring networks and for total carbon mass (TCM) for the SEARCH Daily network (bottom right) using the CMAQ 2002 36 and 12 km Base G2 Actual base case

simulation.B-49 B.4.4.2 EC in April 2002

Reasonable performance is seen for OMC in April, but with an underprediction bias that ranges

from -30% to -50% across the IMPORVE and ST N monitoring networks in the VISTAS region with errors of 50% to 60% (Figure B-18) . The OMC and TCM performance across the SEARCH network in April exhibits near zero bias but with a lot of scatter resulting in errors on

~60%.

Figure B-18. Scatter plots of predicted and observed elemental carbon (EC) concentrations for April 2002 and sites in the VISTAS region using IMPROVE (top left), STN (top right), and SEARCH Daily (bottom left) monitoring networks and for tota l carbon mass (TCM) for the SEARCH Daily network (bottom right) using the CMAQ 2002 36 and 12 km Base G2 Actual base case simulation.B-50 B.4.4.3 EC in July 2002

EC is underestimated in July 2002 in the VIST AS region across all monitoring networks. With the underestimation bias greater at the more rural IMPROVE (-41% and -53%) than urban STN (2% and -23%) networks (Figure B-19). It is in teresting to note that EC model performance at the urban STN and partially urban SEARCH netw orks is much better for the 12 km modeling results (bias of 2% and -60%) than the 36 km results (-23% and -86%).

Figure B-19. Scatter plots of predicted and observed elem ental carbon (EC) concentrations for July 2002 and sites in the VISTAS region using IMPROVE (top left), STN (top right), and SEARCH Daily (bottom left) monitoring networks and for tota l carbon mass (TCM) for the SEARCH Daily network (bottom right) using the CMAQ 2002 36 and 12 km Base G2 Actual base case simulation.B-51 B.4.4.4 EC in October 2002

The EC underprediction bias also occurs in Octobe r (Figure B-20). Again lower bias is seen at the STN networks with the 12 km results (-10%) producing lower bias than the 36 km results (- 31%).

Figure B-20. Scatter plots of predicted and observed elemental carbon (EC) concentrations for October 2002 and sites in the VISTAS region using IMPROVE (top left), STN (top right), and SEARCH Daily (bottom left) monitoring networks and for total carbon mass (TCM) for the SEARCH Daily network (bottom right) using the CMAQ 2002 36 and 12 km Base G2 Actual base case

simulation.B-52 B.4.4.5 EC Monthly Bias and Error

The monthly bias and error performance statistic s for EC across the VISTAS region are shown in Figure B-21a. Bias and error are lower in the first quarter of the year, and then the IMPROVE and SEARCH networks begin to exhibit an underp rediction bias that peaks in the summer and gets lower again with the onset of winter. Surprisingly, the STN network exhibits very low EC bias during the summer to fall.

The Bugle Plot in Figure B-21b shows that a lthough the EC underprediction produces large fractional bias values, because the EC concentrat ions are low the EC performance achieves the model performance goal for all months and networks since it is in the flar ed horn section of the Bugle Plot.B-53 Figure B-21a. Monthly EC fractional bias (top) and fractional gross error (bottom) statistical measures for IMPROVE, STN and SEARCH daily and hourly monitoring sites in the VISTAS region.B-54 Figure B-21b . Bugle Plots of monthly fractional bias (top) and gross error (bottom) for EC in the VISTAS region and comparison to model performance goals (blue) and criteria (yellow).B-55 B.4.5 Other PM 2.5 (Soil) Monthly Model Perfor mance in the VISTAS Region

There are potential model-measurement inco mmensurability issues with the Soil model performance. The IMPROVE and SEARCH networ ks builds up Soil from the elements, whereas in the model Soil is mapped to the other PM 2.5 category that is emissi ons not identified as SO4, NO3, OMC or EC.

B.4.5.1 Soil in January, April, July and October 2002

Soil model performance is poor in January with a systematic overprediction bias of 134% to 157% with similar magnitude of errors (Figure B-22) . The overprediction bi as is not as large in April and no longer systematic (i.e., doesn’t oc cur across all predicted-observed pairs) with fraction bias values of ~20% (IMPROVE) and ~80% (SEARCH) (Figure B-23). Higher

observed values in the measured Soil values in July lowers the model overprediction bias to approximately 40% across the SEARCH network with a -30% underprediction bias across the IMPORVE monitors (Figure B-24). The overprediction bias return s in October with fractional bias of ~100% (IMPROVE) to ~140% (SEARCH).B-56 Figure B-22. Comparison of predicted and observed other PM 2.5 (Soil) concentrations for January 2002 and sites in the VISTAS region using scatter plots (top) and Q-Q plots (bottom) and observations from the IMPROVE (left) and SEARCH Da ily (left) monitoring networks for the CMAQ
2002 12 km Base G2 Actual base case simulation.B-57 Figure B-23. Comparison of predicted and observed other PM 2.5 (Soil) concentrations for April 2002 and sites in the VISTAS region using scatter plots (top) and Q-Q plots (bottom) and observations from the IMPROVE (left) and SEARCH Daily (left) monitoring networks for the CMAQ 2002 12 km
Base G2 Actual base case simulation.B-58 Figure B-24. Comparison of predicted and observed other PM 2.5 (Soil) concentrations for July 2002 and sites in the VISTAS region using scatter plots (top) and Q-Q plots (bottom) and observations from the IMPROVE (left) and SEARCH Daily (left) monitoring networks for the CMAQ 2002 12 km
Base G2 Actual base case simulation.B-59 Figure B-25. Comparison of predicted and observed other PM 2.5 (Soil) concentrations for October 2002 and sites in the VISTAS region using scatter plots (top) and Q-Q plots (bottom) and observations from the IMPROVE (left) and SEARCH Da ily (left) monitoring networks for the CMAQ
2002 12 km Base G2 Actual base case simulation.B-60 B.4.5.2 Soil Monthly Bias and Error

The seasonal dependence of the Soil model performance across the IMPROVE and STN monitors in the VISTAS region is clearly evident in the monthly bias time series plot in Figure B-26a. As noted in the scatter plots in Figur e B-22 through B-25, the modeled Soil values tend

to be between 0 and 2 μ g/m 3 year round, however the observed values are much lower in the winter (0 to 0.5 μ g/m 3 ) and comparable to the modeled values in the summer. These results suggest that the poor Soil model performance in the winter, spring and fall is likely due to incorrect emission temporal adjust ment factors. For example, the effects of wetted surfaces on fugitive dust emissions may not be properly characterized in the emissions inventory.

The Bugle plots of bias and error also display the seasona l dependence of the Soil model performance with the model achieving the mode l performance goals for Soil at the IMPROVE monitoring network for approximately half the year and not achieving both the model performance goals and criteria at the IMPROVE network for the other half (Figure B-26b).

Performance for Soil across the SEARCH network is worse with the model not achieving either the performance goals or criteria for most months.B-61 Figure B-26a. Monthly Soil fractional bias (top) and fractional gross error (bottom) statistical measures for IMPROVE and SEARCH Daily mo nitoring sites in the VISTAS region.B-62 Figure B-26b . Bugle Plots of monthly fractional bias (top) and gross error (bottom) for Soil in the VISTAS region and comparison to model performance goals (blue) and criteria (yellow).B-63 B.4.6 Coarse Mass (CM) and PM 10 Mass Monthly Model Performance in the VISTAS Region

Coarse Mass (CM) measurements are defined as the difference between the PM 10 mass and

PM 2.5 mass measurements. Since IMPR OVE network only speciates the PM 2.5 fraction of PM, then any coarse SO4 or NO3 would be in the CM measurement. In the model, on the other hand, CM is due solely to primary PM emissions that have been speciated to the coarse fractions (PM 2.5-10 ). Thus, there are potential incompatibility issues with the measured and modeled CM

concentrations.

In addition, CM has a much shorter transport dist ances than fine PM. In the modeling this is accounted for though fugitive dust transport factor s (FDTF) that adjust the fugitive dust emissions downward (by ~75% on average in the VI STAS region) to account for the fact that these emissions are deposited locally and are not transported out of the grid cell where they are emitted. However, such local dust emissions can impact an IMPROVE monitor. Thus, much of

the measured CM at the IMPROVE monitors is li kely local in nature an d subgrid-scale to the

model.

B.4.6.1 CM and PM 10 in January, April, July and October 2002

Figures B-27 through B30 display scatter plot s of predicted and observed 24-hour CM and PM 10 concentrations across the IMPROVE monitors in the VISTAS region for, respectively, January, April, July and October. CM e xhibits an underprediction bias of approximately -50% in January and approximately -100% for the other three mont hs with errors from 100% to 130%. The PM 10 performance in January across the IMPORVE monitors is quite good with near zero bias and ~30% error so achieves the most stringent oz one model performance goals (Figure B-27). However, an underprediction bias becomes larger in April (-35%) a nd even larger in July (-60%) only to come back to fairly good model performance again in October.

Figure B-27. Scatter plots of predicted and observed coarse mass (CM) (left) and PM 10 (right) concentrations for January 2002 and observations fr om the IMPROVE monitoring network for the
CMAQ 2002 12 km Base G2 Actual base case simulation.B-64 Figure B-28. Scatter plots of predicted and observed coarse mass (CM) (left) and PM 10 (right) concentrations for April 2002 and observations from the IMPROVE monitoring network for the CMAQ
2002 12 km Base G2 Actual base case simulation.

Figure B-29. Scatter plots of predicted and observed coarse mass (CM) (left) and PM 10 (right) concentrations for July 2002 and observations from the IMPROVE monitoring network for the CMAQ
2002 12 km Base G2 Actual base case simulation.B-65 Figure B-30. Scatter plots of predicted and observed coarse mass (CM) (left) and PM 10 (right)
concentrations for October 2002 and observations fr om the IMPROVE monitoring network for the
CMAQ 2002 12 km Base G2 Actual base case simulation.

B.4.6.5 CM Monthly Bias and Error

Figure B-31a displays the monthly fraction bias and gross error values for CM across the IMPROVE monitors in the VISTAS region. The underprediction bias ranges from -60% to - 120% with errors of 80% to 140%. The Bugle Pl ot (Figure B-31b) shows that these performance statistics are either right at or exceeds th e performance criteria indicating poor CM model performance.B-66 Figure B-31a. Monthly coarse mass (CM) fractional bias (top) and fractional gross error (bottom) statistical measures for IMPROVE monitoring sites in the VISTAS region.B-67 Figure B-31b . Bugle Plots of monthly fractional bias (top) and gross error (bottom) for coarse mass in the VISTAS region and comparison to model performance goals (blue) and criteria (yellow).B-68 B.4.7 PM 2.5 Mass Monthly Model Performa nce in the VISTAS Region

Below we present the CMAQ 2002 Base G2 12km and 36 km model performance for PM 2.5 mass. In the model, PM 2.5 mass is obtained by summing all of the PM species except CM. In the measurement, PM 2.5 is obtained by the total PM on a f ilter collected by a sampler with a 2.5 micron inlet cut point. Thus, the measured PM 2.5 could exclude some SO4 and NO3 if they are

in the coarse mode that would be included in the modeled PM .5 that assumes all SO4 and NO3

are fine.

B.4.7.1 PM2.5 in January 2002

PM 2.5 performance in January is quite good ach ieving the most stringent 1-hour ozone <±15%/<35% 1-hour ozone bias/error pe rformance goal (Figure B-32a). PM 2.5 is slightly overestimated across the IMPROVE, underestimated across the STN and near zero bias across the SEARXCH and FRM monitoring sites.

Figure B-32a. Scatter plots of predicted and observed PM 2.5 mass concentrations for January 2002 and sites in the VISTAS region using IMPROVE (t op left), SEARCH Daily (top right), STN (bottom left) and FRM (bottom right) monitoring networks and the CMAQ 2002 36 and 12 km Base G2 Actual

base case simulation.B-69 Figure B-32a. (concluded) Scatter plots of predicted and observed PM 2.5 mass concentrations for January 2002 and sites in the VISTAS region using IMPROVE (top left), SEARCH Daily (top right), STN (bottom left) and FRM (bottom right) monitoring networks and the CMAQ 2002 36 and 12 km
Base G2 Actual base case simulation.

Figure B-32b. Q-Q plots of predicted and observed PM 2.5 mass concentrations for January 2002 and sites in the VISTAS region using IMPROVE (left) and SEARCH Daily (right) monitoring networks and
the CMAQ 2002 36 and 12 km Base G2 Actual base case simulation.B-70 B.4.7.2 PM 2.5 in April 2002

In April the PM 2.5 model performance is still fairly good with errors in the 30% to 40% range, but with an underprediction bias of fr om -15% (IMPROPVE) to -35% (STN).

Figure B-33a. Scatter plots of predicted and observed PM 2.5 mass concentrations for April 2002 and sites in the VISTAS region using IMPROVE (top left), SEARCH Daily (top right), STN (bottom left) and FRM (bottom right) monitoring networks and the CMAQ 2002 36 and 12 km Base G2 Actual

base case simulation.B-71 Figure B-33b. Q-Q plots of predicted and observed PM 2.5 mass concentrations for April 2002 and sites in the VISTAS region using IMPROVE (left) and SEARCH Daily (right) monitoring networks and
the CMAQ 2002 36 and 12 km Base G2 Actual base case simulation.B-72 B.4.7.3 PM 2.5 in July 2002

The underprediction bias is higher in July (-35% to -50%) with errors higher as well (45% to 55%).

Figure B-34a. Scatter plots of predicted and observed PM 2.5 mass concentrations for July 2002 and sites in the VISTAS region using IMPROVE (top left), SEARCH Daily (top right), STN (bottom left) and FRM (bottom right) monitoring networks and the CMAQ 2002 36 and 12 km Base G2 Actual

base case simulation.B-73 Figure B-34b. Q-Q plots of predicted and observed PM 2.5 mass concentrations for July 2002 and sites in the VISTAS region using IMPROVE (left) and SEARCH Daily (right) monitoring networks and
the CMAQ 2002 36 and 12 km Base G2 Actual base case simulation.B-74 B.4.7.4 PM 2.5 in October 2002

By October the performance has improved with lo wer bias and errors across all the networks.

Figure B-35a. Scatter plots of predicted and observed PM 2.5 mass concentrations for October 2002 and sites in the VISTAS region using IMPROVE (t op left), SEARCH Daily (top right), STN (bottom left) and FRM (bottom right) monitoring networks and the CMAQ 2002 36 and 12 km Base G2 Actual

base case simulation.B-75 Figure B-35b. Q-Q plots of predicted and observed PM 2.5 mass concentrations for October 2002 and sites in the VISTAS region using IMPROVE (left) and SEARCH Daily (right) monitoring networks and
the CMAQ 2002 36 and 12 km Base G2 Actual base case simulation.

B.4.7.5 PM 2.5 Monthly Bias and Error

Figure B-26a displays the monthly time seri es of fractional bias and error for PM 2.5 mass concentrations for the IMPROVE, STN, SEARCH Daily, SEARCH Hourly and FRM networks in the VISTAS region. For the first three and last three months of the year the model performs

well for PM 2.5 with low bias (<±20%) and error (<40% except for SEARCH Hourly). During April through July the underpredi ction bias becomes larger and is in the -20% to -45%) and the errors are also larger.

The Bugle Plots confirm the good performance for PM 2.5 mass concentrations that achieves the model performance goals most of the time and always achieves the model performance criteria (Figure B-36b).B-76 Figure B-36a. Monthly PM 2.5 fractional bias (top) and fractional gross error (bottom) statistical measures in the VISTAS region.B-77 Figure B-36b . Bugle Plots of monthly fractional bias (top) and gross error (bottom) for PM 2.5 in the VISTAS region and comparison to model performance goals (blue) and criteria (yellow).B-78 B.4.8 Sea Salt in the VISTAS Region

Figure B-37 compares scatter plots of predicte d and observed “Sea Salt” for IMPROVE monitors in the VISTAS region and January, April, July and October 2002. The IMPROVE “Sea Salt” is inferred from the chlorine ion measurement [Cl-], whereas CMAQ internally generates its Sea Salt emissions. The model is predicting near ze ro Sea Salt resulting in po or performance. As Sea Salt is not directly used in the 2018 visibility projections this is no t a big cause for concern,

but can affect aerosol concentr ations and is an area that s hould be investig ated further,

Figure B-37. Scatter plots of predicted and observed Sea Salt concentrations in the VISTAS region using IMPROVE monitoring data and for January (top left), April (top right), July (bottom left) and October (bottom right) and the CMAQ 2002 36 and 12 km Base G2 Actual base case simulation.B-79 B.5 Diagnostic Model Evaluation for Gas-Phase and Precursor Species

The CASTNet and SEARCH networks also measure gas-phase species that are PM precursor or related species. The diagnostic evaluation of the 2002 36/12 km Base G2 CMAQ base case simulation for these compounds and the four seasonal months presen ted previously is provided below. Note that the EPA Air Quality Stations (AQS) compliance network also provides routine measurements of many gas-phase precursor (e .g., NOx and SO2) and product (e.g., O3) species as well. However, these sites tend to be urban-oriented and designed to measure maximum potential concentrations of criteria pollutants so would not provide a goo d measure of a model’s ability to estimate regional concentrations at the more rural Class I areas. Consequently the gasphase precursor and product species diagnostic evaluation used measured data from the

CASTNet and SEARCH networks that incl uded some more rural monitoring sites.

The CASTNet network measures weekly aver age samples of SO2, SO4, NO2, HNO3, NO3 and NH4. Because the CASTNet samples are collected in a cartridge and samp led at a later date some of the NO3 may volatize into gaseous HNO3. Thus, we also evaluate against total nitrate (TNO3) that is defined as the sum of NO3 a nd HNO3. The SEARCH Hourly network collects hourly measurements of gaseous SO2, NO, NO2, NOy, O3 and CO and particulate SO4, NO3, and NH4. A comparison of the SO2 and SO4 pe rformance provides insi ght into whether the SO4 formation rate may be too sl ow or fast. For example, if SO4 is underestimated and SO2 is overestimated, that may indicate too slow chemi cal conversion rate. An alyzing the performance for SO4, HNO3, NO3, Total NO3 and NH4 provides insight into the equilibrium of these species. For example, if TNO3 performs we ll but HNO3 and NO3 do not, then there may be issues associated with the par titioning between the gaseous and pa rticle phases of nitrate (e.g., ammonia may be off).B-80 B.5.1 Diagnostic Model Performance in January 2002

Figure B-38a. Scatter plots of predicted and observed NO3 (top left), HNO3 (top right), NH4 (bottom left) and TNO3 (bottom right) from the SEARCH Hourly monitoring network and January 2002 for the
CMAQ 2002 36 and 12 km Base G2 Actual base case simulation.B-81 Figure B-38b. Scatter plots of predicted and observed NO3 (top left), HNO3 (top right), NH4 (bottom left) and TNO3 (bottom right) from the CASTNet monitoring network and January 2002 for the CMAQ
2002 36 and 12 km Base G2 Actual base case simulation.B-82 Figure B-39. Scatter plots of predicted and observed gaseous SO2 (left) and particulate SO4 (right) from the SEARCH Hourly (top) and CASTNet (bottom) monitoring networks and January 2002 for the
CMAQ 2002 36 and 12 km Base G2 Actual base case simulation.B-83 Figure B-40. Scatter plots of predicted and observed gas eous NO (top left), NO2 (top right), NOy (bottom left) and O3 (bottom right) from the SEARCH Hourly monitoring network and January 2002
for the CMAQ 2002 36 and 12 km Base G2 Actual base case simulation.B-84 B.5.2 Diagnostic Model Performance in April 2002

Figure B-41a. Scatter plots of predicted and observed NO3 (top left), HNO3 (top right), NH4 (bottom left) and TNO3 (bottom right) from the SEARCH H ourly monitoring network and April 2002 for the
CMAQ 2002 36 and 12 km Base G2 Actual base case simulation.B-85 Figure B-41b. Scatter plots of predicted and observed NO3 (top left), HNO3 (top right), NH4 (bottom left) and TNO3 (bottom right) from the CASTNet monitoring network and April 2002 for the CMAQ
2002 36 and 12 km Base G2 Actual base case simulation.B-86 Figure B-42. Scatter plots of predicted and observed gaseous SO2 (left) and particulate SO4 (right) from the SEARCH Hourly (top) and CASTNet (bo ttom) monitoring networks and April 2002 for the
CMAQ 2002 36 and 12 km Base G2 Actual base case simulation.B-87 Figure B-43. Scatter plots of predicted and observed gas eous NO (top left), NO2 (top right), NOy (bottom left) and O3 (bottom right) from the SEARCH Hourly monitoring network and April 2002 for
the CMAQ 2002 36 and 12 km Base G2 Actual base case simulation.B-88 B.5.3 Diagnostic Model Performance in July 2002

Figure B-44a. Scatter plots of predicted and observed NO3 (top left), HNO3 (top right), NH4 (bottom left) and TNO3 (bottom right) from the SEARCH Hourly monitoring network and July 2002 for the
CMAQ 2002 36 and 12 km Base G2 Actual base case simulation.B-89 Figure B-44b. Scatter plots of predicted and observed NO3 (top left), HNO3 (top right), NH4 (bottom left) and TNO3 (bottom right) from the CASTNet monitoring network and July 2002 for the CMAQ
2002 36 and 12 km Base G2 Actual base case simulation.B-90 Figure B-45. Scatter plots of predicted and observed gaseous SO2 (left) and particulate SO4 (right) from the SEARCH Hourly (top) and CASTNet (bo ttom) monitoring networks and July 2002 for the
CMAQ 2002 36 and 12 km Base G2 Actual base case simulation.B-91 Figure B-46. Scatter plots of predicted and observed gas eous NO (top left), NO2 (top right), NOy (bottom left) and O3 (bottom right) from the SEARCH Hourly monitoring network and July 2002 for
the CMAQ 2002 36 and 12 km Base G2 Actual base case simulation.B-92 B.5.4 Diagnostic Model Performance in October 2002

Figure B-47a. Scatter plots of predicted and observed NO3 (top left), HNO3 (top right), NH4 (bottom left) and TNO3 (bottom right) from the SEARCH Hourly monitoring network and October 2002 for the
CMAQ 2002 36 and 12 km Base G2 Actual base case simulation.B-93 Figure B-47b. Scatter plots of predicted and observed NO3 (top left), HNO3 (top right), NH4 (bottom left) and TNO3 (bottom right) from the CASTNet monitoring network and October 2002 for the CMAQ
2002 36 and 12 km Base G2 Actual base case simulation.B-94 Figure B-48. Scatter plots of predicted and observed gaseous SO2 (left) and particulate SO4 (right) from the SEARCH Hourly (top) and CASTNet (bottom) monitoring networks and October 2002 for
the CMAQ 2002 36 and 12 km Base G2 Actual base case simulation.B-95 Figure B-49. Scatter plots of predicted and observed gas eous NO (top left), NO2 (top right), NOy (bottom left) and O3 (bottom right) from the SEARCH Hourly monitoring network and October 2002
for the CMAQ 2002 36 and 12 km Base G2 Actual base case simulation.