## PHOTOCHEMICAL MODELING IN SUPPORT OF

ATTAINING THE FEDERAL 8-HOUR OZONE

AIR QUALITY STANDARD IN CENTRAL CALIFORNIA

Volume 1: Model Performance

California Air Resources Board Planning and Technical Support Division Sacramento, California 95814

3/1/07MODEL PERFORMANCE EVALUATION

2 TABLE OF CONTENTS

1 METEOROLOGICAL MODEL PERFORMANCE

................................................... . 3

1.1 Meteorological Model Performance Metrics...........

................................................... .... 3

1.2 Meteorological Model Performance Results ...........

................................................... .... 6

2 AIR QUALITY MODEL PERFORMANCE

................................................... .......... 37

2.1 Air Quality Model Performance Metrics..............

................................................... ..... 37

2.2 Air Quality Model Performance Results ..............

................................................... ..... 43

3 CONCLUSION – CUMULATIVE 1-HOUR AND 8-HOUR DAYS..... ...................... 69

4 RESULTS FOR MODEL SENSITIVITY TESTS................ ..................................... 71

5 REFERENCES.........................................

................................................... ........... 72

6 APPENDIX – INFORMATION AVAILABLE FOR DOWNLOADING... .................. 79MODEL PERFORMANCE EVALUATION

3 For District review purposes, this document summari zes model performance procedures and results for meteorological modeling (Section 1) as well as air quality modeling (Section 2) for the July 1999 and July-Aug ust, 2000, episodes. The model performance evaluations are based on USEPA guidance (1991 and 2005) as well as recommendations from Emery (2001), Tesche (1994) an d Tesche et al. (2001). The third section (Section 3) provides a summary of the performance analysis and Section 4 provides a tabular listing of complete gr aphical and statistical results that can downloaded via ftp from eos.arb.ca.gov.

Of note is that the two episodes have both been ext ended by two days at the beginning of the original episode periods in an effort to inc rease the number of useable days for future year design value calculations.



## 1 Meteorological Model Performance

1.1 Meteorological Model Performance Metrics

Meteorological model performance is assessed both q uantitatively using statistical metrics as well as qualitatively against known conc eptual meteorological flows and observed episodic meteorological features.

1.1.1 

### Quantitative Performance Evaluation

There are a number of statistical and graphical app roaches for evaluating meteorological model outputs. However, none of the m are independently conclusive. Most of these approaches involve comparisons betwee n observed and simulated meteorological parameter values. These analyses po se a difficult challenge, since most of the available meteorological monitoring stations are located in urbanized areas. Thus, the majority of observations tend to represen t those areas versus the full complexity of meteorology throughout the CCOS domai n. Furthermore, since the use of objective analysis and observational nudging forces the meteorological modeling results towards the observations, model performance problem s can increase in areas away from observation locations.

It also needs to be recognized that output from the various meteorological models must be preprocessed for input into the air quality mode l. This preprocessing may inadvertently perturb the meteorological fields. T herefore, meteorological model performance should be based on the air quality mode l input files, rather than the meteorological model outputs.

The SIP modeling domain is geographically very comp lex and the observational data on which meteorological model outputs were evaluated a re not distributed uniformly. Therefore, it is unreasonable to evaluate model per formance for the domain as a whole. For purposes of meteorological model performance an alysis, the CCOS domain isMODEL PERFORMANCE EVALUATION

4 divided into sub-regions, representing areas of sim ilar meteorological features. The graphical and statistical model evaluations will be done for each of these sub-regions.

A number of standard statistical and graphical tech niques are used for meteorological model performance analysis. The most widely used a pplication is the METSTAT program (Tesche, 1994, Tesche et al, 2001). Two gr aphical representations of the METSTAT statistics were used in meteorological mode l performance analysis conducted here: a) “Root Mean Square Error (RMSE) o f Wind Speed” vs. “Gross Error (E) of Wind Direction”, and b) “Bias Error (B)” vs. “Gross Error (E)” for temperature. Equations used for these comparisons were taken fro m the user documentation of the METSTAT program and are given below:

Bias Error (B) : calculated as the mean difference in prediction-o bservation pairings with valid data within a given analysis re gion and for a given time period (hourly or daily):

Here, P and O indicate model predictions and observations, respe ctively. Similarly, I and J are the indices of grid points i n x and y directions, respectively.

Gross Error (E) : calculated as the mean absolutedifference in predictionobservation pairings with valid data within a given analysis region and for a given time period (hourly or daily):

Note that the bias and gross error for winds are ca lculated from the predictedobserved residuals in speed and direction (not from vector components u and v ). The direction error for a given prediction-observat ion pairing is limited to range
from 0 to ± 180 °

.

Root Mean Square Error (RMSE) : calculated as the square root of the mean squared difference in prediction-observation pairin gs with valid data within a given analysis region and for a given time period ( hourly or daily):

The RMSE, as is the gross error, is a good overall measure of model performance. However, since large errors are weighted heavily (d ue to squaring), large errors in small subregions may produce a large RMSE even though the errors may be small and quite acceptable elsewhere. 

# ( ) ∑∑

= = − = J

j I

i i

j i j O P IJ B

1 1 1



# ∑∑

= = − = J

j I

i i

j i j O P IJ E

1 1 1



# ( ) 2/1

1 1 2 1 

    

− = 

# ∑∑

= = J

j I

i i

j i j O P IJ RMSEMODEL PERFORMANCE EVALUATION

5 Table 1-1 shows the criteria used to decide if the results of a given model fall within acceptable performance limits.

Table 1-1 Statistical comparisons between observed and simulated meteorological parameter values. Statist icalcomparisons are made by model performance subregions.

________________________________________________ Parameter Abbreviation Benchmark

________________________________________________

Wind Speed RMSE: < 2 m/s
Bias: < ± 0.5 m/s
IOA: ≥ 0.6

Wind Direction Gross Error: < 30 deg
Bias: < ± 10 deg

Temperature Gross Error: < 2 oK
Bias: < ± 0.5 oK
IOA ≥ 0.8

In an ideal situation, meteorological field evaluat ion would be done independent of the air quality model results. However, in practice, m eteorological field evaluation is limited by the relative paucity of observational data, espe cially aloft. Therefore, base year air quality model performance was also considered in th e selection of meteorological fields used for air quality simulations.

Table 1-2 Graphical analysis of meteorological mode l fields. Time series plots are made for each station and spatial plots are made over the whole modeling domain.

Time-series plots of hourly mean air temperature

Time-series plots of hourly mean wind speeds.

Spatial plots of hourly wind vectors

Spatial plots of hourly air temperaturesMODEL PERFORMANCE EVALUATION

6 1.1.2 

### Qualitative Performance Analyses

Given episode-specific information on the meteorolo gical features that were observed with field measurements, additional subjective anal yses of observed versus predicted mesoscale features can be conducted. Examples of s uch qualitative analyses that will
be considered are described below.

1. Determine and compare modeled and observed horiz ontal flow patterns over the modeling domain. Features to consider include flow splitting, the structure of the sea breeze, urban circulations, local flows such as Fresno and Schultz eddy circulations, slope and drainage flows, up/down val ley flows, and the existence of
cloud formations.

2. Study the 3-D spatial characteristics of the flo w field by using time-height cross sections of wind profiler observations and the simu lated wind field at the wind profiler location.

3. Determine the spatial and temporal characteristi cs of the mixing layer height using available upper air observations, and compare it with the simulated behavior of mixing layer height.

4. Perform some sensitivity tests to see the effect s of certain model parameters on the model results, such as observational nudging vs . analysis nudging, the choice of soil physics, and boundary layer paramete rizations.



### 1.2 Meteorological Model Performance Results

The following two sections present the results of m eteorological model performance for the two modeling episodes, based on the criteria di scussed above. For illustration purposes, a small number of the graphics that were produced are used in the subsequent discussions. However, all of the graphi cs that have been generated are available via ftp per the table in the Appendix.

1.2.1 

### July 1999 Episode (Routine Episode)

The July 1999 simulation covers the period from Jul y 5 th 12Z, 1999 to July 14 th 12Z, 1999. Meteorological model performance is assessed for the 7-day period spanning July 7 th through July 13 th

.MODEL PERFORMANCE EVALUATION

7 The July 1999 episode model performance is assessed for three MM5 runs as follows:

Table 1-3. July-1999 Episode. Three ARB simulatio ns are considered.
Simulation
Number Abbreviation Description
1 C108 7 day simulation; without FDDA
2 C109 7 day simulation; with FDDA; includes all the
available observational data.
3 C110 7 day simulation; with FDDA; but excludes from the FDDA file all known 2-meter station height data (i.e. CIMIS and NWS stations).

The FDDA file for run C109 includes all of the avai lable observational data that are available for this routine field measurement episod e. The Bay Area Air Quality Management District and their contractors, AtMet an d ENVIRON, produced FDDA data for the original ‘core’ episode days, July 9th thro ugh July 12th. For the extended episode days that fall outside of the original core days, ARB FDDA data are used. The third run (c110) has the same model setup as “c109” , except that data from sources utilizing 2-meter station heights are excluded from the C109 FDDA file. Simulation C110 is a sensitivity run to evaluate the effect of 2-meter observational station heights on MM5 performance for this episode.

To calculate model performance statistics, the resu lts from all three MM5 simulations are processed through the METSTAT program. Perform ance statistics and siteaveraged time series are calculated for 5 regions: Bay Area region, Sacramento region, Central San Joaquin Valley, Southern San Joaquin Va lley and Northern San Joaquin Valley. The resulting statistics are presented in soccer plots, where, ideally, model performance statistics fall within the central box of the goal.

Figure 1-1 shows sites-averaged time series for win ds and temperatures in the Bay Area region. There is little difference between th e three simulations. Wind speeds were generally under-predicted over the entire simulatio n period. On the other hand, temperature performance is good, as noted by the si mulated diurnal pattern. The exception to this is that temperatures were over-pr edicted on July 13 th . In terms of model performance statistics for the Bay Area regio n, Figure 1-2 shows the soccer goal plots of daily performance for winds and temperatur e. There was little difference
between c109 and c110.

Figure 1-3 shows sites-averaged time series for win ds and temperatures in the Sacramento region. In general, there is little dif ference between the three simulations. Wind speeds are over-predicted on July 7 th and July 12 th . Otherwise, the wind speed performance is good over the simulation period. On the other hand, temperatures were under-predicted during the day and over-predicted i n the morning. Figure 1-4 shows the soccer goal plots of daily performance for wind s and temperature.MODEL PERFORMANCE EVALUATION

8 Figure 1-5 shows sites-averaged time series for win ds and temperatures in the Central San Joaquin Valley. Wind speeds were generally ove r-predicted. Temperatures were under-predicted during the day and over-predicted i n the morning. There was little difference between the three simulations. Figure 1 -6 shows the soccer goal plots of daily performance for winds and temperature.

Figure 1-7 shows sites-averaged time series for win ds and temperatures in the Southern San Joaquin Valley. Wind speeds were gene rally over-predicted. Temperatures were under-predicted during the day an d over-predicted in the morning. There was little difference between the three simul ations. Figure 1-8 shows the soccer goal plots of daily performance for winds and tempe rature.

Figure 1-9 shows sites-averaged time series for win ds and temperatures in the Northern San Joaquin Valley. Wind speeds were generally ove r-predicted. Temperatures were under-predicted during the day and over-predicted i n the morning. There was little difference between the three simulations. Figure 1 -10 shows the soccer goal plots of daily performance for winds and temperature.

It should be noted that both the ARB and the Bay Ar ea Air Quality Management District have done much work to improve meteorological model performance for this episode. However, little additional progress has been made o ver the past two years and performance statistics still remain outside of the ‘ideal’ range. Alone, however, this is not grounds to dismiss the met simulations as poor. We are hopeful that statistical performance can be improved and will continue to wo rk closely with the districts and other stakeholders, including CCOS contractors, wit h this goal in mind.

Among the three July 1999 simulations, MM5 with obs ervation nudging (c109 and c110) improves the wind speed and wind direction a little over c108, which has no observation nudging. Since there were no significant differenc es between c109 and c110, it is
assumed that the 2-meter station data included in t he FDDA file play no significant role in degrading model performance. As a result, c109 is used as input for the air quality
model.MODEL PERFORMANCE EVALUATION

9 Figure 1-1. Time series of wind speed, direction, and temperat ure for the Bay Area region over the July 7-13, 1999 modeling period.MODEL PERFORMANCE EVALUATION

10 Figure 1-2. MM5 performance for winds and temperature in the Bay Area region.

MM5 Temperature Performance in Bay Area

July 7 - 13, 1999

0 1 2 3 4 5 6

-1 0 1 2 3 4 5 6

Bias (K)Error (K) c108

c109

c110

Benchmark MM5 Wind Performance in Bay Area

July 7 - 13, 1999

0 10 20 30 40 50 60 70

0 0.5 1 1.5 2 2.5

WS RMSE (m/s)WD Gross Error (degree)

c108

c109

c110

BenchmarkMODEL PERFORMANCE EVALUATION

11 Figure 1-3. Time series of wind speed, direction, and temperat ure for the Sacramento region over the July 7-13, 1999 modeling period.MODEL PERFORMANCE EVALUATION

12 Figure 1-4. MM5 performance for winds and temperature in the Sacramento region. MM5 Wind Performance in Sacramento

July 7 - 13, 1999

0 10 20 30 40 50 60 70 80 90

0 0.5 1 1.5 2 2.5

WS RMSE (m/s)WD Gross Error (degree)

c108

c109

c110

Benchmark

MM5 Temperature Performance in Sacramento

July 7 - 13, 1999

0 0.5 1 1.5 2 2.5 3 3.5

-1 -0.5 0 0.5 1 1.5 2

Bias (K)Error (K) c108

c109

c110

BenchmarkMODEL PERFORMANCE EVALUATION

13 Figure 1-5. Time series of wind speed, wind direction, and tem perature for the Central San Joaquin Valley over the July 7-13, 1999 modelin g period.MODEL PERFORMANCE EVALUATION

14 Figure 1-6. MM5 performance for winds and temperature in the Central San Joaquin Valley. MM5 Wind Performance in SJV Central July 7 - 13, 1999

0 10 20 30 40 50 60 70 80 90

0 0.5 1 1.5 2 2.5

WS RMSE (m/s)WD Gross Error (degree)

c108

c109

c110

Benchmark

MM5 Temperature Performance in SJV Central

July 7 - 13, 1999

0 0.5 1 1.5 2 2.5 3 3.5

-1 -0.5 0 0.5 1

Bias (K)Error (K) c108

c109

c110

BenchmarkMODEL PERFORMANCE EVALUATION

15 Figure 1-7. Time series of wind speed, wind direction, and tem perature for the Southern San Joaquin Valley over the July 7-13, 199 9 modeling period.MODEL PERFORMANCE EVALUATION

16 Figure 1-8. MM5 performance for winds and temperature in the Southern San Joaquin Valley. MM5 Wind Performance in SJV Kern July 7 - 13, 1999

0 10 20 30 40 50 60 70 80 90 100

0 0.5 1 1.5 2 2.5

WS RMSE (m/s)WD Gross Error (degree)

c108

c109

c110

Benchmark

MM5 Temperature Performance in SJV Kern

July 7 - 13, 1999

0 0.5 1 1.5 2 2.5 3

-1.5 -1 -0.5 0 0.5 1

Bias (K)Error (K) c108

c109

c110

BenchmarkMODEL PERFORMANCE EVALUATION

17 Figure 1-9. Time series of wind speed, direction, and temperat ure for the Northern San Joaquin Valley over the July 7-13, 1999 modeling pe riod.MODEL PERFORMANCE EVALUATION

18 Figure 1-10. MM5 performance for winds and temperature in the Northern San Joaquin Valley. MM5 Wind Performance in SJV North July 7 - 13, 1999

0 10 20 30 40 50 60 70

0 0.5 1 1.5 2 2.5

WS RMSE (m/s)WD Gross Error (degree)

c108

c109

c110

Benchmark

MM5 Temperature Performance in SJV North

July 7 - 13, 1999

0 0.5 1 1.5 2 2.5 3

-1.5 -1 -0.5 0 0.5 1 1.5

Bias (K)Error (K) c108

c109

c110

BenchmarkMODEL PERFORMANCE EVALUATION

19 1.2.2 

### July-August 2000 Episode (CCOS Episode)

Under the CCOS program, the meteorological modeling group at the National Oceanic and Atmospheric Administration (NOAA) was selected as a contractor to study the July 29, 2000 12Z – Aug 3, 2000 12Z ozone episode that o ccurred during CCOS. Under this CCOS contract, NOAA studied the meteorology of this episode using the MM5 numerical model with various model options and init ial and boundary conditions.

After extensive internal simulations, NOAA produced and distributed an MM5 model output in 2003 that is referred to as the “NOAA pla ceholder” simulation. Subsequently, NOAA produced several additional MM5 outputs. Thre e of these other simulations as well as the placeholder simulation were selected by ARB as candidates for SIP modeling purposes. The last two of these simulatio ns are considered by NOAA to be their ‘best available’ runs. The model setups in a ll of these simulations are identical except as noted in the first four rows of the table below (Table 1-3).

Table 1-3 July-Aug 2000 CCOS Episode. Four 5-day N OAA simulations and two 7-day ARB simulations are considered.
Simulation
Number Abbreviation Description
1 NOAA placeholder 5 day simulation using 5 layer soi l model and observational FDDA file prepared by the Bay Area AQMD
2 NOAA FDDA1 5 day simulation; Same as NOAA placehold er (#1), except using NOAH land-surface model
3 NOAA FDDA2 5 day simulation; Same as NOAA FDDA1 (#2 ), above, except using observational FDDA file prepared by NOAA and with roughness length
doubled.
4 NOAA FDDA3 5 day simulation; Same as NOAA FDDA1 (#2 ), above, except using observational FDDA file prepared by NOAA and with 5 times the roughness length.
5 ARB NO FDDA 7 day simulation
6 ARB FDDA 7 day simulation

As indicated in the last two rows of the table abov e, model simulations were also conducted at ARB. In these two ARB simulations, di fferent model options were used: the Gayno Seaman boundary layer scheme was used; a larger radius of influence was selected; and the model was started approximately t wo days earlier, on July 27, 00Z, in order to provide additional days for Relative Reduc tion Factor calculations performed for air quality analyses. Thus, both ARB runs are for 7 days.MODEL PERFORMANCE EVALUATION

20 The four five-day NOAA outputs along with two seven -day ARB outputs, called ARB NO FDDA and ARB FDDA, were analyzed and compared again st observational data using the comparison methods discussed previously.

Model performance statistics provided in Figures 1- 10 (a-g) and 1-11(a-g) point out that the statistical error of simulation NOAA FDDA3 are within the acceptable limits of EPA standards for wind speed and direction while temper ature predictions are not very good.

The temporal comparison of model variables at the A ngiola site is illustrated in Figures 1-12 (a and b). Comparisons at other stati ons were also made and can be made available (see the reference to other 2000 met eorology information in the Appendix). While previous figures show the station averaged model performance statistics within each subregion, these give a deta iled perspective of model performance
at an individual observation station. As can be se en from the examination of these temporal comparisons, model performance can vary dr amatically from one station to the next. While all NOAA and ARB FDDA simulations appe ar to adequately produce the observed wind field, the NOAA Placeholder model app ears to produce observed temperatures better than the other model runs do.

Figures 1-13(a-f) compare horizontal wind vectors a gainst observations at 21Z on July 29, 2000 (2 PM local time) when the flow field is expected to play an important role
in maximum ozone concentrations. Each model has sl ightly different wind flows, however the flows in the NOAA FDDA2, FDDA3 and Plac eholder simulations seem to be more organized than in the other models.

Since NOAA FDDA2 and NOAA FDDA3 are sensitivity tes ts generated by playing with roughness length, these results will not be conside red in air quality simulations until the effects of these tests are further understood and a ccepted. Presently, it appears that this has an adverse impact on temperature performan ce. ARB is currently working with NOAA to better understand these runs. The figures indicate that the overall performance of NOAA’s placeholder model for all var iables is generally as acceptable
as all other MM5 results that were considered. The refore, the NOAA placeholder model output is used in air quality simulations.MODEL PERFORMANCE EVALUATION

21 Figures 1-11 (a-g): Model performance statistics o f wind speed and direction created for subregions 3, 6, 7, 8, 9, 10 and 11, respective ly.

MM5 Wind Performance Within Bay Area

July 29-Aug 3, 2000

0 10 20 30 40 50 60 70 80 90

0 0.5 1 1.5 2 2.5 3 3.5 4

WS RMSE (m/s)WD Gross Error (degree) Benchmark

ARB NO FDDA

ARB FDDA

NOAA FDDA1

NOAA FDDA2

NOAA FDDA3

NOAA Placeholder

(a)

MM5 Wind Performance Within Sacramento Region

July 29-Aug 3, 2000

0 10 20 30 40 50 60 70 80 90

0 0.5 1 1.5 2 2.5 3 3.5 4

WS RMSE (m/s)WD Gross Error (degree) Benchmark

ARB NO FDDA

ARB FDDA

NOAA FDDA1

NOAA FDDA2

NOAA FDDA3

NOAA Placeholder

(b)MODEL PERFORMANCE EVALUATION

22 MM5 Wind Performance Within SJV Central

July 29-Aug 3, 2000

0 10 20 30 40 50 60 70 80 90

0 0.5 1 1.5 2 2.5 3 3.5 4

WS RMSE (m/s)WD Gross Error (degree) Benchmark

ARB NO FDDA

ARB FDDA

NOAA FDDA1

NOAA FDDA2

NOAA FDDA3

NOAA Placeholder

(c)

MM5 Wind Performance Within SJV Kern

July 29-Aug 3, 2000

0 10 20 30 40 50 60 70 80 90

0 1 2 3 4

WS RMSE (m/s)WD Gross Error (degree) Benchmark

ARB NO FDDA

ARB FDDA

NOAA FDDA1

NOAA FDDA2

NOAA FDDA3

NOAA Placeholder

(d)MODEL PERFORMANCE EVALUATION

23 MM5 Wind Performance Within SJV North

July 29-Aug 3, 2000

0 10 20 30 40 50 60 70 80 90

0 0.5 1 1.5 2 2.5 3 3.5 4

WS RMSE (m/s)WD Gross Error (degree) Benchmark

ARB NO FDDA

ARB FDDA

NOAA FDDA1

NOAA FDDA2

NOAA FDDA3

NOAA Placeholder

(e)

MM5 Wind Performance Within Central Mountain Counti es

July 29-Aug 3, 2000

0 10 20 30 40 50 60 70 80 90 100

0 0.5 1 1.5 2 2.5

WS RMSE (m/s)WD Gross Error (degree) Benchmark

ARB NO FDDA

ARB FDDA

NOAA FDDA1

NOAA FDDA2

NOAA FDDA3

NOAA Placeholder

(f)MODEL PERFORMANCE EVALUATION

24 MM5 Wind Performance Within SJV Above 3000 ft

July 29-Aug 3, 2000

0 10 20 30 40 50 60 70 80 90

0 0.5 1 1.5 2 2.5 3 3.5 4

WS RMSE (m/s)WD Gross Error (degree) Benchmark

ARB NO FDDA

ARB FDDA

NOAA FDDA1

NOAA FDDA2

NOAA FDDA3

NOAA Placeholder

(g)MODEL PERFORMANCE EVALUATION

25 Figures 1-12 (a-g): Same as Figures 1-11, except f or temperature.

MM5 Temperature Performance Within Bay Area

July 29-Aug 3, 2000

0 1 2 3 4 5 6 7

-2 -1 0 1 2 3 4 5

Bias (K)Error (K) Benchmark

ARB NO FDDA

ARB FDDA

NOAA FDDA1

NOAA FDDA2

NOAA FDDA3

NOAA Placeholder

(a)

MM5 Temperature Performance Within Sacramento Regio n

July 29-Aug 3, 2000

0 1 2 3 4 5 6 7

-5 -4 -3 -2 -1 0 1 2 3

Bias (K)Error (K) Benchmark

ARB NO FDDA

ARB FDDA

NOAA FDDA1

NOAA FDDA2

NOAA FDDA3

NOAA Placeholder

(b)MODEL PERFORMANCE EVALUATION

26 MM5 Temperature Performance Within SJV Central

July 29-Aug 3, 2000

0 1 2 3 4 5 6 7

-5 -4 -3 -2 -1 0 1 2 3

Bias (K)Error (K) Benchmark

ARB NO FDDA

ARB FDDA

NOAA FDDA1

NOAA FDDA2

NOAA FDDA3

NOAA Placeholder

(c)

MM5 Temperature Performance Within SJV Kern

July 29-Aug 3, 2000

0 1 2 3 4 5 6 7

-5 -4 -3 -2 -1 0 1 2

Bias (K)Error (K) Benchmark

ARB NO FDDA

ARB FDDA

NOAA FDDA1

NOAA FDDA2

NOAA FDDA3

NOAA Placeholder

(d)MODEL PERFORMANCE EVALUATION

27 MM5 Temperature Performance Within SJV North

July 29-Aug 3, 2000

0 1 2 3 4 5 6 7

-4 -3 -2 -1 0 1 2 3 4

Bias (K)Error (K) Benchmark

ARB NO FDDA

ARB FDDA

NOAA FDDA1

NOAA FDDA2

NOAA FDDA3

NOAA Placeholder

(e)

MM5 Temperature Performance Within Central Mountain Counties July 29-Aug 3, 2000

0 1 2 3 4 5 6 7

-4 -3 -2 -1 0 1 2 3

Bias (K)Error (K) Benchmark

ARB NO FDDA

ARB FDDA

NOAA FDDA1

NOAA FDDA2

NOAA FDDA3

NOAA Placeholder

(f)MODEL PERFORMANCE EVALUATION

28 MM5 Temperature Performance Within SJV Above 3000 f t

July 29-Aug 3, 2000

0 1 2 3 4 5 6 7

-4 -3 -2 -1 0 1 2 3

Bias (K)Error (K) Benchmark

ARB NO FDDA

ARB FDDA

NOAA FDDA1

NOAA FDDA2

NOAA FDDA3

NOAA Placeholder

(g)MODEL PERFORMANCE EVALUATION

29 Figures 1-12 (a and b): Temporal comparisons of wi nd speed, direction and temperature at Angiola station for ARB (a) and NOAA (b) model results

(a)MODEL PERFORMANCE EVALUATION

30 (b)MODEL PERFORMANCE EVALUATION

31 Figures 1-13(a-f): Horizontal variation of wind ve ctors on July 29, 2000 21Z (2 PM local time) compared to the observations for ARB NO FDDA (a), ARB FDDA (b), NOAA FDDA1 (c), NOAA FDDA2 (d), NOAA FDDA3 (e) and NOAA Placeholder models (f).

(a)MODEL PERFORMANCE EVALUATION

32 (b)MODEL PERFORMANCE EVALUATION

33 (c)MODEL PERFORMANCE EVALUATION

34 (d)MODEL PERFORMANCE EVALUATION

35 (e)MODEL PERFORMANCE EVALUATION

36 (f)MODEL PERFORMANCE EVALUATION

37 

## 2 Air Quality Model Performance

2.1 Air Quality Model Performance Metrics

Air quality model results are used to develop strat egies for attaining the federal 8-hour ozone standard. The development of these strategie s relies on the use of relative reduction factors (RRFs). More detailed discussion of RRFs is provided in other documents. However, the use of RRFs requires an ev aluation of relative air quality model response at specific monitoring sites in the base year(s), a baseline year, and a future year.

Adequate model performance is a requirement for use of modeled results. The lack of acceptable performance greatly increases uncertaint y in the use of the modeling results,
and casts doubt on conclusions based on the modelin g. Although it is desirable to include as many days as possible in the RRF calcula tions, our experience has demonstrated that not all modeled days meet the min imum performance standards, and are thus not suitable for use. Therefore only thos e days that satisfy the following model performance criteria will be utilized in subsequent RRF calculations.

The USEPA (1991) and ARB (1990) outline a number of procedures for analysis of base year, air quality model performance. These in clude spatial and time-series plots, and statistical analyses, comparing simulated and o bserved pollutant concentrations, as well as sensitivity analysis of selected input fiel ds. The purpose of the performance analysis is to provide some confidence that the air quality simulations – which are the basis of future-year ozone concentration estimates – are performing properly.

The application of air quality modeling results to demonstrate attainment of the federal 1-hour ozone standard emphasized the simulated unpa ired peak ozone concentration.
Three statistical measures were recommended to eval uate model performance: unpaired peak ratio (UPR), paired mean normalized b ias (NB), and paired gross error (GE). These statistical measures were calculated f or the modeling domain as a whole, and the NB and GE were calculated from all hourly c oncentrations in excess of 60 ppb (to avoid biasing the statistical measures with low concentrations). To meet performance guidelines, recommendations were that t he UPR should be within ± 20%,
NB should be within ± 15%, and the GE less than 35%. However, Californi a’s geography is very complex and modeling domains have evolved to cover large geographic areas. Thus it is recommended that the domains be divided into subregions, and that the performance measures be ca lculated independently for each subregion. The configuration of these subregions i s somewhat arbitrary; however, they should be configured to isolate "common" regions of higher ozone. Figure 2-1 illustrates the proposed subregions for the CCOS do main.

The USEPA (2005) recommends that model performance be evaluated for 8-hour
concentrations as well . The recommended statistic al measures to assess simulated
versus observed maximum 8-hour ozone concentrations include paired (in space, butMODEL PERFORMANCE EVALUATION

38 not time) peak prediction accuracy (PPPA), paired m ean normalized bias (NB), and paired gross error (GE). Although limited performa nce analysis has been completed for 8-hour ozone modeling in California, it seems prude nt at this point to carry forward the 1-hour statistical goals and apply them for the 8-h our standard (UPR within ± 20%, NB
within ± 15%, and the GE less than 35%). However, these li mits may need to be revised as 8-hour SIP modeling progresses and rigor ous model performance evaluations are completed.

While statistical measures for 1-hour model perform ance were typically calculated independently for each modeled day available, the U SEPA also suggests that PPPA, NB, and GE be calculated for each site over all mod eled days. However, because the number of episode days available may be very limite d, the statistical uncertainties in these latter calculations would be large and they a re not recommended or used herein.

In order to have confidence in future year estimate s from air quality models, there must be confidence in the air quality modeling for the b ase year. That is, days not meeting model acceptance criteria provide high uncertainty, and should not be used for the
modeled attainment test.

In addition to the issue of model performance, anal yses conducted by the USEPA (2005) suggest that air quality models respond more to emission reductions at higher predicted ozone values. Correspondingly, the model predicts less benefit at lower concentrations. This is consistent with preliminar y modeling in support of the 8-hour ozone standard conducted by the ARB and the distric ts. These results imply that RRF calculations should be restricted to days with pred icted high ozone concentrations. It is thus reasonable to establish a minimum threshold fo r predicted peak 8-hour ozone concentrations in the baseline year. Days for whic h the predicted daily peak 8-hour
ozone concentrations at a site are less than the th reshold, would not be used for calculating RRFs at that site. Consistent with USE PA’s recommendation, we propose to use a value of 85 ppb for the baseline year thre shold. However, USEPA guidelines
allow the use of the maximum 8-hour concentrations within 15km of a site for this

purpose.

Based on the above discussion, we propose the follo wing model performance based methodology for determining sites and modeled days to be used in the RRF
calculations:MODEL PERFORMANCE EVALUATION

39 Only those modeled days meeting the following crite ria will be used to calculate site-specific RRFs:

1) The modeled daily 8-hour peak ozone concentratio n within 15 km of the site for the base year of the modeling must be within ± 20% of the observed value at the site. 2) The modeled daily 8-hour peak ozone concentratio n within 15 km of the site in the baseline year must be 85 ppb or greater. 3) The subregional 1-hour and 8-hour statistical me asures of NB and
GE must fall within the thresholds of ± 15% and 35%, respectively.

Of these three criteria, only the third is consider ed in this document.

Along with the statistical measures discussed above , the graphical and statistical tests recommended by the USEPA (1991 and 2005) and shown in Tables 2-1 and 2-2 will be used to assess overall model performance. Several sensitivity tests recommended by the USEPA (1991) will also be used (Table 2-3) for qualitative evaluation. While the results of these sensitivity analyses are inherentl y subjective, they are designed to provide confidence that the air quality model is no t only performing well, but is also properly responding to changes in inputs.MODEL PERFORMANCE EVALUATION

40 Figure 2-1 Sub-regions of air quality model perfor mance evaluation (3: Bay area region, 6: metro Sacramento region, 7: central San Joaquin valley region , 8 southern San Joaquin valley region, 9: northern San Joaquin valley region).



### 1 2

3

4 5

6

7

8 9 10

11

12 13

14

15MODEL PERFORMANCE EVALUATION

41 Table 2-1. Statistics for evaluating base year ai r quality model performance for all sub-regions.

− mean normalized bias for all 1-hour ozone concentr ations (60 ppb), unpaired in time and space for all sites

− mean normalized gross error for all 1-hour ozone c oncentrations ( 

### ≥ 60 ppb), unpaired in time and space for all sites

− peak 1-hour ozone concentration ratio, unpaired in time and space

− mean normalized bias for all 8-hour ozone concentr ations ( 

### ≥ 60 ppb), unpaired in
time for all sites

− mean normalized gross error for all 8-hour ozone c oncentrations ( 

### ≥ 60 ppb), unpaired in time for all sites

− peak 8-hour ozone concentration ratio, unpaired in time and spaceMODEL PERFORMANCE EVALUATION

42 Table 2-2. Graphical tools for evaluating base ye ar air quality model performance.

− time-series plots comparing 1-hour measured and si mulated concentrations of ozone, NO, NO2, and CO for each site.

− hourly spatial plots of 1-hour measured and simula ted concentrations of ozone, NO, NO2, and CO for the CCOS modeling domain.

− scatter plot of 1-hour ozone concentrations for ea ch day, and for each subregion of the modeling domain.

Table 2-3. Sensitivity tests for evaluation of Bas e Year air quality simulations. The results of these analyses will be tabulated by subregion.

1 Minimize vertical diffusivity based on land cover

2 Zero anthropogenic emissions

3 Zero biogenic emissions

4 Set lateral ozone boundary conditions to 50 ppb

5 Set lateral ozone boundary conditions to 90 ppb

6 Set initial ozone conditions to 40 ppb everywhere

7 Set initial conditions to 0.1 ppb NO2 and 0.0 NO (r un with all emissions)

8 Set initial conditions to 0.1 ppb NO2 and 0.0 NO (r un with biogenic emissions only)

9 Double biogenic emissions

10 Remove wildfires

11 Zero mobile emissions

12 Set top ozone boundary conditions to 135ppb at 15kmMODEL PERFORMANCE EVALUATION

43 

### 2.2 Air Quality Model Performance Results

The following two sections present the results of a ir quality model performance for the two modeling episodes, based on the criteria discus sed in the previous section. For illustration purposes, only a portion of the graphi cs that were actually produced are presented. All of the graphics that have been gene rated are available via ftp per the table in the Appendix.

2.2.1 

### July 1999 Episode (Routine Episode)

The July 1999 air quality model simulation covers t he 7-day period from July 7, 1999, through July 13, 1999. However, the model performa nce assessment only covers the 5-day, non-spin-up period from July 9 to July 13, 1 999. As discussed previously in Section 1.2.1, the ARB c109 MM5 meteorological simu lation is used for these air quality
simulations.

Tables 2-4 and 2-5 summarize the 1-hour and 8-hour statistical model performance assessment in terms of identifying the days for whi ch simulated results fall within acceptable statistical performance thresholds in ea ch model performance region (performance regions were shown in Figure 2-1). Ea ch cell in the tables represents whether model-simulated results, on a region-wide b asis, are statistically acceptable. The cell is assigned a value of 1 if the model-simu lated results pass the statistical model performance criteria; while a value of 0 means that the model-simulated results for the region do not meet the criteria. If all simulated ozone concentrations in the region are below 60 ppb, then the region-day cell is assigned -99 and the modeling results cannot be used for that region. Total days for each episo de day and region are provided at the bottom row and far right column of the table, respe ctively.

For the Bay Area region, 4 days meet the 1-hour cri teria and only 1 day meets the 8- hour criteria. For the Sacramento region, all 5 da ys meet both the 1-hour and 8-hour criteria. For the Central San Joaquin Valley regio n, 5 days meet the 1-hour criteria and 3 days meet the 8-hour criteria. For the Southern San Joaquin Valley region, only 1 day meets the 1-hour criteria and 4 days meet the 8-hou r criteria. For the Northern San Joaquin Valley region, 2 days meet the 1-hour crite ria and 3 days meet the 8-hour
criteria.

Figures 2-2 through 2-6 show the site-averaged time series of modeled versus predicted CO, ozone, NO and NO 2 for the Bay Area, Sacramento, Central San Joaquin Valley, Southern San Joaquin Valley, and Northern S an Joaquin Valley regions, respectively. The orange and blue lines represent o bservations and model predictions, respectively. The time series for individual stati ons for these five regions have also been plotted and are available via the ftp site and filename indicated in the Appendix.MODEL PERFORMANCE EVALUATION

44 Figure 2-2 shows the hourly averaged CO, ozone, NO and NO 2 for the Bay Area region. Predicted CO concentrations are slightly under-pred icted for most of the simulation period. However, the simulated ozone is generally over-predicted for the entire simulation period. The model captures the magnitud e and diurnal variation of the NO and NO 2 reasonably well.

Figure 2-3 shows the hourly averaged CO, ozone, NO and NO 2 for the Sacramento region. Predicted CO concentrations are over-predi cted for most of the simulation period. Ozone concentrations perform well for the first three days and the last day of the simulation period, but are under-predicted from July 10 to July 12. Predicted NO and NO 2 concentrations generally agree with the observatio ns.

Figure 2-4 shows the hourly averaged CO, ozone, NO and NO 2 for the Central San Joaquin Valley region. Predicted CO concentrations are under-predicted for the entire simulation period. The simulated ozone is generall y under-predicted during the day, but over-predicted in the morning. Both predicted NO a nd NO 2 concentrations are underpredicted for the entire simulation period.

Figure 2-5 shows the hourly averaged CO, ozone, NO and NO 2 for the Southern San Joaquin Valley region. Predicted CO concentrations are generally under-predicted in the morning, but over-predicted in the afternoon. The simulated ozone is generally under-predicted during the day, but over-predicted in the morning. NO concentrations are under-predicted for the entire simulation perio d. In general, NO 2 concentrations are also under-predicted. However, the NO 2 concentrations are over-predicted at night on July 9 and July 11.

Figure 2-6 shows the hourly averaged CO, ozone, NO and NO 2 for the Northern San Joaquin Valley region. Predicted CO concentrations are under-predicted for the entire simulation period. The simulated ozone is generall y predicted well for all days. The model captures the diurnal variation of the NO, but predicted concentrations are generally less than the observed values. NO 2 concentrations also generally agree with
the observations.

Plots of model performance statistics for each regi on are provided in Figures 2-7 and 2- 8, which show the predicted 1-hour and 8-hour unpai red peak ratio and normalized bias in graphical format for each station in each of the five regions.MODEL PERFORMANCE EVALUATION

45 Table 2-4. 1-hour ozone performance by each region over the July 9-13, 1999 modeling period.

Region ID Region Name 7/9/1999 7/10/1999 7/11/1999 7/12/1999 7/13/1999 Total

2 North Coast 1 0 0 1 -99 2

3 BAAQMD 1 1 1 1 0 4

4 MBAQMD 1 0 0 1 0 2

5 Sacramento Valley North 1 0 1 1 0 3

6 Sacramento Region 1 1 1 1 1 5

7 SJVAPCD Central 1 1 1 1 1 5

8 SJVAPCD Kern 0 0 0 0 1 1

9 SJVAPCD North 1 0 1 0 0 2

10 Sierra Nevada Central 0 1 0 0 0 1

11 SJVAPCD Above 3000 ft 0 0 1 1 1 3

12 South Central Coast 1 1 0 1 0 3

13 Sierra Nevada North 1 0 0 1 1 3

14 Desert 0 0 0 0 0 0

15 Nevada 1 0 -99 1 0 2

Total: 10 5 6 10 5 36

Table 2-5. 8-hour ozone performance by each region over the July 9-13, 1999 modeling period.

Region ID Region Name 7/9/1999 7/10/1999 7/11/1999 7/12/1999 7/13/1999 Total

2 North Coast -99 -99 0 1 -99 1

3 BAAQMD 0 0 1 0 0 1

4 MBAQMD 1 -99 0 0 -99 1

5 Sacramento Valley North 1 1 1 1 0 4

6 Sacramento Region 1 1 1 1 1 5

7 SJVAPCD Central 1 1 0 1 0 3

8 SJVAPCD Kern 1 1 1 0 1 4

9 SJVAPCD North 1 0 1 0 1 3

10 Sierra Nevada Central 1 1 0 0 0 2

11 SJVAPCD Above 3000 ft 0 0 1 1 1 3

12 South Central Coast 0 1 0 1 1 3

13 Sierra Nevada North 1 0 0 1 1 3

14 Desert 0 0 0 1 0 1

15 Nevada 1 -99 -99 1 1 3

Total: 9 6 6 9 7 37MODEL PERFORMANCE EVALUATION

46 Figure 2-2. Hourly averaged of CO, ozone, NO and NO 2 for the Bay Area region over the July 7-13, 1999 modeling period.MODEL PERFORMANCE EVALUATION

47 Figure 2-3. Hourly averaged CO, ozone, NO and NO 2 for the Sacramento region over the July 7-13, 1999 modeling period.MODEL PERFORMANCE EVALUATION

48 Figure 2-4. Hourly averaged CO, ozone, NO and NO 2 for the Central San Joaquin Valley over the July 7-13, 1999 modeling period.MODEL PERFORMANCE EVALUATION

49 Figure 2-5. Hourly averaged CO, ozone, NO and NO 2 for the Southern San Joaquin Valley over the July 7-13, 1999 modeling period.MODEL PERFORMANCE EVALUATION

50 Figure 2-6. Hourly averaged CO, ozone, NO and NO 2 for the Northern San Joaquin Valley over the July 7-13, 1999 modeling period.MODEL PERFORMANCE EVALUATION

51 1-ho ur O3 Statistics, B ay A rea

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

Unpair Peak Ratio

1-ho ur O3 Statistics, Sacramento

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

Unpair Peak Ratio

1-ho ur O3 Statistics, C entral SJV

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

Unpair Peak RatioMODEL PERFORMANCE EVALUATION

52 Figure 2-7. Unpaired peak ratio vs. normalized bias for 1-hour ozone for the July 9-13, 1999 modeling period. Each dot represents one-day results for an individual site.

1-ho ur O3 Statistics, So uthern SJV

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

Unpair Peak Ratio

1-ho ur O3 Statistics, N o rthern SJV

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

Unpair Peak RatioMODEL PERFORMANCE EVALUATION

53 8-ho ur O3 Statistics, B ay A rea

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

Unpair Peak Ratio

8-ho ur O3 Statistics, Sacramento

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

Unpair Peak Ratio

8-ho ur O3 Statistics, C entral SJV

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

Unpair Peak RatioMODEL PERFORMANCE EVALUATION

54 Figure 2-8. Unpaired peak ratio vs. normalized bias for 8-hour ozone for the July 9-13, 1999 modeling period. Each dot represents one-day results for an individual site.

8-ho ur O3 Statistics, So uthern SJV

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

Unpair Peak Ratio

8-ho ur O3 Statistics, N o rthern SJV

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

Unpair Peak RatioMODEL PERFORMANCE EVALUATION

55 2.2.2 

### July-August 2000 Episode (CCOS Episode)

The air quality model simulation covers the period from July 27, 2000, to August 2, 2000. The first two days of the simulation are tre ated as a model spin-up period for which model performance is not considered. As a re sult, the statistical model performance assessment only covers the non-spin-up period, from July 29 th through August 2 nd . As described previously in Section 1.2.2, the NO AA placeholder MM5 meteorological simulation is used for these air qua lity simulations.

Performance statistics for 1-hour and 8-hour model performance were calculated for each region and are listed in Table 2-6 and 2-7, re spectively. For days that a region meets the criteria a value of 1 is assigned. A val ue of 0 means that region doesn’t meet the criteria for the respective day and, if there i s no model simulated concentrations above 60ppb, then -99 is assigned. The following p aragraph summarizes the subregional statistical results.

For the Bay Area region, the model performance meet s both the 1-hour and 8-hour criteria on July 29, 2000 and August 2, 2000, but f ails on all the other days. For the Sacramento region, 4 days meet the 1-hour criteria and 3 days meet the 8-hour criteria. For the Central San Joaquin Valley region, all 5 da ys meet both the 1-hour and 8-hour criteria. For the Southern San Joaquin Valley regi on, 4 days meet the 1-hour criteria and all 5 days meet the 8-hour criteria. For the No rthern San Joaquin Valley region, only 2 days meet both the 1-hour and 8-hour criteria.

Figures 2-9 through 2-13 show the site-averaged tim e series of observed versus predicted CO, ozone, NO and NO 2 for the Bay Area, Sacramento, Central San Joaquin Valley, Southern San Joaquin Valley and Northern Sa n Joaquin Valley regions, respectively. The orange and blue lines represent observations and model predictions, respectively. Take note that the start hour of the simulation is 0600 PDT, so the first day has only 18 data points. Also, note that the 1 -hour and 8-hour region-wide statistics are not directly calculated from site-averaged time series, but from the statistics of each station and then arithmetically averaged. The time series for individual stations for these five regions have also been plotted and are a vailable via the ftp site and filename indicated in the Appendix.

Figure 2-9 shows the hourly averaged CO, ozone, NO and NO 2 for the Bay Area region. Predicted CO concentrations are over-predicted for the entire simulation period. Ozone concentrations are significantly over-predicted for the entire simulation period although predictions gradually begin to match observations b y the last day. Predicted NO concentrations generally match the observed concent rations but are under-predicted for the last two days of the simulation period. The mo del predicts NO 2 concentrations that have a significant diurnal variation, where the NO 2 increases in the morning traffic hours and in the evening hours when the atmospheric bound ary layer becomes less turbulent. However, the regional averaged observation does not exhibit this NO 2 variation.MODEL PERFORMANCE EVALUATION

56 Figure 2-10 shows the hourly averaged CO, ozone, NO and NO 2 for the Sacramento region. Predicted CO concentrations generally agre e with the observations. Ozone concentrations show excellent agreement with observ ations for all days. The NO concentrations generally agree with the observation s and clearly reproduce the early morning traffic peak. The model captures the magni tude as well as the diurnal variation of the NO 2

.

Figure 2-11 shows the hourly averaged CO, ozone, NO and NO 2 for the Central San Joaquin Valley region. Predicted CO concentrations are under-predicted for the entire simulation period. Ozone concentrations show excel lent agreement with observations for all the days. The NO concentrations also gener ally agree with the observations and clearly reproduced the early morning traffic peak. The model captures the diurnal variation of the NO 2 , but predicted concentrations are generally less t han the observed
values.

Figure 2-12 shows the hourly averaged CO, ozone, NO and NO 2 for the Southern San Joaquin Valley region. Predicted CO concentrations are under-predicted for the entire simulation period. Ozone concentrations show excel lent agreement with observations for all the days. The NO concentrations also gener ally agree with the observations and clearly reproduced the early morning traffic peak. The model captures the magnitude as well as the diurnal variation of the NO 2

.

Figure 2-13 shows the hourly averaged CO, ozone, NO and NO 2 for the San Joaquin Valley North region. Predicted CO concentrations g enerally match the observed concentrations through July 29 th , but, the sharp peaks in the observations later in the episode are not captured in the simulated concentra tions. These sharp peaks, however, are likely from some high-emitters near th e observation site and cannot be captured by a regional scale model. Ozone concentr ations generally agree with the observations, but some days are over-predicted. Th e NO concentrations also generally agree with the observations and clearly reproduced the early morning traffic peak. The predicted NO 2 concentrations are generally higher than the observ ations.

Figures 2-14 and 2-15 show the correlation between the unpaired peak and the mean normalized bias for each individual site in a regio n. Through experience it has been observed that, if the NB statistical metric is sati sfied, then the GE statistic is also satisfied. Hence, GE statistical results are not p resented.

The results shown above were generated using NOAA’s ‘placeholder’ MM5 meteorology. Since the first ‘placeholder’ version of the NOAA meteorology, CARB and NOAA have been working together to improve the mete orological model performance. A more recent NOAA meteorology, as described in the previous meteorology section, was developed and used as an alternative input to a ir quality modeling simulations. The model performance results using this alternative me teorological field are provided in Tables 2-8 and 2-9 for 1-hour and 8-hour averaging periods, respectively. Using the updated meteorology inputs slightly degrades the 1- hour model performance, butMODEL PERFORMANCE EVALUATION

57 improves the 8-hour model performance. However, as mentioned in section 1.2.2, this alternative wind field as well as the associated me teorological model options and inputs are still in draft form and continue to be investig ated.MODEL PERFORMANCE EVALUATION

58 Table 2-6. 1-hour ozone performance by each region over the July 29-August 2, 2000 modeling period.

Region ID Region Name 7/29/2000 7/30/2000 7/31/2000 8/1/2000 8/2/2000 Total

2 North Coast -99 0 -99 -99 -99 0

3 BAAQMD 1 0 0 0 1 2

4 MBAQMD 0 0 0 1 1 2

5 Sacramento Valley North 1 1 1 1 1 5

6 Sacramento Region 1 1 0 1 1 4

7 SJVAPCD Central 1 1 1 1 1 5

8 SJVAPCD Kern 1 1 1 1 0 4

9 SJVAPCD North 1 0 0 0 1 2

10 Sierra Nevada Central 0 1 0 1 1 3

11 SJVAPCD Above 3000 ft 1 1 1 1 1 5

12 South Central Coast 1 1 0 0 0 2

13 Sierra Nevada North 1 1 1 1 1 5

14 Desert 1 1 0 0 0 2

15 Nevada -99 -99 -99 -99 -99 0

Total: 10 9 5 8 9 41

Table 2-7. 8-hour ozone performance by each region over the J uly 29-August 2, 2000 modeling period.

Region ID Region Name 7/29/2000 7/30/2000 7/31/2000 8/1/2000 8/2/2000 Total

2 North Coast -99 -99 -99 -99 -99 0

3 BAAQMD 1 0 0 0 1 2

4 MBAQMD -99 0 0 1 1 2

5 Sacramento Valley North 1 1 1 1 0 4

6 Sacramento Region 1 0 0 1 1 3

7 SJVAPCD Central 1 1 1 1 1 5

8 SJVAPCD Kern 1 1 1 1 1 5

9 SJVAPCD North 1 0 0 0 1 2

10 Sierra Nevada Central 1 1 0 1 0 3

11 SJVAPCD Above 3000 ft 1 1 1 1 0 4

12 South Central Coast 0 0 0 0 1 1

13 Sierra Nevada North 1 1 1 1 1 5

14 Desert 1 1 1 1 1 5

15 Nevada -99 -99 -99 -99 -99 0

Total: 10 7 6 9 9 41MODEL PERFORMANCE EVALUATION

59 Figure 2-9. Hourly averaged CO, ozone, NO and NO 2 for the Bay Area region over the July 27 – August 2, 2000 modeling period.MODEL PERFORMANCE EVALUATION

60 Figure 2-10. Hourly averaged CO, ozone, NO and NO 2 for the Sacramento region over the July 27 – August 2, 2000 modeling period.MODEL PERFORMANCE EVALUATION

61 Figure 2-11. Hourly averaged CO, ozone, NO and NO 2 for the Central San Joaquin Valley region over the July 27 – August 2, 2000 mod eling period.MODEL PERFORMANCE EVALUATION

62 Figure 2-12. Hourly averaged CO, ozone, NO and NO 2 for the Southern San Joaquin Valley region over the July 27 – August 2, 2000 mod eling period.MODEL PERFORMANCE EVALUATION

63 Figure 2-13. Hourly averaged CO, ozone, NO and NO 2 for the Northern San Joaquin Valley region over the July 27 – August 2, 2000 mod eling period.MODEL PERFORMANCE EVALUATION

64 1-ho ur O3 Statistics, B ay A rea

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

Unpair Peak Ratio

1-ho ur O3 Statistics, Sacramento

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

Unpair Peak Ratio

1-ho ur O3 Statistics, C entral SJV

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

Unpair Peak RatioMODEL PERFORMANCE EVALUATION

65 Figure 2-14. Unpaired peak ratio vs. normalized bias for 1-hou r ozone for the July 29 – August 2, 2000 modeling period. Each dot represents one-day results for an individual
site. 1-ho ur O3 Statistics, So uthern SJV

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

Unpair Peak Ratio

1-ho ur O3 Statistics, N o rthern SJV

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

Unpair Peak RatioMODEL PERFORMANCE EVALUATION

66 8-ho ur O3 Statistics, B ay A rea

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

Unpair Peak Ratio

8-ho ur O3 Statistics, Sacramento

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

Unpair Peak Ratio

8-ho ur O3 Statistics, C entral SJV

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

Unpair Peak RatioMODEL PERFORMANCE EVALUATION

67 Figure 2-15. Unpaired peak ratio vs. normalized bias for 8-hou r ozone for the July 29 – August 2, 2000 modeling period. Each dot represents one-day results for an individual
site. 8-ho ur O3 Statistics, So uthern SJV

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

Unpair Peak Ratio

8-ho ur O3 Statistics, N o rthern SJV

-0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

Unpair Peak RatioMODEL PERFORMANCE EVALUATION

68 Table 2-8. 1-hour ozone performance by each region over the July 27-August 2, 2000 modeling period using the most recent NOAA meteorol ogy (under ARB investigation)

Region ID Region Name 7/29/2000 7/30/2000 7/31/2000 8/1/2000 8/2/2000 Total

2 North Coast -99 0 -99 -99 -99 0

3 BAAQMD 1 1 1 1 1 5

4 MBAQMD 1 1 1 1 1 5

5 Sacramento Valley North 1 1 1 1 1 5

6 Sacramento Region 0 1 1 0 1 3

7 SJVAPCD Central 0 1 1 1 1 4

8 SJVAPCD Kern 0 0 1 0 0 1

9 SJVAPCD North 1 1 1 1 1 5

10 Sierra Nevada Central 0 1 1 0 0 2

11 SJVAPCD Above 3000 ft 0 0 0 1 1 2

12 South Central Coast 1 1 0 0 0 2

13 Sierra Nevada North 1 1 1 0 0 3

14 Desert 0 0 0 0 0 0

15 Nevada -99 -99 -99 -99 -99 0

Total: 6 9 9 6 7 37

Table 2-9. 1-hour ozone performance by each region over the J uly 27-August 2, 2000 modeling period using the most recent NOAA meteorol ogy (under ARB investigation)

Region ID Region Name 7/29/2000 7/30/2000 7/31/2000 8/1/2000 8/2/2000 Total

2 North Coast -99 -99 -99 -99 -99 0

3 BAAQMD 1 0 1 1 1 4

4 MBAQMD -99 1 1 1 1 4

5 Sacramento Valley North 1 1 1 1 1 5

6 Sacramento Region 1 1 1 1 1 5

7 SJVAPCD Central 1 1 1 1 1 5

8 SJVAPCD Kern 1 1 1 1 1 5

9 SJVAPCD North 1 0 0 1 1 3

10 Sierra Nevada Central 1 1 1 0 0 3

11 SJVAPCD Above 3000 ft 0 1 1 1 1 4

12 South Central Coast 1 1 1 1 1 5

13 Sierra Nevada North 1 1 1 1 0 4

14 Desert 1 0 1 1 1 4

15 Nevada -99 -99 -99 -99 -99 0

Total: 10 9 11 11 10 51MODEL PERFORMANCE EVALUATION

69 

### 3 Conclusion – Cumulative 1-hour and 8-hour Days

Per the prior discussion of performance statistics and analyses of model performance, Tables 3-1 and 3-2 provide a summary of 1-hour and 8-hour episode days that meet model performance criteria for both episodes. As n oted previously with regard to these types of tables, for days that a region meets the a ssociated performance criteria a value of 1 is assigned. A value of 0 means that region d oesn’t meet the criteria for the respective day and, if there is no model simulated concentrations above 60ppb, then -99 is assigned.

As is illustrated in the tables, of 10 possible day s per region (5 per episode), 2-10 days are available for each region based on 1-hour metri cs and, with the exception of the North Coast (1 day), 1-9 days are available based o n 8-hour metrics.MODEL PERFORMANCE EVALUATION

70 Table 3-1. Combined Number of Available Days Per Subregion U nder 1-hour Metrics

Region Name July
1999 July-Aug

2000 Total
North Coast 2 0 2

BAAQMD 4 2 6
MBAQMD 2 2 4 Sacramento Valley North 3 5 8 Sacramento Region 5 4 9

SJVAPCD Central 5 5 10

SJVAPCD Kern 1 4 5

SJVAPCD North 2 2 4

Sierra Nevada Central 1 3 4

SJVAPCD Above 3000 ft 3 5 8

South Central Coast 3 2 5

Sierra Nevada North 3 5 8

Desert 0 2 2

Nevada 2 0 2

Total 36 41 77

Table 3-2. Combined Number of Available Days Per Subregion U nder 8-hour Metrics

Region Name July
1999 July-Aug

2000 Total
North Coast 1 0 1
BAAQMD 1 2 3
MBAQMD 1 2 3 Sacramento Valley North 4 4 8 Sacramento Region 5 3 8

SJVAPCD Central 3 5 8

SJVAPCD Kern 4 5 9

SJVAPCD North 3 2 5

Sierra Nevada Central 2 3 5

SJVAPCD Above 3000 ft 3 4 7

South Central Coast 3 1 4

Sierra Nevada North 3 5 8

Desert 1 5 6

Nevada 3 0 3

Total 37 41 78MODEL PERFORMANCE EVALUATION

71 

### 4 Results for Model Sensitivity Tests

(Work in Progress.)MODEL PERFORMANCE EVALUATION

72 

### 5 References

Air Sciences, Inc. (2002) 1996 Fire Emissions Inv entory. Draft Final Report. Lakewood, CO. At: http:/www.wrapair.org/forums/FEJF1/emissions/FEJF19 96EIReport_021208.pdf

Battye, W. and Battye, R. 2002. Development of emis sions inventory methods for wildland fires. U.S. EPA Contract No. 68-D-98-046. On-line link: http://www.epa.gov/ttn/chief/ap42/ch13/related/c13s 01.html

Benjamin, M., Sudol, M., Bloch, L. and A. Winer. 19 96. Low-emitting urban forests: a taxonomic methodology for assigning isoprene and mo noterpene emission rates. Atmospheric Environment. 30 (9): 1437-1452.

Benjamin, M., Sudol, M., Vorsatz, D. and A. Winer. 1997. A spatially and temporally resolved biogenic hydrocarbon emissions inventory f or the California south coast air basin. Atmospheric Environment. 31 (18): 3087-3100.

CARB. 1995. "Sacramento Area Modeling Analysis for the 1994 State Implementation Plan". California Air Resources Board. Technical Su pport Division. Sacramento, CA 95814. April, 1995.

CARB. 2005 ARB Speciation Profiles : (a) California Air Resources Board, "Identification of Volatile Organic Compound Specie s Profiles: ARB Speciation Manual, Second Edition, Volume 1 of 2," August 1991; and (b ) subsequent revisions. The latest ARB speciation profiles are available from ARB's we b site at http://www.arb.ca.gov/emisinv/speciate/speciate.htm

.

CARB. 2005. http://www.arb.ca.gov/airways/ccos/ccos.htm

CARB. “Extension and input refinement to the ARB w ildland fire emissions estimation model,” ARB agreement number 00-729

Carter, W.P.L, 2000. “Documentation of the SAPRC-99 Chemical Mechanism for VOC Reactivity Assessment,” Final Report to California Air Resources Board, Contract No. 92-329, and (in part) 95-308,May 8, 2000, http://pah.cert.ucr.edu/~carter/reactdat.htm

Chang, J. S., J. Shengxin, L. Yinghong, M. Beauharn ois, L., Cheng-Hsuan, and H. HoChun. 1997. The SARMAP Air Quality Model. Planning and Technical Support Division. California Air Resources Board. Sacramento, CA. Apr il, 1997.

Coe, D. 2003. “Assisstance to Rural Counties with Development of Area Source Emission Inventories.” Technical Memoranda. Contra ct No. 00-24CCOS. San JoaquinMODEL PERFORMANCE EVALUATION

73 Valleywide Air Pollution Study Agency and Californi a Environmental Protection Agency
– Air Resources Board. At: www.sonomatech.com/ccosii/ ; user name: “ccosii”; password: “emissions”

Cofer, W., Levine, J., Winstead, E. and Stocks, B. 1991. Trace gas and particulate emissions from biomass burning in temperate ecosyst ems. In: Global Biomass Burning: Atmospheric, Climatic and Biospheric Implications. Levine, J., editor. MIT Press.

Davis, F. W., P. A. Stine, D. M. Stoms, M. I. Borch ert and A. D. Hollander. 1995. Gap analysis of the actual vegetation of California –1. The southwestern region. Madrono 42:
40-78.

Earth Tech. 2000. A User's Guide for the CALMET Met eorological Model (Version 5). Earth Tech. Concord, MA. 01742. January, 2000.

Einfeld, W., Ward, D. and Hardy, C. 1991. Effects o f fire behavior on prescribed fire smoke characteristics: a case study. In: Global Bio mass Burning: Atmospheric, Climatic and Biospheric Implications. Levine, J., editor. MI T Press.

ENVIRON. 2001. The program 'mm5camx_v3'. Personal communication. ENVIRON. Novato, CA. 95945. August, 2001.

ENVIRON. 2004. User's Guide. Comprehensive Air Qual ity Model With Extensions (CAMx) v4.10S. ENVIRON. Novato, CA 95945. August, 2 004.

ENVIRON. 2005a. Personal communication. ENVIRON. Novato, CA. 95945. February,
2005.

ENVIRON. 2005b. The Development of a Photochemical Modeling System for the Bay Area Air Quality Management District to Support Ong oing Air Quality Analysis. BAAQMD. San Francisco, CA. January, 2005.

Fairley, D. and DeMandel, R. 1996. An Analysis of SARMAP Episode Day Representativeness. Final report. Prepared for th e SARMAP Data Analysis Project by the Bay Area Air Quality Management District, San F rancisco, CA, Faust, B. C., Photochemistry of clouds, fogs and aerosols, Enviro n. Sci. Technol., 28A, 217, 1994.

Fujita, E., D. Campbell, R. Keilar, and J. Bowen. 2 001. Central California Ozone Study – Volume III: Summary of Field Operations. Final Repo rt. DRI. California Air Resources Board. Sacxramento, CA 95814. February, 2001.

Fujita, E., D. Campbell, R. Keisler, and J. Bowen. 1999. "Central California Ozone Study – Volume III. Summary of Field Operations". Plann ing and Technical Support Division, California Air Resources Board. Sacramento, CA. 958 14. November, 1999.

Funk, T., Stiefer, P., Chinkin, L. 2001 “Developmen t of Gridded Spatial Allocation Factors for the State of California”. Final Report. Contract No. 00-24CCOS. SanMODEL PERFORMANCE EVALUATION

74 Joaquin Valleywide Air Pollution Study Agency and C alifornia Environmental Protection Agency – Air Resources Board.

Grell, A. G., J. Dudhia, and D. R. Stauffer. 1994. "A Description of the Fifth-Generation Penn State/NCAR Mesosclae Model (MM5)". NCAR Tecnic al Note NCAR/TN398+STR. National Center for Atmospheric Research. Boulder, CO. June, 1994.

Griffith, D., Mankin, W., Coffey, M., Ward, D., Rie bau, A. 1991. FTIR remote sensing of biomass burning emissions of CO2, CO, CH4, CH2O, NO , NO2, NH3, and N2O. In: Global Biomass Burning: Atmospheric, Climatic and B iospheric Implications. Levine, J.,
editor. MIT Press.

Guenther, A. B., P. R. Zimmerman, P. C. Harley, R. K. Monson and R. Fall. 1993. Isoprene and monoterpene emission rate variability – model evaluations and sensitivity analyses. Journal of Geophysical Research. 98(D7): 12609-12617.

Guenther, A. B. R. K. Monson and R. Fall. 1991. Iso prene and monoterpene emission rate variability: observations with eucalyptus and emission rate algorithm development. Journal of Geophysical Research. 96: 10799-10808.

Hayes, T. P., J. J. R. Kinney, and N. J. M. Wheeler . 1984. "California Surface Wind Climatology". Planning and Technical Support Divis ion. California Air Resources Board. Sacramento, CA. 95812. June, 1984.

Karlik, J. and A. McKay. 1999. Development of metho dology and databases for estimating leaf masses in California airsheds. Fina l Report. Contract No. 97-719. State of California Air Resources Board. Sacramento, CA.

Karlik, J. 2002. Validation of databases for modeli ng biogenic volatile organic compound emissions in central California. Draft Final Report . Contract No. 00-16CCOS. San Joaquin Valleywide Air Pollution Study Agency and C alifornia Environmental Protection Agency – Air Resources Board.

Harley, P., V. Fridd-Stroud, J. Greenberg, A. Guent her and P. Vasconcellos. 1996. Emission of 2-methyl-3-buten-2-ol by pines: A poten tially large natural source of reactive carbon to the atmosphere. Journal of Geoph ysical Research. 103: 25479-
25486.

Horie, Y., Sidawi, S. and R. Ellefsen. 1990. Invent ory of leaf biomass and emission factors for vegetation in California’s south coast air basin. Final Technical Report III-C. South Coast Air Quality Management District. Diamon d Bar, CA.

Hunman, R.C., D. J. Jacob, O.R. Cooper, M. J. Evans , C.L. Heald, R.J. Park, F. Fehsenfeld, F. Flock, J. Holloway, G. Hubler, K., K ita, M. Koike, Y. Kondo, A. Neuman, J. Nowak, S. Oltmans, D. Parrish, J. M Roberts, abd T. Ryerson. 2004. OzoneMODEL PERFORMANCE EVALUATION

75 production in transpacific Asian pollution plumes a nd implications for ozone air quality in California. J. Geophys Research, 109:D23S10.

Kleeman, M. 2000. “Updating Point Source Emissions Inventories in the Sacramento Valley and Mountain counties Using Student Assistan ts” Contract No. 00-22CCOS.

Lam, T. Niemeier, D., Jierranaitanakit, K. 2002. “E stimation of Hourly Allocation Factors for the Central California Ozone Study Using State- wide Model Data and Real Time Traffic Data.” Final Draft Report. Contract No. 00- 04PM. San Joaquin Valleywide Air Pollution Study Agency and California Environmental Protection Agency – Air
Resources Board.

Lerhman, D., B. Knuth, and D. Fairly. 2003. "Charac terization of the CCOS 2000 Measurment Period". T&B Systems Contract No. 01-2CC OS. California Air Resources Board -- Planning and Technical Support Division. S acramento, CA 95814. September,
2003.

Lobert, J., Scharffe, D., Hao, W.-M., Kuhlbusch, T. , Seuwen, R., Warneck, P., and Crutzen, P. 1991. Experimental evaluation of biomas s burning emissions: nitrogen and carbon containing compounds. In: Global Biomass Bur ning: Atmospheric, Climatic and Biospheric Implications. Levine, J., editor. MIT Pr ess.

McRae, G.J, W.R. Goodin, and J.H. Seinfeld. 1982, M athematical Modeling of Air Pollution. EQL Report No. 18. Planning and Technica l Support Division, CARB. Sacramento, CA 95814. April, 1982.

McPherson, E. G. 1998. Structure and sustainabilit y of Sacramento’s urban forest. Journal of Arboriculture. 24 (4): 174-190.

Mesinger, F., Z. I. Janjic, S. Nickovic, D. Gavrilo v, and D. G. Deaven, 1988: The stepmountain coordinate: model description and performa nce for cases of Alpine lee cyclogenesis and for a case of an Appalachian redev elopment. Mon. Wea . Rev., 116,
1493-1518.

Myrup, L.O., R.G. Flocchini, and D. Ewell (1981) Tr ansport of Atmospheric Aerosols Above the Sierra Nevada Slopes, Final Report, Contr act A4-127-32. Prepared by University of California Davis, January 15, 1989.

Neff, W.D., J. Jorden. J. Gaynor, D. Wolfe, W. Eckl und, D. Carter, and K. Gage (1991) The use of 915 MHz radar wind profilers in complex terrain and regional air quality studies. Preprints, Seventh Joint Conference on App lications of Air Pollution Meteorology with AWMA. 14-18 January 1991, New Orle ans, LA, American Meteorological Society. Boston, MA., J230-J233.MODEL PERFORMANCE EVALUATION

76 Newchurch, M. J., M. A. Ayoub, S. Oltmans, B. Johns on, and F.J. Schmidlin. 2003.
Vertical distribution of ozone at four sites in the United States. J. of Geophys. Research, 108(D1):ACH 9-1,17.

Nikolov, N. T. 1999. 1-km resolution database of ve getation leaf area index and canopy clumping factor for the western U.S.A. Final Report , U.S.D.A. Forest Service Agreement No. PSW-99-001-RJVA. N&T Services. Oak Ridge, TN.

Nowak, D. J. 1991. Urban forest development and str ucture: Analysis of Oakland, California. PhD dissertation. University of Califor nia, Berkeley, CA.

Pielke, R. A., W. R. Cotton, R. L. Walko, C. J., Tr emback, W. A. Lyons, L. D. Grasso, M. E. Nicholls, M. D. Moran, D. A. Wesley, T. J. Le e, and J. H. Copeland, 1992: A comprehensive meteorological modeling system – RAMS . Meteorology and Atmospheric Physics, 49, 69-91.

Pielke, R. A. and M. Uliasz. 1998. Use of meteorolo gical models as input to Regional and mesoscale air quality models -- limitations an d strengths. 1998. Atmospheric Environment 32:1455-1466.

Pun, B. K., J. F. Louis, and C. Seigneur. 1998. "A Conceptual Model for ozone formation in the San Joaquin Valley". AER Document No. CP049-1-98. Pacific Gas and Electric Co. San Ramon, CA 94583. December, 1998.

Radke, L., Hegg, D., Hobbs, P., Nance, D., Lyons, J ., Laursen, K., Weiss, R., Riggan, P., and Ward, D. 1991. Particulate and trace gas e missions from large biomass fires in north America. In: Global Biomass Burning: Atmosphe ric, Climatic and Biospheric Implications. Levine, J., editor. MIT Press.

Reinhardt, E., Keene, R. and Brown, J. 1997. First Order Fire Effects Model: FOFEM 4.0 User’s Guide. USDA Forest Service, Intermountai n Research Station, General Technical Report INT-GTR-344.

Seaman, N.L., D. R. Stauffer, and A. M. Lario-Gibbs . 1995. A multiscale fourdimensional data assimilation system applied to the San Joaquin Valley during SARMAP. Part I: Modeling design and basic model per formance characteristics. J. of Applied Meteorology 34:1739-1761.

SCAQMD. 2003. "2003 Air Quality Management Plan -- Modeling and Attainment Demonstrations." Final Report -- Appendix V. South Coast Air Quality Management District. Diamond Bar, CA 91765. August, 2003.

Scire J.S., R.J. Yamartino, S.R. Hamma, G.R. Carmic hal, and Y.S. Chang. 1989. CALGRID: A Mesoscale Photochemical Grid Model. Vo lume I: Model FormulationMODEL PERFORMANCE EVALUATION

77 Document. Planning and Technical Support Division, CARB. Sacramento, CA 95814. June, 1989.

Sidawi, S. and Y. Horie. 1992. Leaf biomass density for urban, agricultural and natural vegetation in California’s San Joaquin Valley. Fina l Report. San Joaquin Valley Air Pollution Study Agency.

SJVAQMD. 1994. "The Ozone Attainment Plan". San Jo aquin Valley Unified Air Pollution Control District. Fresno, CA. November, 1994

Skaramarock, W. C., J. B. Klemp, J. Dudhia, G. O. G ill, D. M. Barker, W. Wang, and J. G. Powers, 2005: A description of the Advanced Rese arch WRF Version 2. NCAR Technical Note NCAR/TN-468+STR, June 2005.

Smith, T.B., D.E. Lehrman, D.D Reible, and F.H. Sha ir (1981) The origin and fate of airborne pollutnats within the San Joaquin Valley, Rep. MRI FR-1838, Meteorology Research, Inc., Altadena, CA. Prepared for the Cali fornia Air Resources Board.

Smith, T. B. (1994) Ozone Episode Forecasting in th e San Joaquin Valley. Planning and Managing Regional Air Quality Modeling and Meas urement Studies: A Perspective Through the San Joaquin Valley Air Quality Study an d AUSPEX, ed. by P. A. Solomon. Published by Lewis Publishers, Chelsea, MI in conju nction with Pacific Gas and Electric Company, San Ramon, CA, pp. 507-528.

Susott, R., Ward, D., Babbitt, R. and Latham, D. 19 91. The measurement of trace gass
emissions and combustion characteristics for a mass fire. In: Global Biomass Burning: Atmospheric, Climatic and Biospheric Implications. Levine, J., editor. MIT Press.

Tonneson, G. 2003. Personal Communication. CE-CERT. U.C. Riverside, Riverside, CA. April, 2003.

Ward, D. and Hardy, C. Smoke emissions from wildlan d fires. 1991. Environment International. V17. Pp.117-134.

Wilkinson, J. 2003, “Development of the California Integrated Transportation Network (ITN)”, Draft Final Report. Contract No. 93-2PM. S an Joaquin Valleywide Air Pollution Study Agency and California Environmental Protectio n Agency – Air Resources Board.

Winer, A., Karlik, J. and J. Arey. 1998. Biogenic h ydrocarbon inventories for California: generation of essential databases. Final Report. Co ntract No. 95-309. State of California Air Resources Board. Sacramento, CA.

Winer, A. and Karlik, J. 2001. Development and vali dation of databases for modeling biogenic hydrocarbon emissions in California’s airs heds. Final Report. Contract No. 97-MODEL PERFORMANCE EVALUATION

78 320. California Environmental Protection Agency – A ir Resources Board. Sacramento, CA.

USEPA. 1990. Carbon Bond IV : Morris, R.E. and Meyers, T. E., "User's Guide fo r the Urban Airshed Model," Volume I, Appendix 1: "The Ca rbon Bond IV Chemical Kinetics Mechanism for Urban and Regional Scale Computer Mod eling," EPA-450/4-90-007A,
June 1990.

USEPA. 1991. Guideline for the Regulatory Applicati on of the Urban Airshed Model. OAQPS, USEPA. Research Triangle Park, NC 27711.

USEPA, 1991. "Guideline for Regulatory Application of the Urban Airshed Model". EPA450/4-91-013. USEPA, OAQPS. Research Triangle Park, NC 27711. July, 1991.

USEPA. 1999. "Science Algorithms of the EPA Models- 3 Community Multiscale Air Quality Model (CMAQ) Modeling System. USEPA. EPA/60 0/R-99/030. Office of Research and Development. Washington, D. C. 20460. March, 1999.

USEPA, 2005. "Guidance on the Use of Models and Oth er Analysis in the Attainment Demonstration for the 8-Hour Ozone NAAQS". USEPA, O AR/OAQPS, Research Triangle Park, NC 2771. October, 2005.6 Appendix – Information Available for DownloadingAnonymous ftp to eos.arb.ca.gov, then change directories to /pub/outgoing/model_protocol2Model Episode Task/ItemComplete? File Name on eos.arb.ca.govAQ 1999Regional daily tabulation of 1-hour performance results (1,0,-99) Y Included in documentAQ 1999Station-specific tabulation of 1-hour performance results (1,0,-99) Y 1999 ozone performance byeach station.docAQ 1999Station-specific tabulation of 1-hour performance STATISTICS Y 1999.050c.1hO3.docAQ 1999Station-specific time-series plots of 1-hourozone Y 1999.pdf.zipAQ 1999Station-specific time-series plots of 1-hour precursors (CO, NO,NO2); ok to combine w/ ozoneY 1999.pdf.zipAQ 1999Regional daily tabulation of 8-hour performance results (1,0,-99) Y Included in documentAQ 1999Station-specific tabulation of 8-hour performance results (1,0,-99) Y 1999 ozone performance byeach station.docAQ 1999Station-specific tabulation of 8-hour performance STATISTICS Y 1999.050c.8hO3.docAQ 2000Regional daily tabulation of 1-hour performance results (1,0,-99) Y Included in documentAQ 2000Station-specific tabulation of 1-hour performance results (1,0,-99) Y 2000 ozone performance byeach station.docAQ 2000Station-specific tabulation of 1-hour performance STATISTICS Y 2000.050c.1hO3.docAQ 2000Station-specific time-series plots of 1-hourozone Y 2000.pdf.zipAQ 2000Station-specific time-series plots of 1-hour precursors (CO, NO,NO2); ok to combine w/ ozoneY 2000.pdf.zipAQ 2000Regional daily tabulation of 8-hour performance results (1,0,-99) Y Included in documentAQ 2000Station-specific tabulation of 8-hour performance results (1,0,-99) Y 2000 ozone performance byeach station.docAQ 2000Station-specific tabulation of 8-hour performance STATISTICS Y 2000.050c.8hO3.docMODEL PERFORMANCE EVALUATION80Anonymous ftp to eos.arb.ca.gov, then change directories to /pub/outgoing/model_protocol2Model Episode Task/ItemComplete? File Name on eos.arb.ca.govMet 1999Wind Speed Statistics per Performance Region (RMSE< 2 m/s;Bias::< ±0.5 m/s; IOA:3 0.6)Y Included in documentMet 1999Wind Direction Statistics per Performance Region(Gross Error:< 30deg; Bias:< ±10 deg)Y Included in documentMet 1999Temperature Statistics per Performance Region(GrossError:< 2 K;Bias:< ±0.5 K; IOA3 0.8)Y Included in documentMet 1999Station-specific, time-series plots of hourly mean air temperature Y July1999.met.regionN.pdf,where N is region numberMet 1999Station-specific, time-series plots of hourly mean wind speeds. Y same as aboveMet 1999Domain-wide spatial plots of hourly wind vectors YJuly1999_surface_hourly_wind.EEEE.pdf, whereEEEE is simulation IDMet 1999Domain-wide, spatial plots of hourly air temperatures NMet 2000Wind Speed Statistics per Performance Region (RMSE< 2 m/s;Bias::< ±0.5 m/s; IOA:3 0.6)Y Included in documentMet 2000Wind Direction Statistics per Performance Region(Gross Error:< 30deg; Bias:< ±10 deg)Y Included in documentMet 2000Temperature Statistics per Performance Region(GrossError:< 2 K;Bias:< ±0.5 K; IOA3 0.8)Y Included in documentMet 2000Station-specific, time-series plots of hourly mean air temperature Y <<to be posted>>Met 2000Station-specific, time-series plots of hourly mean wind speeds. Y <<to be posted>>Met 2000Domain-wide spatial plots of hourly wind vectors Y Included in documentMet 2000Domain-wide, spatial plots of hourly air temperatures N