773 San Marin Drive, Suite 2115, Novato, CA 94998 415.899.0700 

Draft Final Report

MODEL PERFORMANCE EVALUATON FOR THE 

JUNE-JULY 2006 OZONE EPISODE FOR THE 

DENVER 8-HOUR OZONE STATE IMPLEMENTATION PLAN 

Prepared for: 

Regional Air Quality Council 

1445 Market Street, Suite 260 

Denver, Colorado 80202 

Prepared by: 

Ralph Morris 

Ed Tai 

Tanarit Sakulyanontvittaya 

ENVIRON International Corporation 

773 San Marin Dr. Suite 2115 

Novato, CA 94998 

Dennis McNally 

Cyndi Loomis 

Alpine Geophysics, LLC 

7341 Poppy Way 

Arvada, Colorado 80007 

August 29, 2008 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\TOC.doc August, 2008 i 

TABLE OF CONTENTS 

Page 

EXECUTIVE SUMARY ........................................................................................................ ES-1 

Overview of Approach.........................................................................................................ES-1 

Model Performance Evaluation ...........................................................................................ES-1 

Conclusions of Model Performance Evaluation ................................................................ES-10 

1. INTRODUCTION................................................................................................................ 1-1 

1.1 Background................................................................................................................ 1-1 

1.2 Purpose....................................................................................................................... 1-2 

1.3 Overview of Modeling Approach .............................................................................. 1-2 

1.4 Emissions Modeling Approach.................................................................................. 1-5 

2. OZONE OPERATIONAL MODEL PERFORMACE EVALUATION ........................ 2-1 

2.1 Introduction................................................................................................................ 2-1 

2.2 Overview of Model Performance Evaluation Approach ........................................... 2-6 

2.3 Operational Ozone Model Performance for the 

 June-July 2006 Modeling Period ............................................................................... 2-7 

2.4. Detailed Performance for Three 3-Day Episodes .................................................... 2-18 

2.5 Ozone Operational Model Performance Evaluation Conclusions ........................... 2-54 

3. DIAGNOSTIC MODEL PERFORMANCE EVALUATION......................................... 3-1 

3.1 Introduction................................................................................................................ 3-1 

3.2 Ozone Precursor Model Performance........................................................................ 3-1 

3.3 Ozone Model Performance Aloft............................................................................. 3-26 

3.4 PM Model Performance Evaluation......................................................................... 3-30 

3.5 Diagnostic Model Performance Conclusions........................................................... 3-36 

 

4. REFERENCES..................................................................................................................... 4-1 

APPENDICES 

Appendix A: Comparison of Predicted and Observed Spatial Distributions of the Daily 

Maximum 8-Hour Ozone Concentrations in the Denver Area during June-July 

2006 for the Final CAMx 2006 Base Case (Run 17) Simulation
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\TOC.doc August, 2008 ii 

TABLES 

Table ES-1. Percent of the monitor-days that the model predicted 

 daily maximum 8-hour ozone concentrations near the 

 monitor is within ±20% of the observed value (total monitor-days = 1008).....ES-3 

Table 1-1. Lambert Conformal Projection (LCP) definition for the 

 Denver 36/12/4 km modeling grids ..................................................................... 1-4 

Table 1-2. Grid definitions for MM5, Emissions and CAMx modeling domains ................ 1-4 

Table 1-3. Summary of SMOKE configuration parameters and data inputs. ....................... 1-7 

Table 1-4. 2006 emissions of all criteria pollutants by county for 

 O&G emissions in the D-J Basin....................................................................... 1-14 

Table 1-5. Summary of CONCEPT parameters and data inputs......................................... 1-15 

Table 1-6. Annual county-level total average weekday VMT from the TDM.................... 1-19 

Table 1-7. BPR speed adjustment coefficients used in estimating congested speeds......... 1-20 

Table 1-8. FHWA Roadway functional classification ........................................................ 1-21 

Table 1-9. Vehicle classifications ....................................................................................... 1-21 

Table 1-10. Fractional Allocation of FHWA Vehicle Classes to 

 MOBILE5 Vehicle Classes................................................................................ 1-21 

Table 2-1. Ozone monitors operating within and near the DMA 

 during 2002-2007................................................................................................. 2-3 

Table 2-2. EPA’s 1-hour ozone performance goals (EPA, 1991)......................................... 2-4 

Table 2-3. 8-hour ozone model performance goal comparing 

 observed daily maximum 8-hour ozone concentrations 

 with predicted values “near the monitor” (EPA, 1999). ...................................... 2-5 

Table 2-4. Model performance goals and criteria for components 

 of fine particle mass............................................................................................. 2-5 

Table 2-5. Definitions of statistical performance measures used in the 

 model evaluation.................................................................................................. 2-6 

Table 2-6. Percent of the monitor-days that the model predicted daily 

 maximum 8-hour ozone concentrations near the monitor is 

 within ±20% of the observed value (total monitor-days = 1008) ...................... 2-16 

Table 2-7. Final CAMx Run 17 2006 base case simulation daily 1-hour 

 and 8-hour statistical performance measures for the June 17-19, 2006 

 episode and comparison with EPA performance goals...................................... 2-19 

Table 2-8. Final CAMx Run 17 2006 base case simulation daily 1-hour 

 and 8-hour statistical performance measures for the July 13-15, 2006 

 episode and comparison with EPA performance goals...................................... 2-29 

Table 2-9. Final CAMx Run 17 2006 base case simulation daily 1-hour 

 and 8-hour statistical performance measures for the July 27-29, 2006 

 episode and comparison with EPA performance goals...................................... 2-41 

Table 3-1a. Locations of monitoring sites where VOC samples 

 were collected during the summer of 2006.......................................................... 3-7 

Table 3-1b. Sampling schedule and times for summer of 2006 VOC 

 sampling study in the DMA................................................................................. 3-7 

Table 3-2. Comparison of predicted and observed morning CB05-VOC/NOx 

 ratios at the downtown Denver CAMP monitoring site..................................... 3-10 

Table 3-3. Comparison of predicted and observed morning 

 paraffin (PAR) and ethane (ETHA) concentrations at the 

 Weld County monitoring sites ........................................................................... 3-24 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\TOC.doc August, 2008 iii 

FIGURES 

Figure ES-1. Hourly ozone performance statistics for June (left) and 

 July (right) 2006 and Unpaired Peak Accuracy (top), Mean 

 Normalized Bias (middle) and Mean Normalized Gross 

 Error (bottom) and comparison against EPA’s model 

 performance goals (red lines).............................................................................ES-3 

Figure ES-2. Comparison of predicted and observed daily maximum 

 8-hour ozone concentrations for the June-July 2006 modeling 

 period and the final CAMx Run 17 2006 base case simulation 

 using the Co-Located predicted value near the monitor ....................................ES-4 

Figure ES-3. Comparison of the predicted and observed daily maximum 

 8-hour ozone concentrations on the two highest observed 

 ozone days during the June-July 2006 modeling period using 

 the final 2006 base case CAMx simulation (Run 17) results.............................ES-5 

Figure ES-4. Predicted and observed hourly ozone concentrations at the 

 Rocky Flats North (left) and Fort Collins West (eight) monitoring 

 sites during July 13-16, 2006 (top) and July 27-29, 2006 

 (bottom) periods.................................................................................................ES-6 

Figure ES-5. Comparison of predicted and observed early morning 

 (5-8am MST) VOC, NOx and CO concentrations and 

 VOC/NOx and CO/NOx ratios at the CAMP monitoring in 

 downtown Denver..............................................................................................ES-8 

Figure ES-6. Comparison of predicted and observed vertical ozone profiles 

 at Boulder for June 15 and July 21, 2006. .........................................................ES-9 

Figure ES-7. Comparison of predicted and observed total PM2.5 mass 

 model performance at FRM (left) and IMPROVE (right) 

 sites in the 12 km modeling domain ................................................................ES-10 

Figure 1-1a. Nested 36/12/4 km modeling domains for the Denver 8-hour 

 ozone modeling study. Blue line domains are for CAMx/SMOKE 

 domains that are nested in the MM5 red line domains ........................................ 1-4 

Figure 1-1b. Nested 12/4 km modeling domains for the Denver CAMx air 

 quality and SMOKE emissions modeling............................................................ 1-5 

Figure 1-2. D-J Basin boundaries overlaid and 2006 oil and gas well locations.................... 1-9 

Figure 1-3. 2006 NOx emissions by source category and by 

 county in the D-J Basin...................................................................................... 1-12 

Figure 1-4. 2006 VOC emissions by source category and by 

 county in the D-J Basin...................................................................................... 1-13 

Figure 1-5. D-J Basin NOx emissions proportional contributions 

 by source category ............................................................................................. 1-13 

Figure 1-6. D-J Basin VOC emissions proportional contributions 

 by source category ............................................................................................. 1-14 

Figure 1-7. DRCOG link-based transportation network used in the 

 CONCEPT on-road mobile source emissions modeling ................................... 1-17 

Figure 1-8. Monthly temporal profiles for facility classes 1 through 14.............................. 1-23 

Figure 1-9. Day of week temporal profiles for June and July .............................................. 1-24 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\TOC.doc August, 2008 iv 

Figure 1-10. Hourly temporal profiles for urban interstates for 

 a week in June and July ..................................................................................... 1-24 

Figure 1-11. Vehicle mix hourly profile for rural interstates in July...................................... 1-25 

Figure 1-12. Vehicle mix hourly profile for urban interstates in July.................................... 1-25 

Figure 2-1. Location of ozone monitoring sites within and near the 

 Denver Metropolitan Area (DMA) (triangle symbols 

 point to monitor identifier)................................................................................... 2-3 

Figure 2-2. Maximum daily maximum 8-hour ozone concentration (ppb) 

 observed during the June-July 2006 episode and three 

 “mini episodes” selected for more detailed analysis: 

 June17-19, July 13-15, and July 27-29. ............................................................... 2-7 

Figure 2-3a. Daily ozone model performance evaluation statistical 

 performance measures in the Denver NAA and comparison 

 with EPA performance goals for June 2006, the final CAMx 

 Run17 2006 base case simulation and 1-hour ozone........................................... 2-9 

Figure 2-3b. Daily ozone model performance evaluation statistical performance 

 measures in the Denver NAA and comparison with EPA performance 

 goals for July 2006, the final CAMx Run17 2006 base case simulation 

 and 1-hour ozone................................................................................................ 2-10 

Figure 2-4a. Daily ozone model performance evaluation statistical performance 

 measures in the Denver NAA and comparison with EPA performance 

 goals for June 2006, the final CAMx Run17 2006 base case simulation 

 and 8-hour ozone................................................................................................ 2-12 

Figure 2-4b. Daily ozone model performance evaluation statistical performance 

 measures in the Denver NAA and comparison with EPA performance 

 goals for July 2006, the final CAMx Run17 2006 base case simulation 

 and 8-hour ozone................................................................................................ 2-13 

Figure 2-5. Compared of observed highest daily maximum 8-hour ozone 

 concentration for each day during June (top) and July (bottom) 

 2006 with concurrent spatially paired prediction from the 

 CAMx Run 17 2006 base case simulation......................................................... 2-15 

Figure 2-6a. Comparison of predicted and observed daily maximum 8-hour ozone 

concentrations for the June-July 2006 modeling period and the 

 final CAMx Run 17 2006 base case simulation using the Maximum 

 predicted value near the monitor........................................................................ 2-17 

Figure 2-6b. Comparison of predicted and observed daily maximum 8-hour ozone 

concentrations for the June-July 2006 modeling period and the 

 final CAMx Run 17 2006 base case simulation using the Closest 

 predicted value near the monitor........................................................................ 2-17 

Figure 2-6c. Comparison of predicted and observed daily maximum 8-hour ozone 

concentrations for the June-July 2006 modeling period and the 

 final CAMx Run 17 2006 base case simulation using the 

 Co-Located predicted value near the monitor.................................................... 2-18 

Figure 2-7a. Spatial distribution of the predicted and observed daily 

 maximum 8-hour ozone concentrations (ppb) in the Denver 

 area on July 17, 2006 for the final CAMx Run 17 2006 

 base case simulation........................................................................................... 2-20 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\TOC.doc August, 2008 v 

Figure 2-7b. Spatial distribution of the predicted and observed daily 

 maximum 8-hour ozone concentrations (ppb) in the Denver 

 area on July 18, 2006 for the final CAMx Run 17 2006 

 base case simulation........................................................................................... 2-21 

Figure 2-7c. Spatial distribution of the predicted and observed 

 daily maximum 8-hour ozone concentrations (ppb) 

 in the Denver area on July 19, 2006 for the final CAMx 

 Run 17 2006 base case simulation..................................................................... 2-22 

Figure 2-8. Time series of predicted and observed hourly ozone 

 concentrations (ppb) during June 17-19, 2006 for the 

 final CAMx Run 17 2006 base case simulation................................................. 2-26 

Figure 2-9a. Comparison of hourly NOx (top), CO (middle) and 

 ozone (bottom) predicted and observed time series at 

 the Welby monitoring site for June 17-19, 2006 and 

 the final CAMx Run17 2006 base case simulation............................................ 2-27 

Figure 2-9b. Comparison of hourly NOx (top) and ozone (bottom) 

 predicted and observed time series at the CAMP monitoring 

 site for June 17-19, 2006 and the preliminary CAMx 

 Run17 2006 base case simulation...................................................................... 2-28 

Figure 2-10a. Spatial distribution of the predicted and observed daily 

 maximum 8-hour ozone concentrations (ppb) in the Denver 

 area on July 13, 2006 for the final CAMx Run 17 2006 

 base case simulation........................................................................................... 2-30 

Figure 2-10b. Spatial distribution of the predicted and observed daily 

 maximum 8-hour ozone concentrations (ppb) in the 

 Denver area on July 14, 2006 for the final CAMx 

 Run 17 2006 base case simulation..................................................................... 2-31 

Figure 2-10c. Spatial distribution of the predicted and observed daily 

 maximum 8-hour ozone concentrations (ppb) in the 

 Denver area on July 15, 2006 for the final CAMx 

 Run 17 2006 base case simulation..................................................................... 2-32 

Figure 2-11. Hourly time series of predicted and observed hourly 

 ozone concentrations for the July 13-15, 2006 period 

 and the final CAMx Run17 2006 base case simulation..................................... 2-38 

Figure 2-12a. Comparison of hourly NOx (top) and ozone (bottom) 

 predicted and observed time series at the Welby monitoring 

 site for July 13-15, 2006 and the final CAMx Run17 2006 

 base case simulation base case simulation......................................................... 2-39 

Figure 2-12b. Comparison of hourly NOx (top) and ozone (bottom) 

 predicted and observed time series at the CAMP monitoring 

 site for July 13-15, 2006 and the final CAMx Run17 2006 

 base case simulation base case simulation......................................................... 2-40 

Figure 2-13a. Spatial distribution of the predicted and observed daily 

 maximum 8-hour ozone concentrations (ppb) in the 

 Denver area on July 27, 2006 for the final CAMx 

 Run 17 2006 base case simulation..................................................................... 2-42 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\TOC.doc August, 2008 vi 

Figure 2-13b. Spatial distribution of the predicted and observed daily 

 maximum 8-hour ozone concentrations (ppb) in the 

 Denver area on July 28, 2006 for the final CAMx 

 Run 17 2006 base case simulation..................................................................... 2-43 

Figure 2-13c. Spatial distribution of the predicted and observed daily 

 maximum 8-hour ozone concentrations (ppb) in the 

 Denver area on July 29, 2006 for the final CAMx 

 Run 17 2006 base case simulation..................................................................... 2-44 

Figure 2-14a. Time series of predicted and observed hourly 

 ozone concentrations (ppb) for the July 27-29, 2006 

 period, monitoring sites to the Northwest of downtown 

 Denver and the final CAMx Run17 2006 base case simulation ........................ 2-47 

Figure 2-14b. Time series of predicted and observed hourly ozone 

 concentrations (ppb) for the July 27-29, 2006 period, 

 monitoring sites mostly south of downtown Denver and 

 the final CAMx Run17 2006 base case simulation............................................ 2-49 

Figure 2-14c. Time series of predicted and observed hourly ozone 

 concentrations (ppb) for the July 27-29, 2006 period, 

 monitoring sites to the North of downtown Denver and 

 the final CAMx Run17 2006 base case simulation............................................ 2-51 

Figure 2-15a. Comparison of hourly NOx (top) and ozone (bottom) 

 predicted and observed time series at the CAMP monitoring 

 site for July 27-29, 2006 and the final CAMx Run17 2006 

 base case simulation........................................................................................... 2-52 

Figure 2-15b. Comparison of hourly NOx (top) and ozone (bottom) 

 predicted and observed time series at the Welby monitoring 

 site for July 27-29, 2006 and the final CAMx Run17 2006 

 base case simulation........................................................................................... 2-53 

Figure 3-1a. Daily model performance statistics for June 2006, 

 hourly NOx concentrations in the DMA (CAMP and WELB) 

 and the final CAMX Run 17 2006 base case simulation..................................... 3-2 

Figure 3-1b. Daily model performance statistics for July 2006, 

 hourly NOx concentrations in the DMA (CAMP and WELB) 

 and the final CAMX Run 17 2006 base case simulation..................................... 3-3 

Figure 3-2a. Daily model performance statistics for June 2006, hourly CO 

 concentrations in the DMA (13 sites) and the final CAMX 

 Run 17 2006 base case simulation....................................................................... 3-4 

Figure 3-2b. Daily model performance statistics for July 2006, hourly CO 

 concentrations in the DMA (13 sites) and the final CAMX 

 Run 17 2006 base case simulation....................................................................... 3-5 

Figure 3-3. Locations of monitoring sites where VOC samples were 

 collected during the summer of 2006................................................................... 3-6 

Figure 3-4a. Comparison of predicted and observed 3-hour average 

 morning (5-8am MST) CB05-VOC, NOx and CO 

 concentrations at the CAMP monitor for the 

 CAMx Run 17 2006 base case simulation......................................................... 3-11 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\TOC.doc August, 2008 vii 

Figure 3-4b. Comparison of predicted and observed 3-hour 

 average morning (5-8am MST) CB05-VOC/NOx 

 and CO/NOx ratio and ethane (ETHA) concentrations 

 at the CAMP monitor for the CAMx Run 17 2006 

 base case simulation........................................................................................... 3-12 

Figure 3-5a. Comparison of predicted and observed 3-hour 

 average morning (5-8am MST) CB05-VOC, NOx and CO 

 concentrations at the WELB monitor for the CAMx 

 Run 17 2006 base case simulation..................................................................... 3-13 

Figure 3-5b. Comparison of predicted and observed 3-hour 

 average morning (5-8am MST) CB05-VOC/NOx 

 and CO/NOx ratio and ethane (ETHA) concentrations 

 at the Welby monitor for the CAMx Run 17 2006 

 base case simulation........................................................................................... 3-14 

Figure 3-6. Comparison of the predicted and observed morning 

 CB05-VOC concentrations at the Ft. Lupton and 

 Platteville sites in Weld County......................................................................... 3-16 

Figure 3-7. Comparison of the predicted and observed morning 

 CB05-VOC concentrations at the Ft. Lupton and 

 Platteville sites in Weld County......................................................................... 3-17 

Figure 3-8a. omparison of predicted and observed 5-8am 

 MST CB05 VOC species at the CAMP monitor on June 19, 2006................... 3-19 

Figure 3-8b. Comparison of predicted and observed 5-8am 

 MST CB05 VOC species at the CAMP monitor on July 13, 2006.................... 3-20 

Figure 3-8c. Comparison of predicted and observed 5-8am 

 MST CB05 VOC species at the CAMP monitor on July 28, 2006.................. 3*-21 

Figure 3-9a. Comparison of predicted and observed 5-8am 

 MST CB05 VOC species at the Welby monitor on June 19, 2006.................... 3-22 

Figure 3-9b. Comparison of predicted and observed 5-8am 

 MST CB05 VOC species at the Welby monitor on July 13, 2006 .................... 3-23 

Figure 3-10. Comparison of predicted and observed CB05 VOC 

 species at the Ft. Lupton monitoring site........................................................... 3-25 

Figure 3-11a. Comparison of predicted and observed vertical 

 distribution of ozone concentrations on June 15, 2006 

 for the CAMx Run 17 2006 base case simulation. ............................................ 3-27 

Figure 3-11b. Comparison of predicted and observed vertical distribution of ozone 

concentrations on June 23, 2006 for the 

 CAMx Run 17 2006 base case simulation......................................................... 3-28 

Figure 3-11c. Comparison of predicted and observed vertical distribution of ozone 

concentrations on June 29, 2006 for the 

 CAMx Run 17 2006 base case simulation......................................................... 3-28 

Figure 3-11d. Comparison of predicted and observed vertical distribution of ozone 

concentrations on July 14, 2006 for the 

 CAMx Run 17 2006 base case simulation......................................................... 3-29 

Figure 3-11e. Comparison of predicted and observed vertical distribution of ozone 

concentrations on July 21, 2006 for the 

 CAMx Run 17 2006 base case simulation......................................................... 3-29 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\TOC.doc August, 2008 viii 

Figure 3-11f. Comparison of predicted and observed vertical distribution of ozone 

concentrations on July 26, 2006 for the 

 CAMx Run 17 2006 base case simulation......................................................... 3-29 

Figure 3-12. Comparison of predicted and observed total PM2.5 

 mass model performance at FRM (left) and IMPROVE (right) 

 sites in the 12 km modeling domain .................................................................. 3-32 

Figure 3-13. Comparison of predicted and observed total sulfate (SO4) 

 model performance at CASTNET (top left), IMPROVE 

 (top right), STN (bottom left) and NADP (bottom right) 

 monitoring sites in the 12 km modeling domain ............................................... 3-33 

Figure 3-14. Comparison of predicted and observed total 

 nitrate (NO3) model performance at CASTNET 

 (top left), CASTNET for total NO3 (NO3+HNO3) 

 (top right), IMPROVE (bottom left) and NADP 

 (bottom right) monitoring sites in the 12 km modeling domain........................ 3-34 

Figure 3-15. Comparison of predicted and observed Organic Carbon 

 (OC; top) and Elemental Carbon (EC; bottom) model 

 performance at IMPROVE (left) and STN (right) monitoring 

 sites in the 12 km modeling domain .................................................................. 3-35 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Exec_Sum1.doc August 2008 ES-1

EXECUTIVE SUMMARY 

Due to violations of the 0.08 parts per million (ppm) 8-hour ozone National Ambient Air Quality 

Standard (NAAQS) based on 2005-2007 air quality data, in November 2007 the Denver 

Metropolitan Area (DMA) reverted to an 8-hour ozone nonattainment area. This requires the 

DMA to develop an 8-hour ozone State Implementation Plan (SIP) that demonstrates the area 

will achieve the 0.08 ppm 8-hour ozone NAAQS by 2010. The Denver Regional Air Quality 

Council (RAQC), in consultation with the Colorado Department of Health and Environment 

(CDPHE) Air Pollution Control Division (APCD) contracted with ENVIRON International 

Corporation and their subcontractor Alpine Geophysics, LLC to develop the photochemical 

modeling databases necessary to demonstrate that the DMA will achieve the 0.08 ppm 8-hour 

ozone NAAQS by 2010. 

OVERVIEW OF APPROACH 

The Comprehensive Air-quality Model with extensions (CAMx; www.camx.com) was set up for 

a June-July 2006 episode on a 36/12/4 km grid with the 4 km domain focused on Colorado. 

Meteorological inputs were prepared using the MM5 meteorological model whose results and 

evaluation are discussed by McNally and co-workers (2008). An initial emissions inventory was 

prepared using the SMOKE emissions modeling system and a preliminary 2006 base case was 

performed. A preliminary model performance evaluation was conducted and diagnostic 

sensitivity tests performed to identify an optimal model configuration for simulating ozone 

formation in the DMA (Morris et al., 2008b). 

A revised CAMx 2006 base case (Run 17) simulation was conducted that included the following 

emission updates from the preliminary 2006 base case simulation reported by Morris and coworkers (2008): 

• Corrections and enhancements to the 2006 emissions for Colorado provided by the 

CDPHE/APCD. 

• Application ConCEPT MV mobile source emissions modeling system that uses diurnally 

varying link-based Vehicle Miles Traveled (VMT), fleet mix and other data to provide 

more detailed on-road mobile source emissions in the DMA. 

• Use of the WRAP Phase III oil and gas production emissions inventory for the DenverJulesburg Basin. 

• Biogenic emission estimates from the MEGAN biogenic emissions model (Guenther and 

Wiedinmyer, 2004). 

This report presents the model performance evaluation for the final 2006 base case simulation 

performed as part of the Denver 8-hour ozone attainment demonstration modeling. 

MODEL PERFORMANCE EVALUATION 

The model performance evaluation of the Denver final 2006 base case simulation performed 

both an operational evaluation that evaluated how well the model predicted the ozone 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Exec_Sum1.doc August 2008 ES-2

observations and a diagnostic evaluation that evaluated the model for ozone precursors, key 

indicator species, particulate matter (PM) and ozone aloft. 

Operational Model Performance Evaluation 

The operational model performance evaluation focused on how well the model predicted the 

observed surface ozone concentrations in the DMA and included graphical displays of model 

performance as well as statistical evaluation metrics and comparisons with model performance 

goals. 

Comparison against Model Performance Goals

One element of an ozone SIP model performance evaluation is to test how well the model 

reproduces the observed ozone concentrations. As part of this, statistical performance metrics 

are calculated that are compared against model performance goals as a method to gauge model 

performance, compare it against other studies and to assist in determining whether the model is 

getting the right answer. EPA’s latest modeling guidance (EPA, 2007) emphasizes using 

graphical and diagnostic evaluation techniques to assure that the photochemical rid model is 

capturing the correct chemical regimes and emissions sources that lead to the high ozone (i.e., 

assuring that the model is getting the right answer for the right reason). 

 

EPA’s 1991 1-hour ozone guidance included three performance goals that have been used for 

over two decades to assist in evaluating ozone models as part of the ozone SIP modeling process: 

• Unpaired Accuracy of the Peak ≤±20%; 

• Normalized Mean Bias ≤±15%; and 

• Normalized Mean Gross Error ≤35%. 

The Mean Normalized Bias and Gross Error statistical measures are calculated using all 

predicted and observed hourly ozone pairs matched by time and location for which the observed 

ozone is 60 ppb or greater. 

Figure ES-1 compares the Denver final 2006 base case CAMx hourly ozone model performance 

against these three EPA model performance goals for each day during the June-July 2006 

episode. The CAMx final 2006 base case simulation achieves the Unpaired Accuracy of the 

Peak performance goal of ≤±20% for 58 of the 60 simulation days of the episode (i.e., 97% of 

the modeled days). There are 58 days with bias and error comparisons during the episode as two 

days had no observed ozone greater than 60 ppb so no statistics could be calculated. Of these 58 

days, 50 of them (86%) achieved EPA’s ≤±15% performance for Mean Normalized Bias and all 

of them achieves EPA’s performance goal for Mean Normalized Gross Error. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Exec_Sum1.doc August 2008 ES-3

Unpaired Peak Accuracy

-30

-20

-10

0

10

20

30

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(%)

Run17

+20%

-20%

Unpaired Peak Accuracy

-30

-20

-10

0

10

20

30

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(%)

Run17

+20%

-20%

Normalized Bias

-30

-20

-10

0

10

20

30

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(%)

Run17

+15%

-15%

Normalized Bias

-30

-20

-10

0

10

20

30

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(%)

Run17

+15%

-15%

Normalized Error

0

10

20

30

40

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(%)

Run17

35%

Normalized Error

0

10

20

30

40

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(%)

Run17

35%

Figure ES-1. Hourly ozone performance statistics for June (left) and July (right) 2006 and 

Unpaired Peak Accuracy (top), Mean Normalized Bias (middle) and Mean Normalized Gross 

Error (bottom) and comparison against EPA’s model performance goals (red lines). 

EPA’s draft 1999 8-hour ozone modeling guidance has a performance goal for daily maximum 

8-hour ozone concentrations that the predicted value near the monitor be within ±20% on most 

monitor-days. This is a particularly important performance metric as it is these exactly same 

predicted daily maximum 8-hour ozone concentrations near the monitor that are used to make 

future year ozone projections. By “near the monitor” we used the same 7 x 7 array of 4 km grid 

cells centered on the monitor as used in the ozone projections and have made this analysis three 

ways by selecting the Maximum predicted value in the array, the predicted value Closest to the 

observed value, and the predicted value Co-Located at the monitor, with this last comparison 

being a particularly stringent test of model performance. Table ES-1 summarizes the results of 

this performance test, with a scatter plot of the predicted and observed daily maximum 8-hour 

ozone concentrations Co-Located at the monitor shown in Figure ES-2. Using the Maximum, 

Closest and Co-Located predicted daily maximum 8-hour ozone concentration near the monitor 

we see that the model achieves the within ±20% of the observed value performance goal 76%, 

89% and 82% of the monitor-days. Thus, the Denver final 2006 base case CAMx simulation 

(Run 17) daily maximum 8-hour ozone predictions achieves EPA’s ≤±20% of the observed value 

on most monitor-days performance goal. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Exec_Sum1.doc August 2008 ES-4

Table ES-1. Percent of the monitor-days that the model predicted daily maximum 8-hour ozone 

concentrations near the monitor is within ±20% of the observed value (total monitor-days = 

1008). 

Maximum Near the Monitor 

Percent Difference # Days % Days 

<-20 9 1% 

-20% to +20 769 76% 

> +20 230 23% 

Closest Near the Monitor 

Percent Difference # days % 

<-20 23 2% 

-20% to +20 902 89% 

> +20 83 8% 

Co-Located At the Monitor 

Percent Difference # days % 

<-20 48 5% 

-20% to +20 829 82% 

> +20 131 13% 

Daily maximum 8-Hour ozone at monitor.

All sites & all days. Subregion = Denver June+July, 2006. Run17 04km

0

20

40

60

80

100

120

0 20 40 60 80 100 120

Observed Ozone (ppb)

Predicted Ozone (ppb)

r

2

=0.2615

O - - O shows quantiles

Figure ES-2. Comparison of predicted and observed daily maximum 8-hour ozone 

concentrations for the June-July 2006 modeling period and the final CAMx Run 17 2006 base 

case simulation using the Co-Located predicted value near the monitor.
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Exec_Sum1.doc August 2008 ES-5

Spatial and Temporal Evaluation of Ozone Model Performance

A detailed analysis of the ozone model performance was assessed by examining the ability of the 

model to predict the observed ozone concentrations for three 3-day episodes during the June-July 

2006 modeling period when 7 of the 9 8-hour ozone exceedance days occurred. The detailed 

ozone model performance revealed that at some days the model failed to reproduce the elevated 

ozone concentrations and model performance was poor. Whereas, on other days the ozone 

model performance was quite good. In general, there were days with good model performance at 

the high southern (e.g., Chatfield) and northwestern (e.g., Rocky Flats North) ozone monitors, 

but there was a general underprediction bias at the northern Fort Collins West monitor. Figure 

ES-3 compares the spatial distribution of the predicted and observed daily maximum 8-hour 

ozone concentrations in the two highest days during the two month modeling period: July 14 and 

29, 2006. On July 14th, the model is correctly predicting elevated ozone concentrations to the 

north-northwest of the DMA with the highest modeled values near the highest observed value at 

Fort Collins West monitor, although the modeled peaks are below the observed ozone peaks. On 

July 29, 2006 the model correctly predicts that the very highest ozone concentrations occur in 

and near the DMA with both the modeled and observed 8-hour ozone peaks in excess of 90 ppb. 

Time series of predicted and observed hourly ozone concentrations for the July 13-16 and July 

27-29 periods and the two highest monitoring sites (RFNO and FTCW)are shown in Figure ES4. In general the model underestimates the observed ozone peaks; the exception to this is at 

Rocky Flats North monitor for July 27-29, 2006 that exhibits very good ozone model 

performance. 

Figure ES-3. Comparison of the predicted and observed daily maximum 8-hour ozone 

concentrations on the two highest observed ozone days during the June-July 2006 

modeling period using the final 2006 base case CAMx simulation (Run 17) results. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Exec_Sum1.doc August 2008 ES-6

RFNO: Rocky Flats North -693.592 21.629

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/13/ 6 7/14/ 6 7/15/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

FTCW: Fort Collins West -682.854 96.141

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/13/ 6 7/14/ 6 7/15/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

RFNO: Rocky Flats North -693.592 21.629

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/27/ 6 7/28/ 6 7/29/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

FTCW: Fort Collins West -682.854 96.141

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/27/ 6 7/28/ 6 7/29/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

Figure ES-4. Predicted and observed hourly ozone concentrations at the Rocky Flats 

North (left) and Fort Collins West (eight) monitoring sites during July 13-16, 2006 (top) and 

July 27-29, 2006 (bottom) periods. 

Diagnostic Model Performance Evaluation 

In addition to the many diagnostic tests performed to test the sensitivity of the model to model 

inputs and options performed as part of the preliminary model performance evaluation discussed 

in previous reports (Morris et al., 2008b; McNally, et al., 2008), the diagnostic model 

performance evaluation also included comparisons against ozone precursors, key indicator 

species, PM species and ozone aloft. 

Ozone Precursor and Indicator Species Comparisons

The CDPHE/APCD collected 3-hour VOC samples at several sites during a few days of the 

June-July 2006 modeling period. Figure ES-5 displays the example comparisons of ozone 

precursor and their ratios at the downtown Denver CAMP monitoring site. In these comparisons, 

the observed VOC species were converted to the CB05 lumped VOC species that is the chemical 

mechanism used in CAMx. Then the modeled and observed CB05 species were summed to get 

the total predicted and observed VOC concentrations, respectively. Note that ethane (ETHA) 

was not included in the total VOC species when summing the CB05 lumped species. Also note 

that since the VOC sampling did not collected measurements for methanol and ethanol, those 

species were also not accounted for when summing the CB05 species. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Exec_Sum1.doc August 2008 ES-7

The model is systematically underpredicting the observed VOC concentrations at the CAMP 

monitoring site, whereas for NOx and CO there are days with underpredictions and days with 

overpredictions, although on average NOx is underpredicted as well. The CAMP monitor is 

located in downtown Denver where we would expect VOC, NOx and CO emissions to be 

dominated by on-road mobile sources. Thus, these comparisons provide a good evaluation of the 

on-road mobile source emissions. However, there are incommensurability differences between 

the observed point measurements at the 4 km grid cell average model predictions. Thus, a better 

indication of the accuracy of the on-road mobile source emissions are the VOC/NOx and 

CO/NOx key indicator ratio comparisons. Comparison of the predicted and observed VOC/NOx 

ratios also provide an indication of whether the model is reproducing the correct chemical regime 

in the DMA. Of the 15 days with morning VOC/NOx comparisons, very good comparisons are 

seen on 11 (73%) of the days. Of the four days in which the model is underpredicting the 

observed VOC/NOx ratio one is July 4th that has atypical traffic patterns that were not simulated 

in the CONCEPT modeling. Two of the other VOC/NOx ratio overprediction days were 

weekend days, although on two other weekend days we saw good performance. The CO/NOx 

ratios appear to be slightly overpredicted on most days which may indicate that MOBILE6 is 

overestimating the on-road mobile source CO emissions, which has also been noted by Pollack 

and co-workers (2004). Ethane (ETHA) is underpredicted by approximately a factor of 10 at the 

CAMP monitor with observed values of ~25 ppbC and predicted values of ~2.5 ppbC. The 

underprediction of ethane is even greater at the two Weld County monitoring sites where the 

CB05 paraffin species is also underpredicted. As ethane is primarily associated with natural gas, 

its underprediction could indicate that organic emissions from natural gas related and oil and gas 

development sources are understated in the inventory. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Exec_Sum1.doc August 2008 ES-8

Predicted vs. Observed VOC at CAMP

0.00

100.00

200.00

300.00

400.00

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

VOC [ppbC]

Predicted Observed

Predicted vs. Observed VOC/NOx Ratio at CAMP

0

1

2

3

4

5

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

VOC/NOx [ppbC/ppb]

Predicted Observed

Predicted vs. Observed NOx at CAMP

0

50

100

150

200

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

NOx [ppb]

Predicted Observed

Predicted vs. Observed CO/NOx Ratio at CAMP

0

5

10

15

20

25

30

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

CO/NOx [ppb/ppb]

Predicted Observed

Predicted vs. Observed CO at CAMP

0

200

400

600

800

1000

1200

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

CO [ppb]

Predicted Observed

Predicted vs. Observed ETHA at CAMP

0

10

20

30

40

50

60

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

ETHA [ppbC]

Predicted Observed

Figure ES-5. Comparison of predicted and observed early morning (5-8am MST) VOC, 

NOx and CO concentrations and VOC/NOx and CO/NOx ratios at the CAMP monitoring in 

downtown Denver. 

In addition to the underestimation of observed ethane and paraffin VOC species noted above, 

carbonyl VOC species were also systematically underpredicted by the model. The 

underprediction of acetaldehyde may be due in part to the SMOKE VOC speciation profile for 

on-road mobile sources not accounting for ethanol blended gasoline whose combustion produces 

higher acetaldehyde than conventional gasoline. However, the reasons for the large 

underprediction of formaldehyde are less clear and since formaldehyde is an important VOC 

species that initiates the radical cycle its underprediction may help explain why the model tends 

to form ozone too slowly and does not obtain as high ozone peaks as observed. A review and 

evaluation of the VOC speciation profiles in the Denver modeling is recommended. 

Ozone Aloft

During six days of the June-July 2006 modeling period, ozonesondes were released from 

Boulder that obtained a measured vertical ozone profile in the atmosphere. One hypothesis for 

the conceptual model on ozone formation in the DMA is that on some days there is a reservoir of 

ozone above the ground that is entrained and mixed to the ground as the mixing height rises. 

The comparison of the modeled vertical ozone profile with the ozonesonde measurements would 

provide an indication of whether the model is capturing this phenomenon. As the ozonesonde 

rises it will move downwind with the prevailing wind. In these comparisons we used the 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Exec_Sum1.doc August 2008 ES-9

modeled vertical ozone profile in the grid cell at the time of the ozonesonde launch, so did not 

account for the horizontal and temporal displacement of the ozonesonde measurements from its 

launch point and start time. Figure ES-6 displays two of the comparisons of the predicted and 

observed vertical ozone profiles for June 15 and July 21, 2006. In general, the model does a 

better job reproducing the observed vertical ozone profiles for the three June days than the three 

July days. For example, on June 15 the model and observed ozone agree well in the lowest 2 km 

of the atmosphere and then deviate from each other. On the other hand, on July 21, 2006 the 

model is underpredicting the observed ozone in the lowest 2 km of the atmosphere by 20 ppb. 

Predicted and Observed Ozonesonde over Boulder 

on June 15, 2006 at 12pm

0

1

2

3

4

5

6

7

8

9

10

0 50 100 150 200 250 300

Ozone [ppb]

Height above Surface [km]

Observed Predicted, Run17.1204

Predicted and Observed Ozonesonde over Boulder 

on July 21, 2006 at 11am MST

0

1

2

3

4

5

6

7

8

9

10

0 20 40 60 80 100

Ozone [ppb]

Height above Surface [km]

Observed Predicted Run17.1204

Figure ES-6. Comparison of predicted and observed vertical ozone profiles at Boulder for 

June 15 and July 21, 2006. 

PM Performance

The Denver June-July 2006 photochemical modeling database was developed for demonstrating 

attainment of the 8-hour ozone standard. Consequently, the focus was on ozone and ozone 

precursor performance. Thus, no effort was spent on optimizing model performance for 

particulate matter (PM) species. In particular, the CDPHE/APCD only provided complete ozone 

precursor emissions (i.e., VOC, NOx and CO) for the state of Colorado. When modeling many 

sources categories, such as on-road mobile using SMOKE-MOBILE6 and CONCEPT MV 

models and processing the CEM data for point sources, we pick up the PM precursor emissions. 

However, for area and non-road mobile source emissions for sources in Colorado we only have 

ozone precursor emissions so are missing many PM related species (e.g., SO2, primary PM and 

ammonia). Thus, we expect to underpredict PM mass and PM species in the model performance 

evaluation. 

Figure ES-7 displays an example evaluation for total PM2.5 mass from the more urban-oriented 

FRM and more rural-oriented IMPROVE monitoring networks. As expected, PM2.5 is 

underestimated most of the time. This underprediction is prevalent across all PM species with 

the lowest underprediction for SO4 and highest for NO3. These results are consistent with 

missing PM precursor emissions from area and non-road sources that would affect NO3 the 

most. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Exec_Sum1.doc August 2008 ES-10

Figure ES-7. Comparison of predicted and observed total PM2.5 mass model 

performance at FRM (left) and IMPROVE (right) sites in the 12 km modeling domain. 

CONCLUSIONS OF MODEL PERFORMANCE EVALUATION 

As noted in EPA’s latest air quality modeling guidance, “by definition, models are simplistic 

approximations of complex phenomena” that “...contain many elements that are uncertain”. 

Consequently, achieving perfect model performance is unattainable and some uncertainties will 

always exist. The Denver final 2006 base case (Run 17) CAMx simulation achieves EPA’s 

performance goals on a vast majority of days during the June-July 2006 modeling episode. The 

model is also exhibiting very good agreement for VOC/NOx ratios in Denver on most days, 

suggesting that the model is simulating the correct chemical regimes. The model performance is 

as good as or better than past ozone SIP modeling in the Denver and many other areas whose 

SIPS have been approved by EPA. Based on the model performance evaluation presented in this 

report, we conclude that the model is performing well enough to reliable project future-year 

ozone concentrations within the normal uncertainties of photochemical grid modeling. Although 

care should be taken that the ozone projections are not unduly affected by the few poor 

performing days. 

The model performance evaluation has identified several areas of future research that could 

improve model performance including a focus on VOC speciation and the presence of aldehydes 

in the inventory, improvements in oil and gas emissions and other sources of natural gas related 

emission sources and better simulation of ozone aloft. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-1

1.0 INTRODUCTION 

1.1 BACKGROUND 

Ozone air quality in the Denver Metropolitan Area (DMA) has been near the 8-hour ozone 

National Ambient Air Quality Standard (NAAQS) of 0.08 ppm (exceedance defined by values of 

85 ppb or higher) for several years. Based on 2005-2007 measured air quality, the DMA 

violated the 0.08 ppm 8-hour ozone NAAQS so is required to prepare an 8-hour ozone State 

Implementation Plan (SIP) that demonstrates attainment by 2010. This report presents the 2006 

base case modeling and model performance evaluation element of the DMA 2010 attainment 

demonstration modeling. 

1.1.1 Denver EAC 8-Hour Ozone SIP 

In December 2002, the Denver Regional Air Quality Council (RAQC) and Colorado Department 

of Health and Environment (CDPHE) Air Pollution Control Division (APCD) and others entered 

into an 8-hour ozone Early Action Compact (EAC) with the U.S. Environmental Protection 

Agency (EPA). EPA’s EAC allows an area to submit an enforceable 8-hour ozone State 

Implementation Plan (SIP) by March 2004 that demonstrates attainment of the 8-hour ozone 

NAAQS by 2007. In return, EPA will defer the classification of an area as nonattainment until 

2007. The RAQC and APCD contracted with ENVIRON International Corporation with their 

subcontract of Alpine Geophysics, LLC to perform the photochemical modeling necessary for 

the Denver 8-hour ozone EAC SIP. At the outset of the EAC modeling, the DMA was in 

attainment of the 8-hour ozone NAAQS. However, the monitored ozone concentrations during 

the summer of 2003 resulted in violations of the 8-hour ozone standard. Because of the EAC, 

EPA classified the DMA as “nonattainment deferred”, with final designation to be determined 

after the 2007 ozone season. More details on the Denver 8-hour EAC SIP are available at: 

http://www.raqc.org/ozone/EAC/ozone-eac.htm

Ozone attainment is based on 8-hour ozone Design Values (DVs) that are defined as the threeyear average of the fourth highest 8-hour ozone concentration during a year at a monitor. Based 

on measured ozone concentrations during the 2005-2007 three-year period, the maximum 8-hour 

ozone DV in the DMA was 85 ppb at the Rocky Flats North (RFNO) monitoring site, which 

exceeds the 8-hour ozone NAAQS. Consequently, in November 2007 the DMA reverted to an 

8-hour ozone nonattainment area and the RAQC/APCD is charged with developing a new 8-hour 

ozone SIP emissions control plan that demonstrates the DMA will achieve attainment of the 0.08 

ppm 8-hour ozone NAAQS by 2010. 

1.1.2 New 8-Hour Ozone NAAQS 

On March 12, 2008, EPA promulgated a new primary ozone NAAQS that has the same form as 

the 0.08 ppm 8-hour ozone NAAQS, but lowers the threshold from 0.08 ppm (85 ppb) to 0.075 

ppm (75 ppb). Of the ~14 ozone monitors in the greater DMA, half have 2005-2007 8-hour 

ozone DVs that are 75 ppb or higher. The current Denver 8-hour ozone SIP modeling effort 

addresses the 0.08 ppm 8-hour ozone NAAQS. The designations of 8-hour ozone nonattainment 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-2

areas under the new 0.075 ppm 8-hour ozone NAAQS will be made in 2010 based on 2007-2009 

observed ozone concentrations with subsequent SIPs due in 2013. Attainment of the new 8-hour 

ozone NAAQS will be required by 2016 (moderate) to 2030 (extreme) depending on the 

nonattainment area classification. 

1.2 PURPOSE 

With the Denver region needing to quickly develop an 8-hour ozone attainment control strategy 

for the SIP, ENVIRON and Alpine were reenlisted to prepare the photochemical modeling 

databases necessary to develop a control plan that demonstrates attainment of the 8-hour ozone 

standard by 2010. The ENVIRON/Alpine Modeling Team are employing the fifth generation 

Mesocale Model (MM5) meteorological model (Anthes and Warner, 1978; Dudhia, 1993), the 

Sparse Matrix Operating Kernel Emissions (SMOKE) modeling system (Coats, 1996) and the 

Comprehensive Air-quality Model with extensions (CAMx) photochemical grid model 

(ENVIRON, 2008) to model ozone in the Denver area for a June-July 2006 modeling period for 

the purposes of demonstrating attainment of the 8-hour ozone standard by 2010. The 8-hour 

ozone modeling activities being performed by the ENVIRON/Alpine Modeling Team consists of 

the following activities: 

• Development of a Denver 8-hour ozone SIP attainment demonstration Modeling Protocol 

(Morris et al., 2007; http://www.ozoneaware.org/documents/DraftFinalProtocolDenver8-

HourOzoneNov282007.pdf); 

• Development of a preliminary 36/12/4 km photochemical modeling database for the 

June-July 2006 episode, the DMA, and initial model performance evaluation, sensitivity 

test modeling and identification of optimal model configuration for simulating ozone in 

the DMA (Morris et al., 2008); 

• Final base case modeling and model performance evaluation for the June-July 2006 

DMA episode (this document); 

• 2010 base case modeling, emission sensitivity tests and ozone source apportionment 

modeling (in progress); and 

• 2010 control strategy modeling (in progress). 

A previous report documented the results of the preliminary application and evaluation of the 

photochemical modeling system for the June-July 2006 period using the initial model 

configuration and includes results of diagnostic tests designed to improve model performance 

and recommendations for a final optimal model configuration for the 8-hour Denver ozone SIP 

(Morris et al., 2008). This report presents the final 2006 base case modeling and model 

performance evaluation for the June-July 2006 episode. 

1.3 OVERVIEW OF MODELING APPROACH 

The Denver 8-Hour ozone SIP modeling includes emissions, meteorological and ozone 

simulations using a nested 36/12/4 km grid with the 4-km grid focused on the state of Colorado 

including the DMA and vicinity. The procedures for conducting the Denver SIP ozone modeling 

are outlined in a Modeling Protocol (Morris et al., 2007). The Denver 8-hour ozone modeling 

approach was to develop an initial 36/12/4 km modeling database for the June-July 2006 episode 

using the MM5 meteorological, SMOKE emissions and CAMx photochemical grid models that 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-3

is described by Morris and co-workers (2008). The preliminary photochemical modeling 

identified an optimal model configuration for the final 2006 base case that is described in this 

report. The emissions were also updated for the final 2006 base case modeling. In addition to 

obtaining revised and corrected emissions from the APCD for Colorado, the following two 

additional emission inventory enhancements were made for the final 2006 base case emissions 

inventory: 

• Generation of refined on-road mobile source emissions using the CONCEPT emissions 

model and link-based Vehicle Miles Travelled (VMT) data from the Denver Regional 

Council of Government (DRCOG) Travel Demand Model (TDM); and 

• Use of the WRAP Phase III oil and gas (O&G) emissions for the Denver-Julesburg 

Basin. 

Below we provide an overview of the modeling approach and summarize emission updates 

performed for the final 2006 base case modeling. Details on the meteorological modeling, 

SMOKE emissions modeling and other aspects of the development of the June-July 2006 

modeling database are presented in the preliminary modeling report (Morris et al., 2008). 

1.3.1 Modeling Domains 

Figure 1-1a displays the MM5 (red) and CAMx (blue) 36/12/4 km modeling domains used in the 

Denver 8-hour ozone modeling study. These modeling domains are based on a Lambert 

Conformal Project (LCP) using the parameters given in Table 1-1. The MM5 modeling domains 

are defined slightly larger than the CAMx domains in order to allow the MM5 model to 

eliminate any boundary artifacts that occur as the MM5 model brings the meteorological 

variables specified in the boundary conditions into dynamic balance with each other. Details on 

the Denver June-July 2006 MM5 meteorological modeling are provided in a companion report to 

this one (McNally et al., 2008). 

The SMOKE, ConCEPT, MEGAN and GloBEIS emissions models use the same modeling 

domains as CAMx. Table 1-2 lists the number of rows and columns and the definition of the X 

and Y origin (i.e., the southwest corner) for the 36/12/4 km domains used by MM5, CAMx and 

the SMOKE, CONCEPT and MEGAN emissions models. In Table 1-2 “Dot” refers to the grid 

mesh defined at the vertices of the grid cells while “Cross” refers to the grid mesh defined by the 

grid cell centers. Thus, the dimension of the dot mesh is equal to the cross mesh plus one. 

Figure 1-1b displays the SMOKE/CAMx 12/4 km modeling domains. The CAMx model was 

first applied to the 36 km continental U.S. domain using boundary conditions (BCs) from a 

global climate air quality model (Figure 1-1a). The CAMx 2006 base case modeling results from 

the 36 km continental U.S. domain simulation are then processed to generate BCs for the CAMx 

12/4 km domain (Figure 1-1b). The CAMx 2006 base case simulations for the 12/4 km domains 

were run using two-way interactive grid nesting (i.e., pollutants can flow back and forth between 

the 12 km and 4 km domains to account for recirculation). 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-4

Table 1-1. Lambert Conformal Projection (LCP) definition for the Denver 36/12/4 km modeling 

grids. 

Parameter Value 

Projection Lambert-Conformal 

1st True Latitude 33 degrees N 

2nd True Latitude 45 degrees N 

Central Longitude -97 degrees W 

Central Latitude 40 degrees N 

Table 1-2. Grid definitions for MM5, Emissions and CAMx modeling domains. 

-2880-2520-2160-1800-1440-1080 -720 -360 0 360 720 1080 1440 1800 2160 2520 2880

-2160

-1800

-1440

-1080

-720

-360

0

360

720

1080

1440

1800

2160

MM5 36 km

CAMx 36 km

MM5 12km

CAMx 12km

MM5 4km

CAMx 4

Figure 1-1a. Nested 36/12/4 km modeling domains for the Denver 8-hour ozone modeling 

study. Blue line domains are for CAMx/SMOKE domains that are nested in the MM5 red line 

domains.

MODEL

COLUMNS

DOT(CROSS) 

ROWS 

DOT(CROSS) 

XORIGIN

(KM) 

YORIGIN

(KM) 

MM5 

 36 km grid 

 12 km grid 

 4 km grid

165 (164) 

187 (186) 

151 (150) 

129 (128) 

157 (156) 

136 (135) 

-2952.0 

-1836.0 

-984.0 

 

-2304.0 

-936.0 

-324.0 

Emissions/CAMx 

 36 km grid 

 12 km grid 

 4 km grid 

(148) 

(167) 

(119)

(112) 

(92) 

(119) 

-2736.0 

-1704.0 

-940.0 

-2088.0 

-624.0 

-940.0 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-5

-1680-1560-1440-1320-1200-1080 -960 -840 -720 -600 -480 -360 -240 -120 0 120 240

-600

-480

-360

-240

-120

0

120

240

360

480

CAMx 12km

CAMx 4km

Denver CAMx 12 and 4 km Domain

Figure 1-1b. Nested 12/4 km modeling domains for the Denver CAMx air quality and SMOKE 

emissions modeling.

1.4 EMISSIONS MODELING APPROACH 

The emission inputs for the Denver June-July 2006 8-hour ozone modeling episode were 

prepared in two phases. In the first phase, a Fast Track model-ready emissions inventory was 

developed in late 2007 using the SMOKE emissions model for all anthropogenic emissions and 

the GloBEIS model for biogenic emissions (Morris et al., 2008). The CDPHE/APCD provided 

2006 emissions for Colorado counties with the rest of the region using the WRAP and other RPO 

2002 emissions projected to 2005. The Fast Track emissions inventory was used for initial 

diagnostic testing and model performance evaluation of the CAMx model (Morris et al., 2008). 

The CDPHE provided revised 2006 base case emissions for Colorado that contained several 

updates and corrections. In addition, emissions for on-road mobile sources in the DMA and oil 

and gas sources were updated as described below. 

1.4.1 SMOKE Emissions Modeling Approach 

CAMx-ready emissions were generated using the Sparse Matrix Operator Kernel Emissions 

(SMOKE) emissions modeling system (Coats, 1996) for all anthropogenic emissions categories 

except on-road mobile sources in the DMA, which used the Consolidated Community Emissions 

Processing Tool (ConCEPT) modeling system (Loomis et al., 2005). The final 2006 base case 

biogenic emissions were based on the Model of Emissions of Gases and Aerosols from Nature 

(MEGAN) biogenic emissions model (Guenther and Wiedinmyer, 2004). 

Emissions inventory development for episodic 8-hour ozone modeling must address several 

source categories including: (a) stationary point sources; (b) area sources; (c) on-road mobile 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-6

sources; (d) non-road mobile sources; (e) biogenic sources; and (f) fire sources. For this 

analysis, these estimates were developed for the June-July 2006 modeling period, the 36, 12 and 

4 km grids (Figure 1-1) and the CB05 chemical mechanism. 

CAMx requires two emission input files: (1) low-level gridded emissions that are emitted 

directly into the first layer of the model from sources at the surface with little or no plume rise; 

and (2) elevated point sources (stacks) with plume rise calculated from stack parameters and 

meteorological conditions. 

1.4.1.1 SMOKE Configuration 

The CAMx-ready emission files for anthropogenic emissions and the Fast Track emissions 

modeling were prepared using the SMOKE emissions modeling system. The SMOKE emissions 

model performs the following tasks: 

Temporal Adjustments: Adjust emission rates for seasonal, day-of-week and hour-of-day 

effects. The temporal allocation factors used are standard SMOKE adjustment factors, 

supplemented with state and county specific adjustments developed during the RPO 

modeling process. 

Chemical Speciation: Emission estimates for total VOC are converted to the more 

detailed chemical speciation used by the Carbon Bond 5 (CB05) chemical mechanism in 

CAMx. Total unspeciated NOx emissions are allocated to NO and NO2 components. 

The SMOKE emission model includes default speciation profiles by SCC codes. 

Spatial Allocation: Area and non-road mobile sources are estimated at the county level, 

and are allocated to the grid cells within each county based on spatial surrogates (e.g., 

population and economic activity). On-road mobile source emissions are also allocated to 

grid cells, using spatial surrogates based on roadway locations and population. The latest 

spatial surrogates available from EPA, dated April 2004, and were used in the emissions 

modeling. (http://www.epa.gov/ttn/chief/emch/spatial/newsurrogate.html) 

Quality Assurance: SMOKE includes quality assurance (QA) and reporting features to 

keep track of the adjustments at each processing stage and ensure that data integrity is not 

compromised. 

The outputs from the emissions model are called the “model-ready” emissions, and are dayspecific, gridded, speciated and temporally (hourly) allocated. SMOKE performs all of the 

processing steps for the anthropogenic emissions. The biogenic emissions were prepared using 

the MEGAN model using hourly MM5 meteorological data and gridded biomass type and 

density information. 

The configuration used in the SMOKE emissions modeling for the final Denver 2006 base case 

simulation is shown in Table 1-3. More details on the SMOKE emissions modeling is provided 

in the preliminary evaluation and sensitivity modeling report (Morris et al., 2008). 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-7

Table 1-3. Summary of SMOKE configuration parameters and data inputs. 

Emissions 

Component Configuration Details 

Model Code SMOKE Version 2.2 www.cmascenter.org

Horizontal 

Grid Mesh 36/12/4 km 

36 km grid 148 x 112 cells 

12 km grid 167 x 92 cells 

4 km grid 119 x 119 

Area Source 

Emissions 

2006 CDPHE/APCD for Colorado 

2005 projected from 2002 RPO 

outside Colorado 

SMOKE processing 

On-Road 

Mobile 

Sources 

SMOKE-MOBILE6 

County HPMS VMT and SMOKEMOBILE6 

In DMA replaced by CONCEPT link-based 

VMT mobile source emissions 

Point Sources 

2006 day-specific CEM 

Projected 2002 RPO outside 

Colorado 

Use 2006 day-specific hourly CEM for 

actual and processed CEM for Typical 

2006 emissions 

Off-Road 

Mobile 

Sources 

2006 CDPHE/APCD for Colorado 

Projected 2002 RPO outside 

Colorado 

O&G Sources WRAP Phase III for D-J Basin 

WRAP Phase II outside of D-J Bar-Ilan et al., 2008 

Emissions 

Data Sources 2006 CDPHE/APCD for Colorado 

 2002 Plan02b WRAP States Projected to 2005 

 2002 Base G CENRAP States Projected to 2005 

 2002 Base M MRPO States Projected to 2005 

 2002 Base G VISTAS States Projected to 2005 

 2006 HPMS for outside Colorado Process with SMOKE-MOBILE6 

 Acid Rain Database for CEM data Large stationary source NOx and SO2 

O&G WRAP Phase III for D-J 

Basin 

Biogenic 

Sources MEGAN Use day-specific hourly MM5 meteorology 

Wildfires 2006 From NCAR 

Temporal 

Adjustments Seasonal, day, hour Based on latest collected information updated for RPO modeling 

Chemical 

Speciation 

Revised CB05 Chemical 

Speciation EPA updated in 2007 

Gridding Spatial Surrogates based on 

landuse EPA updated in 2004 

Quality 

Assurance 

QA Tools in SMOKE and 

CONCEPT; PAVE plots; 

Summary reports 

 

1.4.2 Updated WRAP Phase III O&G Emissions for the D-J Basin 

The Independent Petroleum Association of Mountain States (IPAMS) is sponsoring the 

development of a Phase III regional oil and gas emission inventory for the inter-Mountain West 

jointly with the Western Regional Air Partnership (WRAP), to build on the WRAP Phase I and 

Phase II inventory projects (WRAP Phase III O&G Project). This effort is focused on creating a 

comprehensive criteria pollutant emissions inventory for all activities associated with oil and gas 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-8

field operations in the basins throughout the study region for year 2006 as well as future 

projection years; that includes all point and area sources related to the oil and gas industry. 

The initial region of interest for the WRAP Phase III O&G emission inventory is the DenverJulesburg (D-J) Basin, which includes the Denver and Northern Front Range 8-hour ozone 

nonattainment areas. The 2006 baseline inventory consists of two primary categories: sources 

subject to Air Pollution Emission Notice (APEN) reporting requirements, and sources exempt 

from APEN reporting. The WRAP Phase III project has completed the O&G emissions 

development for the 2006 base and 2010 future year (Bar Ilan et al., 2008a,b) and the reports are 

available on the WRAP website at: 

• http://www.wrapair.org/forums/ssjf/documents/eictts/OilGas/2008-

04_'06_Baseline_Emissions_DJ_Basin_Technical_Memo_(04-30).pdf

• http://www.wrapair.org/forums/ssjf/documents/eictts/OilGas/2008-

04_'10_Projection_Emissions_DJ_Basin_Technical_Memo(04-30).pdf

In general, the inventory was developed using a combination of well count and production 

activity from a commercially available database of oil and gas data maintained by IHS 

Corporation (“the IHS database”), the State of Colorado’s database of permitted sources 

including APENs sources and Regulation 7 reports, and detailed survey responses of oil and gas 

activity from several major participating companies that operate in the D-J Basin. Some 

additional data sources were also used, including American Petroleum Institute (API) technical 

literature, the US Environmental Protection Agency’s (EPA) AP-42 emissions factor technical 

guidance, the US EPA’s NONROAD emissions model, and the US EPA’s Natural Gas Star 

program technical guidance. 

1.4.2.1 Temporal and Geographic Scope 

The WRAP Phase III D-J O&G emissions inventory considers a base year of 2006 for purposes 

of estimating emissions, consistent with the June-July 2006 modeling episode for the Denver 8-

hour ozone SIP modeling. All data requested from participating companies were for these 

companies’ activities in the calendar year 2006. Similarly, all well count and production data for 

the basin obtained from the IHS database were for the calendar year 2006. Emissions from all 

source categories are assumed to be uniformly distributed throughout the year except for heaters 

and pneumatic pumps, which are assigned seasonality fractions as they are typically used 

primarily in winter. 

The geographic scope of this inventory is the D-J Basin, whose boundaries as defined by the US 

Geological Survey (USGS) were used. The USGS boundaries for the D-J Basin were intersected 

with the State of Colorado boundaries so that only the portion of the D-J Basin within Colorado 

was considered for this inventory. The following counties were wholly contained within the 

boundaries of the D-J Basin in this inventory: 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-9

• Adams 

• Arapahoe 

• Boulder 

• Broomfield 

• Crowley 

• Denver 

• Douglas 

• Elbert 

• El Paso 

• Fremont 

• Jefferson 

• Kit Carson 

• Larimer 

• Lincoln 

• Logan 

• Morgan 

• Phillips 

• Pueblo 

• Sedgwick 

• Teller 

• Washington 

• Weld 

• Yuma 

Figure 1-2 shows the boundaries of the D-J Basin, with the 2006 well locations extracted from 

the IHS database overlaid. 

Figure 1-2. D-J Basin boundaries overlaid and 2006 oil and gas well locations. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-10 

1.4.2.2 Well Count and Production Data 

Oil and gas related activity data across the entire D-J Basin were obtained from the IHS Enerdeq 

database queried via online interface. The IHS database uses data from the Colorado Oil and 

Gas Conservation Commission (COGCC) as a source of information for Colorado oil and gas 

activity. Two types of data were queried from the Enerdeq database: production data and well 

data. Production data includes information relevant to producing wells in the basin while well 

data includes information relevant to drilling activity (“spuds”) and completions in the basin. 

1.4.2.3 Sources Subject to APEN and Condensate Tanks Subject to Regulation 7 

On October 31, 2007 a request was made to the APCD for the 2006 Colorado APEN database for 

all oil and gas related emission sources covered by the following SCC and SIC codes: 

• All of the SCCs 202002*, 310*, 404003* (where * indicates all sub-SCCs for the SCC) 

• And only those with the following SICs: 13*, 492*, 4612. 

APEN data for the D-J basin were extracted and sorted by operator. Company specific APEN 

source data were forwarded to participating operators for a completeness review that included 

the following three issues: 

1) Source Categories that were missing from the APEN database, 

2) Specific sources missing from the database, and 

3) Sources within the database known to be no longer operating. 

Following the completeness review and the addition or deletion of sources as appropriate, 

emission rates were reviewed. Emission rates were updated to reflect 2006 actual emissions in 

cases where supporting data were available. Actual emission updates provided by operators 

followed the APCD calculation methodologies from existing permits or required Operation and 

Maintenance Plans. The APCD methodologies are used to update Annual Emission Calculations 

(Minor Sources) and 12-Month Rolling Emission Totals (Synthetic Minor and Major Sources). 

A separate request was made to APCD for a copy of the 2006 Regulation 7 atmospheric storage 

tank reports for year 2006. Within the Ozone Control Area, data from the Regulation 7 reports 

was utilized in place of the APEN data to represent stock tank emissions as the Regulation 7 

reports best reflected actual emissions. The Regulation 7 reports for condensate tanks were in 

the form of monthly reports of condensate throughput for each tank, and emissions for each tank, 

for all companies operating condensate tanks subject to Regulation 7 in the ozone non-attainment 

area. A macro was written in EXCEL to process the reports in such a way that monthly 

condensate throughput (bbl) and emissions (lb-VOC) could be extracted and summed. 

Confirmation was obtained that CDPHE’s annual Regulation 7 condensate tank emissions 

summary for 2006 was in reasonable agreement with the extracted emissions from the monthly 

Regulation 7 reports. 

GIS analysis was used to intersect the boundary of the ozone non-attainment area with the 

latitude/longitude coordinates of all APENs sources. Those sources falling within the ozone 

non-attainment area were filtered to remove any sources that were condensate tanks, based on 

SCC and SCC description. For purposes of summing all permitted oil and gas sources’ 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-11 

emissions in the D-J Basin, emissions from the remaining APENs sources (excluding condensate 

tanks in the ozone non-attainment area) were added to the summary emissions from all 

Regulation 7 condensate tank reports. 

1.4.2.4 APEN Exempt Sources 

Survey forms consisting of 11 Excel spreadsheets were forwarded to participating operators in 

the D-J basin. Each spreadsheet contained a request for specific data related to one of the 

following APEN exempt source categories: 

• Well blowdowns 

• Well completions 

• Drilling rigs 

• Exempt engines 

• Fugitive emissions 

• Heaters 

• Gas composition analysis for the basin 

• Pneumatic devices 

• Pneumatic pumps 

• Water tanks 

• Workover rigs 

The companies participating in the survey process for the D-J Basin represented 50% of well 

ownership in the basin, 63% of gas production in the basin, and 58% of oil production in the 

basin. This represented a sufficiently large percentage of oil and gas activity in the basin that it 

was felt that the responses obtained from the participating companies would be representative of 

all oil and gas operations in the basin. 

In addition to the source categories listed above, emissions from three additional APEN exempt 

source categories were estimated based on additional information requests from the participating 

companies: 

• APEN exempt atmospheric storage tanks 

• Truck loading activities 

• Flaring from condensate tanks 

1.4.2.5 WRAP Phase III D-J Basin O&G Summary Results

Results from the combined permitted sources (APENs sources excluding condensate tanks in the 

ozone non-attainment area, and condensate tanks in the ozone non-attainment area from the 

Regulation 7 reports), and the combined unpermitted sources are presented below on a county 

level and as summaries for the entire D-J Basin as a series of pie charts and bar graphs. 

Figure 1-3 shows that NOx emissions are primarily concentrated in Weld and Yuma counties, 

which is not surprising given that it is the areas of large concentrations of well locations. Figure 

1-4 shows that VOC emissions are primarily concentrated in Weld county only. Production 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-12 

activity in Yuma County is mostly dry gas, and therefore a smaller proportion of total VOC 

emissions occur in Yuma County. 

Figure 1-4 shows that compressor engines and drilling rigs combined account for almost 80% of 

NOx emissions. Similarly, Figure 1-5 shows that permitted and unpermitted condensate tanks 

and pneumatic devices account for approximately 81% of VOC emissions. Table 1-4 

summarizes the WRAP Phase III emissions for the D-J Basin and 2006 base case by county. A 

majority of the VOC (78%) and NOx (59%) D-J Basin O&G emissions come from Weld 

County. 

0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

Adams

Arapahoe

Boulder

Broomfld

Crowley

Denver

Elbert

Fremont

Jefferson

Kit Cars

Larimer

Lincoln

Logan

Morgan

Phillips

Pueblo

Sedgwick

Washngtn

Weld

Yuma

County

NOx Emissions (tons/year)

Other Categories

Glycol Dehydrator

Compressor Engines

Workover rigs

Venting -

recompletions

Pneumatic devices

Permitted Fugitives

Exempt engines

Drill rigs

Figure 1-3. 2006 NOx emissions by source category and by county in the D-J Basin. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-13 

0

20,000

40,000

60,000

80,000

100,000

120,000

140,000

Adams

Arapahoe

Boulder

Broomfld

Crowley

Denver

Elbert

Fremont

Jefferson

Kit Cars

Larimer

Lincoln

Logan

Morgan

Phillips

Pueblo

Sedgwick

Washngtn

Weld

Yuma

County

 VOC Emissions (tons/year)

Other Categories

Glycol Dehydrator

Compressor Engines

Venting - recompletions

Venting - initial completions

Venting - blowdowns

Truck loading of condensate liquid

Small condensate tanks

Pneumatic pumps

Pneumatic devices

Large condensate tanks

Permitted Fugitives

Unpermitted Fugitives

Drill rigs

Figure 1-4. 2006 VOC emissions by source category and by county in the D-J Basin. 

Drill rigs

25%

Exempt engines

14%

Other Categories

1%

Workover rigs

3%

Heaters

3%

Compressor Engines

55%

Figure 1-5. D-J Basin NOx emissions proportional contributions by source category. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-14 

Large condensate tanks

50%

Other Categories

1%

Pneumatic devices

14%

Permitted Fugitives

1%

Venting - recompletions

1%

Truck loading of 

condensate liquid

1%

Venting - initial 

completions

1%

Small condensate tanks

16%

Pneumatic pumps

1%

Venting - blowdowns

2%

Unpermitted Fugitives

9%

Glycol Dehydrator

1%

Compressor Engines

3%

Figure 1-6. D-J Basin VOC emissions proportional contributions by source category. 

Table 1-4. 2006 emissions of all criteria pollutants by county for O&G emissions in the D-J 

Basin. 

County 

NOx 

[tons/yr] 

VOC 

[tons/yr] 

CO 

[tons/yr] 

SOx 

[tons/yr] 

PM 

[tons/yr] 

Adams 2,286 3,005 939 13 19

Arapahoe 742 408 253 0 4

Boulder 129 803 76 1 4

Broomfield 14 193 10 0 0

Crowley 63 1 85 0 1

Denver 32 103 19 0 2

Douglas 0 0 0 0 0

Elbert 43 363 27 0 1

El Paso 0 0 0 0 0

Fremont 16 329 9 0 1

Jefferson 6 0 10 0 0

Kit Carson 10 139 6 0 1

Larimer 37 651 23 0 1

Lincoln 14 462 11 0 0

Logan 491 1,382 183 2 9

Morgan 672 883 672 132 4

Phillips 40 47 26 0 1

Pueblo 0 0 0 0 0

Sedgwick 1 11 0 0 0

Teller 0 0 0 0 0

Washington 284 4,509 207 1 9

Weld 12,310 64,111 8,393 51 421

Yuma 3,592 4,359 1,993 24 158

Totals 20,783 81,758 12,941 226 636
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-15 

1.4.3 ConCEPT On-Road Mobile Source Emissions for the DMA 

CONCEPT is an emissions model that performs the three key features of emissions processing: 

temporal allocation of the emissions (to hourly), spatial allocation of the hourly emissions to the 

grid cells in the modeling domain, and emissions speciation to develop emissions for species 

used by the air quality model. CONCEPT MV runs MOBILE6 to generate on-road mobile 

emissions in a very detailed manner. The basic steps in the calculation of on-road emissions 

using CONCEPT MV are as follows (additional details in Table 1-5): 

o Activity Data Preparation. CONCEPT estimates and grids link-level emissions using the 

output from Transportation Demand Models (TDMs). The TDM Transformation Tool, 

or T3, to processes the traffic demand model vehicle types, road networks, and vehicle 

activity to the file formats required by CONCEPT. 

o Temporal Allocation. Transportation demand model (TDM) data are typically provided 

for multi-hour periods. CONCEPT uses the total VMT hourly, daily, and monthly 

temporal profiles to split the multi-hour volumes to hourly volumes per link. 

o Speed Adjustment. CONCEPT calculates hourly volume/capacity ratios and applies 

adjustments to the free-flow speeds to account for congestion. 

o Spatial Allocation. The vehicle activity data for each link for each hour are spatially 

allocated to grid cells using an overlay of the link network on the model grid. 

o VMT Mix Profiles. VMT data are split into eight vehicle classes using the vehicle mix 

temporal profiles that vary by hour of day, day of week, and month of year. 

o Execute MOBILE6. MOBILE6 is executed by CONCEPT-MV for each link in each grid 

cell using the link-specific speed and grid-specific temperature and humidity from the 

MM5 modeling data files, and county-specific MOBILE6 model inputs. 

o Calculate link-level emissions. Emissions for each vehicle class, emission type, and 

pollutant for each link in each grid cell are estimated as the product of the emission factor 

and the VMT for each vehicle class. 

o Speciate the Emissions. CONCEPT MV applies the appropriate speciation profiles by 

pollutant and emission mode to generate the speciated emissions and create model-ready 

files for CMAQ or AERMOD. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-16 

Table 1-5. Summary of CONCEPT parameters and data inputs. 

Emissions Component Configuration Details 

Activity data preparation T3 used to format TDM 

data 

TransCAD output from DRCOG 

Temporal allocation Total volume and vehicle 

mix profiles generated from 

automated traffic counter 

data in the DMA 

The traffic counter data was provided by the 

Department of Transportation. A discussion 

of the traffic counter data analysis is provided 

in Appendix A. 

Speed adjustment BPR volume-delay applied 

by road type 

Curve coefficients provided by DRCOG 

Spatial allocation Links gridded to the 4km 

nest by CONCEPT 

VMT mix profiles Vehicle mix profiles 

generated from automated 

traffic counter data in the 

DMA 

The traffic counter data was provided by the 

Department of Transportation 

Execute MOBILE6 MOBILE6 inputs provided 

by CDPHE 

CONCEPT uses a customized version of M6 

(developed by Air Improvement Resource 

[AIR] under contract to LADCO) that 

summarizes the database output across 

model years within vehicle classes, and into 

the eight MOBILE5 vehicle classes employed 

in CONCEPT 

Speciate emissions The CB05 speciation 

profiles were developed for 

EPA 

QA Output reports from both T3 

and CONCEPT. 

Comparison with county 

totals generated by Alpine 

Geophysic’s application of 

the SMOKE model 

T3 summaries of VMT by county and time 

period provided to CONCEPT. CONCEPT 

reports of emissions (in tons) by county, 

emissions by vehicle type and emissions 

mode, summaries of M6 runs, total emissions 

in tons, hourly VMT, and summaries of 

adjusted and unadjusted speeds. 

The transportation demand model (TDM) data used in the CONCEPT modeling were provided 

by the Denver Regional Council of Governments (DRCOG) from their Integrated Regional 

Model. The activity data included: 

• Link Characteristics: Start and end node coordinates, link length, free-flow speeds, 

roadway type, hourly capacity, and county. 

• Link Volumes: Period-level directional total weekday average volumes for the following 

ten time periods: 630 – 700, 700 – 800, 800 – 900, 900 – 1130, 1130 – 1500, 1500 – 

1700, 1700 – 1800, 1800 – 1900, 1900 – 1900 – 2300, 2300 – 630. The period-level 

volume was pre-processed to whole hours, and then CONCEPT further disaggregated the 

volumes to hourly values using the temporal profile inputs. 

• Speed Adjustment: Bureau of Public Roads (BPR) speed adjustment curve coefficients 

A plot of the 2005 transportation network is presented in Figure 1-7. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-17 

Figure 1-7. DRCOG link-based transportation network used in the CONCEPT on-road mobile 

source emissions modeling. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-18 

Area Type 

The area types in the TDM output were represented by numeric codes that were translated as 

follows: 

Code Description Urban/Rural 

1 CBD Urban 

2 Fringe Urban 

3 Urban Urban 

4 Sub-Urban Urban 

5 Rural Rural 

Roadway Type

The TDM functional classes were provided as numeric codes that were translated to HPMS 

roadway types as follows: 

Code Urban/Rural Functional Class HPMS Type (Code) 

1 Rural freeway Principal Arterial - Interstate (01) 

2 Rural major regional arterial Principal Arterial - Other (02) 

3 Rural principal arterial Principal Arterial - Other (02) 

4 Rural minor arterial Minor Arterial (06) 

5 Rural collector Major Collector (07) 

6 Rural ramp Ramp (added to CONCEPT for modeling 

purposes) (03) 

8 Rural zone connector Local System (09) 

1 Urban freeway Principal Arterial - Interstate (11) 

2 Urban major regional arterial Principal Arterial - Other Freeways or 

Expressways (12) 

3 Urban principal arterial Principal Arterial - Other (14) 

4 Urban minor arterial Minor Arterial (16) 

5 Urban collector Collector (17) 

6 Urban ramp Ramp (added to CONCEPT for modeling 

purposes) (13) 

8 Urban zone connector Local System (19) 

The VMT was calculated as the volume times the link length. Note that the VMT from the TDM 

centroid connectors was included as local road VMT. 

Growth

DRCOG provided TDM output for 2005, 2015, and 2020. The 2005 transportation data were 

grown to 2006, and the 2015 data were back casted to 2010 using the county-level growth and 

back casting factors interpolated between the 2005 and 2015 county-level VMT. 

The county total VMT from the TDM and the 2006 and 2010 interpolated totals are presented in 

Table 1-6. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-19 

Table 1-6. Annual county-level total average weekday VMT from the TDM. 

County 2005 

2006 

Interpolated 

2010 

Interpolated 2015 2020 

Adams 10,922,773 11,289,709 12,757,453 14,592,132 16,440,119

Arapahoe 10,931,247 11,253,596 12,542,993 14,154,738 15,673,382

Boulder 5,737,221 5,824,590 6,174,064 6,610,906 7,014,825

Broomfield 2,006,314 2,040,202 2,175,753 2,345,192 2,620,702

Clear Creek 36,082 38,101 46,175 56,267 70,371

Denver 14,430,681 14,731,101 15,932,782 17,434,883 18,292,519

Douglas 7,877,685 8,144,715 9,212,837 10,547,989 11,777,265

Gilpin 14,933 15,170 16,119 17,305 18,872

Jefferson 13,148,925 13,385,173 14,330,163 15,511,400 16,614,625

Weld 2,746,625 2,879,718 3,412,088 4,077,550 4,651,258

Total 67,852,486 69,602,074 76,600,424 85,348,362 93,173,938

Speed Adjustment

After CONCEPT disaggregated the total volumes into hourly values, the free-flow speeds were 

adjusted to congested speeds as appropriate using the most common volume-delay function (the 

Bureau of Public Roads curve, or BPR curve). The general form of this function is defined as: 

( ) ⎥⎦

⎤ ⎢⎣⎡ +

= B

ff

a

C A V

S

S

1 *

where: 

Sa = actual link speed (mph) 

Sff = reported link free flow speed (mph) 

V = total link volume (vehicles OR vehicles per hour) 

C = total link capacity (vehicles OR vehicles per hour) 

A, B = curve calibration coefficients 

The A and B coefficients employed were the revised coefficients in DRCOG’s Highway 

Skimming and Assignment Documentation, December 2004. The roadway-specific coefficients 

are presented in Table 1-7. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-20 

Table 1-7. BPR speed adjustment coefficients used in estimating congested speeds. 

A coefficients 

Area 

Type Freeway 

Major 

Arterial Principal 

Minor 

Arterial Collector 

CBD 0.70 0.10 0.20 0.20 0.20 

Fringe 0.70 0.10 0.30 0.30 0.30 

Urban 0.40 0.25 0.40 0.40 0.40 

Suburban 0.40 0.25 0.40 0.40 0.40 

Rural 0.40 0.40 0.40 0.40 0.40 

B Coefficients 

CBD 5.50 7.50 5.00 5.00 5.00 

Fringe 5.50 7.50 5.00 5.00 5.00 

Urban 7.50 7.50 5.50 5.50 5.50 

Suburban 7.50 7.50 5.50 5.50 5.50 

Rural 5.00 5.00 5.00 5.00 5.00 

The speeds of local roads were not adjusted. 

Weekday/weekend Temporal Adjustment

CONCEPT disaggregates period-level VMT into hourly values by assigning the relative 

proportion of VMT for a given hour within a period based on the hourly temporal profiles. 

Because the TDM data are reflective of weekday volumes, we generated a second set of inputs to 

CONCEPT for the weekend day simulations where the period-level VMT were summed to 

create a daily total. This daily total was then fed to CONCEPT, which then disaggregated the 

VMT according to a true weekend temporal distribution. 

Additional Temporal Adjustments 

We received from the Colorado Department of Transportation 2005 and 2006 total volume and 

vehicle classification data in the DMA from automated traffic counters. The volume data 

consisted of the total number of vehicles counted on a roadway by day and hour. The vehicle 

mix profiles were generated using vehicle classification data provided by CDOT that counted the 

number of vehicles of each different class by day and hour. Data from both years was merged 

together in the temporal analysis; in other words, no distinction was made by year. 

All the observation sites in the total volume and vehicle classification data were assigned to a 

Federal Highway Administration (FHWA) roadway function class. The cross-reference between 

the CDOT roadway classifications and FHWA classifications are presented in Table 1-8. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-21 

Table 1-8. FHWA Roadway functional classification. 

CDOT Road Type FHWA Classification and Code

Collector Urban Collector (17) 

Freeway Urban Interstate (11) 

Interstate Urban Interstate (11) 

Major Regional Urban Principal Arterial - Other Freeways or Expressways (12) 

Minor Arterial Urban Minor Arterial (16) 

Other Principal Arterial Urban Principal Arterial - Other Freeways or Expressways (12) 

Principal Urban Major Arterial (14) 

Principal Arterial Urban Major Arterial (14) 

The vehicle classes in the classification data are somewhat different than the vehicle classes in 

MOBILE5 or MOBILE6. They are presented in Table 1-9. 

Table 1-9. Vehicle classifications. 

FHWA 

Class VTRIS Vehicle Type 

1 Motorcycle 

2 Passenger cars 

3 Other 2-axle, 4-tire single unit vehicles 

4 Buses 

5 2-axle, 6-tire single-unit vehicles 

6 3-axle, 6-tire single-unit vehicles 

7 4+ axle single-unit vehicles 

8 4 or less axle combination vehicles 

9 5-axle combination vehicles 

10 6+ axle combination vehicles 

11 5-axle multi-trailer vehicles 

12 6-axle multi-trailer vehicles 

13 7+ axle multi-trailer vehicles 

14 Unclassified 

15 Unclassifiable 

The estimated fraction of each FHWA vehicle class assigned to the MOBILE5 vehicle classes is 

shown in Table 1-10. 

Table 1-10. Fractional Allocation of FHWA Vehicle Classes to MOBILE5 Vehicle Classes. 

FHWA 

Class LDGV LDGT1 LDGT2 HDGV LDDV LDDT HDDV MC 

1 0 0 0 0 0 0 0 1 

2 0.52225 0.35340 0.11183 0.00748 0.00085 0.00180 0.00238 0 

3 0.51365 0.34758 0.11956 0.00689 0.00084 0.00190 0.00957 0 

4 0 0 0 0.16928 0 0 0.83072 0 

5 0 0.24070 0.19405 0.12262 0 0.00287 0.43976 0 

6 0 0.24070 0.19405 0.12262 0 0.00287 0.43976 0 

7 0 0.24070 0.19405 0.12262 0 0.00287 0.43976 0 

8 0 0.00031 0.00701 0.02044 0 0.00010 0.97214 0 

9 0 0.00031 0.00701 0.02044 0 0.00010 0.97214 0 

10 0 0.00031 0.00701 0.02044 0 0.00010 0.97214 0 

11 0 0.00031 0.00701 0.02044 0 0.00010 0.97214 0 

12 0 0.00031 0.00701 0.02044 0 0.00010 0.97214 0 

13 0 0.00031 0.00701 0.02044 0 0.00010 0.97214 0 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-22 

The CDOT data was read into a Microsoft SQL Server database. The total volume and vehicle 

classification data was processed in the following steps: 

1) All counts across lanes in the same roadway direction were totaled. Different directions 

at site were treated separately. 

2) All counts (either total volume or count for each vehicle class) were averaged for each 

site-direction pair by hour, day of week (i.e., Sunday through Saturday), month, and 

roadway classification. This means that at most five values were averaged together, 

corresponding to the total number of days in a week during one month. In other words, 

all Monday counts during January for hour 10 were averaged together at each sitedirection pair. This averaging by site first was done so that sites that may not have had as 

long a period of observation would be weighted equally in the average across the sites. 

3) The counts were averaged across the sites. These averages were calculated by roadway 

function class, vehicle class, month, day of week, and hour of the day. 

4) Any holes in the data (where either data was missing or the data was not representative) 

were filled in. 

5) The FHWA vehicle classes were mapped into MOBILE5 vehicle classes. 

To calculate the temporal profiles, the total volume by roadway class was tallied for the year, 

month, and day of week. The monthly profile fractions were calculated by dividing the monthly 

totals by the yearly total. The day of week profiles were calculated by dividing the day of week 

totals by the monthly total for each month. Finally the hourly totals were calculated by dividing 

the hourly counts by the day of week totals (summed over all 24 hours) for each day of week and 

month. Each row in the temporal profiles sum to 1. 

The vehicle mix profiles were calculated as the fractions contributed by each vehicle class to the 

total hourly volume. So for each roadway function class, month, day of week, and hour, the 

hourly vehicle mix profiles were calculated by dividing each vehicle class count by the total 

volume for that hour. 

Before proceeding with the analysis, the following data was dropped to ensure the integrity of 

the profiles: 

• Dropped 65 duplicate 2005 vehicle classification records corresponding to 10/29/2005 at 

sites 3, 103, 108, 124, 215, 245, 247, 256, 504, 507, 512, 609, 103608, 103694, 103712, 

and 11/13/2005 at site 107. 

• Removed duplicated “05” records from the total volume data, where identical volume 

counts were recorded with the year “2005.” 

• Replaced corrupted 2006 classification table stations 10, 12 & 103387 with corrected 

data, and dropped 144 records of other corrupted records in this dataset. 

• Dropped the vehicle classification sites 7, 215 & 247 corresponding to FHWA facility 

class 16 (Minor Arterial) because according to CDOT, they were not representative of the 

Denver Area. 

• Dropped all vehicle classification records for which the total hourly volume was less than 

3. 

• Dropped all records for which more than 25% of the hours in a day had zero volume. 

• Dropped all records for which there were not 24 hours of observations in a day. 

• Dropped sites for which there was not a complete week in a given month. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-23 

• For the total volume monthly profiles, dropped all sites for which there were not 12 

months worth of data after incomplete weeks were dropped. 

There was enough data to create profiles for the following facility classes: 

01 – Rural Interstates 

02 – Rural Principal Arterials – Other 

06 – Rural Minor Arterials 

11 – Urban Interstates 

12 – Urban Principal Arterials – Other Freeways or Expressways 

14 – Urban Principal Arterials – Other 

We did not have enough data from the remaining classes: 

07 (Rural Major Collector), 08 (Rural Minor Collector), 09 (Rural Local), 16 (Urban Minor 

Arterial), 17 (Urban Collector), or 19 (Urban Local). For the total volume profiles, we used the 

profiles generated for class 06 and applied them to classes 07, 08, and 09. We applied the total 

volume profiles from class 14 to classes 16, 17, and 19. 

Figures 1-8 through 1-10 display the monthly, July day of week, and July urban interstate hourly 

total volume temporal profiles. Figures 1-11 and 1-12 display the vehicle mix hourly profiles for 

rural and urban interstates for a week in July. 

Denver Monthly Traffic Timeseries by Facility Class

0

0.02

0.04

0.06

0.08

0.1

0.12

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Month

Fraction of Total Traffic

FC1

FC2

FC6

FC11

FC12

FC14

Figure 1-8. Monthly temporal profiles for facility classes 1 through 14. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-24 

Denver Day of Week Profiles for Urban Interstates (NFC=11)

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

Sun Mon Tue Wed Thu Fri Sat

Day

Fraction of Total Traffic

Jun

Jul

Figure 1-9. Day of week temporal profiles for June and July. 

Denver Hourly Profiles for Urban Interstates (FC=11)

0

0.02

0.04

0.06

0.08

1 5 9 13 17 21 1 5 9 13 17 21 1 5 9 13 17 21 1 5 9 13 17 21 1 5 9 13 17 21 1 5 9 13 17 21 1 5 9 13 17 21

Hour

Fraction of Total Traffic

Jun

Jul

Figure 1-10. Hourly temporal profiles for urban interstates for a week in June and July. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-25 

Denver Rural Interstates (NFC=1), July

Hourly Class Fractions Sunday through Saturday

0

0.1

0.2

0.3

0.4

0.5

0.6

1 5 9 13 17 21 1 5 9 13 17 21 1 5 9 13 17 21 1 5 9 13 17 21 1 5 9 13 17 21 1 5 9 13 17 21 1 5 9 13 17 21

HDDV HDGV LDDT LDDV LDGT1 LDGT2 LDGV MC

Figure 1-11. Vehicle mix hourly profile for rural interstates in July. 

Denver Urban Interstates (NFC=11), July

Hourly Class Fractions Sunday through Saturday

0

0.1

0.2

0.3

0.4

0.5

0.6

1 5 9 13 17 21 1 5 9 13 17 21 1 5 9 13 17 21 1 5 9 13 17 21 1 5 9 13 17 21 1 5 9 13 17 21 1 5 9 13 17 21

HDDV HDGV LDDT LDDV LDGT1 LDGT2 LDGV MC

Figure 1-12. Vehicle mix hourly profile for urban interstates in July. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec1_MPE_Intro1.doc August 2008 1-26 

Combining ConCEPT DMA and SMOKE Colorado On-Road Mobile Emissions

The SMOKE-MOBILE6 and CONCEPT on-road mobile source emissions modeling both 

produced on-road mobile source emissions for the area covered by the DRCOG link-based VMT 

network. To avoid double counting on-road mobile source emissions in the DMA, a mask was 

developed that identified all 4 km grid cells that were covered by the DRCOG link-based 

network. The SMOKE-MOBILE6 on-road mobile source emissions in the area covered by the 

DMA mask were then eliminated so that they were replaced by the CONCEPT derived on-road 

mobile source emissions when the SMOKE-MOBILE6 and CONCEPT files were merged 

together. 

 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-1 

2.0 OZONE OPERATIONAL MODEL PERFORMACE EVALUATION 

2.1 INTRODUCTION 

This Chapter describes the operational model performance evaluation for ozone and the final 

CAMx 2006 base case simulation (Run 17) of the June-July 2006 modeling period using the 

final model configuration. That is, the ozone operational model initial evaluation focuses on 

how well the model reproduces the observed surface ozone concentrations during the June-July 

2006 modeling period. In Chapter 3 we present a diagnostic model performance evaluation that 

focuses on the model performance for ozone precursors (VOC, NOx and CO), ozone aloft and 

particulate matter (PM) species. 

2.1.1 Purpose of the Model Performance Evaluation 

The purpose of the model performance evaluation is to establish a reliable CAMx 8-hour ozone 

modeling database for the DMA that can be used for projecting future year 8-hour ozone 

concentrations. In general terms, this process consists of the following cycle: 

• Exercise the modeling system for a series of base case simulations attempting to 

replicate the time and space behavior of the observed 1-hour and 8-hour ozone 

concentration fields as well as concentrations of precursor and product species; 

• Identify sources of error and/or compensating biases, through evaluation of 

preprocessor models (MM5, SMOKE, CONCEPT, GloBEIS, MEGAN), air quality 

model inputs, concentrations aloft, mass budgets and conservation, process analysis, 

etc; 

• Through a documented process of diagnostic and sensitivity investigation, pinpoint 

and correct the performance problems via model refinement, additional data 

collection and/or analysis, or theoretical considerations; 

• Re-run the model for the refined base case and re-evaluate performance until 

adequate, justifiable performance is achieved or the modeling period is declared 

unsuited for further use based on documented performance problems. 

In practice this process is limited by time and resource constraints. The preliminary model 

performance evaluation and diagnostic sensitivity tests performed most of the elements listed 

above (Morris et al., 2008b), whereas this document focuses on the last item; the documentation 

of the model performance of the final 2006 base case simulation and justification for its use in 

the 8-hour ozone attainment demonstration modeling. 

2.1.2 Approach for the Operational Ozone Model Evaluation 

The CAMx performance evaluations follows the procedures recommended in the EPA 

photochemical modeling guidance documents (EPA, 1991; 1999; 2005a; 2007). The evaluation 

was carried out in two sequential phases, beginning with the simplest comparisons of modeled 

and observed ground-level ozone concentrations presented in this Chapter follows by potentially 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-2 

more illuminating analyses presented in Chapter 3 (e.g., examination of available precursor and 

product species, comparisons of pollutant ratios and groupings, comparison against PM species, 

deposition and visibility). The CAMx evaluation will use most of the following six means for 

assessing photochemical model performance as specified in EPA’s guidance are as follows: 

• Use of computer generated graphics; 

• Use of ozone metrics in statistical comparisons; 

• Comparison of predicted and observed precursor emissions or species concentrations; 

• Comparison of observed and predicted ratios of indicator species; 

• Comparison of predicted source category contribution factors with estimates obtained 

using observational models as available; and 

• Use of retrospective analyses in which air quality differences predicted by the model are 

compared with observed trends. 

The Denver model performance evaluation focuses mainly on the first three of these elements for 

evaluating model performance, with some presentation of the fourth element that compares some 

predicted and observed indicator ratios presented in Chapter 3. The Denver ozone modeling did 

not perform any ozone source apportionment modeling for the 2006 base case, although such 

modeling was performed for the 2010 base case emissions scenario that is reported with the 2010 

modeling. In addition, we did not perform any retrospective modeling to compare the modeling 

system’s response to past changes in emissions. Although in hindsight, the Denver 8-hour ozone 

EAC SIP modeling of the June-July 2002 episode using essentially the same MM5/CAMx 

modeling system did perform some prospective modeling where the modeled changes in ozone 

between 2002 and 2007 compared favorable with the observed changes. The maximum 2007 

projected 8-hour ozone Design Value predicted by the CAMx model using the 2002 EAC 

modeling database was 85 ppb at the Rocky Flats North monitor, which agreed exactly with the 

observed 2005-2007 8-hour ozone Design Value at that site. 

2.1.2.1 Available Aerometric Data for the Evaluations 

The model performance evaluation is limited by the availability of monitoring data in the region. 

Table 2-1 and Figure 2-1 display the ozone monitoring network in the DMA and vicinity. Also 

shown in Table 2-1 are the years of operation of the Denver ozone monitoring sites. Note that 

the Chatfield (CHAT) site was moved in 2004. The 2010 8-hour ozone Design Value 

projections will be based on current year Design Values from the 2005-2007 period, so only sites 

with ozone observations from those years will have monitor-specific Design Value projections. 

Although a two-year Design Value (2006-2007) was used for the Fort Collins West monitor that 

started in 2006. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-3 

Table 2-1. Ozone monitors operating within and near the DMA during 2002-2007. 

ID Number Site Name Site ID 2002 2003 2004 2005 2006 2007 Lat Long 

080013001 Welby WELB x x x x x x 39.838 -104.950 

080050002 Highland HIGH x x x x x x 39.568 -104.957 

080130011 S. Boulder Creek SBC x x x x x x 39.957 -105.238 

080310002 CAMP CAMP x x x 39.751 -104.988 

080310014 Carriage CARR x x x x x x 39.752 -105.031 

080350002 Chatfield #1 CHAT x x x 39.538 -105.065 

080350004 Chatfield #2 CHAT x x x x 39.534 -105.070 

080410013 USAF Academy ACAD x x x x x x 38.958 -104.817 

080410016 Manitou Springs MANI x x x x 38.853 -104.901 

080590002 Arvada ARVA x x x x x x 39.800 -105.100 

080590005 Welch WELC x x x x x x 39.639 -105.139 

080590006 RFN RFNO x x x x x x 39.913 -105.189 

080590011 NREL NREL x x x x x x 39.744 -105.178 

080590012 Lookout Mountain LOOK x 39.727 -105.247 

080690007 RMNP RMNP x x x x x x 40.277 -105.545 

080690011 FtC West FTCW x x 40.593 -105.141 

080691004 Ft. Collins FTCO x x x x x x 40.577 -105.079 

081230007 Greeley GREE x 40.416 -104.692 

081230009 Weld Co. Tower WCTO x x x x x x 40.386 -104.737 

-800 -760 -720 -680 -640 -600

-120

-80

-40

0

40

80

120

SBC

ACAD

 ARVA

NREL 

RMNP

FTCW

 GREE

NIWOT

CHAT CHAT

MANI

FTCO

WCTO

 CARR

WELB

HIGH

CAMP

 RFNO

WELC

LOOK

Figure 2-1. Location of ozone monitoring sites within and near the Denver 

Metropolitan Area (DMA) (triangle symbols point to monitor identifier).

 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-4 

During 2006, two of the routine monitoring sites (CAMP and WELB) also collected NO2 

measurements in the DMA and 13 monitors collected CO as well. 

We are aware of three “field study” campaigns in the general Denver area during 2006 that 

provides additional data to evaluate the modeling system: 

• The CDPHE/APCD collected VOC samples during 2006 that will assist in evaluating this 

important ozone precursor. Most VOC samples were collected in the morning (6-9am 

MDT) with samples for the following sites available during some days of the June-July 

2006 modeling episode: 

• CAMP 

• Welby 

• Fort Lupton 

• Platteville 

• Rocky Flats North 

• Fort Collins West 

• During portions of the 2006 episode (end of July), NOAA launched daily ozonesondes at 

numerous sites throughout the U.S., including Boulder. The ozonesonde measurements 

will provide valuable information on the vertical structure of ozone concentrations, 

including the potential identification of an ozone reservoir aloft. 

• Finally, the NPS and CDPHE collected special measurements as part of the ROMANS 

study during 2006. 

The comparisons against these field study data are done in the Chapter 3 diagnostic evaluation 

section. 

2.1.2.2 Model Performance Goals and Criteria 

As discussed in the Denver Modeling Protocol (Morris et al., 2007) and preliminary model 

performance evaluation (Morris et al., 2008b), we are using several model performance goals 

and criteria based on EPA guidance documents and past studies to assist in gauging the 2006 

base case model performance. The model performance goals and criteria are not meant to be 

pass/fail tests, but rather to help frame the model performance and put it into contact. EPA’s 1-

hour ozone modeling guidance presented three model performance goals for hourly ozone that 

are listed in Table 2-2. 

Table 2-2. EPA’s 1-hour ozone performance goals (EPA, 1991). 

Performance Statistics Performance Goal 

Unpaired Peak Accuracy <±20% 

Mean Normalized Bias <±15% 

Mean Normalized Gross Error <35% 

EPA’s 1999 draft 8-hour ozone modeling guidance presented a useful performance goal that 

compares the observed daily maximum 8-hour ozone concentrations with modeled values “near 

the monitor”, with the goal being that most of the observed and modeled value near the monitor 

pairs be within ±20% of each other. This performance goal is particularly relevant because the 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-5 

modeled daily maximum 8-hour ozone concentrations near the monitor are the very values used 

in making the 8-hour ozone projections. As used in the Denver EAC SIP (Morris et al., 2003), 

we have developed three approaches for defining “near the monitor”. For two of the approaches 

we define “near” as the same NX by NY array of cells centered on the monitor as used in EPA’s 

8-hour ozone attainment test (e.g., 7 x 7 for 4-5 km grid) and the two tests differ in only which 

estimated value is selected from this array of cells. For the third test, we select the estimated 

value at the monitor (i.e., spatially paired). The three methods for defining “near the monitor” 

are as follows: 

Maximum: Select the maximum estimated daily maximum 8-hour ozone concentration 

near the ozone monitor for each day. This is the same approach used in EPA’s 8-hour 

ozone attainment test. 

Closest: Select the estimated 8-hour ozone concentrations near the monitor that matches 

the observed value best. 

Spatially Paired: Select the estimated 8-hour ozone concentrations at the monitoring 

location. 

Table 2-3 summarizes the performance goal for daily maximum 8-hour ozone concentrations 

using these three approaches for “near the monitor”. 

Table 2-3. 8-hour ozone model performance goal comparing observed daily maximum 8-hour 

ozone concentrations with predicted values “near the monitor” (EPA, 1999). 

“Near The Monitor” Threshold Goal 

Maximum modeled daily maximum 8-hour ozone 

concentrations within a 7 x 7 array of 4 km grid cells around 

monitor 

<±20% Most pairs within ±20% 

Closest modeled daily maximum 8-hour ozone concentrations 

within a 7 x 7 array of 4 km grid cells around monitor to the 

observed value at the monitor. 

<±20% Most pairs within ±20% 

Spatially Paired modeled daily maximum 8-hour ozone 

concentration at the location of the monitoring site 

<±20% Most pairs within ±20% 

For PM species, several of the RPOs conducting regional haze modeling have adopted three 

levels of model performance goals and criteria for PM species fractional bias and gross error as 

listed in Table 2-4. Note that we are not suggesting that these performance goals be adopted as 

guidance. Rather, we are just using them to frame and put the PM model performance into 

context and to facilitate model performance intercomparison across episodes, species, models, 

sensitivity tests and studies. 

Table 2-4. Model performance goals and criteria for components of fine particle mass. 

Fractional Bias Fractional Error Comment 

<15% <35% Goal for PM model performance based on ozone model 

performance, considered excellent performance 

<30% <50% Goal for PM model performance, considered good 

performance 

<60% <75% Criteria for PM model performance, considered average 

performance. Exceeding this level of performance 

indicates fundamental concerns with the modeling 

system and triggers diagnostic evaluation. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-6 

The above performance statistics are based use the Unpaired Peak Accuracy and normalized and 

fractional bias and gross error metrics. These statistical performance metrics are defined in 

Table 2-5. 

Table 2-5. Definitions of statistical performance measures used in the model evaluation. 

Statistical 

Measure 

Shorthand 

Notation 

Mathematical Expression Notes 

Accuracy of 

Paired Peak 

Ap

peak

peak

O

P − O Predicted value P is located at 

same monitor as observed peak 

Unpaired Peak 

Accurcay 

UPA 

peak

peak

O

P − O Predicted value P taken as the 

unpaired maximum anywhere in the 

local region 

Mean Fractional 

Error 

MFE 

∑= +

− N

i i i

i i

P O

P O

N 1

2 Reported as % 

Mean Absolute 

Gross Error 

MAGE ∑=

−

N

i

Pi Oi N 1

1

Mean 

Normalized 

Gross Error 

MNGE 

∑=

N −

i i

i i

O

P O

N 1

1 Reported as % 

Mean Bias MB ∑( ) =

−

N

i

Pi Oi N 1

1 Reported as concentration 

Mean 

Normalized Bias 

MNB ( ) ∑=

− N

i i

i i

O

P O

N 1

1 Reported as % 

Mean 

Fractionalized 

Bias 

(Fractional Bias) 

MFB 

∑= ⎟

⎟

⎠

⎞

⎜

⎜

⎝

⎛

+

− N

i i i

i i

P O

P O

N 1

2 Reported as % 

2.2 OVERVIEW OF MODEL PERFORMANCE EVALUATION APPROACH 

The model performance evaluation of the final CAMx Run 17 2006 base case simulation of the 

June-July 2006 Denver ozone episode makes use of graphical, statistical and diagnostic 

evaluation tools. We first present a series of standard ozone model performance statistics for the 

entire June-July 2006 episode and compare them with the performance goals. Although such 

comparisons are not a comprehensive evaluation, they do identify if something is greatly wrong 

and provide a context for comparison model performance with other studies. The general CAMx 

ozone model performance over the entire episode is then discussed. 

We then focus on the ozone model performance for three 3-day episodes when observed 

exceedances of the 8-hour ozone standard occurred in the Denver NAA. The details of the ozone 

model performance for these 3-day episodes are presented. Figure 2-2 displays the region-wide 

maximum 8-hour ozone concentration in the Denver NAA for the June-July 2006 modeling 

period, with the three 3-day episodes selected for more detailed analysis indicated by the red 

dotted lines at the 85 ppb ozone NAAQS level. These three 3-day episodes include 7 of the 9 8-

hour ozone exceedance days during the June-July 2006 modeling period and are as follows: 

• June 17-19, 2006 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-7 

• July 13-15, 2006 

• July 27-29, 2006 

Figure 2-2. Maximum daily maximum 8-hour ozone concentration (ppb) observed during the 

June-July 2006 episode and three “mini episodes” selected for more detailed analysis: June17-

19, July 13-15, and July 27-29. 

2.3 OPERATIONAL OZONE MODEL PERFORMANCE FOR THE JUNE-JULY 

2006 MODELING PERIOD 

Figures 2-3 and 2-4 display the daily ozone modeling performance statistical metrics during 

June-July 2006 and the final CAMx Run 17 2006 base case simulation for, respectively, 1-hour 

and 8-hour ozone concentrations in the Denver NAA, and compares them with EPA’s 1-hour 

ozone performance goals for Unpaired Peak Accuracy (≤±20%), Mean Normalized Bias 

(≤±15%) and Mean Normalized Gross Error (≤35%). EPA (1991) developed these ozone 

statistical performance goals for 1-hour ozone, but we have also applied them for 8-hour ozone 

performance for testing purposes as well. The ozone statistical performance measures are 

calculated using all ozone predicted and observed pairs (matched by time and location) for which 

the observed values is greater than a threshold values; a 60 ppb and 40 ppb observed threshold 

value was used for the 1-hour and 8-hour ozone performance measures, respectively. 

There are five panels of daily ozone model performance presented in these figures for the JuneJuly 2006 modeling period. The top panel presents the observed region-wide daily maximum 8-

hour ozone concentration and the predicted value that is spatially paired at the same site. Not 

surprising, the modeled predicted daily maximum 8-hour ozone concentrations at the site with 

the observed maximum is almost always lower. The second panel presents the Unpaired Peak 

Accuracy metric and compares it to the ≤±20% EPA performance goal. The third panel is the 

Average Peak Paired Accuracy which is essentially the Normalized Mean Bias of predicted and 

observed daily maximum 8-hour ozone concentrations paired at each monitoring site. The fourth 

and fifth panels show the normalized bias and gross error statistical performance metrics that are 

compared to EPA’s ≤±15% and ≤35% performance goals, respectively. 

2.3.1 Ozone Performance Statistics for 1-Hour Ozone 

Denver Highest Observed 8-Hour Ozone during June-July 2006

50

55

60

65

70

75

80

85

90

95

100

1-Jun-063-Jun-065-Jun-067-Jun-069-Jun-06

11-Jun-0613-Jun-0615-Jun-0617-Jun-0619-Jun-0621-Jun-0623-Jun-0625-Jun-0627-Jun-0629-Jun-06

1-Jul-063-Jul-065-Jul-067-Jul-069-Jul-06

11-Jul-0613-Jul-0615-Jul-0617-Jul-0619-Jul-0621-Jul-0623-Jul-0625-Jul-0627-Jul-0629-Jul-0631-Jul-06

Ozone (ppb)

1st

Date
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-8 

The 1-hour ozone Unpaired Peak Accuracy performance metric achieves the ≤±20% 

performance goal for all days of the June-July 2006 modeling period except two days (second to 

top time series in Figure 2-3). On June 19, 2008 the modeled region-wide maximum 8-hour 

ozone concentrations is -20.8% below the observed value so just barely does not meet EPA’s 

≤±20% performance goal. And on July 24, 2006, the 1-hour ozone Unpaired Peak Accuracy is 

22.9% that slightly exceeds the ≤±20% performance goal. The modeled unpaired peak on this 

day occurs to the southeast of Denver (see Appendix A) where no monitors exist so the presence 

of this higher modeled value can not be verified by the current monitoring network in the Denver 

NAA. In fact, more than two thirds of the time (73% of the days) the 1-hour ozone Unpaired 

Peak Accuracy metric achieves a more stringent ≤±10% performance goal (that is the final 

CAMx 2006 base case simulation modeled peak 1-hour ozone concentrations in the Denver 

NAA is within 10% of the observed peak on a vast majority of the days during the June-July 

2006 modeling period). 

The third daily time series panel in Figure 2-3 presents the 1-hour ozone Average Accuracy of 

the Peak performance measure. This is the Mean Normalized Bias for daily maximum 1-hour 

ozone concentrations across all sites in the Denver NAA matched by location and day (but may 

not match by time of day). There is no performance goal for this metric, but it appears most days 

during the June-July 2006 modeling period are within ±20%. 

The bottom two panels in Figure 2-3 displays the daily Mean Normalized Bias and Mean 

Normalized Gross Error performance metrics for 1-hour ozone for which EPA has performance 

goals of within ≤±15% and ≤35%, respectively. The Mean Normalized Gross Error always 

achieves the ≤35% performance goal and, in fact, is always ≤20% and frequently ≤10%. These 

are quite good performance metrics for Mean Normalized Gross Error. 

Of the 30 days during June 2006, 26 days achieve the <±15% performance goal for 1-hour ozone 

Normalized Bias (i.e., 87 percent of the days in June 2006 achieve the 1-hour ozone bias 

performance goal). Of the four days in June that do not achieve the bias performance goal, three 

are due to an underprediction bias of -16.8% (June 13), -17.2% (June 17) and -16.4% (June 24) 

and one is due to an overprediction bias of 18.1% (June 28). 

The 1-hour ozone Mean Normalized Bias metrics for June are fairly balanced with days of both 

underprediction and overprediction bias (Figure 2-3a). However, in July a normalized bias 

undeprediction occurs on most days. Even with this underprediction bias in July, of the 28 days 

with normalized bias metrics, 24 days (86%) had normalized bias that achieves EPA’s ≤±15% 

performance goal. Note that normalized bias values were not calculated for July 8-9, 2006 

because there were no observed hourly ozone concentrations above the 60 ppb threshold and July 

31st was also missing because it was not simulated for the full day because it was at the end of 

the episode and meteorological data for the end of the day were not available due to the 

conversion from GMT to MST time; besides it was a low ozone day (all observed ozone < 70 

ppb) so would not provide any additional data for the Design Value projections. The normalized 

bias on July 10, 13, 22 and 23 did not achieve the 1-hour ozone performance goal for normalized 

bias with normalized bias values of, respectively, -19.7%, -18.1%, -17.95% and -19.7%. The 

July 10th poor performance is not a concern since this is a low observed and modeled ozone day, 

however July 13, 22 and 23 are observed ozone exceedance or near exceedance days so the 

underprediction bias is a concern. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-9 

Denver Run17 2006 Base Case 1hr O3

Unpaired Peak Accuracy

-30

-20

-10

0

10

20

30

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(%)

Run17

+20%

-20%

Average Paired Peak Accuracy

-30

-20

-10

0

10

20

30

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(%)

Run17

Normalized Bias

-30

-20

-10

0

10

20

30

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(%)

Run17

+15%

-15%

Normalized Error

0

10

20

30

40

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(%)

Run17

35%

Peak Observed and Paired Predicted

0

20

40

60

80

100

120

140

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(ppb)

Observed

Run17

120 ppb

Figure 2-3a. Daily ozone model performance evaluation statistical performance measures in 

the Denver NAA and comparison with EPA performance goals for June 2006, the final CAMx 

Run17 2006 base case simulation and 1-hour ozone. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-10 

Denver Run 17 2006 Base Case 1hr O3

Unpaired Peak Accuracy

-30

-20

-10

0

10

20

30

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(%)

Run17

+20%

-20%

Average Paired Peak Accuracy

-30

-20

-10

0

10

20

30

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(%)

Run17

Normalized Bias

-30

-20

-10

0

10

20

30

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(%)

Run17

+15%

-15%

Normalized Error

0

10

20

30

40

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(%)

Run17

35%

Peak Observed and Paired Predicted

0

20

40

60

80

100

120

140

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(ppb)

Observed

Run17

120 ppb

Figure 2-3b. Daily ozone model performance evaluation statistical performance measures in 

the Denver NAA and comparison with EPA performance goals for July 2006, the final CAMx 

Run17 2006 base case simulation and 1-hour ozone.
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-11 

2.3.2 Daily Ozone Performance Statistics for 8-Hour Ozone Concentrations 

Figure 2-4 displays the same daily statistical performance metric time series for running 8-hour 

ozone concentrations as shown for 1-hour ozone in Figure 2-3, only using a 40 ppb observed 

ozone cutoff instead of 60 ppb as used for the 1-hour ozone metrics. Also shown in Figure 2-4 

are EPA’s 1-hour ozone performance goals for Unpaired Peak Accuracy and Mean Normalized 

Bias and Gross Error that are compared to the 8-hour ozone statistics for reference. The 

Unpaired Peak Accuracy measure for 8-hour ozone achieves the ≤±20% performance goal on 27 

of the 30 days (90% of the days) in June (Figure 2-4a). On June 19, 22 and 28 the 8-hour ozone 

Unpaired Peak Accuracy performance statistics were -23.8%, +20.6% and +26.1%, respectively. 

On June 19, 2006, the model predicts relatively low ozone (50-75 ppb) throughout the domain 

and completely fails to capture the observed ozone exceedances at RFNO (95 ppb), FTCW (88 

ppb) and SBC (87 ppb). June 19, 2006 is a very poor performing day for the model. An 

examination of the maps comparing the predicted and observed spatial distribution of the 8-hour 

ozone concentrations on June 22, 2006 (Appendix A) reveals that there is a predicted cloud of 

elevated 8-hour ozone concentrations of 75-80 ppb that occurs south of Denver between the 

monitors in the DMA and Colorado Springs. Within the DMA, both the observed and modeled 

8-hour ozone concentrations are between ~40 ppb and ~60 ppb. On June 28 the modeled 8-hour 

ozone peak is 86 ppb and occurs southeast of the DMA away from the ozone monitors; again 

within the DMA itself there is reasonable agreement between the predicted and observed daily 

maximum 8-hour ozone concentrations. The modeled high ozone southeast of the DMA is not 

supported by the observations in Colorado Springs, but its presence can not be confirmed or 

refuted by the current DMA ozone monitoring network. During July 2006, the CAMx Run 17 

2006 base case simulations achieves EPA’s ≤±20% performance goal on all 30 days of the 

month (Figure 2-4b). 

With the exception of an overprediction bias on June 16 and 28, the Average Accuracy of the 

Peak performance metrics are less than ±20% for 28 days in June and less than ±20% for all 30 

days in July (Figure 2-4). The 8-hour ozone Mean Normalized Bias values in June 2006 are 

characterized by an overprediction bias with 11 days exceeding the ≤±15% 1-hour performance 

goal. An examination of the time series plots of hourly ozone in Appendix C reveals that this 

overprediction bias is mainly due to overestimation of the nighttime low hourly ozone values. 

Unlike for 1-hour ozone performance statistics, where June was performing better than July, for 

the 8-hour ozone performance statistics July is performing better (Figure 2-4b) than June (Figure 

2-4a). All days in July the 8-hour ozone Unpaired Peak Accuracy meets the ≤±20% performance 

goal. With the exception of July 3 (+16.3%) and 12 (+16.2%) over 90% of the days in July 

exhibit normalized bias values for 8-hour ozone concentrations that achieve the ≤±15% 1-hour 

performance goal. As for June, all days achieve the gross error performance goal by a wide 

margin. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-12 

Denver Run 17 2006 Base Case 8hr O3

Unpaired Peak Accuracy

-30

-20

-10

0

10

20

30

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(%)

Run17

+20%

-20%

Average Paired Peak Accuracy

-30

-20

-10

0

10

20

30

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(%)

Run17

Normalized Bias

-30

-20

-10

0

10

20

30

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(%)

Run17

+15%

-15%

Normalized Error

0

10

20

30

40

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(%)

Run17

35%

Peak Observed and Paired Predicted

50

60

70

80

90

100

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(ppb)

Observed

Run17

85 ppb

Figure 2-4a. Daily ozone model performance evaluation statistical performance measures in 

the Denver NAA and comparison with EPA performance goals for June 2006, the final CAMx 

Run17 2006 base case simulation and 8-hour ozone.
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-13 

Denver Run 17 2006 Base Case 8hr O3

Unpaired Peak Accuracy

-30

-20

-10

0

10

20

30

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(%)

Run17

+20%

-20%

Average Paired Peak Accuracy

-30

-20

-10

0

10

20

30

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(%)

Run17

Normalized Bias

-30

-20

-10

0

10

20

30

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(%)

Run17

+15%

-15%

Normalized Error

0

10

20

30

40

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(%)

Run17

35%

Peak Observed and Paired Predicted

0

20

40

60

80

100

120

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(ppb)

Observed

Run17

85 ppb

Figure 2-4b. Daily ozone model performance evaluation statistical performance measures in 

the Denver NAA and comparison with EPA performance goals for July 2006, the final CAMx 

Run17 2006 base case simulation and 8-hour ozone.
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-14 

2.3.3 Daily Maximum 8-Hour Ozone Performance 

The Unpaired Peak Accuracy performance metric compares the daily maximum observed ozone 

at any site in the Denver monitoring network with the model predicted maximum ozone 

anywhere within the Denver subregion (Figure 2-1). This is metric is somewhat biased because 

the monitoring network represents a limited spatial extent that may not record the actual daily 

maximum ozone concentration on that day, whereas the model has predictions throughout the 

region whose maximum may not occur at the locations of a monitor. In this case, a “perfect 

model” could exhibit an overprediction bias for this metric just because the modeled ozone peak 

occurs away from the monitoring sites. Thus, in this section we examine predicted and observed 

spatial paired daily maximum 8-hour ozone metrics, which is a particularly stringent test of 

model performance. 

Figure 2-5 displays the highest observed daily maximum 8-hour ozone concentration at any 

monitoring in the Denver network along with the spatially paired predicted daily maximum 8-

hour ozone concentration at the same monitor. The modeled daily maximum 8-hour ozone 

concentration at the site with the highest observed value tends to follow the observed day-to-day 

variations quite well, but it usually has an underprediction bias especially on the highest days. 

This is not surprising as small spatial displacement in the winds may slightly miss-locate an 

urban ozone plume so that it does not directly impact the ozone monitor. EPA accounts for such 

spatial displacements of the modeled peak from the observed peak in their attainment 

demonstration test by using the highest modeled 8-hour ozone near the ozone monitor, where 

near is defined as approximately within approximately 15 km. Thus, for making 8-hour ozone 

projections at a monitor EPA guidance recommends selecting the highest modeled daily 

maximum 8-hour ozone concentration from a 7 x 7 array of 4 km grid cells centered on the 

monitoring site (EPA, 2007). 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-15 

Peak Observed 8-Hour Ozone and Paired Predicted

50

55

60

65

70

75

80

85

90

95

100

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(ppb)

Observed Run17

Peak Observed 8-Hour Ozone and Paired Predicted

50

55

60

65

70

75

80

85

90

95

100

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(ppb)

Observed Run17

Figure 2-5. Compared of observed highest daily maximum 8-hour ozone concentration for 

each day during June (top) and July (bottom) 2006 with concurrent spatially paired prediction 

from the CAMx Run 17 2006 base case simulation. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-16 

As noted above, EPA’s draft 8-hour ozone modeling guidance listed a performance goal for 

predicted daily maximum 8-hour ozone concentrations near a monitor of within ±20% for most 

of the monitor-days (EPA, 1999). By near the monitor we have presented three approaches (see 

Table 2-3) for selecting the modeled daily maximum 8-hour ozone concentration to pair with the 

observed value, the Maximum and Closest value within a 7 x 7 array of 4 km grid cells centered 

on the monitor and selection of the modeled value at the monitor location (Spatially Paired). 

Figure 2-6 and Table 2-6 present the results of the comparisons of predicted and observed daily 

maximum 8-hour ozone concentrations and compares them with the ≤±20% performance goal. 

Using the closest modeled daily maximum 8-hour ozone concentration near the monitor to the 

observed value, 89% of the daily maximum 8-hour ozone concentrations prediction and observed 

monitor-days are within ±20% of each other. And when comparing predicted/observed daily 

maximum 8-hour ozone concentrations co-located at the monitor, 82% of the monitor-day pairs 

are within ±20% of each other. Thus, the CAMx Run 17 2006 base case simulation satisfies the 

performance goal from EPA’s 1999 draft 8-hour ozone modeling guidance that most predicted 

daily maximum 8-hour ozone monitor-days be within ±20% of the observed value. 

Table 2-6. Percent of the monitor-days that the model predicted daily maximum 8-hour ozone 

concentrations near the monitor is within ±20% of the observed value (total monitor-days = 

1008). 

Maximum Near the Monitor 

Percent Difference # Days % Days 

<-20 9 1% 

-20% to +20 769 76% 

> +20 230 23% 

Closest Near the Monitor 

Percent Difference # days % 

<-20 23 2% 

-20% to +20 902 89% 

> +20 83 8% 

Co-Located At the Monitor 

Percent Difference # days % 

<-20 48 5% 

-20% to +20 829 82% 

> +20 131 13% 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-17 

Daily maximum 8-Hour ozone near monitor.

All sites & all days. Subregion = Denver June+July, 2006. Run17 04km

0

20

40

60

80

100

120

0 20 40 60 80 100 120

Observed Ozone (ppb)

Predicted Ozone (ppb)

r

2

=0.2032

O - - O shows quantiles

Figure 2-6a. Comparison of predicted and observed daily maximum 8-hour ozone 

concentrations for the June-July 2006 modeling period and the final CAMx Run 17 2006 base 

case simulation using the Maximum predicted value near the monitor. 

Nearest daily maximum 8-Hour ozone.

All sites & all days. Subregion = Denver June+July, 2006. Run17 04km

0

20

40

60

80

100

120

0 20 40 60 80 100 120

Observed Ozone (ppb)

Predicted Ozone (ppb)

r

2

=0.61

O - - O shows quantiles

Figure 2-6b. Comparison of predicted and observed daily maximum 8-hour ozone 

concentrations for the June-July 2006 modeling period and the final CAMx Run 17 2006 base 

case simulation using the Closest predicted value near the monitor.
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-18 

Daily maximum 8-Hour ozone at monitor.

All sites & all days. Subregion = Denver June+July, 2006. Run17 04km

0

20

40

60

80

100

120

0 20 40 60 80 100 120

Observed Ozone (ppb)

Predicted Ozone (ppb)

r

2

=0.2615

O - - O shows quantiles

Figure 2-6c. Comparison of predicted and observed daily maximum 8-hour ozone 

concentrations for the June-July 2006 modeling period and the final CAMx Run 17 2006 base 

case simulation using the Co-Located predicted value near the monitor.

2.4 DETAILED PERFORMANCE FOR THREE 3-DAY EPISODES 

In this section we perform a more detailed ozone model performance evaluation for three 3-day 

episode that occurred during the June-July 2006 modeling period. During these three 3-day 

episodes, 7 of the 9 8-hour ozone exceedance days from June-July 2006 modeling period 

occurred. This episodic ozone model performance evaluation makes use of performance 

statistics, spatial maps and time series plots that are provided for the entire June-July 2006 

modeling period in Appendices B and C. 

2.4.1 June 17-19, 2007

The first 3-day episode examined is the June 17-19, 2006 episode during which 8-hour ozone 

exceedances occurred at the RFNO (94 ppb), SOBC (87 ppb) and FTCW (87 ppb) monitoring 

sites on June 19. Elevated ozone also occurred at the Chatfield site (81 ppb) on June 17, and 

numerous monitors measured 8-hour ozone concentrations in excess of 70 ppb during this 3-day 

episode. Table 2-7 compares the ozone model performance statistical measures with EPA’s 

goals (EPA, 1991) for monitoring sites in the DMA and vicinity and the three episode days. 

With the exception of 1-hour ozone normalized bias on June 17 (-17.2%) and unpaired peak 

accuracy on June 19 for 1-hour (-20.8%) and 8-hour (-23.8%) ozone, all of the ozone modeling 

performance statistics achieve EPA’s goals. However, there is a consistent and pronounced 

underprediction bias in the ozone performance statistics. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-19 

Table 2-7. Final CAMx Run 17 2006 base case simulation daily 1-hour and 8-hour statistical 

performance measures for the June 17-19, 2006 episode and comparison with EPA 

performance goals. 

Performance 2006 

Statistic 

EPA 

Goal June 17 June 18 June 19 

1-Hour Ozone 

Unpaired Peak ≤±20% -17.2 -2.2 -20.8 

Normalized Bias ≤±15% -17.2 -13.0 -11.9 

Normalized Error ≤35% 17.3 13.0 15.9 

8-Hour Ozone 

Unpaired Peak ≤±20% -15.0 -8.9 -23.8 

Normalized Bias ≤±15% -6.1 -1.3 2.1 

Normalized Error ≤35% 12.4 12.1 17.5 

Figures 2-7 display the spatial distribution of the predicted and observed daily maximum 8-hour 

ozone concentrations on July 17-19, 2006 and the preliminary CAMx (Run1.1204) base case 

simulation. The spatial distribution of the modeled daily maximum 8-hour ozone concentrations 

on June 17th are extremely flat, ranging from 50 to 70 ppb (Figure 2-7a). This is compared to 

observed values that are also fairly flat, but about 10 ppb higher, ranging from 67 to 81 ppb. At 

the Chatfield and Highland monitors, where the highest observed values occur on this day (80-81 

ppb), the model is predicting less than 70 ppb (60-70 ppb). 

On June 18th the variations in spatial distribution of the modeled (50-70 ppb) and observed (63-

76 ppb) daily maximum 8-hour ozone concentrations were both fairly flat, with the modeled 

values lower than observed (Figure 2-7b). 

On the June 19th, the 8-hour ozone exceedance day for this 3-day episode, both the modeled and 

observed 8-hour ozone concentrations are higher. The model correctly simulates the observed 

slightly elevated daily maximum 8-hour ozone concentrations at the southern ozone monitors 

(74-76 ppb at CHAT and HIGH). However, the highest observed 8-hour ozone concentrations at 

RFNO (95 ppb) is underpredicted by the model by approximately 20 ppb, as is the ozone 

exceedance at FCTW (88 ppb). 

Time series of predicted and observed hourly ozone concentrations for the June 17-19, 2006 

period and several key monitoring sites are shown in Figure 2-8. The modeled diurnal variations 

of the hourly ozone concentrations at the ARVA and NREL sites northwest of Denver are much 

flatter than the observed values. There are nighttime ozone increases in both the modeled and 

observed ozone concentrations. At ARVA the model fails to capture the observed daytime peaks 

and nighttime lows, and at NREL the nighttime lows are captured, but the daytime peaks are 

underestimated. The underprediction of the daytime peaks is also apparent at the RFNO 

monitoring site, particular on the June 19th exceedance day. At RFNO, the modeled ozone 

increase begins to rise with the observed values at 9am, but flattens out at under 80 ppb at 11am 

as the observed values keep on rising to over 100 ppb. This may be due to too fast rise of the 

modeled mixing heights, a failure to capture a subsidence inversion and/or failure of the model to 

capture high ozone aloft. 

At two sites south of Denver (CHAT and HIGH) the observed daytime peaks are underestimated 

on June 17-18, but reproduced quite well on June 19th. The nighttime lows are represented 

reasonably well at these two sites. Similar results are seen at the two Fort Collins sites north of 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-20 

Denver with the modeled daytime peaks underestimated by the model. The model does identify 

June 19th as the worst ozone day, but ozone formation at the Fort Collins sites is too slow and 

peaks15-20 ppb below the observed ozone peaks. 

Figure 2-9 displays the predicted and observed hourly time series plots for NOx (top), CO 

(middle) and ozone (bottom) for two sites in downtown Denver, WELB and CAMP. At WELB 

the model is estimating higher daytime NOx than observed. At CAMP, the model matches the 

diurnal variations of the observed NOx concentrations reasonably well, suggesting that the 

mobile source NOx emissions in downtown Denver are reasonably well characterized. 

Figure 2-7a. Spatial distribution of the predicted and observed daily maximum 8-hour ozone 

concentrations (ppb) in the Denver area on July 17, 2006 for the final CAMx Run 17 2006 base 

case simulation. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-21 

Figure 2-7b. Spatial distribution of the predicted and observed daily maximum 8-hour ozone 

concentrations (ppb) in the Denver area on July 18, 2006 for the final CAMx Run 17 2006 base 

case simulation.
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-22 

Figure 2-7c. Spatial distribution of the predicted and observed daily maximum 8-hour ozone 

concentrations (ppb) in the Denver area on July 19, 2006 for the final CAMx Run 17 2006 base 

case simulation.
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-23 

ARVA: Arvada -687.213 8.568

0

10

20

30

40

50

60

70

80

90

100

110

120

 6/17/ 6 6/18/ 6 6/19/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

NREL: National Renewable E -694.376 2.926

0

10

20

30

40

50

60

70

80

90

100

110

120

 6/17/ 6 6/18/ 6 6/19/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-24 

RFNO: Rocky Flats North -693.592 21.629

0

10

20

30

40

50

60

70

80

90

100

110

120

 6/17/ 6 6/18/ 6 6/19/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

CHAT: Chatfield -687.313 -20.937

0

10

20

30

40

50

60

70

80

90

100

110

120

 6/17/ 6 6/18/ 6 6/19/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-25 

HIGH: Littleton -677.368 -18.104

0

10

20

30

40

50

60

70

80

90

100

110

120

 6/17/ 6 6/18/ 6 6/19/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

FTCW: Fort Collins West -682.854 96.141

0

10

20

30

40

50

60

70

80

90

100

110

120

 6/17/ 6 6/18/ 6 6/19/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-26 

FTCO: Fort Collins South -677.801 94.012

0

10

20

30

40

50

60

70

80

90

100

110

120

 6/17/ 6 6/18/ 6 6/19/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

Figure 2-8. Time series of predicted and observed hourly ozone concentrations (ppb) during 

June 17-19, 2006 for the final CAMx Run 17 2006 base case simulation. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-27 

80013001: Welby -674.141 11.613

0

30

60

90

120

150

 6/17/ 6 6/18/ 6 6/19/ 6 Date

NOX [ppb]

Minimum to Maximum Range Observed Predicted

80013001: Welby -674.141 11.613

0

200

400

600

800

1000

1200

1400

1600

1800

2000

 6/17/ 6 6/18/ 6 6/19/ 6

Date

CO [ppb]

Minimum to Maximum Range Observed Predicted

WELB: Welby -674.141 11.613

0

10

20

30

40

50

60

70

80

90

100

110

120

 6/17/ 6 6/18/ 6 6/19/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

Figure 2-9a. Comparison of hourly NOx (top), CO (middle) and ozone (bottom) predicted and 

observed time series at the Welby monitoring site for June 17-19, 2006 and the final CAMx 

Run17 2006 base case simulation.
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-28 

80310002: CAMP -678.182 2.310

0

30

60

90

120

150

 6/17/ 6 6/18/ 6 6/19/ 6 Date

NOX [ppb]

Minimum to Maximum Range Observed Predicted

80310002: CAMP -678.182 2.310

0

100

200

300

400

500

600

700

800

900

1000

1100

1200

1300

1400

1500

 6/17/ 6 6/18/ 6 6/19/ 6

Date

CO [ppb]

Minimum to Maximum Range Observed Predicted

CAMP: 2105 Broadway -678.182 2.310

0

10

20

30

40

50

60

70

80

90

100

110

120

 6/17/ 6 6/18/ 6 6/19/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

Figure 2-9b. Comparison of hourly NOx (top) and ozone (bottom) predicted and observed time 

series at the CAMP monitoring site for June 17-19, 2006 and the preliminary CAMx Run17 2006 

base case simulation.
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-29 

2.4.2 July 13-15, 2006 

The July 13-15, 2006 episode was a very severe 3-day ozone episode in the DMA with 11 

instances of monitors exceeding the 85 ppb 8-hour ozone NAAQS and with two exceedance days 

each at the SOBC, RFNO and FCTW monitoring sites. Table 2-8 summarizes the 1-hour and 8-

hour ozone performance statistics for this 3-day episode. Although there is a large ozone 

underprediction bias on July 13th,so that the EPA normalized bias goal is not achieved, 

performance on July 14-15 is better achieving EPA’s performance goals for all metrics. 

Table 2-8. Final CAMx Run 17 2006 base case simulation daily 1-hour and 8-hour statistical 

performance measures for the July 13-15, 2006 episode and comparison with EPA performance 

goals. 

Performance 2006 

Statistic 

EPA 

Goal July 13 July 14 July 15 

1-Hour Ozone 

Unpaired Peak ≤±20% -18.6 -0.3 -7.6 

Normalized Bias ≤±15% -181 -12.7 -5.3 

Normalized Error ≤35% 18.1 13.4 9.8 

8-Hour Ozone 

Unpaired Peak ≤±20% -18.5 -9.5 -6.5 

Normalized Bias ≤±15% -6.1 -6.4 7.7 

Normalized Error ≤35% 14.8 13.3 12.7 

The poor performance on July 13th is also reflected in the comparisons of the spatial distribution 

of the observed and predicted daily maximum 8-hour ozone concentrations in Figure 2-10a. The 

high observed ozone at the monitoring sites south of Denver (86-92 ppb) is underpredicted by 

the model by approximately 20 ppb. 

Better performance is seen on July 14, 2006 (Figure 2-10b). The model correctly places a large 

expanse of elevated ozone concentrations stretching from northwest Denver to Fort Collins that 

also impacts the Niwot Ridge and RMNP sites where ozone exceedances are observed (91 ppb). 

The modeled ozone peak (88 ppb) is located very close to the monitored ozone peak (97 ppb) at 

FTCW monitoring site. If the modeled ozone was about 10 ppb higher and extended a little 

further west, model performance would be excellent on this day. 

On July 15th the modeled elevated ozone cloud is located to the northeast of Denver, when it 

should be located to the northwest of the city. This is clearly a case of spatial displacement due 

to errors in the wind fields (Figure 2-10c).
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-30 

Figure 2-10a. Spatial distribution of the predicted and observed daily maximum 8-hour ozone 

concentrations (ppb) in the Denver area on July 13, 2006 for the final CAMx Run 17 2006 base 

case simulation. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-31 

Figure 2-10b. Spatial distribution of the predicted and observed daily maximum 8-hour ozone 

concentrations (ppb) in the Denver area on July 14, 2006 for the final CAMx Run 17 2006 base 

case simulation. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-32 

Figure 2-10c. Spatial distribution of the predicted and observed daily maximum 8-hour ozone 

concentrations (ppb) in the Denver area on July 15, 2006 for the final CAMx Run 17 2006 base 

case simulation. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-33 

The time series of predicted and observed hourly ozone concentrations for the July 13-15, 2006 

3-day episode are shown in Figure 2-11. The model does a better job of reproducing the 

observed diurnal variation of the observed hourly concentrations at the ARVA and NREL sites 

just northwest of Denver on July 13-15 than the June 17-19 episode, but still shaves the observed 

daytime ozone peaks. The 8-hour ozone exceedances at the RFNO monitor on July 14-15 are 

underestimated by over 20 ppb. 

The modeled ozone at RMNP is nearly constant 60 ppb except for a 20 ppb increase on July 14 

to 80 ppb. The observed values are more variable (typically 50 ppb to 70 ppb) and also exhibit 

an increase on July 14th that is twice as high (40 ppb) as the modeled increase (20 ppb). As 

noted in the spatial comparisons in Figure 2-10b), the modeled elevated ozone cloud is displaced 

slightly eastward. This is a fairly common attribute of the MM5 winds inability to fully simulate 

the upslope flows during the day. 

Ozone model performance at the monitoring sites south of Denver (WELC, CHAT and HIGH) is 

quite good on July 14-15, with the observed daytime ozone underestimated on July 13. 

At the monitoring sites north of Denver (FTCO, FTCW and WCTO), daytime ozone is 

underestimated on July13th at all three monitors with better performance seen on July 14-15, 

albeit with a daytime underestimation bias. 

Figure 2-12 compares the predicted and observed hourly NOx, CO and ozone concentrations at 

the WELB and CAMP central Denver sites. At these sites the NOx, CO and ozone 

concentrations are reproduced quite well, with daytime modeled NOx values higher than 

observed at WELB, but matched quite well at CAMP. As NOx and CO at these two sites will be 

dominated by on-road mobile source emissions, these results suggest that the on-road mobile 

source NOx and CO emissions are reasonably well characterized.
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-34 

NREL: National Renewable E -694.376 2.926

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/13/ 6 7/14/ 6 7/15/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

ARVA: Arvada -687.213 8.568

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/13/ 6 7/14/ 6 7/15/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-35 

RFNO: Rocky Flats North -693.592 21.629

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/13/ 6 7/14/ 6 7/15/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

RMNP: Rocky Mountain NP -719.909 64.528

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/13/ 6 7/14/ 6 7/15/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-36 

WELC: Welch -692.154 -8.925

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/13/ 6 7/14/ 6 7/15/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

HIGH: Littleton -677.368 -18.104

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/13/ 6 7/14/ 6 7/15/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-37 

CHAT: Chatfield -687.313 -20.937

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/13/ 6 7/14/ 6 7/15/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

FTCO: Fort Collins South -677.801 94.012

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/13/ 6 7/14/ 6 7/15/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-38 

FTCW: Fort Collins West -682.854 96.141

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/13/ 6 7/14/ 6 7/15/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

WCTO: Weld County Tower -651.000 70.457

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/13/ 6 7/14/ 6 7/15/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

Figure 2-11. Hourly time series of predicted and observed hourly ozone concentrations for the 

July 13-15, 2006 period and the final CAMx Run17 2006 base case simulation. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-39 

80013001: Welby -674.141 11.613

0

50

100

150

200

 7/13/ 6 7/14/ 6 7/15/ 6 Date

NOX [ppb]

Minimum to Maximum Range Observed Predicted

80013001: Welby -674.141 11.613

0

200

400

600

800

1000

1200

1400

1600

1800

2000

 7/13/ 6 7/14/ 6 7/15/ 6

Date

CO [ppb]

Minimum to Maximum Range Observed Predicted

WELB: Welby -674.141 11.613

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/13/ 6 7/14/ 6 7/15/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

Figure 2-12a. Comparison of hourly NOx (top) and ozone (bottom) predicted and observed 

time series at the Welby monitoring site for July 13-15, 2006 and the final CAMx Run17 2006 

base case simulation base case simulation. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-40 

80310002: CAMP -678.182 2.310

0

50

100

150

200

 7/13/ 6 7/14/ 6 7/15/ 6 Date

NOX [ppb]

Minimum to Maximum Range Observed Predicted

80310002: CAMP -678.182 2.310

0

200

400

600

800

1000

1200

1400

1600

1800

2000

 7/13/ 6 7/14/ 6 7/15/ 6

Date

CO [ppb]

Minimum to Maximum Range Observed Predicted

CAMP: 2105 Broadway -678.182 2.310

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/13/ 6 7/14/ 6 7/15/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

Figure 2-12b. Comparison of hourly NOx (top) and ozone (bottom) predicted and observed 

time series at the CAMP monitoring site for July 13-15, 2006 and the final CAMx Run17 2006 

base case simulation base case simulation. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-41 

2.4.3 July 27-29, 2006 

The July 27-29, 2006 episode was the most severe 3-day ozone episode that occurred during 

2006. 8-hour ozone exceedances occurred on all three days and there were a total of 10 monitordays of exceedances in the Denver area, including 2 days at the key RFNO monitoring site. 

Table 2-9 displays the 1-hour and 8-hour performance statistics for July 27-29, 2006 all of which 

achieve EPA’s performance goals, although with an underprediction bias of the 1-hour ozone 

statistics. 

Table 2-9. Final CAMx Run 17 2006 base case simulation daily 1-hour and 8-hour statistical 

performance measures for the July 27-29, 2006 episode and comparison with EPA performance 

goals. 

Performance 2006 

Statistic 

EPA 

Goal July 27 July 28 July 29 

1-Hour Ozone 

Unpaired Peak ≤±20% -11.8 7.8 -1.2 

Normalized Bias ≤±15% -9.6 -9.7 -9.1 

Normalized Error ≤35% 13.1 14.8 12.4 

8-Hour Ozone 

Unpaired Peak ≤±20% -10.3 -6.2 -6.3 

Normalized Bias ≤±15% 1.0 -1.8 3.2 

Normalized Error ≤35% 13.6 12.3 14.2 

The comparison of the predicted and observed spatial distributions of daily maximum 8-hour 

ozone concentrations on July 27th confirm the model underprediction bias on this day (Figure 

2-13a). The model predicts elevated ozone mainly to the north and northwest of downtown, 

whereas the peak observed values are south of Denver. Although the modeled 8-hour ozone 

peak (78 ppb) is in the direction of an observed 82 ppb ozone peak in Weld County. Clearly the 

model is failing to capture what caused the observed high ozone south of Denver on this day. 

Better ozone performance is seen on July 28th (Figure 2-13b). The model correctly locates the 

elevated ozone cloud northwest of Denver, although ozone formation appears to be slower in 

then model than observed so the peak ozone occurs farther downwind. The model elevated 

ozone cloud also appears to be slight displaced westward only glancing the FTCW monitor 

where high ozone was recorded (87 ppb). 

July 29, 2006 was the most adverse ozone day during 2006 in the DMA. High ozone values 

were recorded at numerous monitors with exceedances even recorded at the Carriage monitor 

close to downtown. On other days we noted the too slow ozone formation in the model that 

caused the modeled ozone to occur too far downwind and away from Denver, such is not the 

case on this day. The highest modeled ozone in excess of 85 ppb is estimated south of 

downtown Denver where observed values of 83-97 are recorded. Elevated predicted ozone is 

estimates at RFNO (~80 ppb) where an exceedance is observed (97 ppb). And high modeled 

values are estimated to stretch to the FCTW monitor (~75 ppb), where another ozone exceedance 

is observed (95 ppb). 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-42 

Figure 2-13a. Spatial distribution of the predicted and observed daily maximum 8-hour ozone 

concentrations (ppb) in the Denver area on July 27, 2006 for the final CAMx Run 17 2006 base 

case simulation. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-43 

Figure 2-13b. Spatial distribution of the predicted and observed daily maximum 8-hour ozone 

concentrations (ppb) in the Denver area on July 28, 2006 for the final CAMx Run 17 2006 base 

case simulation. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-44 

Figure 2-13c. Spatial distribution of the predicted and observed daily maximum 8-hour ozone 

concentrations (ppb) in the Denver area on July 29, 2006 for the final CAMx Run 17 2006 base 

case simulation. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-45 

Figure 2-14 displays the time series of predicted and observed hourly ozone concentrations at 

key sites for the July 27-29, 2006 episode. For sites to the northeast of Denver (Figure 2-14a), 

the ozone model performance is quite good. The model follows the observed ozone variations at 

the NREL and ARVA well, although the hourly ozone peaks on July 29th are underestimated a 

little. The observed hourly ozone at RFNO is tracked well on July 27-29, 2006, again the hourly 

ozone peak on July 29th is underpredicted slightly. Further northeast from Denver at the SOBC, 

the ozone model performance is also quite good. 

At the CARR site just west of downtown Denver and WELC site to the southwest of downtown 

Denver, ozone model performance is also good, albeit the peaks on July 29th are underestimated 

(Figure 2-14b). South of Denver at the CHAT and HIGH monitoring sites the observed ozone 

on July 27th is underestimated, but model performance for July 28 and 29 is quite good. 

At sites north of Denver (Figure 2-14c), ozone model performance is not as good as seen for the 

other sites with the daytime ozone peaks too low. As seen on some of the other days, the model 

reproduces the observed rate of ozone formation in the morning but then stalls out around noon 

while the observed values continue to rise. 

Figure 2-15 compares the predicted and observed hourly NOx, CO and ozone time series at the 

CAMP and WELB sites in Denver. The model reproduces the observed NOx and CO 

concentrations reasonable well, but with an overprediction bias during the day at WELB. The 

ozone is also reproduced well at these two sites. The model fuels to reproduce the very high 

observed morning NOx spike on July 28th at both sites and overpredicts the CO spikes on July 

29th at WELB but less so at CAMP, whether this is an emissions or mixing issue can not be 

discerned. 

.
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-46 

ARVA: Arvada -687.213 8.568

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/27/ 6 7/28/ 6 7/29/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

NREL: National Renewable E -694.376 2.926

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/27/ 6 7/28/ 6 7/29/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-47 

RFNO: Rocky Flats North -693.592 21.629

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/27/ 6 7/28/ 6 7/29/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

SOBC: South Boulder Creek -697.351 26.869

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/27/ 6 7/28/ 6 7/29/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

Figure 2-14a. Time series of predicted and observed hourly ozone concentrations (ppb) 

for the July 27-29, 2006 period, monitoring sites to the Northwest of downtown Denver 

and the final CAMx Run17 2006 base case simulation. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-48 

WELC: Welch -692.154 -8.925

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/27/ 6 7/28/ 6 7/29/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

CARR: Carriage -681.834 2.717

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/27/ 6 7/28/ 6 7/29/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-49 

HIGH: Littleton -677.368 -18.104

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/27/ 6 7/28/ 6 7/29/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

CHAT: Chatfield -687.313 -20.937

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/27/ 6 7/28/ 6 7/29/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

Figure 2-14b. Time series of predicted and observed hourly ozone concentrations (ppb) 

for the July 27-29, 2006 period, monitoring sites mostly south of downtown Denver and 

the final CAMx Run17 2006 base case simulation.
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-50 

WCTO: Weld County Tower -651.000 70.457

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/27/ 6 7/28/ 6 7/29/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

FTCO: Fort Collins South -677.801 94.012

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/27/ 6 7/28/ 6 7/29/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-51 

FTCW: Fort Collins West -682.854 96.141

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/27/ 6 7/28/ 6 7/29/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

Figure 2-14c. Time series of predicted and observed hourly ozone concentrations (ppb) 

for the July 27-29, 2006 period, monitoring sites to the North of downtown Denver and 

the final CAMx Run17 2006 base case simulation.
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-52 

80310002: CAMP -678.182 2.310

0

50

100

150

200

 7/27/ 6 7/28/ 6 7/29/ 6 Date

NOX [ppb]

Minimum to Maximum Range Observed Predicted

80310002: CAMP -678.182 2.310

0

200

400

600

800

1000

1200

1400

1600

1800

2000

 7/27/ 6 7/28/ 6 7/29/ 6

Date

CO [ppb]

Minimum to Maximum Range Observed Predicted

CAMP: 2105 Broadway -678.182 2.310

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/27/ 6 7/28/ 6 7/29/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

Figure 2-15a. Comparison of hourly NOx (top) and ozone (bottom) predicted and observed 

time series at the CAMP monitoring site for July 27-29, 2006 and the final CAMx Run17 2006 

base case simulation. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-53 

80013001: Welby -674.141 11.613

0

50

100

150

200

 7/27/ 6 7/28/ 6 7/29/ 6 Date

NOX [ppb]

Minimum to Maximum Range Observed Predicted

80013001: Welby -674.141 11.613

0

200

400

600

800

1000

1200

1400

1600

1800

2000

 7/27/ 6 7/28/ 6 7/29/ 6

Date

CO [ppb]

Minimum to Maximum Range Observed Predicted

WELB: Welby -674.141 11.613

0

10

20

30

40

50

60

70

80

90

100

110

120

 7/27/ 6 7/28/ 6 7/29/ 6

Date

O3 [ppb]

Minimum to Maximum Range Observed Predicted

Figure 2-15b. Comparison of hourly NOx (top) and ozone (bottom) predicted and observed 

time series at the Welby monitoring site for July 27-29, 2006 and the final CAMx Run17 2006 

base case simulation. 
 F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec2_MPE_Ozone1.doc August 2008 2-54 

2.5 OZONE OPERATIONAL MODEL PERFORMANCE EVALUATION 

CONCLUSIONS 

The ozone operational model performance evaluation of the CAMx Run 17 2006 base case 

simulation of the June-July 2006 period has found some days when the model reproduces the 

observed ozone quite well, where as others where the model does not simulate the ozone well. A 

vast majority of the modeling days achieve EPA’s ozone model performance goals, lending some 

confidence that the model is performing well enough for use in making future year ozone 

projections. However, we should be taken that such ozone projections are not relying solely on 

poorly performing model days. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-1

3.0 DIAGNOSTIC MODEL PERFORMANCE EVALUATION 

3.1 INTRODUCTION

In the previous section we performed an operational model performance evaluation of the final 

CAMx Run 17 2006 base case simulation that addressed how well the model reproduces the 

surface hourly and 8-hour ozone concentrations in the Denver Metropolitan Area (DMA) (i.e., 

does the model get the right answer). In this section we perform a diagnostic model performance 

evaluation that compares the model predictions against ozone precursors, ozone aloft, key 

indicator species and tries to address the issue of whether the model is getting the right answer 

for the right reason. We also present in this section an evaluation of the base case simulation for 

particulate matter (PM) species. However, because the focus of the Denver June-July 2006 

modeling was on ozone, very little effort was devoted to developing the PM precursor emissions 

inventory and no effort was made to optimize the model configuration for PM performance. 

3.2 OZONE PRECURSOR MODEL PERFORMANCE 

The performance for ozone precursors is examine using the routine hourly NOx and CO 

monitoring network in the DMA and using special study Volatile Organic Compound (VOC) 

grab sampling at several sites conducted by the CDPHE/APCD that occurred during portions of 

the June-July 2006 episode. 

3.2.1 Routine Performance for Hourly NOx and CO Concentrations 

Figure 3-1 displays the same daily model performance statistics for hourly NOx concentrations 

in the DMA as used for ozone. NOx is collected at two sites in the DMA (CAMP and WELB) 

and is a primary emitted pollutant. We would not expect the model to produce model 

performance for NOx anywhere near as good for ozone as NOx is highly influenced by subgridscale emissions variations that can not be captured by the model. Although the ozone model 

performance goals are provided in Figure 3-1, they are not expected to be achieved for NOx 

model performance. For example, the CAMP monitor is located at the intersection of several 

major downtown streets that is affected by local mobile sources, whereas the model is predicting 

a 4 km by 4 km average concentration. Given this, it is not surprising that the model usually 

underestimates the maximum hourly NOx concentrations at the monitor (top panel in Figure 3-

1). Over all hours the model is exhibiting an overprediction bias in hourly NOx concentrations 

that is mainly due to an underprediction of the daytime low NOx concentrations at Welby shown 

in the previous Chapter. Because the normalized bias performance metric normalizes the 

difference in the predicted and observed hourly NOx concentrations by the observation, as the 

observed NOx concentration approaches zero it tends to blow up and become very large. 

Figure 3-2 displays model performance statistics for hourly CO concentrations that are collected 

at 13 sites in the DMA. Again because CO is a primary emitted species that is emitted mainly by 

mobile sources, it can exhibit large spatially variations within short distances that is not captured 

by a photochemical grid model using a 4 km grid resolution. Even with these caveats, the model 

appears to estimate reasonable levels of CO concentrations that are similar to the observed values 

on average.
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-2

Denver Run 17 2006 Base Case Hourly NOx.

Unpaired Peak Accuracy

-100

-50

0

50

100

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(%)

Run17

+20%

-20%

Average Paired Peak Accuracy

-100

-50

0

50

100

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(%)

Run17

Normalized Bias

-50

0

50

100

150

200

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(%)

Run17

+15%

-15%

Normalized Error

0

50

100

150

200

250

300

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(%)

Run17

35%

Peak Observed and Paired Predicted

0

50

100

150

200

250

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(ppb)

Observed

Run17

Figure 3-1a. Daily model performance statistics for June 2006, hourly NOx 

concentrations in the DMA (CAMP and WELB) and the final CAMX Run 17 2006 base 

case simulation.
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-3

Denver Run 17 2006 Base Case Hourly NOx.

Unpaired Peak Accuracy

-100

-50

0

50

100

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(%)

Run17

+20%

-20%

Average Paired Peak Accuracy

-100

-50

0

50

100

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(%)

Run17

Normalized Bias

-50

0

50

100

150

200

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(%)

Run17

+15%

-15%

Normalized Error

0

50

100

150

200

250

300

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(%)

Run17

35%

Peak Observed and Paired Predicted

0

50

100

150

200

250

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(ppb)

Observed

Run17

Figure 3-1b. Daily model performance statistics for July 2006, hourly NOx 

concentrations in the DMA (CAMP and WELB) and the final CAMX Run 17 2006 base 

case simulation.
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-4

Denver Run 17 2006 Base Case Hourly CO.

Unpaired Peak Accuracy

-60

-40

-20

0

20

40

60

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(%)

Run17

+20%

-20%

Average Paired Peak Accuracy

-50

0

50

100

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(%)

Run17

Normalized Bias

-40

-20

0

20

40

60

80

100

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(%)

Run17

+15%

-15%

Normalized Error

0

20

40

60

80

100

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(%)

Run17

35%

Peak Observed and Paired Predicted

0

500

1000

1500

2000

2500

6/16/26/36/46/56/66/76/86/9

6/106/116/126/136/146/156/166/176/186/196/206/216/226/236/246/256/266/276/286/296/30

(ppb)

Observed

Run17

Figure 3-2a. Daily model performance statistics for June 2006, hourly CO 

concentrations in the DMA (13 sites) and the final CAMX Run 17 2006 base case 

simulation.
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-5

Denver Run 17 2006 Base Case Hourly CO.

Unpaired Peak Accuracy

-100

-50

0

50

100

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(%)

Run17

+20%

-20%

Average Paired Peak Accuracy

-50

0

50

100

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(%)

Run17

Normalized Bias

-50

0

50

100

150

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(%)

Run17

+15%

-15%

Normalized Error

0

50

100

150

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(%)

Run17

35%

Peak Observed and Paired Predicted

0

500

1000

1500

2000

2500

3000

7/17/27/37/47/57/67/77/87/9

7/107/117/127/137/147/157/167/177/187/197/207/217/227/237/247/257/267/277/287/297/30

(ppb)

Observed

Run17

Figure 3-2b. Daily model performance statistics for July 2006, hourly CO 

concentrations in the DMA (13 sites) and the final CAMX Run 17 2006 base case 

simulation.
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-6

3.2.2 VOC and Key Indicator Model Performance 

During the summer of 2006, the CDPHE/APCD collected 3-hour grab canister samples at several 

sites in the DMA as shown in Figure 3-3 and Table 3-1. Morning VOC samples were collected 

at the CAMP and Welby sites in the metropolitan Denver as well as Fort Lupton and Platteville 

up in Weld County. Afternoon VOC samples were collected at the Rocky Flats North and Fort 

Collins West high ozone (downwind) monitors. The VOC samples at the two Denver sites 

(CAMP and WELB) will provide an evaluation of the model’s on-road mobile source emissions 

inventory, whereas the morning samples at the two Weld County sites will provide an evaluation 

of the oil and gas VOC emissions inventory that dominates the VOC inventory in Weld County. 

Figure 3-3. Locations of monitoring sites where VOC samples were collected during the 

summer of 2006.
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-7

Table 3-1a. Locations of monitoring sites where VOC samples were collected during the 

summer of 2006. 

2006 Ozone Precursor Monitoring Sites 

AQS # Site Name Address Latitude Longitude Elevation

08-001-3001 Welby 3174 E. 78th Ave., 

Denver, 80229 39 50 21 104 56 56 1550 m 

08-031-0002 CAMP 2105 Broadway, Denver, 

80205 39 45 04 104 59 14 1591 m 

08-123-0008 Platteville 1004 Main St., Platteville, 

80651 40 12 40 104 49 25 1476 m 

08-123-0011 Fort Lupton Kahil St. & Fulton Ave., 

Fort Lupton, 80621 40 04 28 104 48 53 1498 m 

08-059-0006 Rocky Flats - N 16600 W. Highway 128, 

Broomfield, 80021 39 54 46 105 11 18 1796 m 

08-069-0011 Fort Collins - West 3416 W. LaPorte Ave., 

Fort Collins, 80525 40 35 32 105 08 27 1575 m 

Table 3-1b. Sampling schedule and times for summer of 2006 VOC sampling study in the 

DMA. 

2006 Ozone Precursor Monitoring Sites

AQS # Site Name Sample Period Time Period 

08-001-3001 Welby 16 June – 31 July 2006, every 3rd day A.M. --- 06:00 – 09:00 (MDT) 

08-031-0002 CAMP 16 June – 31 July 2006, every 3rd day A.M. --- 06:00 – 09:00 (MDT) 

08-123-0008 Platteville 16 June – 31 July 2006, every 3rd day A.M. --- 06:00 – 09:00 (MDT) 

08-123-0011 Fort Lupton 16 June – 31 July 2006, every 3rd day A.M. --- 06:00 – 09:00 (MDT) 

08-059-0006 Rocky Flats - N 28 June – 13 July 2006, on-call basis P.M. --- 13:00 – 16:00 (MDT) 

08-069-0011 Fort Collins - West 19 July – 28 July 2006, on-call basis P.M. --- 13:00 – 16:00 (MDT) 

These VOC canister samples were speciated to obtain individual VOC species concentrations 

along with total nonmethane hydrocarbon (TNMHC) concentrations that were compared against 

the model results to determine how well the model predicted total VOC concentrations, 

concentrations of individual VOC species as well as predicting the key VOC/NOx indicator 

species ratios. However, before the measured VOC concentrations could be compared against 

the modeled values the following activities had to be performed: 

• Time shift of the measured data to the MST time zone being used in the modeling; 

• Adjustment of the VOC species to account for concentrations below the minimum 

detection level (MDL) whereby a fairly standard practice of setting VOC species 

concentrations that are below diction to half of the MDL; and 

• Speciated the VOC speciation samples to the CB05 VOC species used in the modeling. 

The speciation of the VOC samples into the CB05 species was performed by the research group 

at Colorado University under the direction of Dr. Jana Milford whose assistance was greatly 

appreciated. 

The conversions of the measured VOC species concentrations to the CB05 species was necessary 

in order to obtain an “apples-to-apples” comparison of VOC species and total VOC between the 

model predictions and observations. The raw VOC measurements can not be directly compared 

with the modeled VOC concentrations. There are portions of several VOC species that are 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-8

considered nonreactive (NR) in the CB05 VOC classification system so are dropped in the model 

VOC species. In addition, the VOC sampling did not speciate their samples for methanol 

(MeOH) and ethanol (EtOH) that are two species in the CB05 chemical mechanism so those two 

species were dropped from the comparison. In addition, ethane (ETHA) is an explicit species in 

CB05 but is considered nonreactive by EPA’s reactivity classification scheme so is not included 

in the definition of “total VOC” used in this evaluation. However, the model performance for 

ethane is evaluated separately. Thus, the total VOC comparisons presented below will not 

include all VOC compounds, but they will include consistent compounds between the 

observations and model predictions so are an appropriate comparison from a model performance 

evaluation perspective. However, when looking at the VOC/NOx ratios and trying to interpret 

results in terms of whether ozone formation is more VOC-limited or NOx-limited the user should 

be aware that not all VOC compounds are present in the “total VOC” concentrations used in the 

model performance evaluation. 

3.2.2.1 VOC Performance in Denver Urban Area 

VOC samples were collected at the CAMP and Welby monitoring sites that are located within 

the heart of the DMA. VOC, NOx and CO concentrations at these two urban sites will be 

dominated by on-road mobile sources. Consequently, the model performance at these two sites 

will help assess the accuracy of the on-road mobile source emissions inventory. As noted above, 

there are a lot of local- and microscale effects (e.g., streets) that result in high spatial variability 

in the observed VOC, NOx and CO concentrations that can not be captured by the model 

predictions that are averaged across a 4 km by 4 km grid cell with a layer 1 top of approximately 

36 m AGL. Thus, in evaluating model performance for ozone precursors at these sites, more 

emphasis is placed on the performance for VOC/NOx and CO/NOx ratios then for VOC, NOx 

and CO concentrations. Since VOC, NOx and CO are all dominated by on-road mobile sources 

at these two sites, the subgrid-scale effects on the three pollutants should be very similar. Given 

that there is more confidence in the on-road mobile source NOx emissions than the VOC and CO 

emissions, then an evaluation of the model at these sites for VOC/NOx and CO/NOx ratios 

provides a good indication of the accuracy of the on-road mobile source VOC and CO emissions. 

The performance for VOC/NOx ratios will also provide an indication of whether the model is 

predicting the correct chemical regimes and will predict ozone responses to changes in VOC and 

NOx emissions in the correct fashion (i.e., getting the right answer for the right reason). 

Figure 3-4a compares the predicted and observed morning 3-hour average VOC, NOx and CO 

concentrations at the CAMP monitor. Although there are a few outliers, in general the observed 

VOC concentrations are approximately 200 ppbC that is approximately a factor of two higher 

than the modeled values (~100 ppbC). Similarly, the observed NOx concentrations are 

approximately 100 ppb that are approximately a factor of 2 higher than predicted (~50 ppb). 

With one exception, the observed VOC and NOx concentrations are always the same or higher 

than the predicted values. The exception is on July 4, 2006 when the observed value is 10 ppb 

that is twice as low is the next lowest NOx observation and an order of magnitude lower than the 

average observed NOx concentrations at this site. July 4 has very unusual and atypical traffic 

patterns and it is not surprising that it is an outlier compared to the other days. On-road mobile 

source emissions representative of a Sunday were used for July 4, but that likely did not fully 

capture the unusual and atypical traffic patterns for this holiday day. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-9

Of the 15 days with morning samples at the CAMP monitor, there is excellent agreement with 

the predicted and observed VOC/NOx ratios for 11 of the 15 days (73% of the days; Figure 3-4b, 

top). On the four days when there is not excellent agreement with the predicted and observed 

VOC/NOx ratios at the CAMP monitor, one is the July 4th outlier day discussed previously. On 

July 22, 2006 (a Saturday) the model predicts a 2.5 VOC/NOx ratio when an 8.3 ratio was 

observed. Whereas on June 25 (a Sunday) and July 22 (a Thursday) VOC/NOx ratios of 5.9 and 

5.1 were observed with modeled values of 3.8 and 3.2, respectively. 

The model is generally overpredicting the observed morning CO/NOx ratio at the CAMP 

monitor (Figure 3-4b, middle). With a few exceptions, this overprediction is not large, but is 

systematic and may indicate an overrestimation of the on-road mobile source CO emissions. 

With the exception of July 19, 2006, observed morning ethane (ETHA) concentrations at the 

CAMP monitoring are underpredicted by a large amount at the CAMP monitor with average 

observed values near 25 ppbC and average modeled values an order magnitude lower (~2.5 

ppbC). This is undoubtedly partly due to EPA’s classification of ethane as a nonreactive VOC 

species so it is not included in a standard VOC emissions inventory. Although ethane was 

included in the WRAP Phase III oil and gas emissions inventory for the Denver-Julesburg Basin, 

it is likely missing for some sources in the DMA so it is not surprising that it is underestimated 

by the model. Since ethane has very low reactivity this should not adversely affect ozone 

formation, model performance or response to controls, but it certainly affects ethane model 

performance. 

The comparison of VOC, NOx and CO concentrations and VOC/NOx and CO/NOx ratios at the 

Welby monitor are shown in Figure 3-5. As noted in Chapter 2, there appear to be more issues 

with subgrid-scale variability at the Welby monitor then at CAMP. There is also more missing 

data at Welby so there are less VOC/NOx and CO/NOx ratio comparisons. Results at Welby are 

similar to CAMP with the model underestimating VOC and NOx concentrations, but generally 

predicting the same magnitude of CO concentrations as observed (Figure 3-5a). Although there 

is good agreement with the modeled and observed VOC/NOx ratios on some days (June 19 and 

July 7, 10 and 16) the model underpredicts the observed VOC/NOx ratio on June 16 and 22 and 

July 1 and 4. 

Table 3-2 summarizes the predicted and observed morning VOC/NOx ratio comparisons at the 

CAMP monitoring site in downtown Denver that also includes the day of week. As noted above, 

for most days these comparisons are very good, which is likely due to the extra effort invested in 

the Denver modeling of on-road mobile sources using the CONCEPT emissions model. Both the 

observed and predicted VOC/NOx ratios are higher on the weekend days, with Sunday being 

higher than Saturday, and lower on weekdays. The model underestimates the observed 

VOC/NOx ratio on Sunday June 25 by -25%, but agrees quite well on Sunday July 16. Similar 

results are seen for the two Saturday comparisons with good agreement on July 1 (+13%) and an 

underprediction bias on July 22 (-38%). Thus, the model appears to be capturing much of the 

weekend effect on mobile sources, but may not be capturing the full effect on all days. However, 

with just four days of weekend comparisons it is hard to draw firm conclusions. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-10

Table 3-2. Comparison of predicted and observed morning CB05-VOC/NOx ratios at the 

downtown Denver CAMP monitoring site. 

VOC/NOx Ratios 

Date 

Day of 

Week Observed Predicted 

Percent 

Difference 

June 16 Fri 3.31 2.56 -22% 

June 19 Mon 1.58 2.18 +38% 

June 22 Thurs 8.30 2.52 -70% 

June 25 Sun 5.85 3.81 -35% 

June 28 Wed 2.27 2.01 +13% 

July 1 Sat 3.04 3.42 +13% 

July 4 Holiday 17.74 2.18 -88% 

July 7 Fri 2.57 2.21 +36% 

July 10 Mon 2.07 2.82 +36% 

July 13 Thurs 2.42 2.39 -1% 

July 16 Sun 3.68 3.81 +3% 

July 19 Wed 1.64 2.39 +46% 

July 22 Sat 5.13 3.17 -38% 

July 25 Tue 2.60 2.20 -15% 

July 28 Fri 2.00 2.50 +25% 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-11

Predicted vs. Observed VOC at CAMP

0.00

100.00

200.00

300.00

400.00

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

VOC [ppbC]

Predicted Observed

Predicted vs. Observed NOx at CAMP

0

50

100

150

200

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

NOx [ppb]

Predicted Observed

Predicted vs. Observed CO at CAMP

0

200

400

600

800

1000

1200

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

CO [ppb]

Predicted Observed

Figure 3-4a. Comparison of predicted and observed 3-hour average morning (5-8am 

MST) CB05-VOC, NOx and CO concentrations at the CAMP monitor for the CAMx Run 

17 2006 base case simulation. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-12

Predicted vs. Observed VOC/NOx Ratio at CAMP

0

2

4

6

8

10

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

VOC/NOx 

[ppbC/ppb]

Predicted Observed

Predicted vs. Observed CO/NOx Ratio at CAMP

0

5

10

15

20

25

30

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

CO/NOx [ppb/ppb]

Predicted Observed

Predicted vs. Observed ETHA at CAMP

0

10

20

30

40

50

60

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

ETHA [ppbC]

Predicted Observed

Figure 3-4b. Comparison of predicted and observed 3-hour average morning (5-8am 

MST) CB05-VOC/NOx and CO/NOx ratio and ethane (ETHA) concentrations at the 

CAMP monitor for the CAMx Run 17 2006 base case simulation.
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-13

Predicted vs. Observed VOC at Welby

0.00

100.00

200.00

300.00

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

VOC [ppbC]

Predicted Observed

Predicted vs. Observed CO at Welby

0

200

400

600

800

1000

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

CO [ppb]

Predicted Observed

Predicted vs. Observed NOx at Welby

0

50

100

150

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

NOx [ppb]

Predicted Observed

Figure 3-5a. Comparison of predicted and observed 3-hour average morning (5-8am 

MST) CB05-VOC, NOx and CO concentrations at the WELB monitor for the CAMx Run 

17 2006 base case simulation.
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-14

Predicted vs. Observed VOC/NOx Ratio at Welby

0

2

4

6

8

10

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

VOC/NOx 

[ppbC/ppb]

Predicted Observed

Predicted vs. Observed CO/NOx Ratio at Welby

0

10

20

30

40

50

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

CO/NOx [ppb/ppb]

Predicted Observed

Predicted vs. Observed ETHA at Welby

0

10

20

30

40

50

60

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

ETHA [ppbC]

Predicted Observed

Figure 3-5b. Comparison of predicted and observed 3-hour average morning (5-8am 

MST) CB05-VOC/NOx and CO/NOx ratio and ethane (ETHA) concentrations at the 

Welby monitor for the CAMx Run 17 2006 base case simulation.
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-15

3.2.2.2 VOC Comparisons in Weld County 

Unlike the two Denver VOC sampling sites, NOx and CO were not collected at the two Weld 

County sites so we can only make model-observed comparisons for total CB05-VOC and 

Ethane, which are shown in Figure 3-6. At Ft. Lupton the modeled VOC ranges from 40 to 226 

ppbC, whereas the observed values range from 122 to 981 ppbC. Most modeled VOC values at 

Ft. Lupton are between 100-200 ppbC, whereas most observed values range from 250 to 700 

ppbC. Similar results are seen at Platteville where the modeled VOC ranges from 50 to 200 

ppbC, whereas the observed values range from 200 to 2,000 ppbC. 

The model is also underpredicting the observed ethane concentrations at the two Weld County 

sites (Figure 3-7). At Ft. Lupton the observed ethane values range from 24 to 240 ppbC, with 

average values of around 150 ppbC. Whereas the model ethane predictions range from 6 to 43 

ppb, with average values of approximately 25 ppbC that are over a 100 ppbC lower than 

observed on average. Even larger ethane underprediction bias is seen at Platteville with 

observed values of 35 to 600 ppbC and modeled values of 9 to 80 ppbC. 

The model systematic underprediction of ethane in Denver and Weld County suggest that there 

are missing sources of natural gas in the inventory. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-16

Predicted vs. Observed VOC at Ft Lupton

0.00

200.00

400.00

600.00

800.00

1000.00

1200.00

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

VOC [ppbC]

Predicted Observed

Predicted vs. Observed VOC at Platteville

0.00

500.00

1000.00

1500.00

2000.00

2500.00

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

VOC [ppbC]

Predicted Observed

Figure 3-6. Comparison of the predicted and observed morning CB05-VOC 

concentrations at the Ft. Lupton and Platteville sites in Weld County. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-17

Predicted vs. Observed ETHA at Ft Lupton

0

50

100

150

200

250

300

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

ETHA [ppbC]

Predicted Observed

Predicted vs. Observed ETHA at Platteville

0

100

200

300

400

500

600

700

606166061960622606256062860701607046070760710607136071660719607226072560728

Date

ETHA [ppbC]

Predicted Observed

Figure 3-7. Comparison of the predicted and observed morning CB05-VOC 

concentrations at the Ft. Lupton and Platteville sites in Weld County. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-18

3.2.3 Speciated VOC Comparisons

Figure 3-8 compares the speciated CB05 VOC concentrations for three days at the CAMP 

monitor. There is similarity in the speciated VOC model performance so just 3 days are 

reproduced. Paraffin (PAR) are reproduced well on June 19, but underestimated by 30-40 ppbC 

on July 13 and 28. Olefins (OLE) are underestimated by approximately a factor of two. The two 

aromatic species (TOL and XYL) are overestimated on June 19 but underestimated on the two 

July days. Formaldehyde (FORM) is underpredicted by approximately a factor of 2 on all three 

days. Acetaldehyde (ALD2) is greatly underestimated by the model with observed values of ~9 

ppbC and modeled values a factor of ~6 lower at ~1.5 ppbC. Higher aldehydes are 

overestimated on June 19 and underestimated on the other two days. Ethene (ETH) is 

underestimated by 20-40%. ISOP and TERP, that are mainly biogenic VOC species, are almost 

nonexistent in the observed and modeled VOC concentrations at CAMP. 

CB05 VOC species performance results at the Welby monitor (Figure 3-9) are similar to what 

was seen at CAMP. PAR is reproduced reasonable well on June 19, but underpredicted by 90 

ppb on July 13 (samples for July 28 were missing). Formaldehyde is again underestimated by a 

factor of two, and acetaldehyde is underestimated by approximately a factor of 6. Ethane is 

underpredicted by a factor of 3 on June 19 and a factor of 20 on July 13. 

The ETHA and PAR underestimation bias suggests missing natural gas emissions in the 

emissions inventory. This issue is discussed in more detail below when looking at the Weld 

County monitors. The underprediction of FORM is of concern since it is an important VOC 

species that initiates the radical cycle. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-19

Predicted and Observed CB05 Species at CAMP on 

June 19, 2006 (5-8AM MST)

0

20

40

60

80

100

120

PAR

OLE

TOL

XYL

FORM

ALD2

ETH

ISOP

ETHA

IOLE

ALDX

TERP

Concentration [ppb]

Observed Run17.1204

Predicted and Observed CB05 Species at CAMP on 

June 19, 2006 (5-8AM MST)

0

2

4

6

8

10

PAR

OLE

TOL

XYL

FORM

ALD2

ETH

ISOP

ETHA

IOLE

ALDX

TERP

Concentration [ppb]

Observed Run17.1204

Figure 3-8a. Comparison of predicted and observed 5-8am MST CB05 VOC species at 

the CAMP monitor on June 19, 2006. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-20

Predicted and Observed CB05 Species at CAMP on 

July 13, 2006 (5-8AM MST)

0

20

40

60

80

100

120

PAR

OLE

TOL

XYL

FORM

ALD2

ETH

ISOP

ETHA

IOLE

ALDX

TERP

Concentration [ppb]

Observed Run17.1204

Predicted and Observed CB05 Species at CAMP on 

July 13, 2006 (5-8AM MST)

0

2

4

6

8

10

PAR

OLE

TOL

XYL

FORM

ALD2

ETH

ISOP

ETHA

IOLE

ALDX

TERP

Concentration [ppb]

Observed Run17.1204

Figure 3-8b. Comparison of predicted and observed 5-8am MST CB05 VOC species at 

the CAMP monitor on July 13, 2006.
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-21

Predicted and Observed CB05 Species at CAMP on 

July 28, 2006 (5-8AM MST)

0

20

40

60

80

100

120

PAR

OLE

TOL

XYL

FORM

ALD2

ETH

ISOP

ETHA

IOLE

ALDX

TERP

Concentration [ppb]

Observed Run17.1204

Predicted and Observed CB05 Species at CAMP on 

July 28, 2006 (5-8AM MST)

0

2

4

6

8

10

PAR

OLE

TOL

XYL

FORM

ALD2

ETH

ISOP

ETHA

IOLE

ALDX

TERP

Concentration [ppb]

Observed Run17.1204

Figure 3-8c. Comparison of predicted and observed 5-8am MST CB05 VOC species at 

the CAMP monitor on July 28, 2006.
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-22

Predicted and Observed CB05 Species at Welby on 

June 19, 2006 (5-8AM MST)

0

50

100

150

PAR

OLE

TOL

XYL

FORM

ALD2

ETH

ISOP

ETHA

IOLE

ALDX

TERP

Concentration [ppb]

Observed Run17.1204

Predicted and Observed CB05 Species at Welby on 

June 19, 2006 (5-8AM MST)

0

2

4

6

8

10

PAR

OLE

TOL

XYL

FORM

ALD2

ETH

ISOP

ETHA

IOLE

ALDX

TERP

Concentration [ppb]

Observed Run17.1204

Figure 3-9a. Comparison of predicted and observed 5-8am MST CB05 VOC species at 

the Welby monitor on June 19, 2006.
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-23

Predicted and Observed CB05 Species at Welby on 

July 13, 2006 (5-8AM MST)

0

50

100

150

PAR

OLE

TOL

XYL

FORM

ALD2

ETH

ISOP

ETHA

IOLE

ALDX

TERP

Concentration [ppb]

Observed Run17.1204

Predicted and Observed CB05 Species at Welby on 

July 13, 2006 (5-8AM MST)

0

2

4

6

8

10

PAR

OLE

TOL

XYL

FORM

ALD2

ETH

ISOP

ETHA

IOLE

ALDX

TERP

Concentration [ppb]

Observed Run17.1204

Figure 3-9b. Comparison of predicted and observed 5-8am MST CB05 VOC species at 

the Welby monitor on July 13, 2006.
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-24

VOC species at the two Weld County monitors are dominated by ethane (ETHA) and paraffin 

(PAR). Table 3-3 compares the predicted and observed PAR and ETHA concentrations at Ft. 

Lupton and Platteville and Figure 3-10 compares the other CB05 VOC species at Ft. Lupton and 

three days. The large underprediction of PAR and ETHA at these sites (typically -70% to -80%) 

indicate missing natural gas emissions in the inventory (Table 3-3). In addition to 

underestimating FORM and ALD2, as seen at the two Denver monitoring sites, the aromatic 

species are also underestimates at the Weld County sites, which suggest that gasoline combustion 

VOC emissions may be underestimated in the inventory as well. 

Table 3-3. Comparison of predicted and observed morning paraffin (PAR) and ethane (ETHA) 

concentrations at the Weld County monitoring sites. 

Site Date Observed Predicted Difference Difference 

(ppbC) (ppbC) (ppbC) (%) 

PAR 

Ft. Lupton June 19 403 119 284 -70% 

Ft. Lupton July 13 321 67 254 -79% 

Ft. Lupton July 28 455 105 350 -77% 

Platteville June 19 484 224 260 -54% 

Platteville July 28 779 267 512 -66% 

ETHA 

Ft. Lupton June 19 88 18 70 -80% 

Ft. Lupton July 13 41 9 32 -77% 

Ft. Lupton July 28 96 13 83 -77% 

Platteville June 19 97 31 66 -68% 

Platteville July 28 161 34 127 -79% 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-25

Predicted and Observed CB05 Species at Ft Lupton on 

June 19, 2006 (5-8AM MST)

0

1

2

3

4

5

PAROLETOL

XYL

FORM

ALD2

ETH

ISOP

ETHA

IOLE

ALDX

TERP

Concentration [ppb]

Observed Run17.1204

Predicted and Observed CB05 Species at Ft. Lupton on 

July 13, 2006 (5-8AM MST)

0

1

2

3

4

5

PAR

OLE

TOL

XYL

FORM

ALD2

ETH

ISOP

ETHA

IOLE

ALDX

TERP

Concentration [ppb]

Observed Run17.1204

Predicted and Observed CB05 Species at Ft. Lupton on 

July 28, 2006 (5-8AM MST)

0

1

2

3

4

5

PAR

OLE

TOL

XYL

FORM

ALD2

ETH

ISOP

ETHA

IOLE

ALDX

TERP

Concentration [ppb]

Observed Run17.1204

Figure 3-10. Comparison of predicted and observed CB05 VOC species at the Ft. 

Lupton monitoring site. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-26

3.3 OZONE MODEL PERFORMANCE ALOFT 

NOAA collected ozonesonde measurements launched at Boulder that overlapped with our 

modeling period for six days. An ozonesonde collects a vertical profile of ozone concentrations 

starting at the launch point (Boulder) and rising at a given rate that moves horizontally 

downwind by the prevailing winds. When matching the modeled vertical ozone profile with the 

ozonesonde data, we used the predicted ozone concentrations within the grid cell above the 

launch point of the ozonesonde. Thus, we did not match up the horizontal location or time of the 

ozonesonde with the modeling results. Although the ozonesonde measurements went up to 20-

30 km, we only present the results for the lowest 10 km of the atmosphere as that is what is 

important for tropospheric ozone modeling. 

One element of the Conceptual Model of high ozone events in the Denver areas is that on some 

days there may be a reservoir of ozone aloft that is entrained into the mixed layer as it rises 

during the day. Thus, it will be interesting to see whether elevated ozone exists aloft that could 

be entrained into the mixed layer. 

Figure 3-11 compares the predicted and observed vertical ozone profiles at Boulder for the six 

days that ozonesonde measurements were available during the June-July 2006 modeling period. 

One general observation is that the model exhibits much less vertical variations in ozone 

concentrations than observed. This may be due in part by the fact that the model is just looking 

at vertical variations in ozone over the launch point, whereas the ozonesonde is measuring 

vertical, horizontal and temporal changes in ozone as the ozonesonde moves downwind. 

On June 15th there is good agreement between the predicted and observed ozone in the lowest 2 

km of the atmosphere (75-80 ppb). Between 2 km and 3 km AGL the observed ozone drops 

from 80 to 50 ppb, whereas the modeled values stay constant at 75-80 ppb. At ~5 km AGL the 

observed ozone rises from 50 to 90 ppb between 5 km and 6 km AGL and again matches the 

modeled value at 75-80 ppb between 6 km and 7 km. At 7 km the observed ozone rises to 250 

ppb at 10 km AGL. This elevated ozone aloft is too high to be entrained in the mixing layer and 

is likely due to stratospheric ozone intrusion that is decoupled from the tropospheric ozone. 

A comparison of the predicted and observed vertical ozone profiles for the morning of June 23 

are given in Figure 3-11b. The model and observations agree on the reservoir of 75-80 ppb 

ozone above the morning inversion in the 1-2 km AGL height. However, above 2 km the 

observed values drop to a minimum of ~40 ppb at approximately 4 km AGL, whereas the 

modeled vertical ozone profile is constant at 75-80 ppb. 

On June 29th the model estimates a nearly constant 65-75 ppb variation in ozone in the vertical 

from the surface to 10 km AGL (Figure 3-11c). The observed vertical profile is also relatively 

constant but has more variations from 55 ppb at the surface to 90 ppb at 10 km AGL. 

On July 14, 2006 the model is predicting a constant ~70 ppb vertical ozone profile in the lowest 

3 km of the atmosphere, whereas the observed values are 10-15 ppb higher (Figure 3-11d). 

Similarly, on July 21 the model (~55 ppb) is approximately 20 ppb lower than observed (~75 

ppb) in the lowest 1 km of the atmosphere. Between 2 and 4 km AGL there is good agreement 

between the modeled and observed vertical profile with a steady increase of from 55 ppb to 70 

ppb (model) and 80 ppb (observed). However, above ~4 km AGL the model exhibits increasing 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-27

ozone with height, but the observed ozone drops ~50 ppb with the observed value at 10 km AGL 

(55 ppb) being about 30 ppb lower than the model (85 ppb). 

On July 26 the modeled ozone is below the observed ozone throughout the vertical profile 

(Figure 3-11f). The modeled ozone ranges from ~55 ppb at the surface and increases to ~65 ppb 

at 10 km AGL, whereas the observed values range form about 60 ppb at the surface increasing to 

over 100 ppb at 5 km AGL and then down to 75 ppb at 10 km AGL. 

Maximum afternoon mixing heights in the Denver area would be expected to be in the 2,000 to 

5,000 m AGL range. There does not appear to be any evidence of a reservoir of elevated ozone 

aloft in the June ozonesonde measurements. However, the July ozonesonde measurements show 

elevated ozone aloft that is not captured by the model. Although these results are inconclusive, 

one potential explanation contributing to the underprediction of surface ozone concentrations at 

the high monitoring sites is the failure of the model to simulate the high ozone aloft that is 

entrained down to the surface as the mixing height rises. 

Predicted and Observed Ozonesonde over Boulder 

on June 15, 2006 at 12pm

0

1

2

3

4

5

6

7

8

9

10

0 50 100 150 200 250 300

Ozone [ppb]

Height above Surface [km]

Observed Predicted, Run17.1204

Figure 3-11a. Comparison of predicted and observed vertical distribution of ozone 

concentrations on June 15, 2006 for the CAMx Run 17 2006 base case simulation. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-28

Predicted and Observed Ozonesonde over Boulder 

on June 23, 2006 at 9am MST

0

1

2

3

4

5

6

7

8

9

10

0 20 40 60 80 100 120 140

Ozone [ppb]

Height above Surface [km]

Observed Predicted Run17.1204

Figure 3-11b. Comparison of predicted and observed vertical distribution of ozone 

concentrations on June 23, 2006 for the CAMx Run 17 2006 base case simulation.

Predicted and Observed Ozonesonde over Boulder 

on June 29, 2006 at 9am MST

0

1

2

3

4

5

6

7

8

9

10

0 50 100 150 200

Ozone [ppb]

Height above Surface [km]

Observed Predicted Run17.1204

Figure 3-11c. Comparison of predicted and observed vertical distribution of ozone 

concentrations on June 29, 2006 for the CAMx Run 17 2006 base case simulation.
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-29

Predicted and Observed Ozonesonde over Boulder 

on July 14, 2006 at 11am MST

0

1

2

3

4

5

6

7

8

9

10

0 20 40 60 80 100 120

Ozone [ppb]

Height above Surface [km]

Observed Predicted Run17.1204

Figure 3-11d. Comparison of predicted and observed vertical distribution of ozone 

concentrations on July 14, 2006 for the CAMx Run 17 2006 base case simulation.

Predicted and Observed Ozonesonde over Boulder 

on July 21, 2006 at 11am MST

0

1

2

3

4

5

6

7

8

9

10

0 20 40 60 80 100

Ozone [ppb]

Height above Surface [km]

Observed Predicted Run17.1204

Figure 3-11e. Comparison of predicted and observed vertical distribution of ozone 

concentrations on July 21, 2006 for the CAMx Run 17 2006 base case simulation.
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-30

Predicted and Observed Ozonesonde over Boulder 

on July 26, 2006 at 9am MST

0

1

2

3

4

5

6

7

8

9

10

0 20 40 60 80 100 120

Ozone [ppb]

Height above Surface [km]

Observed Predicted Run17.1204

Figure 3-11f. Comparison of predicted and observed vertical distribution of ozone 

concentrations on July 26, 2006 for the CAMx Run 17 2006 base case simulation.

3.4 PM MODEL PERFORMANCE EVALUATION 

In this section we compare the CAMx Run 17 2006 base case modeling results against 

particulate matter (PM) species and PM related species concentrations. However, the Denver 

June-July 2006 modeling database was set up for ozone modeling, not PM modeling. In fact, the 

CDPHE/APCD only provided 2006 base case emissions in Colorado for ozone precursor species 

(i.e., VOC, NOx and CO) and PM10 emissions for some categories . Thus, some of the PM 

emissions and PM related species are missing in Colorado. Where PM emissions data was 

available it was included, but given the time constraints of the study and the focus on ozone, no 

attempt was made to spend the extra resources and time to fill in the missing PM precursor 

emissions in Colorado. 

Outside of Colorado the projected WRAP emissions inventory was used that includes PM 

precursors. However, these data were replaced with the CDPHE ozone precursor inventory in 

Colorado. For on-road mobile sources, the MOBILE6 emission factor model includes PM 

species with the SMOKE and CONCEPT emissions modeling so on-road mobile source 

emissions include PM precursors. For large stationary point sources we used CEM data that 

includes NOx and SO2 emissions. The MEGAN biogenic emissions model also includes PM 

precursor emissions. So the main source of missing PM emissions in Colorado is within the nonroad mobile and area source categories. Given this, we would expect the model to underestimate 

PM concentrations. In particular, because most of the ammonia emissions are in the area source 

category, we would expect particulate nitrate (NO3) and ammonium (NH4) to be 

underestimated. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-31

Total PM2.5 mass measurements were collected at FRM and IMPROVE monitoring networks 

within the 12 km modeling domain. Figure 3-12 displays the predicted and observed scatter 

plots of 24-hour average PM2.5 mass concentrations for the two networks and the June-July 2006 

modeling period. Not surprising, given the missing PM precursor emissions in Colorado for 

some source categories, there is an underprediction tendency for total PM2.5 mass with 

fractional bias values of -90% to -100%. 

Better performance is seen for SO4 concentrations, albeit with an underestimation tendency 

(Figure 3-13). The fractional bias for SO4 across the CASTNET, IMPROVE and STN networks 

in the 12 km domain are -74%, -58% and -64%, respectively. Although this underprediction bias 

is due in part to missing SO2 and SO4 emissions, it is also affected by too much wet deposition 

of SO4 as evident by the overprediction of wet deposited SO4 at the NADP sites. Note that the 

MM5 overstatement of the summer time convective rainfall in the Four Corners states is a 

common occurrence and much of the WRAP MM5 modeling effort was devoted to limiting the 

overstated summer rainfall. For the Denver MM5 modeling of the June-July 2006 period, we did 

not observe overstated convective rainfall in the Denver area. However, that does not mean 

MM5 did not overstate rainfall in other areas of the western U.S. that is likely causing the 

overstated of wet SO4 deposition across the NADP monitoring sites. 

Particulate nitrate (NO3) is greatly underestimated by the model (Figure 3-14). A large part of 

this underprediction is due to missing ammonia emissions in Colorado; NO3 particles can not 

form unless there is a buffering species and ammonia is the most common buffering agent in the 

western U.S. Without ammonia, all of the oxidized NOx will stay as nitric acid (HNO3). In the 

top right panel of Figure 3-14 the modeled total nitrate (NO3+HNO3) is compared against the 

CASTNet measured total nitrate and a much better comparison is seen. Although there still is an 

underprediction bias in total nitrate, it is much lower than for NO3 and comparable to the other 

species. 

Figure 3-15 displays the model performance for organic carbon (OC) and elemental carbon (EC) 

across the IMPROVE and STN monitoring networks. Note that the OC and EC measurements 

are more uncertain than the other PM species. Also note that although the monitors measure OC, 

the model predicts organic carbon matter (OCM) that includes other elements besides carbon. 

For the OC model evaluation the observed OC was multiplied by 1.4 for comparison with the 

modeled OC in Figure 3-15. 

As expected, both EC and OC are underestimated due to missing emissions, with the 

underestimation comparable to the other species. The exception is EC at the STN network that 

exhibits fairly reasonable fractional bias (-29%). This is likely because the EC emissions were 

included in the on-road mobile source emissions and the urban-oriented STN sites are likely 

dominated by on-road mobile sources. 

Because of the deficient PM precursor inventory, the PM evaluation is inconclusive. As 

expected the model is underpredicting, and the underprediction appears to be consistent across 

the PM species. Given this, we elected not to expend any effort evaluating the model using the 

NPS ROMANS study database until better PM precursor emissions can be developed.
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-32

Figure 3-12. Comparison of predicted and observed total PM2.5 mass model 

performance at FRM (left) and IMPROVE (right) sites in the 12 km modeling domain. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-33

Figure 3-13. Comparison of predicted and observed total sulfate (SO4) model 

performance at CASTNET (top left), IMPROVE (top right), STN (bottom left) and NADP 

(bottom right) monitoring sites in the 12 km modeling domain.
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-34

Figure 3-14. Comparison of predicted and observed total nitrate (NO3) model 

performance at CASTNET (top left), CASTNET for total NO3 (NO3+HNO3) (top right), 

IMPROVE (bottom left) and NADP (bottom right) monitoring sites in the 12 km modeling 

domain.
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-35

Figure 3-15. Comparison of predicted and observed Organic Carbon (OC; top) and 

Elemental Carbon (EC; bottom) model performance at IMPROVE (left) and STN (right) 

monitoring sites in the 12 km modeling domain.
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec3_MPE_Diag1.doc August 2008 3-36

3.5 DIAGNOSTIC MODEL PERFORMANCE CONCLUSIONS 

The diagnostic model performance evaluation provides additional support that the Denver JuneJuly 2006 modeling database is working sufficiently well for projecting future year changes in 

ozone concentrations in response to changes in emissions. It has also identified areas of further 

analysis and refinement can be achieved to obtain a better photochemical modeling database. 

The mostly good agreement between the predicted and observed VOC/NOx ratios at the 

metropolitan Denver sites suggests that we are characterizing the on-road mobile sources well 

and capturing the correct chemical regimes in Denver. However, the comparisons for ethane and 

paraffin at the Denver sites, and especially the Weld County sites, suggest that natural gas 

emission sources are likely understated. In addition, the VOC speciation is an area that should 

be reviewed in the future with the model generally underestimating carbonyls (e.g., 

formaldehyde). 

The ozone aloft performance evaluation was inconclusive due to only having six comparisons 

available. But the results do lend credence to the hypotheses that the model is not predicting as 

high ozone aloft that is brought to the ground through vertical mixing. The PM evaluation was 

also inconclusive due to missing PM precursor emissions in the database, which is also an area 

for future improvements. 

 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec4_Refs1.doc August 2008 4-1 

4.0 REFERENCES 

Anthes, R.A. 1977. “A Cumulus Parameterization Scheme Utilizing a One-Dimensional Cloud 

Model”, Mon. Wea. Rev., Vol. 105, pp. 270-286. 

Anthes, R.A. and T.T. Warner. 1978. “The Development of Mesoscale Models Suitable for Air 

Pollution and Other Mesometeorological Studies”, Mon. Wea. Rev., Vol. 106, pp, 1045-

1078. 

Arnold, J.R., R.L. Dennis, and G.S. Tonnesen. 2003. “Diagnostic Evaluation of Numerical Air 

Quality Models with Specialized Ambient Observations: Testing the Community 

Multiscale Air Quality Modeling System (CMAQ) at Selected SOS 95 Ground Sites”, 

Atmos. Environ., Vol. 37, pp. 1185-1198. 

Arnold, J.R.; R.L. Dennis, and G.S. Tonnesen. 1998. Advanced techniques for evaluating 

Eulerian air quality models: background and methodology. In: Preprints of the 10th Joint 

Conference on the Applications of Air Pollution Meteorology with the Air & Waste 

Management Association. January, Phoenix, Arizona. American Meteorological Society, 

Boston, Massachusetts, paper no. 1.1, pp. 1-5. 

Barna, M., W. Malm, B. Schichtel, K. Gebhart and M. Rodriguez. 2008. Prediction Total 

Nitrogen Deposition at National Parks. Presented at Air & Waste Management 

Association’s Aerosol & Atmospheric Optics: Visual Air Quality and Radiation 

conference, Moab, Utah, April 28 – May 2. 

Bar-Ilan, A., R. Friesen, A. Pollack and A. Hoats. 2007. “WRAP Area Source Emissions 

Inventory Projections and Control Strategy Evaluation Phase II”. ENVIRON 

International Corporation, Novato, CA. July. 

http://www.wrapair.org/forums/ssjf/documents/eictts/OilGas/2007-

07_Phase%20II_O&G_Draft_Final_Report-v7-23.pdf. 

Bar-Ilan, A., J. Grant, R. Friesen, A. Pollack, D. Henderer, Daniel Pring and Kathleen Sgamma. 

2008a. Development of Baseline 2006 Emissions from Oil and Gas Activity in the 

Denver Julesburg Basin. ENVIRON International Corporation, Novato, California. 

April 30. (http://www.wrapair.org/forums/ssjf/documents/eictts/OilGas/2008-

04_%2706_Baseline_Emissions_DJ_Basin_Technical_Memo_(04-30).pdf). 

Bar-Ilan, A., J. Grant, R. Friesen, A. Pollack, D. Henderer, Daniel Pring and Kathleen Sgamma. 

2008a. Development 2010 Oil and Gas Emissions Projections for the Denver Julesburg 

Basin. ENVIRON International Corporation, Novato, California. April 30. 

(http://www.wrapair.org/forums/ssjf/documents/eictts/OilGas/2008-

04_%2710_Projection_Emissions_DJ_Basin_Technical_Memo(04-30).pdf).pdf). 

Barchet, W.R., and R.L. Dennis. 1990. “ANAPAP Model Evaluation, Volume 1: Protocol.” 

Prepared for the U.S. Environmental Protection Agency, prepared by Battelle Pacific 

Northwest Laboratories, Richland, WA. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec4_Refs1.doc August 2008 4-2 

Boylan, J. W. 2004. “Calculating Statistics: Concentration Related Performance Goals”. 

Presented at the EPA PM Model Performance Workshop, Chapel Hill, NC. February. 

Carter, W.P.L. 1999. “Documentation of the SAPRC-99 Chemical Mechanism for VOC 

Reactivity Assessment”. Draft report to the California Air Resources Board, Contracts 

92-329 and 95-308, September. 

Coats, C.J. Jr. 1996. High-performance algorithms in the Sparse Matrix Operator Kernel 

Emissions (SMOKE) modeling system. Proc. Ninth AMS Joint Conference on 

Applications of Air Pollution Meteorology with AWMA. Amer. Meteor. Soc., Atlanta, 

GA, 584-588. 

Coe D. and S. Reid. 2003. “Research and Development of Ammonia Emission Inventories for 

the Central States regional Air Planning Association”. Final Report. Sonoma 

Technology, Inc. (STI). October. 

Dudhia, J. 1993. "A Non-hydrostatic Version of the Penn State/NCAR Mesoscale Model: 

Validation Tests and Simulation of an Atlantic Cyclone and Cold Front". Mon. Wea. 

Rev., Vol. 121. pp. 1493-1513. 

Emery, C., E. Tai, and G. Yarwood. 2001. “Enhanced Meteorological Modeling and 

Performance Evaluation for Two Texas Episodes”. Prepared for the Texas Natural 

Resources Conservation Commission. Prepared by ENVIRON International Corp, 

Novato, CA. 

ENVIRON and UCR. 2004. “Modeling Protocol for the CENRAP 2002 Annual Emissions and 

Air Quality Modeling.” ENVIRON International Corporation and University of 

California at Riverside. November. 

ENVIRON. 2008. User’s Guide – Comprehensive Air Quality Model with Extensions – 

Version 4.40. ENVIRON International Corporation, Novato, CA http://www.camx.com. 

 February. 

EPA. 2007. “Guidance on the Use of Models and Other Analyses for Demonstrating 

Attainment of Air Quality Goals for Ozone, PM2.5 and Regional Haze”. U.S. 

Environmental Protection Agency, Research Triangle Park, NC. EPA-454/B-07-002. 

April. 

EPA. 2005a. “Guidance on the Use of Models and Other Analyses in Attainment 

Demonstrations for the 8-hr Ozone NAAQS”. U.S. Environmental Protection Agency, 

Atmospheric Sciences Modeling Division, Research Triangle Park, N.C. October. 

EPA. 2005b. “Technical Support Document for the Final Clean Air Interstate Rule – Air Quality 

Modeling”. U.S. Environmental Protection Agency, Office of Air Quality and Planning 

Standards, Research Triangle Park, North Carolina, 27711. March. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec4_Refs1.doc August 2008 4-3 

EPA. 2005c. “Regional Haze Regulations and Guidelines for Best Available Technology 

(BART) Determinations”. Fed. Reg./Vol. 70, No. 128/Wed. July. Rules and Regulations, 

pp. 39104-39172. 40 CFR Part 51, FRL-7925-9, RIN AJ31. 

EPA. 2003a. “A Conceptual Model to Adjust Fugitive Dust Emissions to Account for Near 

Source Particle Removal in Grid Model Applications”. Prepared by Tom Pace, U.S. 

EPA, August. 

http://www.epa.gov/ttn/chief/emch/invent/statusfugdustemissions_082203.pdf. 

EPA. 2003b. National emission inventory QA and augmentation. Memorandum. Prepared by 

Anne Pope, U.S. EPA, June. 

http://www.epa.gov/ttn/chief/emch/invent/qaaugmementationmemo_99nei_60603.pdf

EPA. 2001. “Guidance Demonstrating Attainment Air Quality Goals for PM2.5 and Regional 

Haze”. Draft Final (17 February 2005), U.S. Environmental Protection Agency, 

Atmospheric Sciences Modeling Division, Research Triangle Park, N.C. 

EPA. 1999. “Draft Guidance on the Use of Models and Other Analyses in Attainment 

Demonstrations for the 8-hr Ozone NAAQS”. Draft (May 1999), U.S. Environmental 

Protection Agency, Office of Air Quality Planning and Standards, Research Triangle 

Park, N.C 

EPA. 1991. "Guidance for Regulatory Application of the Urban Airshed Model (UAM), 

"Office of Air Quality Planning and Standards, U.S. Environmental Protection Agency, 

Research Triangle Park, N.C. 

ERG. 2006a. “WRAP Point and Area Source Emissions Projections for the 2018 Base Case”. 

Eastern Research Group, Inc., Sacramento, California. January. 

http://www.wrapair.org/forums/ssjf/documents/eictts/docs/WRAP_2018_EI-Version_1-

Report_Jan2006.pdf. 

ERG. 2006b. “Mexico National Emissions Inventory, 1999”. Eastern Research Group, Inc, 

Sacramento, California and TransEngineering, El Paso, Texas. October. 

http://www.epa.gov/ttn/chief/net/mexico/1999_mexico_nei_final_report.pdf. 

Fahey, K.M., and S.N. Pandis. 2001. Optimizing model performance: variable size reduction in 

cloud chemistry modeling. Atmos. Environ. Vol. 35, pp. 4471-4478. 

Gao, Y.-R., Y.-H. Kuo, J. Dudhia, and D.B. Barsons. 2000. Four-dimensional variational data 

assimilation of heterogeneous mesoscale observations for a strong convective case. Mon. 

Wea. Rev., 128, 619-643. 

Gephart, K.A., B.A. Schitel, M.G. Barna, M.A. Rodriguez and J. Hand. 2007. Meteorological 

Issues Associated with Nitrogen and Sulfur Deposition at Rocky Mountain National 

Park, Colorado. National Park Service, Air Resources Division, Fort Collins, Colorado. 

Presented at Air and Waste Management Association Meeting, Pittsburgh, PA, Paper No. 

492. June. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec4_Refs1.doc August 2008 4-4 

Gery, M. W., G.Z. Whitten, J.P. Killus, and M.C. Dodge. 1989. A photochemical mechanism 

for urban and regional-scale computer modeling. J. Geophys. Res. 94, 12925-12956. 

Grell, G. A., J. Dudhia, and D. R. Stauffer. 1994. "A Description of the Fifth Generation Penn 

State/NCAR Mesoscale Model (MM5). NCAR Tech. Note, NCAR TN-398-STR, 138 pp. 

Guenther, A. and C. Wiedinmyer. 2004. User’s Guide to the Model of Emissions of Gases and 

Aerosols from Nature (MEGAN). National Center for Atmospheric Research (NCAR), 

Boulder, Colorado (http://acd.ucar.edu/~christin/megan1.0_userguide.pdf). 

Jacob, D.J., R. Park and J.A. Logan. 2005. “Documentation and Evaluation of the GEOSCHEM Simulation for 2002”. Provided to the VISTAS Group. Harvard University. 

June. 

Johnson, M. 2007. Meteorological Model Performance Evaluation of an Annual 2002 MM5 

(Version 3.6.3) Simulation. Iowa Department of natural Resources, Air Quality Bureau. 

November. http://www.iowadnr.gov/air/prof/progdev/modeling.html. 

Kemball-Cook, S., Y. Jia, C. Emery, R. Morris, Z. Wang and G. Tonnesen. 2004. "Comparison 

of CENRAP, VISTAS and WRAP 36 km MM5 Model Runs for 2002, Task 3: 

Meteorological Gatekeeper Report.” 

http://pah.cert.ucr.edu/aqm/cenrap/ppt_files/CENRAP_VISTAS_WRAP_2002_36km_M

M5_eval.ppt. December. 

Kemball-Cook, S. Y Jia, C. Emery, R. Morris, Z. Wang, G. Tonnesen. 2004. “2002 Annual 

MM5 Simulation to Support WRAP CMAQ Visibility Modeling for the Section 308 

SIP/TIP: MM5 Sensitivity Simulations to Identify a More Optimal MM5 Configuration 

for Simulating Meteorology in the Western United States”. Prepared for the Western 

Regional Air Partnership, by ENVIRON International Corporation and UC Riverside, 

CE-CERT. December. 

http://pah.cert.ucr.edu/aqm/308/reports/mm5/DrftFnl_2002MM5_FinalWRAP_Eval.pdf

Loomis, C., J. Wilkinson, J. Haasbeek, A. Pollack and M. Jansen. 2005. ConCEPT – 

Consolidated Community Emissions Processing Tool, An Open-Source Tool for the 

Emissions Modeling Community. Presented at the 14th International Emission Inventory 

Conference “Transforming Emission Inventories – Meeting Future Challenges Today”, 

Las Vegas, Nevada, April 11-14. 

(http://www.epa.gov/ttn/chief/conference/ei14/session11/loomis.pdf) 

MACTEC. 2006. “Documentation of the Base G 2002 Base Year, 2009 and 2018, Emission 

Inventories for VISTAS”. Prepared for Visibility Improvement State and Tribal 

Association of the Southeast (VISTAS). MACTEC, Inc., Gainesville, Florida. 

Mansell, G.E. and T. Dinh. 2003a. “Emission Inventory Report, Air Quality Modeling Analysis 

for the Denver Early Action Ozone Compact: Development of the 2002 Base Case 

Modeling Inventory”. Prepared for Denver Regional Air Quality Council prepared by 

ENVIRON International Corporation, Novato, CA. September. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec4_Refs1.doc August 2008 4-5 

Mansell, G.E. and T. Dinh. 2003b. “Emission Inventory Report Addendum, Air Quality 

Modeling Analysis for the Denver Early Action Ozone Compact: Development of the 

2002 Base Case Modeling Inventory”. Prepared for Denver Regional Air Quality 

Council. Prepared by ENVIRON International Corporation, Novato, CA. September. 

Mansell, G.E. and T. Dinh. 2003c. “Emission Inventory Report, Air Quality Modeling Analysis 

for the Denver Early Action Ozone Compact: Development of the 2007 Base Case 

Modeling Inventory”. Prepared for Denver Regional Air Quality Council. Prepared by 

ENVIRON International Corporation, Novato, CA. October. 

Mansell, G., S. Lau, J. Russell and M. Omary. “Fugitive Wind Blown Dust Emissions and 

Model Performance Evaluation – Phase II”. ENVIRON International Corporation, 

Novato California and University of California at Riverside. May. 

http://www.wrapair.org/forums/dejf/documents/WRAP_WBD_PhaseII_Final_Report_05

0506.pdf. 

McNally, D., T.W. Tesche and R. Morris. 2003. “Air Quality Modeling Analysis For The 

Denver Early Action Ozone Compact: Evaluation of MM5 Simulations of The Summer 

’02 Denver Ozone Season and Embedded High 8-hr Ozone Episodes”. Alpine 

Geophysics, LLC, Ft. Wright, Kentucky. July. 

McNally, D., G. Schewe, J. Johnson and R. Morris. 2008. “Evaluation of Preliminary MM5 

Meteorological Model Simulation for the June-July 2006 Denver Ozone SIP Modeling 

Period Focused on Colorado”. Prepared by Alpine Geophysics, LLC and ENVIRON 

International Corporation. Prepared for Denver Regional Air Quality Council (RAQC). 

February. 

Morris, R.E. and G.E. Mansell. 2003a. Preliminary CAMx Base Case ozone modeling for the 

June-July 2002 Denver Ozone Episode. Memorandum. Prepared for Gerald Dilley, 

Denver Regional Air Quality Council prepared by ENVIRON International Corporation, 

Novato, CA. October. 

Morris, R.E., G.E. Mansell, D.A. McNally and T.W. Tesche. 2003b. Update on Ozone 

Modeling to Support Denver 8-Hour Early Action Compact. Presented at Denver EAC 

Modeling Review Panel (MRP) Meeting October, Denver, Colorado. 

http://www.raqc.org/ozone/EAC/MRP/Oct17/Environ_101703.pdf 

Morris, R.E., B. Koo, S. Lau, T.W. Tesche, D. McNally, C. Loomis, G. Stella, G. Tonnesen and 

Z. Wang. 2004a. “VISTAS Emissions and Air Quality Modeling – Phase I Task 4cd 

Report: Model Performance Evaluation and Model Sensitivity Tests for Three Phase I 

Episodes”. Prepared for the VISTAS Technical Analysis Committee. Prepared by 

ENVIRON International Corporation, Alpine Geophysics, LLC, and the University of 

California, Riverside (CE-CERT). 

Morris, R.E., D. McNally, T. Tesche, G. Tonnesen, J. Boylan and P. Brewer. 2004b. “Regional 

Haze Modeling over the VISTAS States: Preliminary Verification of Models-3/CMAQ 

for the 2002 Annual Period.” Presented at AWMA Visibility Conference, Asheville, 

North Carolina. October. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec4_Refs1.doc August 2008 4-6 

Morris R.E., T.W. Tesche and G.S. Tonnesen. 2004c. “Quality Assurance Project Plan (Draft) 

for the Visibility Improvements for State and Tribal Associations for the Southeastern 

States (VISTAS) Emissions and Air Quality Modeling”. Revision 0. ENVIRON 

International Corporation, Novato, CA. November. 

Morris, R.E., G.E. Mansell, E. Tai, D.A. McNally and T.W. Tesche. 2004d. “Air Quality 

Modeling Analysis for the Denver Early Action Compact: Preliminary Photochemical 

Base Case Modeling and Model Performance Evaluation for the Summer ’02 Denver 

Ozone Season and Embedded High 8-Hour Ozone Episodes”. Prepared for Denver 

Regional Air Quality Council, Denver, Colorado. Prepared by ENVIRON International 

Corporation, Novato, CA. November.

Morris, R.E., G.E. Mansell, E. Tai, D.A. McNally and T.W. Tesche. 2004f. “Air Quality 

Modeling Analysis for the Denver Early Action Compact: 2007 Base Case, Control 

Strategy and Sensitivity Analysis Modeling”. Prepared for Denver Regional Air Quality 

Council, Denver, Colorado. Prepared by ENVIRON International Corporation, Novato, 

CA. January.

Morris, R.E., G.E. Mansell, E. Tai, D.A. McNally and T.W. Tesche. 2004g. Air Quality 

Modeling Analysis for the Denver Early Action Compact: 2007 Emissions Reduction 

Sensitivity Modeling. Prepared for Colorado Department of Health and Environment and 

Denver Regional Air Quality Council, Denver, Colorado. Prepared by ENVIRON 

International Corporation, Novato, CA. February. 

Morris, R.E., G.E. Mansell, E. Tai, D.A. McNally and T.W. Tesche. 2004h. “Air Quality 

Modeling Analysis for the Denver Early Action Compact: 2007 Control Strategy 

Modeling for the Denver EAC”. Prepared for the Colorado Department of Health and 

Environment and Denver Regional Air Quality Council, Denver, Colorado. Prepared by 

ENVIRON International Corporation, Novato, CA. February. 

Morris R.E., D.E. McNally, T.W. Tesche, G.S. Tonnesen, J.W. Boylan and P. Brewer. 2005a. 

“Preliminary Evaluation of the Community Multiscale Air Quality Model for 2002 over 

the Southeastern United States”. J. Air and Waste Manage. Assoc. 55:1694-1708. 

November. 

Morris R.E. and G.M. Stella. 2005b. “Quality Assurance Project Plan (Draft) for the 

Association for Southeastern Integrated Planning (ASIP) Emissions and Air Quality 

Modeling to Address 8-Hour Ozone and PM 2.5 Nonattainment in the Southeastern 

States”. Revision 0. Prepared by ENVIRON International Corporation, Novato, CA. 

November. 

Morris, R.E., A. Hoats, S. Lau, B. Koo, G. Tonnesen, C-J. Jung and M. Omary. 2005c. “Air 

Quality Modeling Analysis for CENRAP – Preliminary 2002 Base Case CMAQ and 

CAMx Modeling of the Continental US 36 km Domain and Model Performance 

Evaluation”. Prepared by ENVIRON International Corporation, Novato, CA. April. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec4_Refs1.doc August 2008 4-7 

Morris R.E., E. Tai, Y. Jia and M. Jimenez. 2005d. “Photochemical Modeling of the Tulsa and 

Oklahoma City 8-Hour Ozone Early Action Compact (EAC) State Implementation Plan 

(SIP) – Technical Support Document”. Prepared by ENVIRON International 

Corporation, Novato, CA. March. 

Morris R.E. and G. Tonnesen. 2006. “Quality Assurance Project Plan (Draft) for Central 

Regional Air Partnership (CENRAP) Emissions and Air Quality Modeling – Revision 3”. 

 ENVIRON International Corporation, Novato, CA. March. 

Morris, R.E., B. Koo, A. Guenther, G. Yarwood, D. McNally, T.W. Tesche, G. Tonnesen, J. 

Boylan and P. Brewer. 2006. Model sensitivity evaluation for organic carbon using two 

multi-pollutant air quality models that simulate regional haze in the Southeastern United 

States. Atmos. Env. 40 (2006) 4960-4972. 

Morris, R., B. Wang, E. Tai, D. McNally, C. Loomis, G. Stella and T.W. Tesche. 2007. 

Modeling Protocol for the Denver 8-Hour Ozone Attainment Demonstration Modeling. 

ENVIRON International Corporation, Novato, CA. November. 

http://www.ozoneaware.org/documents/DraftFinalProtocolDenver8-

HourOzoneNov282007.pdf. 

Morris, R., S. Kemball-Cook, D. Strohm and L. Parker. Modeling Protocol for the Unita Basin 

Air Quality Study. 2008a. Prepared for Independent Petroleum Association of Mountain 

States (IPAMS). Prepared by ENVIRON International Corporation, Novato, California. 

March 17. 

Morris R., E. Tai, L. Parker, J. Johnson, D. McNally and C. Loomis. 2008b. Initial Ozone 

Model Performance Evaluation of the June-July 2006 Denver Ozone Episode and 

Diagnostic Testing and Analysis. ENVIRON International Corporation, Novato, 

California. Prepared for Denver Regional Air Quality Council (RAQC), Denver, 

Colorado. May 13. 

Nenes, A., C. Pininis, and S.N. Pandis. 1998. ISORROPIA: A new thermodynamic model for 

multiphase multicomponent inorganic aerosols. Aquatic Geochemistry, 4, 123-152. 

Nenes, A., C. Pininis, and S.N. Pandis. 1998. Continued development and testing of a new 

thermodynamic module for urban and regional air quality models. Atmos. Environ., 33, 

1533-1560. 

Nielsen-Gammon, J. W., R.T. McNider, A.B. White, W. Angevin, and K. Nnupp. 2005. 

Mesoscale model performance with assimilation of wind profiler data: Sensitivity to 

assimilation parameters and network configuration. J. Appl. Meteorol. 

Pechan. 2005a. “Electric Generating Unit (EGU) Growth Factor Comparison”. Prepared for 

CENRAP Emissions Inventory Workgroup. E.H. Pechan and Associates, Inc. Durham, 

North Carolina. January. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec4_Refs1.doc August 2008 4-8 

Pechan. 2005b. Technical Memorandum: Updates to Source Classification Code (SCC) to 

speciation profile cross-reference table. Prepared for CENRAP Emissions Inventory 

Workgroup. E.H. Pechan and Associates, Inc. Durham, North Carolina. April. 

Pechan and CEP. 2005c. “Consolidated of Emissions Inventories”. (Schedule 9; Work Item 3). 

 E.H. Pechan and Associates, Inc. Durham, North Carolina. Carolina Environmental 

Program, University of North Carolina, Chapel, Hill, North Carolina. April. 

Pechan 2005d. “Development of Growth and Control Inputs for CENRAP 2018 Emissions, 

Draft Technical Support Document”. E.H. Pechan and Associates, Inc. Durham, North 

Carolina. Carolina Environmental Program, University of North Carolina, Chapel, Hill, 

North Carolina. May. 

Pechan and CEP. 2005e. “Refinements of CENRAP’s 2002 Emissions Inventories”. (Schedule 

9; Work Item 3). E.H. Pechan and Associates, Inc. Durham, North Carolina. Carolina 

Environmental Program, University of North Carolina, Chapel, Hill, North Carolina. 

August. 

Pechan and CEP. 2004. “Methods for Consolidation of Emissions Inventories”. (Schedule 9; 

Work Item 3). Prepared for CENRAP Emissions Inventory Workgroup. E.H. Pechan and 

Associates, Inc. Durham, North Carolina. Carolina Environmental Program, University 

of North Carolina, Chapel, Hill, North Carolina. 

Pilinis, C., K.P. Capaldo, A. Nenes, and S.N. Pandis. 2000. “MADM - A new multicomponent 

aerosol dynamics model.” Aerosol Sci. Tech., Vol. 32(5), pp. 482-502. 2000. 

Pollack, A., C. Lindhjem, T. Stoeckenius, C. Tran, G. Mansell, M. Jimenez, G. Wilson and S. 

Coulter-Burke. 2004. Evaluation of the U.S. EPA MOBILE6 Highway Vehicle 

Emissions Factor Model, Final Report, CRC Project E-64. ENVIRON International. 

Pollack, A.K, L. Chan, P. Chandraker, J. Grant, C. Lindhjem, S. Rao, J. Russell and C. Tran. 

2006. “WRAP Mobile Source Emission Inventories Update”. ENVIRON International 

Corporation, Novato, CA. May. 

http://www.wrapair.org/forums/ef/UMSI/0606_WRAP_Mobile_Source_EI_Final_Report

.pdf

Reid S., S. Brown, D. Sullivan, H. Arkinson, T. Funk, and P. Stiefer. 2004a. “Research and 

Development of Planned Burning Emission Inventories for the Central States Regional 

Air Planning Association”. Sonoma Technology, Inc. (STI). July. 

Reid, S., D. Sullivan, B. Penfold, T. Funk, T. Tamura, P. Stiefer, S. Raffuse and H. Arkinson. 

2004. “Emission Inventory Development for Mobile Sources and Agricultural Dust 

Sources form the Central States”. Sonoma Technology, Inc. (STI). September. 

Reynolds, S. D., C.L., Blanchard, S.D., Ziman. 2003. Understanding the effectiveness of 

precursor reductions in lowering 8-hr ozone concentrations. Journal of the Air & Waste 

Manage. Association, 53, 195-205. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec4_Refs1.doc August 2008 4-9 

Rodriguez, M., M. Barna, K. Gebhart, B. Schichtel and W. Malm. 2008. Regional Air Quality 

Model Simulation of the Rocky Mountain Atmosphere Nitrogen and Sulfur Study: 

Preliminary Results and Analysis. Presented at Air & Waste Management Association’s 

Aerosol & Atmospheric Optics: Visual Air Quality and Radiation conference, Moab, 

Utah, April 28 – May 2 

Roth, P.M., S.D. Reynolds, and T.W. Tesche. 2005. Air quality modeling and decisions for 

ozone reduction strategies. Journal of the Air & Waste Manage. Association, 56, 1-16. 

Russell, A.G., and R.L. Dennis. 2000. NARSTO critical review of photochemical models and 

modeling. Atmos. Environ., Vol. 34, No. 12-14, pp. 2283-2324. 

Seaman, N.L. 2000. Meteorological modeling for air quality assessments. Atmos. Environ., 

Vol. 34, No. 12-14, 2231-2260. 

Seaman, N.L. 1996. “Study of Meteorological Variables Needed in Air-Quality Modeling”. 

Annual progress report prepared for the Coordinating Research Council (CRC), Interim 

Report, Project A-11, prepared by the Department of Meteorology, Penn State 

University, State College, PA. 

Seaman, N.L. 1995. “Status of Meteorological Pre-Processors for Air Quality Modeling”. 

Presented at International Conf. on Particulate Matter, Air and Waste Mgt. Assn., 

Pittsburgh, PA. 

Seaman, N. L., and D. R. Stauffer. 1996. "SARMAP meteorological model final report". 

Prepared for the San Joaquin Valleywide Air Pollution Study Agency. Prepared by the 

Department of Meteorology, Pennsylvania State University, University Park, PA. 

Seaman, N.L., D.R. Stauffer, and T.W. Tesche. 1992. "The SARMAP Meteorological Model: A 

Four-Dimensional Data Assimilation Technique Used to Simulate Mesobeta-Scale 

Meteorology During a High-Ozone Episode in California", International Specialty 

Conference on Tropospheric Ozone Nonattainment and Design Value Issues, U.S. 

EPA/AWMA, October, Boston, MA. 

Seaman, N.L., D.R. Stauffer, and L.M. Lario. 1995. A multiscale four-dimensional data 

assimilation system applied to the San Joaquin Valley during SARMAP. Part I: Modeling 

design and basic performance characteristics. J. Appl. Meteo., Vol. 34, pp. 1739-1761. 

Seaman, N.L., D.R. Stauffer, and D.E. McNally. 1996. "Application of the MM5-FDDA 

Meteorological Model to the Southern California SCAQS-1997 Domain: Preliminary 

Test Using the SCAQS August 1987 Case". Presented at Ninth Joint Conference on 

Applications of Air Pollution Meteorology, American Meteorological Society, JanuaryFebruary, Atlanta, GA. 

Sistla, G., W. Hao, J-Y Ku, G. Kallos, K. Zhang, H. Mao, and S.T. Rao. 2001. An operational 

evaluation of two regional-sale ozone air quality modeling systems over the eastern 

United States. Bull. Amer. Meteorological Soc., 82, (5), 945-964. 
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec4_Refs1.doc August 2008 4-10 

Stauffer, D.R. and N.L. Seaman. 1990. Use of four-dimensional data assimilation in a limitedarea mesoscale model. Part I: Experiments with synoptic data. Mon. Wea. Rev., 118, 

1250-1277. 

Strader, R., F.W. Lurmann and S.N. Pandis. 1999. Evaluation of secondary organic aerosol 

formation in winter. Atmos. Environ. Vol. 33, pp. 4849-4864. 

Strait, R., S. Roe and J. Vukovich. 2004. “2002 Base Case Modeling Inventory – Work Plan.” 

E.H. Pechan and Associates, Inc. Durham, North Carolina. July. 

Tesche, T.W., and D.E. McNally. 1993. “Operational Evaluation of the CAL-RAMS 

Meteorological Model for LMOS Episode 1: 26-28 June 1991". Prepared for the Lake 

Michigan Air Directors Consortium, prepared by Alpine Geophysics, LLC, Crested 

Butte, CO. 

Tesche, T.W., et al. 2002. “Operational Evaluation of the MM5 Meteorological Model over the 

Continental United States: Protocol for Annual and Episodic Evaluation”. Prepared for 

the U. S. Environmental Protection Agency, Office of Air Quality Planning and 

Standards. Prepared by Alpine Geophysics, LLC, Ft. Wright, KY. 

Tesche, T.W., D.E. McNally, C. Loomis, R.W. Morris and G.E. Mansell. 2003. “Revised 

Ozone Modeling Protocol -- Air Quality Modeling Analysis for the Denver Early Action 

Ozone Compact: Modeling Protocol, Episode Selection, and Domain Definition”. 

Prepared by Alpine Geophysics, LLC, Ft. Wright, Kentucky and ENVIRON International 

Corporation, Novato, CA. Prepared for Mr. Gerald Dilley, Denver Regional Air Quality 

Council. May. 

Tesche, T.W., R.E. Morris, G. Tonnesen, D.E. McNally, J. Boylan, and P. Brewer. 2005a . 

CMAQ/CAMx annual 2002 performance evaluation over the eastern U.S. Atmospheric 

Environment. 

Tesche, T.W., D.E. McNally, C.F. Loomis, G.M. Stella, and J.G. Wilkinson. 2005b. “Modeling 

Support for the Houston 8-hr Ozone Attainment Demonstration: 8-Hr Ozone Modeling 

Protocol”, prepared for the Houston 8-hr Ozone Coalition and the Texas Commission on 

Air Quality, prepared by Alpine Geophysics, LLC, Ft. Wright, KY. 

Tonnesen, G., R.E. Morris, J. Vimont, M. Uhl, K. Briggs and T. Moore. 2003. “The WRAP 

Regional Modeling Center – Application and evaluation of regional visibility models.” 

Presented at AWMA Annual Meeting and Exhibition. June. 

Tonnesen, G., Z. Wang, M. Omary, C-J. Chien, Y. Wang, R. Morris, S. Kemball-Cook, Y. Jia, S. 

Lau, B. Koo, Z. Adelman, A. Holland and J. Wallace. 2005. “Final Report for the 

Western Regional Air Partnership (WRAP) Regional Modeling Center (RMC) for the 

Project Period March 1, 2004, through February 28, 2005”. Prepared for WRAP 

Regional Modeling Center (RMC). August. 

http://pah.cert.ucr.edu/aqm/308/reports/final/2004_RMC_final_report_main_body.pdf
F:\Denver_O3_2008\Reports\2006_Base_Case\Draft_Final\Sec4_Refs1.doc August 2008 4-11 

Tonnesen, G., Z. Wang, M. Omary, C-J. Chien, Y. Wang, R. Morris, S. Kemball-Cook, Y. Jia, S. 

Lau, B. Koo, Z. Adelman, A. Holland and J. Wallace. 2006. “Final Report for the 

Western Regional Air Partnership (WRAP) 2002 Visibility Model Performance 

Evaluation”. Prepared for WRAP Regional Modeling Center (RMC). February. 

http://pah.cert.ucr.edu/aqm/308/reports/final/2002_MPE_report_main_body_FINAL.pdf

Wesely, M.L. 1989. Parameterization of surface resistances to gaseous dry deposition in 

regional-scale numerical models. Atmos. Environ., 23, 1293-1304. 

Yarwood, G., S. Shepard, G. Wilson and A. Guenther. 2007. “User’s Guide to the Global 

Biosphere Emissions and Interactions System (GloBEIS) – Version 3.2”. Prepared by 

ENVIRON International Corporation, Novato, CA. November. 

(http://www.globeis.com/data/Users_Guide_Globeis_v3.2.Jan23_2008.pdf). 

Xiu, A. and J.E. Pleim. 2000. Development of a land surface model. Part I: application in a 

mesoscale meteorology model. J. App. Met., 40, pp. 192-209. 
APPENDIX A 

Comparison of Predicted and Observed Spatial Distributions of the Daily Maximum 

8-Hour Ozone Concentrations in the Denver Area during June-July 2006 for the 

Final CAMx 2006 Base Case (Run 17) Simulation 
Denver 2006 Base Case (run17.1204)