May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 5.doc 5-1

5.0 MODEL INPUT PREPARATION PROCEDURES 

In this section we describe the procedures to be used to develop the CMAQ model inputs 

for the Phase II 2002 annual and potentially additional ~60 days episodic simulations. We also 

discuss how the CAMx model inputs will be generated if the optional task to run an alternative 

model to corroborate the CMAQ results is selected. The development of the CMAQ 

meteorological and emissions inputs are discussed first followed by the science options to be 

used by CMAQ. The procedures for developing the initial and boundary conditions and 

photolysis rates inputs are then discussed along with the model application procedures. 

5.1 Meteorological Inputs to Emissions and Air Quality Models 

The emissions and air quality models require certain meteorological input data including 

wind fields, estimates of turbulent eddy dispersion, humidity, temperature, clouds, and actinic 

flux. Spatially gridded and hourly varying meteorological data are needed to estimate biogenic, 

mobile source emissions, and plume-rise for large, elevated point sources. Meteorological data 

are needed to drive chemical transport models for solving atmospheric diffusion and chemistry 

equations for model species. Because observed data are not available for the full gridded model 

domain, numerical meteorological models are used to provide these inputs. 

The National Center for Atmospheric Research (NCAR)/Pennsylvania State University 

(PSU) Fifth-Generation Mesoscale Model (MM5) (v3.6) is being used by the VISTAS 

meteorological modeling contactor, Baron Advanced Meteorological Systems (BAMS) (Olerud, 

2004) to simulate meteorology at a 36-km resolution for calendar year 2002 over the entire 

continental United States and including portions of Canada, Mexico, and the Atlantic and Pacific 

Oceans. MM5 is also being applied over the southeastern U.S. using a 12 km resolution grid. 

The MM5 is a three-dimensional prognostic meteorological model that is used not only for 

meteorology studies but also for air quality studies. Some of the physics used in the simulation 

include one-way nesting; nonhydrostatic dynamics; four-dimensional data assimilation of wind, 

temperature, and mixing ratio; explicit treatment of moisture; cumulus cloud parameterization; 

vertical mixing of momentum in the mixed layer; PBL process parameterization; atmospheric 

radiation; sea ice treatment; and snow cover (see Chapter 2 for more details). 

After the MM5 simulation is completed, the MM5 output files are transferred to the 

emissions and air quality modeling team and analyzed by the Meteorological Gatekeeper. The 

Meteorological Gatekeeper performs two main roles. To provide an independent evaluation of 

the 2002 MM5 simulation that also serves to determine whether the MM5 data have been 

transferred correctly from the VISTAS meteorological modeling contractor. And to process the 

2002 MM5 output using Version 2.2 of the Models-3 CMAQ Meteorological-Chemical Interface 

Processor (MCIP) to generate meteorological fields that will be used for emissions processing 

and air quality simulation. 

5.1.1 MCIP Reformatting Methodology 

 

The Models-3 Community Multiscale Air Quality (CMAQ) modeling system is designed 

to simulate multiscale (urban and regional) and multi-pollutant (oxidants, acid deposition, and 

particles) air quality problems. But before running the CMAQ Chemical Transport Model 

(CCTM), the MM5 generated meteorological data must be pre-processed and converted to 

TN Regional Haze SIP Appendix C.1 - 95
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 5.doc 5-2

Models-3 consistent data structures. MCIP version 2.2 will be used to preprocess the MM5 

meteorological output. The “pass through” option in MCIP will be used in the VISTAS Phase II 

modeling. One of MCIP’s functions is to translate meteorological parameters from the output of 

the Pennsylvania State University/National Center for Atmospheric Research (PSU/NCAR) 

Mesoscale Modeling System Generation 5 (MM5) to the Models-3 input/output applications 

program interface (I/O API format) which is required for operation of Models-3 CMAQ 

processors. Some other necessary parameters not available from the meteorological model are 

estimated with appropriate diagnostic algorithms in the program. The key functions of MCIP 

include: 

1. Reading in meteorological model output files 

2. Extraction of meteorological data for CTM window domain 

3. Interpolation of coarse meteorological model output for finer grid 

4. Collapsing of meteorological profile data if coarser vertical resolution data is requested 

5. Computation or passing through surface and PBL parameters 

6. Diagnosing of cloud parameters 

7. Computation of species-specific dry deposition velocities 

8. Generation of coordinate dependent meteorological data for the generalized coordinate 

CCTM simulation 

9. Output meteorological data in Models-3 I/O API format 

The MCIP processor transforms the data into I/O API format while also calculating several 

new data fields (e.g. low, middle, and high cloud fractions) that are not readily available in the 

raw MM5 output. It also interpolates temperature and wind speed to observation height (1.5m 

and 10m, respectively). The MCIP processor culls a minimum of six cells about the domain 

periphery to minimize edge effects in the MM5 simulation. MCIP can be used to further reduce 

the rows or columns in the MM5 data so that the domain definition for the MCIP output files 

precisely matches the domain used in the air quality modeling. MCIP also allows MM5 layers to 

be “collapsed” (i.e., some layers can be aggregated). When feasible it is desirable to use the same 

layer structure in the air quality model as in the MM5 to prevent errors associated with 

aggregating layer data and to maintain consistency between data produced by the meteorological 

model and those used by the chemistry-transport model. However, due to computational costs 

associated with using large number of vertical layers, vertical layer collapsing is typically used to 

reduce the total number of layers used by the CCTM. In the VISTAS Phase II modeling we will 

collapse from 34 layers in MM5 output into 19 layers for the CMAQ air quality simulations. The 

first 8 layers of CMAQ, up to approximately 450 m AGL, will match the MM5 vertical layer 

structure exactly. The MM5 layers are then “doubled up” in CMAQ, up to a height of 

approximately 3,500 m AGL. The region top for CMAQ is the same as used by MM5, 100mb 

(approximately 15 km AGL). The 36 km analysis domain contains 148 columns, 112 rows, and 

19 layers. The 12 km analysis domain covers 168 columns, 177 rows, and 19 layers. More 

details on the CMAQ modeling domain definitions are provided in Chapter 4 with the vertical 

layer structure of MM5 and MCIP/CMAQ shown at Table 4-3. 

5.1.2 Products of the Meteorological Input Development Process 

The meteorological input development process produces three two-dimensional and four 

three-dimensional daily meteorological and geophysical output data in the Models-3 I/O API 

format. These CCTM-ready meteorological input files are used in both emissions processing and 

TN Regional Haze SIP Appendix C.1 - 96
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 5.doc 5-3

the CCTM simulations. The met fields are 36 km and 12 km horizontal resolution on a Lambert 

Conformal Projection (LCP) coordinate system with 19 vertical sigma layers extending from the 

surface to the 100 mb pressure level. The data files include three-dimensional gridded fields of 

u- and v-wind components, vertical velocity, temperatures, Jacobian, Jacobian weighted air 

density, total air density, water vapor, cloud water content, rain water content, ice and snow 

mixing ratio, layer heights, and vertical exchange coefficients. Two-dimensional gridded fields 

of latitude and longitude, squared map-scaled factor, surface temperatures and pressures, 1.5 and 

10 meter temperature, planetary boundary heights, rainfall, total cloud fraction, snow cover, 

deposition velocities, u* and w*, surface roughness length, as well as dominant land use 

category are also developed. 

Table 5-1 shows the configuration to be used in MCIP version 2.2 for processing the 

2002 MM5 output to produce CCTM-ready meteorology input files. 

Table 5-1. MCIP V2.2 Configuration used In the VISTAS Phase II Modeling. 

Module or option Values or 

setting 

Additional Information 

PBL value computation option 1 Use PBL value from input meteorology 

Radiation fields 1 Use radiation fields from input 

meteorology 

Dry deposition option 2 Use Models-3 (Pleim) dry deposition 

routine 

Output interval 60 Unit is in minutes 

Vertical layer structure 19 layers See Chapter 4 

5.2 Development of Emissions Model Inputs and Resultant Inventories 

 The base year emissions inventory for Phase II of the VISTAS modeling will be founded 

on revised 2002 emissions developed by VISTAS emission inventory contractors in NEI Input 

Format (NIF) 3.0. These revised emissions were reviewed by VISTAS stakeholders and 

considered complete in March of 2004. 

Non-VISTAS state emissions are based on inventories obtained by the Study Team and 

determined to be representative of the 2002 episode year. Western State base year point and area 

source emissions for 2002 were provided by the WRAP RPO. Additionally, an inventory of 

point source resolved agricultural fire emissions were provided by WRAP and utilized in the 

modeling. The CENRAP RPO provided VISTAS with an inventory of 2002 area source 

ammonia emissions which were also used in developing the remaining non-VISTAS emissions. 

For the remaining U.S. domain, point source projections were based on EPA’s 2001 

modeling platform inventories and area source, nonroad mobile, and fire emissions were based 

on EPA’s preliminary 2002 NEI. VMT and MOBILE input files were taken from EPA’s 

preliminary 2002 FTP site and used in running MOBILE6 nationally. 

TN Regional Haze SIP Appendix C.1 - 97
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 5.doc 5-4

Mexican and Canadian emissions are be based on the latest available inventories 

obtainable by the Study Team in formats lending themselves to emissions modeling. At this time, 

these inventories are the same as in Phase I. 

For purposes of air quality model validation, actual 2002 calendar year emissions for 

EGU and fire activity will be used, while during strategy and future year emission runs, “typical 

year” emissions for these categories will be processed. 

A final revised 2002 VISTAS state emission inventory is expected by September 2004 

and will be used in the final model performance demonstration and configuration expected to 

begin in October 2004. Non-VISTAS state emissions are expected to be based on State 

submittals to EPA under the CERR and will be augmented with additional data provided by 

RPO, State, and international sources. As in the initial modeling, the revised modeling will use 

actual 2002 calendar year emissions for EGU and fires, while “typical year” emissions for these 

categories will be processed during the strategy runs. 

 These emissions will then be converted to Inventory Data Analyzer (IDA) formatted 

versions and the data will be processed for air quality modeling using Version 2.0 of the Sparse 

Matrix Operating Kernel Emissions (SMOKE) model. Included in these runs will be the 

temporal, spatial, and speciation profiles and cross-reference data currently provided with the 2.0 

release of the model augmented with any recommended and approved emission profile data 

provided by the emissions inventory contractor or obtained from EPA prior to initial emissions 

modeling. The processing will be adjusted for each run to account for the specific air quality 

model (AQM) input required by CMAQ. 

5.2.1 Emissions Modeling Methodology

Emissions inventory development for photochemical modeling must address several 

source categories including: (a) stationary point sources, (b) area sources, (c) on-road mobile 

sources, (d) non-road mobile sources, and (e) biogenic sources. For this analysis, these estimates 

must be developed to support the episode that is being modeled (i.e., the historical base year 

when the episode actually occurred; 2002). 

Development of an emissions inventory customized for the VISTAS region requires a 

merging of: (a) the most recent pertinent regional inventory and (b) available high-resolution, 

locale-specific emissions estimated by local, state, and regional agencies in the VISTAS region. 

Local air regulatory and transportation planning agencies are generally the best sources of 

domain specific activity and control factors to use in developing the base year emissions. Often, 

these local emissions data sets come from a variety of sources, frequently in different formats. 

 The study team will acquire emissions estimate data from Emissions Inventory 

Contractor, in the NIF 3.0 format for purposes of generating the emission inventory base year 

files necessary for Phase II of the VISTAS contract. These data will be augmented with highway 

mobile source data submitted to the VISTAS emission inventory contractors from the VISTAS 

participating States. 

Contacts with VISTAS’ emission inventory contractors and the U.S. EPA will be 

established and formal requests made for inventory corrections, updates and ancillary data 

TN Regional Haze SIP Appendix C.1 - 98
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 5.doc 5-5

pertinent to the modeling of emissions in their jurisdictions. Where feasible and consistent with 

project resources and schedule, these updated data sets will be acquired and will be used to 

create day-specific modeling inventories specific to the VISTAS domain for the base year 

episodes to be modeled. 

5.2.2 Set-up of SMOKE Over the VISTAS Domain 

 SMOKE will be configured to generate point, area, nonroad, highway, and biogenic 

source emissions. In addition, certain subcategories, such as fires and EGUs will be maintained 

in separate source category files in order to allow maximum flexibility in producing alternate 

strategies. Settings for each of the source categories are discussed in relevant sections below. 

With the exception of biogenic and highway mobile source emissions that are generated using 

the, respectively, BEIS and MOBILE6 modules in SMOKE, pre-computed annual emissions will 

be processed using the month, day, and hour specific temporal profiles of the SMOKE model. 

 To produce an emissions inventory to support annual modeling, representative time 

periods will be selected and modeled. Area, nonroad, and point sources will be modeled as a 

block of Thursday, Friday, Saturday, Sunday, Monday one per month (total of 60 days modeled). 

On-Road motor vehicles will be represented by a entire single week for each month. This 

selection criteria allows for the representation of day-of-the-week variability in the on-road 

motor vehicles, and models a representation of the meteorological variability in each month. 

Holidays will be modeled as Sundays. A list of modeled holidays is provided in Table 5-2. The 

biogenic emissions will be modeled on a day specific basis (365 days). 

Table 5-2. SMOKE Modeled Holidays. 

Date Julian Day Holiday Description 

January 1, 2002 2002001 New Year's Day 

March 29, 2002 2002089 Good Friday 

May 27, 2002 2002147 Memorial Day 

July 4, 2002 2002185 July 4th 

September 2, 2002 2002245 Labor Day 

November 28, 2002 2002332 Thanksgiving Thurs 

November 29, 2002 2002333 Thanksgiving Fri 

December 24, 2002 2002358 Christmas Eve 

December 25, 2002 2002359 Christmas Day 

 Population will be used as a gridding default for all source categories when the assigned 

surrogate would cause SMOKE to drop emissions. This can be a case when the county-level 

emission inventories are prepared using surrogates other than those available for modeling 

purposes. 

 The domain for the Phase II episode will be identical to the Phase I domain, which is 

based on the EPA’s 36-km national CMAQ domain, illustrated in Figure 5-1 below (details on 

the modeling domains are provided in Chapter 4). 

TN Regional Haze SIP Appendix C.1 - 99
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 5.doc 5-6

Figure 5-1. EPA 36-km National CMAQ Domain. 

The parameters for the SMOKE runs are as follows: 

Episodes: 2002 Calendar Base Year. Optional tasks for episodic modeling of up to 60 

days that will likely be from 2003. 

Future Years: To be determined. 

Output Time Zone: Greenwich Mean Time (zone 0) 

Projection: Lambert Conformal with Alpha=33, Beta=45, Gamma=-97, and center at (-

97,40). 

Domain: 

• 36 Kilometer Grid: Origin at (-2736, -2088) kilometers with 148 rows by 112 

columns and 36-km square grid cells. 

• 12 Kilometer Grid: Origin at (108, -1620) kilometers with 168 rows by 177 columns 

and 12-km square grid cells. 

Layer Structure: The CMAQ layer structure will be 19 layers, with specific layer 

positions defined in the meteorology files (see Chapter 4). 

CMAQ Model Species: The CMAQ initial configuration will be for the CB-IV chemical 

mechanism with PM. The model species in the emission input files will be: CO, NO, 

TN Regional Haze SIP Appendix C.1 - 100
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 5.doc 5-7

NO2, ALD2, ETH, FORM, ISOP, NR, OLE, PAR, TERPB, TOL, XYL, NH3, SO2, 

SULF, PEC, PMFINE, PNO3, POA, PSO4, and PMC. 

Meteorology Data: Daily (25-hour). SMOKE requires the following five types of MCIP 

outputs: (1) Grid cross 2-d, (2) Grid cross 3-d, (3) Met cross 2-d, (4) Met cross 3-d, and 

(5), Met dot 3-d. These files need to match the grid projection and overlap with the 

emissions modeling region but can be larger in the horizontal directions than the 

modeling region shown in Figure 1. Therefore, the data files for the 36 Kilometer grid 

domain will be at least 90 columns by 132 rows 

Elevated Sources: All sources will be treated by SMOKE as potentially elevated. No 

plume-in-grid sources will be modeled. Wildfire emissions will be handled as point 

sources. 

Producing 365 day-specific input files for all source categories places a burden on 

available computing facilities, data management systems, and would adversely affect the Phase 

II schedule. Selecting representative model days for some or all of the source categories reduces 

the processing and file handling requirements to a more manageable level, and in most cases 

does not compromise the accuracy of the emissions files. 

Other current or recent projects undertaken by EPA, WRAP and LADCo have used a 

selection approach for all of the source categories (except biogenics) that use a representative 

weekday/Saturday/Sunday either for each month or each season to model all of the emissions 

files. In an attempt to better represent the level of temporal and spatial detail available for each 

source category, we have developed a more detailed strategy. 

Biogenic emissions will be modeled for each episode day, using the daily meteorology. 

Point sources, including CEM and fire emissions will be modeled for each episode day to take 

advantage of the available day-specific emissions and meteorology. Area sources, including 

non-road mobile and dust emissions do not utilize meteorological data, and are temporally 

allocated by monthly, daily and hourly profiles. Reviewing these profiles indicate that maximum 

temporal definition can be achieved by selecting representative Thursday, Friday, Saturday, 

Sunday, and Monday profiles for each month. 

Motor vehicle emissions are influenced by meteorological variability, but the processing 

requirements for daily motor vehicle emissions were determined to be prohibitive under the 

current schedule. Rather than utilizing averaged meteorological data or pre-calculated motor 

vehicle emissions, a single week per month was selected for modeling. This week was selected 

from mid-month, to try to best represent the average temperature ranges for the month, and also 

adjusted to exclude holidays that would require atypical processing. The area source modeling 

dates were also selected from these ranges to simplify data handling procedures. 

2002 On-Road Mobile Sources Represented by the Following Weeks: 

 January 13-19 

 February 10-16 

 March 10-16 

 April14-20 

 May 12-18 

TN Regional Haze SIP Appendix C.1 - 101
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 5.doc 5-8

 June 9-15 

 July 14-20 

August 11-17 

September 15-21 

October 13-19 

November 10-16 

December 15-21 

5.2.3 Development of Point Source Emissions 

 Stack parameters are often more important to the reliability of the air quality modeling 

results than the emissions rates themselves. Stack parameter data are frequently incorrect, 

especially in some of the current regional modeling inventories and careful QA is required to 

assure that the point source emissions are properly located both horizontally and vertically on the 

modeling grid. To screen for simple, but potentially serious inventory errors such as these, the 

study team has modified procedures originally developed by EPA to quality assure, augment, 

and where necessary, revise, stack parameters to examine the accuracy of the point source 

emissions, as well as standardize procedures to identify and correct stack data errors. These 

procedures will be implemented in the NIF to IDA conversion step of the inventory 

development. Additionally, SMOKE has a number of built-in QA procedures designed to catch 

missing or out-of-range stack parameters. These procedures will also be invoked in the 

processing of the point source data. 

For the Phase II initial baseline modeling, we will be separating the point source 

emissions into EGU and non-EGU categories. The non-EGU category will not be using any day 

or hour-specific emissions. All non-EGU point source emissions will be temporally allocated to 

month, day, and hours using annual emissions and source category code (SCC) based allocation 

factors. These factors will be based on the cross-reference and profile data supplied with the 

SMOKE 2.0 version and will be supplemented with relevant data provided to the study team by 

VISTAS or its contractors. 

For EGU sources with EPA reported CEM data or with hourly emissions provided by 

stakeholders, actual hourly data will be used. For those sources where EPA CEM data are 

utilized, NOx, SO2, and heat input-based hour-specific profiles were developed and applied to 

NOx, SO2, and all other emissions, respectively. This ensured that the annual emission values 

provided by the EI contractor were maintained, but distributed using hourly to annual profiles. 

For sources providing hour-specific data and where they were approved by the State in which 

they operated, those data were substituted for EPA CEM-based emissions and distributions. 

To temporally allocate the remaining EGU point sources, the NOx, SO2, and heat input 

data were collected from the 2002 Continuous Emissions Monitoring (CEM) datasets, and used 

to develop unit-level temporal distributions. The hour, day of week, and monthly specific 

temporal profiles will be used in conjunction with the EI supplied emissions data to calculate 

hourly EGU emissions by unit. 

TN Regional Haze SIP Appendix C.1 - 102
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 5.doc 5-9

All point sources will be spatially allocated in the domain based on the stationary source 

geographic coordinates. If a point source is missing its latitude/longitude coordinates, the source 

will be placed in the center of its respective county. 

5.2.4 Development of Area and Non-Road Source Emissions 

All area and non-road source emissions will be temporally allocated to month, day, and 

hours using annual emissions and source category code (SCC) based allocation factors. These 

factors will be based on the cross-reference and profile data supplied with the SMOKE 2.0 beta 

version and will be supplemented with relevant data provided to the study team by VISTAS or 

its contractors. Area and non-road sources will be spatially allocated in the domain based on 

SCC-based spatial allocation factor files. If an area or non-road source SCC does not have an 

existing cross-reference profile assigned to it, the county-level emissions will be allocated by 

population density in the respective county. 

A crustal PM transport factor will be applied to fugitive dust emission sources that have 

been identified in U.S. EPA modeling to have only a portion of its mass transportable from the 

source of the emission generation. The EPA’s studies indicate that 60 to 90 percent of PM 

emissions from fugitive dust sources do not reach an elevated level necessary to be transported 

or modeled in an episodic simulation. For this reason, the VISTAS emissions contractor will 

apply county-specific fugitive dust emissions transport factors to these sources to adjust PM 

emissions accordingly. The fugitive dust emissions adjustment is done by the VISTAS 

emissions contractor prior to receipt by the emissions and air quality modeling Team. Thus, no 

additional adjustment will be made in the emissions modeling. (for adjustment factors see: 

http://www.epa.gov/ttn/chief/emch/invent/statusfugdustemissions_082203.pdf). 

5.2.5 Development of On-Road Mobile Source Emissions 

The MOBILE6 module of SMOKE will be used to develop the base year on-road mobile 

source emissions estimates for CO, NOx, PM, and VOC emissions. The MOBILE6 parameters, 

vehicle fleet descriptions, and VMT estimates will be combined with gridded, episode-specific 

temperature data to calculate the gridded, temporalized emission estimates. Of note, whereas the 

on-network emissions estimates are spatially allocated based on link location and subsequently 

summed to the grid cell level, the off-network emissions estimates are spatially allocated based 

on a combination of the FHWA version 2.0 highway networks and population. For the VISTAS 

36/12 km modeling, no link based data will be used. The MOBILE6 emissions factors are based 

on episode-specific temperatures predicted by the meteorological model. Further, the MOBILE6 

emissions factors model accounts for the following: 

• Hourly and daily minimum/maximum temperatures; 

• Facility speeds; 

• Locale-specific inspection/maintenance (I/M) control programs, if any; 

• Adjustments for running losses; 

• Splitting of evaporative and exhaust emissions into separate source categories; 

• VMT, fleet turnover, and changes in fuel composition and Reid vapor pressure 

(RVP). 

TN Regional Haze SIP Appendix C.1 - 103
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 5.doc 5-10

The primary input to MOBILE6 is the MOBILE shell file. The MOBILE shell contains the 

various options (e.g. type of inspection and maintenance program in effect, type of oxygenated 

fuel program in effect, alternative vehicle mix profiles, RVP of in-use fuel, operating mode) that 

direct the calculation of the MOBILE6 emissions factors. 

 

5.2.6 Development of Biogenic Source Emissions 

 A revised version of a commonly used biogenic emissions model, the Biogenic 

Emissions Inventory System (BEIS), has recently been developed and tested by EPA over two 

separate modeling domains/episodes. This version of the model (BEIS-3, v0.9) contains several 

changes over BEIS-2, including the following: 

• Vegetation input data -- are now based on a 1-km Biogenic Emissions Landuse 

Database (BELD3) vegetation data base, 

• Emission factors – many updates including some recent NARSTO modifications, 

• Environmental algorithm -- includes a sunlit/shaded leaf solar radiation model. 

A series of sensitivity modeling simulations has been completed and concluded that the more 

recent BEIS-3 methodology will impact base case model ozone predictions in most parts of the 

U.S. The preliminary tests have also shown that the newer biogenic emissions do not appear to 

have a large effect on: 1) the control signal response, 2) relative reduction factors resulting from 

a projected emissions change, or 3) overall regional model performance in the eastern U.S. 

For this particular application of BEIS-3, version 0.9 as currently incorporated in the 

SMOKE processor will be used. This means that: 1) soil NO emissions shall be prepared 

without the input of specific soil moisture and precipitation data and 2) MEOH emissions will 

not be modeled explicitly. Otherwise, the modeling should be identical to a BEIS-3 (v1.0) 

application. 

The BELD-3 landuse data on a Lambert conformal grid at 1-km resolution have already 

been developed, are available, and will be used to estimate biogenic emissions in this study. The 

BEIS model also requires as input hourly, gridded temperature and solar radiation data to 

estimate biogenic emissions, and these data will be derived from the MM5 predictions. 

 5.2.7 Wildfires, Prescribed Burns, Wind Blown Dust and Sea 

 Salt Source Emissions 

Wildfires and Prescribed Burns

Wildfire and prescribed burn emissions will be handled separately from the standard area 

source input files. The study team expects to receive monthly estimates of fire emissions from 

the EI contractor, which include burn acreage and biomass loading information for the VISTAS 

states. Depending on the completeness and quality of the data received, attempts will be made to 

calculate spatial and temporal distributions of the fire emissions, rather than relying on standard 

distribution profiles. Also, the study team will attempt to calculate vertical distribution of the 

fire emissions, based on fire size and biomass involvement. The SMOKE 2.0 can model fire 

plume rise if provided with the following variables: 

TN Regional Haze SIP Appendix C.1 - 104
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 5.doc 5-11

PTOP – Top of the fire plume profile (meters above ground level) 

PBOT – Bottom of the fire plume profile (meters above ground level) 

Lay1 – The percent of the emissions entrained in the first modeling layer 

The WRAP Fire Emissions Joint Forum Emissions Inventory Report (FEJF,2002) has 

documented an approach to calculating these plume descriptors. In this method, the fires are 

assigned to one of 5 size categories, based on the total burn acreage, and the biomass fuel 

loading. These categories are then used to calculate representative hourly plume profiles. These 

profiles are then used by SMOKE2.0 to distribute the vertical emissions for the fires. To 

successfully model fires as elevated point sources, the data provided by the EI contractor will 

need to include both the day or days on which the fire occurs, and a spatial identifier of the fire 

location. At a minimum, a latitude and longitude of the fire location can be used, while a 

polygon coverage would be preferable. 

 In addition, wildfire and prescribed burn data, including emissions estimates and plume 

rise distributions, will be obtained from the WRAP contractor and used to supplement the 

inventory for the WRAP states. 

Windblown Dust

PM10 and PM2.5 emissions from wind erosion of natural geogenic sources (SCCs 

2730100000 [total] and 2730100001 [dust devils]) will be excluded from the resulting modeling 

files using a 100 percent reduction in the control packets. 

Sea Salt

CMAQ currently treats sea salt as an inert PM species. That is, the sea salt is not allowed 

to chemically interact with other species, such as producing particulate sodium nitrate. There are 

plans to update CMAQ to have chemically active sea salt, but it is unclear whether such an 

update will occur during the Phase II modeling. Accordingly, the initial Phase II modeling will 

be conducted without any sea salt emissions. If CMAQ is updated to treat chemically active Sea 

Salt, or if CAMx is run using its full-science options, then Sea Salt emissions will be generated 

using appropriate procedures (e.g., as was done in Phase I). 

5.2.8 Speciation and Reformatting of Emissions 

SMOKE will be run to speciate the emissions estimates according to the requirements of 

the Carbon Bond Mechanism version four (CBM-IV, CB-IV or CB4). Note that VISTAS is 

evaluating whether the CB-IV or SAPRC99 photochemical mechanisms are more appropriate for 

the Phase II modeling and may switch to the SAPRC99 mechanism for the revised base case 

modeling that will be initiated in September 2004. The SMOKE model will also reformat the 

emissions estimates for use in CMAQ modeling. For each model-ready emissions inventory, 

SMOKE will produce at a minimum five (5) separate air quality model-ready files: low-level 

point source, area source, elevated point source, mobile source, and biogenics. Other source 

categories, such as EGU and fire emissions may also be handled as separate air quality modelready files.

 

TN Regional Haze SIP Appendix C.1 - 105
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 5.doc 5-12

5.2.9 Development of Modeling Inventories 

 The emissions inventories modeled for the VISTAS Phase II can be grouped into three 

distinct types: 1) 2002 annual inventories, 2) Future Year episodic inventories, using Phase I 

definitions, and 3) Alternate Base Year Episodic Inventories. In all cases, the Study Team 

expects to receive the emissions inventory data for the VISTAS states from the Emissions 

Inventory contractor, add non-VISTAS states and Canadian and Mexican data acquired from 

alternate EPA and/or RPO sources, and produce the CMAQ ready emissions files. 

 5.2.9.1 2002 Annual Inventories 

 The initial 2002 annual base inventory will be produced in two phases, coinciding with 

the delivery of the MM5 model results for the first 8 months and last 4 months of the 2002 

calendar year. The initial inventory will consist of all of the required source categories: area, 

nonroad, point, motor vehicle and biogenics. 

 A revised 2002 annual inventory will be modeled later in the project, using updated 

emissions as supplied by the EI contractor. This may require re-production of all or some of the 

source categories. A “typical” inventory will be produced, based on the revised 2002, and 

replacing the revised actual emissions for EGUs and fires with “typical” EGU and fire emissions, 

as provided by the EI contractor. The Study Team will also produce up to 3 future year scenario 

annual runs, based on the 2002 meteorology. The composition of these future year runs is yet to 

be determined. 

5.2.9.2 Future Year Phase I Episodic Inventories 

 Utilizing emissions data provided by the EI contractors, the Study Team will model up to 

2 alternate future-year scenarios, using 2 of the Phase I episodes and the accompanying MM5 

meteorological data: (1) January 2- January 21, 2002, and (2) July 7 to July 28, 2001. This may 

require re-modeling of all of the source categories, with the exception of the biogenics. 

 5.2.9.3 Alternate Year Episodic Inventories 

To capture visibility conditions that are not otherwise represented by 2002 conditions, we 

may perform episodic air quality modeling for up to 60 additional episode days selected from 

2003 or another year. Emissions inventories for these additional days will be prepared by the 

Study Team and the requisite meteorological modeling results would be provided prior to any 

modeling. The additional episode days will be modeled for a single base case inventory. 

5.2.10 Products of the Emissions Inventory Development Process

 In addition to the CMAQ-ready input files generated for each hour of the days modeled 

in the Phase II annual run, a number of quality assurance (QA) files will be prepared and used to 

check for gross errors in the emissions inputs. Importing the model-ready emissions into PAVE 

and looking at both the spatial and temporal distribution of the emission provides insight into the 

quality and accuracy of the emissions inputs. 

TN Regional Haze SIP Appendix C.1 - 106
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 5.doc 5-13

C Visualizing the model-ready emissions with the scale of the plots set to a very low value, 

we can determine whether there are areas omitted from the raw inventory or if emissions 

sources are erroneously located in water cells. 

C Spot-check the holiday emissions files to confirm that they are temporally allocated like 

Sundays. 

C Producing pie charts emission summaries that highlight the contribution of each 

emissions source component (e.g. nonroad mobile). 

C Normalizing the emissions by population for each state will illustrate where the 

inventories may be deficient and provide a reality check of the inventories. 

C Spot check vertical allocation of point sources using PAVE. 

We will use state inventory summaries prepared prior to the emissions processing to compare 

against SMOKE output report totals generated after each major step of the emissions generation 

process. 

 To check the chemical speciation of the emissions to CB-IV terms and the vertical 

allocation of the emissions, we will compare reports generated with SMOKE reports to target 

these specific areas of the processing. For speciation, we will compare the inventory import state 

totals versus the same state totals with the speciation matrix applied. 

 For checking the vertical allocation of the emissions, we will create reports by source, 

hour, and layer for randomly selected states in the domain. We will create these reports for a 

representative weekday in each of the episodes for each of these selected states. 

 The quantitative QA analyses often reveal significant deficiencies in the input data or the 

model setup. It may become necessary to tailor these procedures to track down the source of 

each major problem. As such, we can only outline the basic quantitative QA steps that we will 

perform in an attempt to reveal the underlying problems with the inventories or processing. 

Following are some of the reports that may be generated to review the processed emissions: 

C State and county totals from inventory for each source category 

C State and county totals after spatial allocation for each source category 

C State and county totals by day after temporal allocation for each source category for 

representative days 

C State and county totals by model species after chemical speciation for each source 

category 

C State and county model-ready totals (after spatial allocation, temporal allocation, and 

chemical speciation) for each source category and for all source categories combined 

C If elevated source selection is chosen by user, the report indicating which sources have 

been selected as elevated and plume-in-grid will be included. 

TN Regional Haze SIP Appendix C.1 - 107
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 5.doc 5-14

C Totals by source category code (SCC) from the inventory for area, mobile, and point 

sources 

C Totals by state and SCC from the inventory for area, mobile, and point sources 

C Totals by county and SCC from the inventory for area, mobile, and point sources 

C Totals by SCC and spatial surrogates code for area and mobile sources 

C Totals by speciation profile code for area, mobile, and point sources 

C Totals by speciation profile code and SCC for area, mobile, and point sources 

C Totals by monthly temporal profile code for area, mobile, and point sources 

C Totals by monthly temporal profile code and SCC for area, mobile, and point sources 

C Totals by weekly temporal profile code for area, mobile, and point sources 

C Totals by weekly temporal profile code and SCC for area, mobile, and point sources 

C Totals by diurnal temporal profile code for area, mobile, and point sources 

C Totals by diurnal temporal profile code and SCC for area, mobile, and point sources 

C PAVE plots of gridded inventory pollutants for all pollutants for area, mobile, and point 

sources 

5.3 Quality Model Configuration and Input Preparation 

5.3.1 CMAQ Science Configuration 

This section described the model configuration and science options to be used in the 

Phase II modeling effort. The recommendations are based on testing and model evaluations of 

several models or model configurations carried out in the VISTAS Phase I study. Table 5-3 

summarizes the proposed configuration for CMAQ. The latest version of CMAQ is currently 

Version 4.3 that was released October 2003. However, EPA is expected to release an updated 

version of CMAQ in mid 2004, and this version will likely be used in the Phase II modeling. In 

addition, we will perform simulations for at least January and July using the CMAQ 4.3 and the 

new versions of CMAQ to determine if the CMAQ updates substantially change the model 

predictions or performance. 

In the CMAQ base configuration we will run both the 36 km and 12 km grids using oneway grid nesting where the boundary conditions for the 12 km grid simulation are extracted from 

the 36 km run using the CMAQ BCON processor. The base configuration of CMAQ will use 19 

vertical layers up to a region top of 100 mb (approximately 15 km AGL). 

TN Regional Haze SIP Appendix C.1 - 108
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 5.doc 5-15

The PPM advection solver would be used along with the spatially varying (Smagorinsky) 

horizontal diffusion approach. K-theory will be used for vertical diffusion with a minimum 

eddy diffusion constant (Kz) of 0.1 m2

/s. Note that the CMAQ default minimum Kz value is 1.0 

m

2

/s. However, based on theoretical considerations and the Phase I testing and evaluation a 0.1 

m

2

/s minimum Kz values was selected for Phase II modeling. 

The MCIP2.2 will be used to process the MM5 data using the “pass through” option. 

The AERO3/ISORROPIA aerosol chemistry scheme will be used for inorganic aerosol 

thermodynamics. During the Phase I testing of the CMAQ model it was noted that the 

AERO3/ISORROPIA aerosol modules failed to conserve sulfur and nitrogen mass. A mass 

conservation patch developed by the Georgia Institute of Technology was added to the model 

that renormalized the total sulfur, reactive nitrogen and reduced nitrogen species after the call to 

the aerosol modules to conserve sulfur and nitrogen mass in the model. Sensitivity simulations 

were performed with and without the mass conservation patch. Although the mass conservation 

patch did not affect model performance, the VISTAS science team believes that mass 

conservation is a fundamental characteristic of the real-world atmosphere so have retained the 

mass conservation patch for the Phase II modeling. The SORGAM scheme, which includes a 

reversible thermal equilibrium, will be used for secondary organic aerosols. 

Phase I testing included comparisons of the three gas phase chemistry mechanisms: the 

Carbon Bond Mechanism version 4 (CB4, Gery et al., 1989), a recent update of CB4 (CB4-2002, 

Jeffries et al., 2002), and the SAPRC99 (Carter, 1999). Although Phase I testing did not show 

notably better performance for any one mechanism, the use of either CB4-2002 or SAPRC99 

chemistry in the Phase II modeling is desirable because these mechanisms employ the most 

recent available chemical kinetics data and are more scientifically defensible than the older CB4 

mechanism. The computational cost of the SAPRC99 chemistry is nearly twice that of the CB4. 

Since CB4-2002 is only implemented in the slower SMVGEAR chemistry solver, the 

computational penalty of using CB4-2002 chemistry over using CB4 with the EBI chemistry 

solver is even greater than that for SAPRC99. Given that the initial testing of the CMAQ model 

in the first half of 2004 will be more of a shakedown of the modeling system, including the 2002 

MM5 data and initial 2002 emission estimates, and the three chemical mechanisms produce 

similar model performance, then we propose to use the CB4 chemical mechanism for the initial 

2002 base case modeling. In the mean time, under Task 7 of the Phase II work effort the 

sensitivity of the CMAQ model to emission reductions using the CB4 and SAPRC99 chemistries 

will be evaluated using the January 2002 and July 2001 Phase I episodes to determine whether 

the two mechanisms have different responses. Based on this evaluation, and considerations of 

whether CB4-2002 has been implemented in CMAQ using the faster EBI chemistry solver, a 

decision will be made on the choice of the gas-phase photochemical mechanism for the revised 

2002 base case modeling to be initiated in September 2002. 

Finally, recent environmental chamber experiments at UCR suggest that both CB4-2002 

and SAPRC99 may underestimate the reactivity of aromatic compounds. UCR and others expect 

to test new versions of CB4 and SAPRC chemistry during 2004 independently of the VISTAS 

work effort. Although updates in aromatic chemistry will primarily affect urban areas, it is 

desirable to perform Phase II modeling using the most up-to-date version of the chemistry 

possible. We will report any pertinent findings or new chemical mechanisms to VISTAS as they 

become available. 

TN Regional Haze SIP Appendix C.1 - 109
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 5.doc 5-16

Table 5-3. Proposed Phase II Model Configuration for the CMAQ.

Model Option CMAQ 

Model Version Version 4.4 (April 2004) 

Horizontal Resolution 36/12 km 

No. Vertical Layers NZ = 19 

Horizontal Advection PPM 

Vertical Advection PPM 

Horizontal Diffusion Spatially Varying 

Vertical Diffusion KV (Eddy Diffusion) 

MM5 Configuration Pleim-Xiu/ACM 

MM5 Processing MCIP2.2 Pass Through 

Gas-Phase Chemistry CB4 

Gas-Phase Chemistry Solver MEBI/Hertel 

Secondary Organic Aerosol SORGAM 

Aqueous-Phase Chemistry RADM 

Aerosol Chemistry AE3/ISORROPIA 

Dry Deposition Pleim-Xiu 

Plume-in-Grid Off 

Initial Concentrations CMAQ Default 

Boundary Conditions Initial: Seasonal 2001 GEOS-CHEM 

Revised: 3-Hourly 2002 GEOS-CHEM 

Emissions NEI 1999 v2 

CMU NH3 Adjustments 

5.3.2 Spin-Up Initialization 

For the 2002 annual CMAQ modeling, the model will be exercised separately for four 

quarters. The 2002 MM5 modeling started on December 17, 2001 at 12Z. Thus, allowing for 12 

hours of spin up of the MM5 model, CMAQ will be initialized at 00Z on December 18, 2001. 

This results in a 13 day spin up period for CMAQ and the first quarter run segment of 2002. For 

the other quarter run segments of 2002, CMAQ will be initialized with a 15 day spin up period. 

5.3.3 Boundary Conditions 

The Phase I modeling included comparisons of several schemes for producing boundary 

conditions for the 36-km domain, including the following: 

• Fixed, default values at each edge, with vertical variation based on CMAQ default data. 

• Clean boundary conditions. 

• Seasonal mean output from the GEOS-CHEM model for a 2001 simulation 

In addition, for the July 2001 episode we examined the following three boundary conditions: 

• Fixed CMAQ default values; 

• 3-hourly temporally and spatially varying BCs based on the 2001 GEOS-CHEM output 

• Monthly average GEOS-CHEM output for July 2001 

TN Regional Haze SIP Appendix C.1 - 110
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 5.doc 5-17

Sensitivity experiments showed that model performance in the VISTAS states during conditions 

of high visibility impairment was relatively insensitive to variations in the boundary conditions, 

although differences in concentrations did occur. However, it is likely that boundary conditions 

will have a larger effect on visibility for clean days and for future conditions as visibility begins 

to improve. Moreover, boundary conditions will have a more significant effect on natural 

background visibility. In fact the BCs had a larger effect on the cleaner January 2002 model 

estimates than the dirtier July 1999 and 2001 episodes. Therefore, as a long-term effort we 

recommend that boundary conditions for the 36 km continental US domain be obtained from a 

global model such as GEOS-CHEM. However, because global model results are not yet 

available for 2002, in the initial Phase II modeling we will use seasonal average diurnally 

varying data from a 2001 GEOS-CHEM data to provide boundary conditions for the 36 km 

domain. If 2002 GEOS-CHEM results become available, the BCs will be updated with high 

temporal resolution (e.g., 3-hourly) day-specific GEOS-CHEM output. Boundary conditions for 

the 12 km nest will be obtained from the CMAQ 36-km simulation. 

5.3.4 Photolysis Rates 

Several chemical reactions in the atmosphere are initiated by the photodissociation of 

various trace gases. To accurately represent the complex chemical transformations in the 

atmosphere, accurate estimates of these photodissociation rates must be made. The Models-3 

CMAQ system includes the JPROC processor, which calculates a table of clear-sky photolysis 

rates (or J-values) for a specific date. JPROC uses default values for total aerosol loading and 

provides the option to use default column O3 data or to use TOMS data for total column O3. 

Previously, in the Phase I modeling we used TOMS data, and we will continue to use TOMS 

data for Phase II. There are a few days in 2002 for which TOMS data is not available, and we 

will used default column O3 data or previous days data for those days. 

JPROC produces a "look-up" table provides the photolysis rates as a function of latitude, 

altitude, and time (in terms of the number of hours of deviation from local noon, or hour angle). 

In the current CMAQ implementation, the J-values are calculated for six latitudinal bands (10o, 

20o, 30o, 40o, 50o, and 60o N), seven altitudes (0 km, 1 km, 2 km, 3 km, 4 km, 5 km, and 10 km), 

and hourly values up to "8 hours of deviation from local noon. During model calculations, 

photolysis rates for each model grid cell are estimated by first interpolating the clear-sky 

photolysis rates from the look-up table using the grid cell latitude, altitude, and hour angle, 

followed by applying a cloud correction factor. 

The photolysis rates input file must be prepared as separate look-up tables for each 

simulation day. The modeling team has already prepared scripts to automate the production of 

photolysis rate files for each day of the annual simulation. Photolysis files are ASCII files, and 

these will be visually checked for selected days to verify that photolysis are within the expected 

ranges. 

5.4 CAMx Model Inputs 

If the optional task to apply the CAMx model to corroborative the CMAQ results were 

selected, then the model inputs would be prepared in a manner consistent with the CMAQ inputs. 

The same modeling domain and grid structure as used for the CMAQ modeling of 2002 would 

TN Regional Haze SIP Appendix C.1 - 111
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 5.doc 5-18

be used for CAMx (see Chapter 4). Meteorological inputs would be generated using the 

MM5CAMx processor using options similar to the MCIP “pass through” option and with a 0.1 

m2

/s minimum Kz. The CAMx initial and boundary conditions (IC/BC) inputs would be 

generated from the CMAQ IC/BC inputs using the CMAQ-to-CAMx IC/BC processors. The 

CAMx emission inputs would be generated using the CMAQ-ready emission inputs and the 

CMAQ-to-CAMx emission processor. 

TN Regional Haze SIP Appendix C.1 - 112
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 6.doc 6-1 

6.0 QUALITY ASSURANCE PLAN

In this section we discuss the quality assurance procedures that will be used in the Phase 

II modeling. More details are provided in the VISTAS Phase II Quality Assurance Project Plan 

(QAPP) that is under preparation. 

6.1 Quality Assurance Objectives 

 In December 2002, the USEPA publish extensive guidance on developing a Quality 

Assurance Project Plan (QAPP) for modeling studies (EPA, 2002). The objective of a QAPP is 

to ensure that a modeling study is scientifically sound, robust, and defensible. The new EPA 

guidance suggests that a QAPP should include the following elements : 

• a systematic planning process including identification of assessments and related 

performance criteria; 

• peer reviewed theory and equations; 

• a carefully designed life-cycle development process that minimizes errors; 

• clear documentation of assumptions, theory, and parameterization that is detailed enough 

so others can fully understand the model output; 

• input data and parameters that are accurate and appropriate for the problem; 

• output data that can be used to help inform decision making. 

• documentation of any changes from the original quality assurance plan; 

 Moreover, the EPA guidance specifies that different levels of QAPP may be required 

depending on the intended application of the model, with a modeling study designed for 

regulatory purposes requiring the highest level of quality assurance. 

 The QAPP also provides a valuable resource for project management. It can be used to 

document data sources and assumptions used in the modeling study, and it can be used to guide 

project personnel through the data processing and model application process to ensure that 

choices are consistent with the project objectives. 

 The modeling team has already developed QA documents and procedures in the Phase I 

effort. For the Phase II effort these will be enhanced and modified to assure that it conforms to 

the EPA QAPP guidance. 

 The guidance document also addresses model development, coding and selection of 

models, and model performance requirements. For the Phase II modeling we are using an 

existing EPA sponsored model. Thus, our QAPP will focus primarily on documenting data 

sources and QA of data processing performed by the model team. In addition, because no official 

EPA guidance currently exists for visibility model performance, a major objective of our QAPP 

will be to propose and define model performance evaluation procedures. QA objectives for 

specific aspects of the project are discussed below, and these will be incorporated into a QAPP 

that conforms to the EPA guidance document for modeling studies. 

TN Regional Haze SIP Appendix C.1 - 113
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 6.doc 6-2 

6.2 Emissions Model Inputs and Outputs 

Emissions Quality Assurance (QA) and Quality Control (QC) are the single most critical 

step in performing air quality modeling studies. Because emissions processing is tedious, time 

consuming and involves complex manipulation of many different types of large data sets, errors 

are frequently made in emissions processing and, if rigorous QA measures are not in place, these 

errors may remain undetected. 

As part of the VISTAS Phase II QA effort, an “Emissions Gatekeeper” function will be 

implemented similar to that used effectively in Phase I. The Study Team envisions the role of 

this Gatekeeper as one to perform quality assurance activities on the following emission 

inventory (EI) data: 

(1) EI data obtained from the VISTAS emissions inventory contractors; and 

(2) The emission inventory to be used for modeling outside of the States in the VISTAS 

region. 

Specifically, the Emissions Gatekeeper will review the content and format of the provided 

emission inventories ensuring an appropriate appraisal of the emissions data and estimates for 

the VISTAS States. Other tasks will include any additional translation from mass emissions files 

into the emissions modeling input file structure necessary for modeling. The Study Team will 

supplement these activities with QA checks on the intermediate and model output files using 

internal and public domain visualization and diagnostic packages. 

We propose to continue with multistep emissions the QA/QC approach applied in the 

Phase I modeling. This includes the initial emissions QA/QC by the Emissions Gatekeeper 

described above, as well as QA/QC by the Emissions Modeler during the processing of 

emissions and then additional QA/QC by the air quality modeler of the processed model ready 

emission files. This multistep process with three separate groups involved in the QA/QC of the 

emissions is much more likely to catch any errors prior to the air quality model simulations. 

6.2.1 Emissions Modeling QA/QC 

EMS and EPA Input Screening Error Checking Algorithms: Although the SMOKE emissions 

model will be used for emissions processing, some of the more advanced EMS input error 

checking algorithms will be used to screen the data and identify potential emission input errors. 

Additionally, EPA has issued a revised stack QA and augmentation procedures memorandum 

that will be used to identify and augment any outlying stacks. 

SMOKE error messages: SMOKE provides various cautionary or warning messages during the 

emissions processing. We will redirect the SMOKE output to log files and review the log files 

for serious error messages. An archive of the log files will be maintained so that the error 

messages can be reviewed at a later date if necessary. 

SMOKE emissions summaries: We will use QA functions built into the SMOKE processing 

system to provide summaries of processed emissions as daily totals according to species, source 

category and county and state boundaries. These summaries will then be compared with 

TN Regional Haze SIP Appendix C.1 - 114
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 6.doc 6-3 

summary data prepared for the pre-processed emissions, e.g., state and county totals for 

emissions from the augmented emissions data. 

6.2.2 QA of the Model-Ready Emissions Impacts 

The goal of the post-processed emissions summary QA is to detect possible errors in the 

final, model-ready binary emissions files by preparing summary plots that characterize spatial 

and temporal patterns in the emissions data. This step is designed to catch errors that may be 

missed in the internal SMOKE QA procedures. We will use a QA/QC post-processing program 

that read the CMAQ-ready I/O API emissions file formats for each of the major source 

categories (mobile, area, point, biogenic, fire) and produce the following plots. 

Spatial Summary: We will sum the emissions for all layers and for all 24 hours that is used to 

prepare a PAVE plot showing the daily total emissions spatial distribution. For a 20 day 

simulation this produces approximately 20 days x 20 species x 5 emissions categories = 2,000 

plots. In our base case simulations these plots will be presented as tons per day. The objective of 

this step is to identify errors in spatial distribution of emissions. 

Vertical Profile: For point sources the emissions total for each layer will be summed and plotted 

to show the vertical distribution of emissions. These plots show the emissions on the x-axis for 

each model layer on the y-axis. The objective of this step is to identify possible errors in vertical 

distribution of emissions. 

Short Term Temporal Summary: The total domain emissions for each hour will be accumulated 

and time series plots prepared that display the diurnal variation in total hourly emissions. The 

objective of this step is to identify errors in temporal profiles. 

Long Term Temporal Summary: The total domain emissions for each day will be accumulated 

and displayed as time series plots that show the daily total emissions across the domain as a 

function of time. The objective of this step is to identify particular days for which emissions 

appear to be inconsistent with other days for no reason (e.g., not a weekend) and compare against 

the general trend. 

Control Strategy Spatial Displays: Spatial summary plots of the daily total emissions differences 

between a control strategy and base case emissions scenarios will be generated. These plots can 

be used to immediately identify a problem in a control strategy. For example, if a VISTAS states 

SO2 control strategy is being analyzed and there are changes in emissions for other pollutants or 

for SO2 outside of the VISTAS states problems in emissions processing can be identified prior to 

the air quality model simulation. 

6.3 Meteorological Model Outputs 

As part of the VISTAS Phase II QA effort, a “Meteorological Gatekeeper” function will 

be implemented similar to that used effectively in Phase I (Tesche and McNally, 2002a,b,c). The 

task of the Gatekeeper is to provide an independent review and quality assurance of the 

meteorological modeling and related data sets developed by the VISTAS meteorological 

modeling contractor (BAMS) and used subsequently by the emissions and air quality modeling 

teams. This Gatekeeper QA review serves two specific purposes: (a) to ensure that any potential 

TN Regional Haze SIP Appendix C.1 - 115
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 6.doc 6-4 

problems with the data sets (should they exist) are identified and corrected in a timely manner, 

and (b) to provide the study team with information to support ongoing CMAQ model 

performance testing and sensitivity analyses. In the case of meteorology, the Gatekeeper’s 

independent QA analysis of the MM5 meteorological data sets serves to provide direct assistance 

to the emissions and air quality modeling team as it undertakes to ratify the SMOKE model 

outputs and to diagnose CMAQ model performance and sensitivity analyses. 

In addition to having personal responsibility for the quality and chain of custody of the 

meteorological data sets supplied by other VISTAS contractors, the Meteorological Gatekeeper 

will be responsible for ensuring and maintaining the integrity of the data files uploaded to the 

project website. This website, hosted by UCR, serves as the repository of data for the 

ENVIRON/UCR/Alpine modeling centers and for the VISTAS TAWG participants. In 

performing the Gatekeeper quality assurance activity, one of the first steps is to conduct an 

independent operational evaluation on the MM5 model results at 36 km and 12 km grid scale. 

This evaluation covers surface and aloft wind direction, temperature, mixing ratio, precipitation, 

and planetary boundary layer (PBL) depths on a continental scale (36 km) and subregional scale 

(12 km) basis. The specific techniques to be used are described in the MM5 model performance 

protocol prepared for EPA for annual modeling (McNally and Tesche, 2002). The Gatekeeper 

will also perform supplemental, ad hoc analysis of pertinent MM5 fields (e.g., PBL depths) 

where that might be useful to the emissions and air quality modeling teams. Another task of the 

Gatekeeper will be to exercise the Meteorological Chemistry Interface Processor (MCIP) version 

2.2 is to read the MM5 outputs from BAMS and produce binary input files for the CMAQ 

Chemical Transport Model (CCTM) to provide the complete set of parameters necessary in the 

emissions processing and air quality modeling. 

In summary, the quality assurance plan for the meteorological data will include the 

following elements: 

3⁄4 Upon receiving the MM5 and MCIP 2.2 output files from BAMS, we will verify the 

integrity of the file transfer (e.g., no missing and/or corrupted files); 

 

3⁄4 Since the CMAQ modeling domain is a subset of the MM5 domain, we will verify 

that the modeling domain and vertical layer structures in the MCIP files are identical 

to the CMAQ modeling domain; 

3⁄4 We will select several days of the MM5 output and reprocess the MM5 files with 

MCIP v2.2 using the predetermined MCIP options. We will then compare the MCIP 

files with those provided by BAMS to verify that we obtain identical results from the 

MCIP processing. 

3⁄4 We will create horizontal and vertical plots of temperature, pressure, precipitation, 

modeled flow patterns, PBL heights, etc. to assess whether the MCIP output fields are 

reasonable; 

TN Regional Haze SIP Appendix C.1 - 116
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 6.doc 6-5 

3⁄4 The VISTAS 2002 MM5 simulation will be evaluated using the same surface 

observations, subdomains and procedures as used to evaluate the WRAP 2002 MM5 

simulation as an independent QA and evaluation of the database. 

3⁄4 We will make the plots available on the VISTAS website for viewing and download. 

6.4 Air Quality Model Inputs and Outputs 

Key aspects of QA for the CMAQ input and output data include the following: 

• Verification that correct configuration and science options are used in compiling and 

running each model of the in the CMAQ modeling system, where these include the 

MCIP, JPROC, ICON, BCON and the CCTM. 

• Verification that correct input data sets are used when running each model. 

• Evaluation of CCTM results to verify that model output is reasonable and consistent with 

general expectations. 

• Processing of ambient monitoring data for use in the model performance evaluation. 

• Evaluation of the CCTM results against concurrent observations. 

• Backup and archiving of critical model input data. 

 The most critical element in the QA plan for CMAQ simulations is the QA/QC of the 

meteorological and emissions input files. The major QA issue specifically associated with the air 

quality model simulations is verification that the correct science options were specified in the 

model itself and that the correct input files were used when running the model. For the CMAQ 

model we employ a system of naming conventions using environment variables in the compile 

and run scripts that guarantee that correct inputs and science options are used. We also employ a 

redundant naming system so that the name of key science options or inputs are included in the 

name of CMAQ executable program, in the name of the CMAQ output files, and in the name of 

the directory in which the files are located. This is accomplished by using the environment 

variables in the scripts to specify the names and locations of key input files. For example, if a 

model simulation is performed using the CB4 mechanism, all compile and run scripts contain the 

variable definition “$MECH = CB4”, and this variable is hard coded into the script for the 

executable name, the output file name, and the output directory name. This procedure produces 

long file/directory names but it effectively prevents mistakes or makes mistakes readily apparent 

if they do occur. 

 

 A second key QA procedure is to never “recycle” run scripts, i.e., we always preserve the 

original runs scripts and directory structure that were used in performing a model simulation. For 

example, if we perform simulation with the SAPRC mechanism, instead of editing the original 

scripts to specify “$MECH = SAPRC” we will create a parallel directory structure with a new set 

of scripts to perform the SAPRC simulations. This provides a permanent archive of the scripts 

that were used in performing model simulations. In addition, output from the model simulation 

TN Regional Haze SIP Appendix C.1 - 117
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 6.doc 6-6 

will be directed to a log file that provides a record of input file names, warning messages etc that 

will be archived. 

 We will also perform a post-processing QA of the CMAQ output files similar to that 

described for the emissions processing. We will generate animated gif files using PAVE that can 

be viewed to search for unexpected patterns in the CMAQ output files. In the case of model 

sensitivity studies, the animated gifs will be prepared as difference plots for the sensitivity case 

minus the base case. Often, errors in the emissions inputs can be discovered by viewing the 

animated GIFs. Finally, we will produce 24 hour average plots for each day of the CMAQ 

simulations. This provides a summary that can be useful for more quickly comparing various 

model simulations. 

TN Regional Haze SIP Appendix C.1 - 118
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-1 

7.0 MODEL PERFORMANCE EVALUATION 

7.1 OVERVIEW

As noted in Chapter 1, Appendix A presents a summary of the Scope of Work (SOW) for the 

VISTAS Phase II Emissions and Air Quality Modeling activities. This Phase II SOW delineates the 

tasks, approach and schedule for carrying out the various technical activities described in greater 

detail in this modeling protocol. The SOW in Appendix A is broken down into a base effort that 

focuses on annual modeling of 2002 and several optional tasks to be evaluated by VISTAS. The 

ultimate objective of the Phase II work effort in 2004 is to obtain a properly evaluated operational 

emissions and air quality modeling system for the 2002 annual calendar year so that control strategy 

evaluation can begin in 2005. Given the formal contractual role that the SOW plays in guiding the 

work products of the Emissions and Air Quality Modeling Team in Phase II, where ambiguities 

might arise regarding the Task 4a/4b model evaluation effort, the SOW takes precedence over this 

Protocol Document. Consistent with the spirit of a modeling protocol for regulatory decisionmaking, this document lays out the ‘roadmap’ for achieving an adequately tested modeling system 

for regulatory usage. But, obviously, this does not mean that every analysis identified in this chapter 

will be carried out or is indeed even possible given the Phase II schedule and resources, the existing 

aerometric data bases, and present technology constraints. The roadmap guides the way to the 

desired destination – in this case, an evaluated, operational PM/regional haze modeling system – but 

does not commit the driver to exploring every side street and back country road along the way. 

Indeed, one expectation of the VISTAS TAWG is a close working relationship with the modeling 

team to ensure that the available resources and schedule are applied most efficiently in reaching the 

aforementioned goal. 

 This chapter describes a range of model testing methodologies potentially available to the 

Emissions and Air Quality Modeling study team in its efforts to adequately evaluate the performance 

of the CMAQ air quality modeling system for the 2002 annual period. Since one cannot know at 

this juncture the specific performance problems that may arise in the initial 2002 CMAQ testing 

(Task 4a) or the final year 2002 evaluation (Task 4b), we set forth in this chapter a broad range of 

methods and techniques that may be brought to bear in examining CMAQ model performance. We 

identify the core operational evaluation procedures, recommended in EPA (2001) guidance, that will 

be performed in Tasks 4a and 4b. We also describe a broad range of additional performance testing 

methods that may be worth considering, if necessary. Implementation of one or several of these 

various techniques would fall under Optional Task 14 (Enhanced Model Performance Evaluation). 

The Optional Task 14 represents a means for marshalling additional VISTAS funding, or perhaps 

supplemental funding from other stakeholder groups, for the purpose of enhancing the core 

operational evaluation presently budgeted for Tasks 4a and 4b. Should portions or all of Optional 

Task 14 be invoked, we strongly recommend that this be guided by a cooperative effort by the 

modeling team and the TAWG to ensure that the highest priority “supplemental evaluations” be 

selected and properly carried out. 

 Clearly, not all of the supplemental evaluative techniques identified in this chapter will 

ultimately be performed. There are three reasons for this: 

3⁄4 The Phase II SOW places clear limits on the resources available to perform model 

evaluation analyses. Accordingly, some evaluation steps, while desirable, simply 

TN Regional Haze SIP Appendix C.1 - 119
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-2 

may not be possible given current funding levels; 

3⁄4 The Phase II SOW places stringent schedule demands on the model evaluation. A 

number of the model performance evaluation methods introduced in this chapter (e.g. 

Weight of Evidence analyses, diagnostic testing with individual measurement 

networks, PM indicator species and ratios analyses, use of the alternative CAMx 

model) could very likely require more time to carry out given their quasi-research 

nature. Since VISTAS Phase II is not a model research and development effort, but 

rather an operational evaluation of existing modeling systems for regulatory 

decision-making, some interesting, but time consuming analyses simply may not be 

possible given the present schedule; and 

3⁄4 To conform to the EPA PM guidance documents requirements for PM model testing, 

it may not be necessary to conduct many of the diagnostic and Weight of Evidence 

tests identified in this protocol. Indeed, an adequate evaluation of the Phase II 

modeling system may be possible through straightforward application of the core 

operational performance evaluation procedures identified in EPA’s 2001 draft 

guidance. 

 At a minimum, the evaluation of the CMAQ modeling system for the Phase II annual 2002 

simulation (and for any additional episodes that might be chosen by VISTAS) will be consistent with 

EPA’s draft guidance on PM model testing. This guidance essentially calls for an operational 

evaluation of the model focusing on a specific set of gas phase and aerosol chemical species and a 

suite of statistical metrics for quantifying model response over the annual cycle. The emphasis is on 

assessing: (a) How accurately the model predicts observed concentrations? and, (b) How accurately 

does the model predict responses of predicted air quality to changes in inputs? States are 

encouraged to utilize the evaluation procedures set forth in the earlier 1991 guidance document 

(EPA, 1991) for gas phase species and the newer (2001) guidance of PM species. Thus, in carrying 

out the initial Task 4a operational evaluation and the subsequent Task 4b final evaluation, we will 

implement the suggested EPA performance testing methodologies for the key gas phase and aerosol 

species. Since these methods are explicitly presented in EPA’s guidance document, there is no need 

to repeat them here. 

 Subject to the availability of time and Optional Task 14 resources, the Phase II evaluation 

will also attempt to employ other testing methods beyond those in the EPA guidance document. In 

addition, if the initial Task 4a evaluation evolves smoothly without problems arising in the early 

performance testing efforts, resources and/or time conserved in such a situation might then be used 

to underwrite on or more scientifically rigorous model performance investigations (e.g., those 

afforded by the rich observational networks such as the continuous speciation SEARCH 

measurements and the EPA Supersite data). To this end, we discuss in this chapter various 

supplemental performance testing methods that might be worth pursuing given adequate, additional 

resources and time. 

 We conclude by again emphasizing that most important goal of the Phase II CMAQ 

evaluation is to determine whether the aggregate modeling system (model codes plus input data sets 

and observational data for testing) offers sufficiently reliable and accurate results that public 

decision-makers may have reasonable confidence in using the model to help choose between 

TN Regional Haze SIP Appendix C.1 - 120
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-3 

alternative regional haze reduction scenarios. If the CMAQ model evaluation, as outlined in this 

chapter, provides sufficient evidence that the modeling system is operating reliably and in 

conformance with measurements and scientific expectations, then specific justifications explaining 

why the model is acceptable for developing regional haze strategies will be offered in the 2002 

CMAQ Modeling Summary Report. Conversely, should the evaluation determine that the modeling 

system suffers from important flaws or errors that undermine its reliability or use, these findings will 

also be documented, together with recommendations regarding the use of alternate methods, steps to 

improve the model and/or data base, or other approaches. 

7.2 Context for the VISTAS Phase II Model Evaluation 

 We begin the discussion of the Phase II modeling evaluation methodology by reviewing how 

the CMAQ model output is used in regional haze applications to project changes in visibility1

. 

When designing a model performance evaluation, it is important to understand how the modeling 

results will ultimately be used. EPA has published two versions of draft guidance for fine particulate 

and regional haze modeling (EPA, 2000; 2001), utilizing a Fine Particulate Guidance Workgroup to 

provide technical input in the development of both documents2

. More recently, EPA has provided 

an informal update on the PM/regional haze modeling guidance (Timin, 2002) and conducted a PM 

model evaluation workshop (see, for example, Timin, 2004; Boylan, 2004) shedding additional light 

on what the final guidance document might contain. 

A key concept in EPA’s guidance for addressing regional haze issues is that the modeling 

results should be used in a relative sense to scale or roll back the observed particulate matter (PM) 

concentrations from which light extinction is estimated. Adopting the recommendations from the 

Federal Land Managers’ Air Quality Related Values Workgroup (FLAG), estimates of visibility are 

obtained from observed fine particulate concentrations using the following IMPROVE reconstructed 

mass extinction equation (FLAG, 2000): 

bext = 3{f(RH)[(NH4)2SO4]} + 3{f(RH)[NH4NO3]} 

 + 4{f’(RH)[OC]} + 10[EC] + 1[IP] 

+ 0.6[CM] + brayleigh 

where: 

 bext is the estimated extinction coefficient (Mm-1); 

 [SO4] is the sulfate concentration assumed to be ammonium sulfate; 

 [NO3] is the particulate nitrate concentration assumed to be ammonium nitrate; 

 [OC] is the organic carbon concentration; 

 [EC] is the elemental carbon concentration; 

 

1

 Details on the definitions of model performance evaluation statistical measures and graphical 

displays in three reports to VISTAS: (a) the Phase I Task 2 model configuration report (ENVIRON, 

2003b), (b) the Task 4a/b model sensitivity and CMAQ configuration report (ENVIRON, 2003d), 

and (c) the Final Phase I Evaluation and Model Sensitivity Analysis report (ENVIRON, 2004). 

Additional details are also presented in Table 7-4. 

2 Members of the VISTAS modeling team participated on this work group over the two-year span of 

its activities. 

TN Regional Haze SIP Appendix C.1 - 121
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-4 

[IP] is the inorganic primary fine particulate (< 2.5 :) concentration excluding primary 

sulfates and nitrates; 

[CM] is the coarse particulate (> 2.5 : and < 10 :) concentration; 

brayleigh is the light-scattering due to Rayleigh scattering (assumed to be 10 Mm-1); 

f(RH) is a relative humidity adjustment factor for the sulfate and nitrates; and 

f’(RH) is a relative humidity adjustment factor for OC that is assumed to be 1.0. 

The relative humidity (RH) values used in the above equation are monthly- specific and 

Class I area-specific values based on a long term average (EPA, 2003a). 

The regional haze rule expresses reasonable progress in terms of changes in the deciview 

()dV) from the current to future year conditions. Deciview (dV) is expressed as the natural 

logarithm of the extinction coefficient (bext) to Rayleigh scattering: 

Deciview = 10 ln(bext/10) 

)dV = 10 ln[(bext)future / (bext)base] 

In this framework, changes in visibility in terms of percentage changes in extinction over a 

natural “clean” background visibility rather than changes in deciview. Changes in deciview or 

extinction are essentially mathematically identical with a 1% change in deciview being 

approximately equivalent to a 10% change in extinction. Both visibility parameters will be 

calculated in the Phase II model applications. 

The VISTAS Phase II model testing will concentrate on an operational evaluation of those 

model predictions that are most necessary for estimating visibility (i.e., NH4, SO4, NO3, OC, EC, IP 

and CM). Where feasible and supported by sufficient measurement data, we will also evaluate the 

modeling system for its ability to accurately estimate gas-phase oxidant and precursor/product 

species since correct, unbiased simulation of gas-phase photochemistry is a necessary element of 

reliable regional haze predictions. This evaluation will be carried out across the full VISTAS 

domain for the entire year and also on subdomains and month-by-month basis to help build 

confidence that the modeling system is operating correctly. With this context in mind, we next turn 

to the philosophy of the model evaluation process. 

7.3 Multi-Layered Model Testing Process 

EPA’s “Draft Guidance for Demonstrating Attainment of Air Quality Goals for PM2.5 and 

Regional Haze” (EPA, 2001) affirms the recommendations of numerous modeling scientists over the 

past decade (see, for example, Dennis et al., 1990; Tesche et al., 1990, 1994; Seigneur et al., 1998, 

2000; Russell and Dennis, 2000; Arnold et al., 2003; Boylan et al., 2003; Tonnesen, 2003) that a 

comprehensive, multi-layered approach to model performance testing should be performed, 

consisting of the four components: operational, diagnostic, mechanistic (or scientific) and 

probabilistic. As applied to regional PM/visibility models, this multi-layered framework may be 

viewed conceptually as follows: 

> Operational Evaluation: Tests the ability of the model to estimate PM concentrations 

(both fine and coarse) and the components at PM10 and PM2.5 including the quantities 

TN Regional Haze SIP Appendix C.1 - 122
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-5 

used to characterize visibility (i.e., sulfate, nitrate, ammonium, organic carbon, elemental 

carbon, PM2.5, and PM10). This evaluation examines whether the measurements are 

properly represented by the model predictions but does not necessarily ensure that the 

model is getting “the right answer for the right reason”; 

> Diagnostic Evaluation: Tests the ability of the model to predict visibility and extinction, 

PM chemical composition including PM precursors (e.g., SOx, NOx, and NH3) and 

associated oxidants (e.g., ozone and nitric acid); PM size distribution; temporal variation; 

spatial variation; mass fluxes; and components of light extinction (i.e., scattering and 

absorption); 

> Mechanistic Evaluation: Tests the ability of the model to predict the response of PM 

and visibility to changes in variables such as emissions and meteorology; and 

> Probabilistic Evaluation: Takes into account the uncertainties associated with the 

model predictions and observations of PM and visibility. 

Within the constraints of the Phase II schedule and budget resources the VISTAS Phase II effort will 

attempt to include elements of each of these components. The operational evaluation will obviously 

receive the greatest attention since this is the primarily thrust of EPA’s 2001 PM guidance. 

However, we will consider, where feasible and appropriate, diagnostic and mechanistic tests (e.g., 

use of probing tools, indicator species and ratios, aloft model evaluations, urban vs. rural 

performance analyses), traditional sensitivity simulations to explore uncertainty, and the use of 

alternative science platforms (e.g., CAMx). The scope of these additional diagnostic and 

mechanistic tests will be shaped by the resources available in Optional Task 14 and the timing of 

when such analyses are commissioned relative to the Phase II schedule and resources. 

Before discussing the types of testing procedures available for the above evaluation 

components, we first identify the surface and aloft data sets that are available to support these 

comparisons. 

7.4 Development of Consistent Evaluation Data Sets 

 7.4.1 Surface Measurements 

 The ground-level model evaluation database will be developed using several routine and 

research-grade databases. The first is the routine gas-phase concentration measurements for ozone, 

NO, NO2 and CO archived in EPA’s Aerometric Information Retrieval System (AIRS/AQS) 

database. Other sources of information come from the various PM monitoring networks in the U.S., 

with particular emphasis in the central and eastern U.S. These include the: (a) Interagency 

Monitoring of Protected Visual Environments (IMPROVE), (b) Clean Air Status and Trends 

Network (CASTNET), (c) Southeastern Aerosol Research and Characterization (SEARCH), (d) EPA 

PM2.5 and PM10 Mass Networks (EPA-FRM), (e) EPA Speciation Trends Network (STN); (f) 

National Acid Deposition Network (NADP) and (g) EPA Supersites (EPA-SPEC) networks. 

Typically, these networks provide ozone, other gas phase precursors and product species, PM, and 

visibility measurements. Noteworthy is the continuous speciated PM2.5 measurements available 

TN Regional Haze SIP Appendix C.1 - 123
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-6 

from the SEARCH network for 2002 modeling period. For 2003 episode modeling, additional 

continuous PM speciated data will be available from the FOCUS network. 

 As an example, the IMPROVE network gives daily (24-hour) average mass concentrations 

every 3 days for SO4, NO3, organic carbon (OC), elemental carbon (EC), soil (IP), CM, PM2.5 and 

PM10. These data are available at approximately 38 sites in the VISTAS 12 km domain. In addition, 

hourly values of light extinction and deciview are available at several of these sites. The SEARCH 

network provides 24-hour as well as continuous (hourly) speciated measurements of PM2.5

components and other specifics from 8 stations, depending on the time period (Hansen et al., 2003). 

We will use data from these and the other observational databases listed in Table 7-1, supplemented 

with the routine AIRS/AQS data, as appropriate, for CMAQ model performance testing. 

 Care must be taken in selecting data that are representative of regional concentrations and the 

prediction of regional haze at the sensitive Class I areas within the VISTAS (see Table 7-3). For 

example, the criteria and other pollutants in the AIRS/AQS database are typically urban-oriented. 

Thus, would not expect CMAQ predictions for these species using a 36/12 km grid mesh to be 

ideally suited to simulating urban-scale concentrations. With other data sets (e.g., SEARCH) it may 

be possible to reproduce the urban-to-rural gradients in PM2.5 components. While finer grid meshes 

(e.g., 4 km) might be more appropriate, high-resolution modeling at 4 km scale on an annual time 

frame for the VISTAS region is prohibitive given current computer technology and model run times. 

 Accordingly, some selectivity is needed in assembling pertinent measurement sites for comparisons 

with regional-scale model predictions and the interpretation of the modeled and measured 

comparisons must be made with care. 

 

 Another important consideration is that different PM monitoring networks may use different 

measurement approaches that “measure” different amounts of the same species that are also different 

from the modeled species. For example, the IMPROVE network only speciates PM2.5 so any sulfate 

or nitrate in the coarse mode (PM2.5-10) is included in the CM species. As was done in Phase I, the 

model will be evaluated separately for each network. While Phase II is clearly not aimed at new 

model development or algorithm refinement, information discerned from model performance across 

the various networks should be useful in later model refinement activities and network design 

improvements. Finally, the mapping of the modeled species to the monitored data will also have to 

be performed in a consistent fashion. 

 The VISTAS Phase I air quality data assessment report (ENVIRON, 2003c) provides more 

details on the ambient monitoring data available for Phase II modeling. 

 7.4.2 Aloft Measurements 

In recent years, the use of instrument aircraft in support of regulatory monitoring and research 

programs has become much more commonplace. Indeed, in the upper Midwest, the Lake Michigan 

Air Directors Consortium (LADCo) has been centrally involved in aircraft programs to support 

model development and applications studies for seventeen (17) years, beginning with pioneering 

flights in 1987. Supplementing the long-term sampling performed by LADCo in the Midwest, there 

have been other occasional intensive airborne sampling campaigns throughout the eastern U.S. (e.g., 

the 1999 SOS field program which provided aloft data for our evaluation of CMAQ for the July ’99 

episode), that have produced very useful information for air quality model performance testing. 

TN Regional Haze SIP Appendix C.1 - 124
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-7 

Fortunately, during CY-2002, there were at least two mature airborne field programs underway in 

the eastern U.S. One was centered over the Midwest, the other on the mid-Atlantic coast. A brief 

characterization of these potentially valuable CMAQ model evaluation data sets is given here. Note 

that the advanced modeling evaluation using non-routine data sets like aircraft data, would be 

covered under Optional Task 14 (see Appendix A). 

During 2002, the Wisconsin Department of Natural Resources (WDNR) and the Midwest 

RPO (MRPO) (who funded the Jacko aircraft) collaborated on the support of airborne sampling 

using two aircraft that, along with ground-based measurements, provided a 3-dimensional 

representation of air pollution concentrations across the upper Midwest with some flight paths 

extending south to include the Mammoth Cave, KY and Dolly Sods, WV Class I areas in the 

VISTAS domain. The goal of the WDNR/MRPO flights was to collect aloft air quality and 

meteorological data to support model evaluation and data analyses. The aircraft flights were aimed 

at: (1) characterizing high fine particle and ozone episodes, (2) characterizing air quality over the 

Class I areas in the upper Midwest (Isle Royale National Park and Seney National Wildlife Refuge 

in northern Michigan) on both clean and hazy days, and (3) characterizing urban areas in the 

Midwest. 

As indicated in Table 7-2, airborne sampling was performed over a broad region of the 

Midwest (including portions of the VISTAS states) from 1 June to 22 November. Lasting 3-5 hours, 

the WDNR and Jacko aircraft sampled a variety of aerometric parameters (depending upon the flight 

and aircraft) including wind speed, wind direction temperature, dew point, relative humidity, 

pressure, O3, NO, NO2, NOx, NOy, speciated VOCs, carbonyls, HNO3, NH3, Hg, SO4, OC, EC, 

PM2.5, and light scattering (Neph). Still photographs documenting visibility were also collected. 

Presently, the full WDNR/MRPO aircraft database, from the first flights in 1987 to the recent 

sampling in 2003 is being aggregated into a master data base archive. 

 At the University of Maryland, researchers have been using ground-based monitors, 

radiosondes, profilers, and instrumented aircraft to make observations each year since 1992. 

Parameters measured included meteorology; selected trace gases; fine particulate chemistry, 

microphysics and optical properties across broad regions of the middle Atlantic coast. During 2002, 

the University Research Foundation’s Aztec-F aircraft instrument suite included O3, NO, CO, SO2 

samplers, as well as a NO2 closed-path tunable diode laser system, and a differential GPS-based 

meteorology (T, RH) and horizontal wind (u and v horizontal components) data system. Aztec-F 

flights were made from 23 May to 3 October, typically lasting 3 hours. 

The volume of aircraft information available for CMAQ performance testing during 2002 is 

quite significant and, as discussed later, this data set will be used as extensively as possible. 

Historically, aircraft data sets have been used only sporadically in evaluating model performance 

aloft. To our knowledge, the VISTA Phase II evaluation under optional Task 14 could represent the 

most thorough utilization of aircraft data for regional model ozone/PM model evaluation yet 

attempted. 

7.5 Model Evaluation Tools 

This section introduces the various statistical measures, graphical tools, and related analytical 

procedures that have proven useful over the years in evaluating grid-based chemical transport 

TN Regional Haze SIP Appendix C.1 - 125
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-8 

models. Many of the methodologies mentioned below were utilized to one degree or another in 

Phase I. Where appropriate, they will also be used in the 2002 annual and episodic evaluation of the 

CMAQ modeling system. However, as with Phase I, while we plan on calculating a rich variety of 

statistical performance metrics, only a very limited subset of these measures will actually be relied 

upon to form judgments concerning model acceptability and in the final reporting. 

 The current methodologies and statistical metrics may be augmented, as necessary and 

appropriate, by new measures that may become available during the course of the Phase II modeling. 

Definitions of the statistical measures for gas-phase species such as ozone are provided in the Phase 

I Task 4 report (ENVIRON, 2003d); measures geared more toward fine particulate aerosol species 

are identified later in this chapter. In sum, the Phase II evaluation will employ similar methods and 

evaluation tools used in Phase I but the evaluation will remain open to emergent methods for testing 

model performance where necessary and consistent with the project schedule and resources.

 7.5.1 Statistical Performance Metrics 

EPA’s 2001 PM and regional haze guidance suggests a suite of metrics for use in evaluating 

model performance. The standard set of statistical performance measures suggested by EPA for 

evaluating fine particulate models include: (a) normalized bias; (b) normalized gross (unsigned) 

error; (c) fractional bias; (d) fractional gross error; and (e) fractional bias in standard deviations. 

These measures are subsumed within the list of metrics that were calculated on a routine basis in 

Phase I using the UCR analysis package. (These are identified in Table 7-4). The UCR evaluation 

software will generate these statistical measures for each model simulation performed for each 

analysis region (see below). In parallel, the AG MAPS software will be used to generate 

complimentary statistical measures which combine the methods recommended in the EPA ozone 

guidance (EPA, 1991; 1999) and the newer 2001 PM guidance. From past regional PM model 

evaluations we have found the fractional bias and fractional error to be the most useful summary 

measures and we will focus mainly upon them in Phase II, but not to the exclusion of others that are 

found to yield discriminating power. For ozone and other gas phase species (NO, NO2, SO2) we will 

include use the traditional statistical measures (Tesche et al., 1990; EPA, 1991, 1999) develop with 

MAPS as an adjunct to the UCR analysis measures. 

Typically, the statistical metrics are calculated at each monitoring site across the full 

computational domain for all simulation days. In the Phase II CMAQ evaluation, we will stratify the 

performance statistics across relevant space and time scales. As part of the Task 4a/b operational 

evaluation, the gas-phase and aerosol statistical measures shown in Table 7-4 will be computed for 

the full 36 km and 12 km domains, as well as for the individual RPOs (including VISTAS of 

course) and on other subdomains as appropriate. Temporally, we will compute the statistical 

measures for the appropriate averaging times: 1 hr for ozone, and gas-phase precursors such as NO, 

NO2, CO, SO2; 8-hr for ozone, and 24 hr for sulfate, nitrate, PM and other aerosol species. These 

results will then be averaged over annual, monthly, and seasonal periods for display, further 

analysis, and reporting. Should it become necessary as part of model performance diagnosis, we 

will consider aggregating the statistics in other ways, e.g., (a) day vs. night, (b) weekday vs. 

weekend, (c) precipitation vs. non-precipitation days, (d) month of the year, and (e) the 20% 

haziest/cleanest days, in order to help elucidate model performance problems. Absent performance 

difficulties, these supplemental time/space analyses would only be considered if supported under 

directed Optional Task 14 funding. In subregional performance testing, the focus would likely be on 

TN Regional Haze SIP Appendix C.1 - 126
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-9 

the Class I areas and sites where enhanced monitoring (EPA STN locations, the hourly/daily 

SEARCH sites) within the VISTAS 12 km domain (Hansen et al., 2003) is available. 

As part of the operational evaluation, the metrics defined in Table 7-4 will be calculated for 

each gas phase species and each fine particulate species in the extinction equation as well as 

separately for SO4, NO3 and ammonium (NH4) on both the 36 km and 12 km domains. In any 

diagnostic evaluations that are performed, we will examine the model’s ability to estimate the 

gaseous species listed above from EPA’s guidance (EPA, 2001). However, in reality ambient 

gaseous species in 2002 are principally available for ozone, NO2, SO2, and CO. Since most of the 

gaseous air quality monitors are located in urban settings, these data sets will be of somewhat 

restricted value in evaluating the 36/12 km regional-scale CMAQ predictions. Under Optional Task 

14, we would consider the merits of assembling a rural-oriented gaseous species observational 

model performance evaluation database for use with CMAQ. This database could be composited 

with measurement from rural sites from the AQS, CASTNet, NPS, STN, and SEARCH networks. If 

commissioned under Optional Task 14 by VISTAS, this rural vs. urban scale operational evaluation 

would be conducted to augment the more traditional operational evaluation using all valid data sets. 

 

 7.5.2 Graphical Representations 

The Task 4a/b core operational air quality model evaluation will utilize numerous graphical 

displays to facilitate quantitative and qualitative comparisons between CMAQ predictions and 

measurements, many of which were used in the Phase I Final Report (ENVIRON, 2004). Together 

with the statistical metrics listed in Table 7-4, the graphical procedures are intended to help: (a) 

identify obviously flawed model simulations, (b) guide the implementation of Task 4a/4b 

performance improvements in the 2002 model input files in a logical, defensible manner, and (c) to 

help elucidate the similarities and differences between the alternative CMAQ simulations. These 

graphical tools are intended to depict the model’s ability to predict the observed fine particulate and 

gaseous species concentrations. 

The Phase I modeling helped to refine the suite of graphical tools most effectively in 

assessing model performance and the differences between the baseline CMAQ runs and sensitivity 

experiments. The core graphical displays to be considered for use in Phase II include the following: 

> Spatial mean concentration time series plots; 

> Time series plots at monitoring locations; 

> Ground-level gas-phase and particulate concentration maps (i.e., tile plots); 

> Concentration scatterplots stratified by station, by time, and by network; 

> Bias and error stratified by concentration; 

> Bias and error stratified by time; 

> Histogram plots of the statistical metrics, stratified by day, by pollutant, by subregion 

(e.g., 12 km vs. 36 km, by RPO), and by monitoring network; and 

> Quantile-Quantile (Q-Q) plots. 

These graphical displays will be generated, were appropriate for the full annual cycle as well as for 

monthly and seasonal periods. The displays will be generated with a consistent suite of products 

including the UCR analysis tools, Alpine MAPS software and ENVIRON evaluation software. 

TN Regional Haze SIP Appendix C.1 - 127
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-10 

 

TN Regional Haze SIP Appendix C.1 - 128
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-11 

7.5.3 MAPS/Flying Data Grabber Routines 

This section describes the procedures we would implement for the aloft gas phase and PM 

model evaluation with CMAQ using aircraft data should Optional Task 14 be commissioned by 

VISTAS. This aloft performance evaluation would employ aircraft data sets from various sampling 

programs carried out over the Midwest and eastern U.S. during 2002. Details on how these data are 

used to evaluate CMAQ performance aloft together with findings from our aloft model evaluations 

with the 10-21 July 1999 episode are described in the Phase I Final Report (ENVIRON, 2004). 

The principal challenge in using the aircraft data for meteorological and photochemical 

model evaluations relates to the ‘incommensurability’ of Lagrangian aircraft observations with 

Eulerian (i.e., fixed location) volume-averaged model estimates (see, for example, Hanna, 1994). 

Aircraft data are essentially continuous, high frequency Lagrangian samples having response times 

on the order of 30 seconds or less. In contrast, CMAQ model estimates represent hourly-averaged 

values. Thus, the aircraft data must be averaged in some manner to yield quantities that are at least 

qualitatively comparable to the air quality fields estimated by CMAQ in the grid volume(s) through 

which the aircraft passes. The objective is to develop hourly-average time series of measurements 

and model estimates that are as nearly comparable as possible. 

The procedures proposed for processing the aloft meteorological and air quality observations 

and CMAQ model predictions have been described in several science reports prepared in connection 

with the Lake Michigan Ozone Study (Tesche and McNally, 1993a-d, 2001) and more recently the 

Houston-Galveston 1-hr ozone SIP modeling that utilized the TexAQS 2000 data base (Tesche and 

Jeffries, 2002). These methods, formalized within AG’s Flying Data Grabber (FDG) model, were 

employed in Phase I for the 13-21 July 1999 episode. The methods used are substantial extensions 

of the techniques pioneered a decade ago by Schere and Wayland (1989) for the Regional Oxidant 

Model (ROM2.0) evaluation against the NEROS database and by Barchet and Dennis (1990) for the 

RADM/ADOM evaluation (Dennis et al., 1990). 

For a typical aircraft (or helicopter flight), the Flying Data Grabber first identifies the 

specific time interval during which the aircraft was located in a given CMAQ model grid cell along 

the flight path. The observations are then integrated to produce mean, standard deviation, bias and 

error estimates for the variable measured within each grid cell of the flight path. This averaging 

process produces an observed, averaged time series for the above statistical quantities along the 

flight path. Note, that these time intervals are characteristically much smaller than the one-hour 

model averaging time. Flight path statistics, together with the mean modeled and observed 

horizontal winds along the flight paths are also produced by the FDG. The maximum and minimum 

values during each time segment within a grid cell are also recorded. 

The FDG methodology further assumes that the air quality model estimates vary 

approximately linearly during each hour. It is then straightforward to construct an estimated time 

series of the modeled values that corresponds to the above-described observed time series derived 

from the aircraft data. The measurements and model estimates, now on roughly comparable time 

and space scales, are subsequently processed with the MAPS statistical/graphical software tools 

described in the Phase I Task 4a report (ENVIRON, 2003d). The statistics of principal interest are 

the mean values of the observed and modeled concentrations together with estimates of bias and 

TN Regional Haze SIP Appendix C.1 - 129
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-12 

imprecision (i.e., gross error). A variety of graphical representations are also produced to facilitate 

evaluation an intercomparison. 

7.5.4 Use of Multiple Evaluation Packages

In Phase I model evaluation (ENVIRON, 2004) relied principally on the UCR evaluation 

package that was originally developed for WRAP and then enhanced by VISTAS. This package 

produce scatter plots by site, day or all sites and days, time series plots and statistical measures. In 

Phase I modeling an error was introduced in the extension of the package to treat 12 km grids that 

caused some delays as we attempted to resolve the 36 km and 12 km performance issues. In Phase II 

we will attempt to prevent such a reoccurrence by using additional software evaluation packages 

specifically, the Alpine MAPS package will be used to not only corroborate the UCR results but also 

provide some additional model evaluation measures and displays (e.g. spatial average time series 

plots). In addition; ENVIRON’s evaluation software that focuses on the evaluation of the model at 

Class I areas for the 20% best and 20% worst days will be employed to identify how the model is 

predicting PM species on these key days and identify any days from the 20% worst and 20% best 

days3

 where model performance is so suspect that use of the modeling results on these days to 

project visibility improvements may not be desirable. 

 7.5.5 Probing Tools and Allied Methods 

 The CMAQ model evaluation in Tasks 4a and 4b will employ routine operational evaluation 

methods and standard statistical metrics (Table 7-4) and graphical displays to support the assessment 

of whether the model is shown to perform with sufficient accuracy and reliably for its intended 

purpose. Ideally, this operational evaluation will confirm that the modeling system is performing 

consistent with its scientific formulation, technical implementation, and at a level that is at least as 

reliable as other current state-of-science methods. Should unforeseen model performance problems 

arise in the initial or refined year 2002 model simulations, it may be necessary to draw into the 

evaluation supplemental diagnostic tools to aid in model testing. These diagnostic techniques are 

loosely referred to as “probing tools”. The actual need for their use, if any, can only be determined 

once the initial 2002 CMAQ operational evaluation is completed. Should such diagnostic methods 

actually be needed, their usage would be funded by Optional Task 14 and/or Optional Task 15 

(Contingency). The scope of these additional analyses into the Task 4a/b evaluation process will be 

closely coordinated with the VISTAS TAWG, with due consideration being given to the Phase II 

schedule and resources. Below, we identify the types of probing tools that could be brought to bear 

under Optional Tasks 14 and/or 15 should their use become necessary at some later time. 

Current ‘One-Atmosphere” models such as CMAQ and CAMx have been outfitted with a 

number of “probing tools” that have proven to be very useful in testing and improving model 

performance and in evaluating emissions control strategies. Among the probing tools available in 

one or both models are: (a) ozone source apportionment technology (OSAT) algorithms, (b) process 

analysis (PA), and (c) the direct decoupled method (DDM) for sensitivity analysis. In addition, the 

 

3

 Mention here and elsewhere in this protocol of evaluation efforts for the 20% best and worst days 

is in addition to the planned evaluation of all days in the annual cycle. While EPA visibility 

guidance addresses the 20% days, and our evaluation is tailored to follow the guidance, the overall 

CMAQ evaluation will address all days in the year. 

TN Regional Haze SIP Appendix C.1 - 130
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-13 

project team is implementing PM Sour apportionment techniques in CMAQ and CAMx. VISTAS 

may choose to evaluate these tools as part of the Phase II modeling exercise. It is currently unclear 

what funding will be available under Optional Tasks 14 and/or 15 for these options. 

Source Apportionment Technology: CAMx contains a suite of “source attribution” 

methods. One such method is Ozone Source Apportionment Technology (OSAT). OSAT tracks 

ozone formation based on how groups of ozone precursors contributed to ozone formation. Thus, 

OSAT decides whether ozone formation is NOx or VOC limited in each grid cell at each time step, 

and bases ozone attributions on the relative amounts of the limiting precursor from different sources 

that are present in that grid cell at that time step. These incremental ozone attributions are integrated 

throughout the model run. The method is generally applicable and has been widely used to aid 

model diagnosis in the performance testing phase and to guide control strategy formulations as well. 

 

A new PM Source Apportionment Technology (PSAT) is being implemented in CAMx funded by 

the MRPO that is in its final stages of testing. A Tagged Species Source Apportionment (TSSA) 

approach has also been implemented in CMAQ and tested for nitrate (Tonneson, 2004, personal 

communication). 

Decoupled Direct Method (DDM): Various forms of the Decoupled Direct Method (DDM) 

have been installed in CMAQ and CAMx, based on the original work of Dunker and co-workers 

(Dunker, 1981; 1984; Dunker et al., 2002) and researchers at Georgia Institute of Technology (GIT). 

 In general, the DDM method: (a) calculates first order sensitivities dC/dP where C is a 

concentration output and P an input parameter4

, (b) promotes accuracy by using consistent numerical 

methods and the same time steps for concentrations and sensitivities, (c) optimizes the code for 

efficiency, but not at expense of accuracy, and (d) calculates sensitivities with respect to parameters 

representing pollutant sources – emissions, BCs and ICs. Finally, the DDM provides a flexible and 

powerful user interface for defining various sensitivities including: 

> Emissions resolved by geographic area. 

> Emissions resolved by source category. 

> BCs optionally resolved by boundary edge (N, S, E, W, Top). 

> All sensitivities available relative to sources of individual species (NO, PAR, etc.) or 

species group (VOC, NOx or ALL). 

> Simultaneously calculate sensitivities to many initial condition, boundary condition 

and emissions parameters. 

In recent comparisons between CAMx DDM sensitivities and brute-force sensitivities 

(calculated from +/- 20% perturbations) Dunker et al., (2002a,b) reported that sensitivities of ozone 

with respect to area source NOx and VOC emissions were calculated and results indicated that the 

agreement between DDM and brute force sensitivities is excellent. DDM implementation into 

CMAQ is reported by Kumar (2003). 

Process Analysis (PA): Photochemical air quality model simulations are usually evaluated 

primarily in terms of their ability to simulate observed O3 data. There is an increasing awareness that 

 

4 Recent research by Prof. Russell and coworkers at GIT has led to the extension of the CMAQ 

DDM method to include second order sensitivity coefficients (see, Hakami et al., 2003). 

TN Regional Haze SIP Appendix C.1 - 131
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-14 

chemical mechanisms, and air quality models must also be evaluated in terms of their ability to 

simulate the fundamental chemical processes that control O3 formation and the sensitivity of O3 to 

emissions reductions (Arnold et al., 1998). Process analysis is a method for explaining model 

simulations by adding algorithms to the AQM to store the integrated rates of species changes due to 

individual chemical reactions and other sink and source processes (Jeffries and Tonnesen, 1994; 

Tonnesen, 1995). By integrating these rates over time and outputting them at hourly intervals, 

process analysis provides diagnostic outputs that can be used to explain a model simulation in terms 

of the budgets of free radicals, production and loss of odd oxygen and O3, and conversion of NOx to 

inert forms, as well as the effects of transport and other sink and source terms. Of particular 

importance to the VISTAS modeling, process analysis can also improve model diagnosis and 

performance evaluation efforts by identifying processes that are ‘out of balance’ (Tesche and 

Jeffries, 2002), by identifying situations for which the model formulation and/or implementation 

should not be expected to apply and by suggesting how ambient data can be used to evaluate model 

accuracy for key terms in the chemical processing of VOC and NOx (e.g., Imre et al., 1998). 

Process Analysis (PA) is implemented in both CMAQ and CAMx and each model supports 

three complementary aspects of the method: (a) the integrated process rate (IPR), (b) integrated 

reaction rate (IRR) and (c) chemical process analysis (CPA). Several versions of process analysis 

(PA) have been implemented in air quality models (AQMs) including both trajectory models 

(Tonnesen, 1990, 1995) and grid models (Jang et al., 1995, Tonnesen and Dennis, 2000; Arnold et 

al., 1998; and Wang, 1997). 

The fundamental approach in all versions of PA is similar: The AQM is modified to 

calculate the integral over time of the individual sink and source processes and each chemical 

reaction. These integrated sink/source process rates (IPR) and integrated reaction rates (IRR) can 

then be stored to a file and analyzed using a post-processor, or some processing can be performed 

internally in the model and a more limited set of process diagnostic information is output directly by 

the AQM. Chemical process analysis (CPA) is an improvement on the IRR method whereby some 

of the processing of IRR information is internalized within the AQM to output chemically 

meaningful parameters directly (e.g., budget terms for O3, NOx and odd oxygen). 

 Process analysis measures for aerosol chemistry have not been analyzed as much as for 

ozone chemistry. Although the ozone chemistry process analysis is directly related to secondary 

sulfate and nitrate formation, there is additional process analysis information available in the aerosol 

modules that are not extracted in either CMAQ or CAMx. In particular, information on sulfate 

formation and oxidants from the aqueous-phase module and on the sulfate/nitrate equilibrium from 

the aerosol thermodynamics module would be a useful addition to the current process analysis 

output. 

Because application of all three of these probing tools--source apportionment, DDM, and 

Process Analysis—are computational intensive and require a fair amount of analysis time to reap the 

benefits of using the methods, they do not lend themselves directly to annual simulations. However, 

each method has potential for use in addressing key episodic periods or geographical locations in the 

VISTAS domain where performance in the 2002 simulation may present a problem or where 

particular attention needs to be focused on emissions controls (a specific Class 1 area for example). 

In such focused applications, one or more of these probing tools may indeed serve a purpose and 

will be considered where appropriate.

TN Regional Haze SIP Appendix C.1 - 132
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-15 

TN Regional Haze SIP Appendix C.1 - 133
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-16 

7.6 Phase II Model Evaluation Procedures 

EPA guidance (EPA, 2001, pg. 227) suggests that the performance evaluation focus on two 

aspects: 

 > How well is the model able to replicate observed concentrations of components of 

PM2.5, and total observed mass of PM2.5? and 

 > How accurately does the model characterize the sensitivity of changes in component 

concentrations to changes in emissions? 

Recognizing that the former is much easier to accomplish than the latter, EPA goes on to declare that 

testing of a model’s reliability in estimating the actual effects of emissions changes is the more 

important. Over the past 20 years, a substantial body of information and analytical techniques has 

been developed to address the first aspect. Unfortunately, even today there are little rigorous 

methods available for quantifying the accuracy and precision of a model’s predictions of ozone, PM 

or visibility changes as the result of emissions changes. In this section we explain how the VISTAS 

Phase II testing will address the first aspect of the performance evaluation, i.e., how does the model 

compare against observed data. In section 7.9 we consider the second performance consideration. 

 7.6.1 Assessment of Ground-Level Gas-Phase and Aerosol Species 

Given that visibility is expressed in terms of extinction built off of individual components of 

fine particulate matter, the model should be evaluated separately for each of the key fine particulate 

matter components that make up the extinction coefficient. Current EPA guidance suggests that the 

model should also be evaluated for ammonium as well as several key gas-phase species that are 

important for fine particulate modeling. For particulate species this includes SO4 and/or S, NH4, 

NO3, mass associated with SO4, mass associated with NO3, elemental carbon (EC), organic carbon 

(OC), IP, mass of individual constituents of IP, and coarse matter (CM). The gaseous species

include ozone (O3), HNO3, NO2, PAN, NH3, NOy, SO2, CO, and H2O2. 

As indicated above, for some of the VISTAS subregions there are very few measurements, if 

any, for most of the gaseous species. But ozone measurements are available and should be 

integrated into the model performance evaluation. Given the importance of the photochemistry and 

the radical cycle in forming secondary fine particulate matter, the ozone evaluation will provide 

some insight into the model’s ability in this area. However, given the coarse grid resolution to be 

used (36 km and 12 km) perhaps greater weight should be given to the model evaluation results at 

the more rural ozone monitors (e.g., CASTNet) compared to urban ozone monitors. In addition to 

the list of PM and gaseous species from EPA’s guidance document listed above, the Task 4a/4b 

evaluation will also examine model’s ability to estimate PM2.5 mass, PM10 mass, extinction (bext) and 

deciview (dV) using the equation above to obtain extinction from fine particulate concentrations. 

At some of the IMPROVE sites there are also direct measurements of hourly extinction using 

transmissometer or nephelometer instruments.5

 Thus, it would be scientifically interesting to 

 

5 The Phase I modeling did not utilize the hourly extinction data in the operational evaluation of the 

CMAQ and CAMx models due to schedule and resource constraints and because the models 

TN Regional Haze SIP Appendix C.1 - 134
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-17 

evaluate the model estimated extinction with the hourly measured values at these sites. If such a 

comparison were to be made, day-specific hourly and site-specific RH and f(RH) values could be 

used to convert model estimated fine particulate matter to extinction rather than the monthly f(RH) 

EPA recommends for projecting visibility improvements (EPA, 2003a). Furthermore, there have 

been some discrepancies reported in the past between the extinction calculated using the 

reconstructed mass data and the hourly measurements. Thus, it would be interesting to calculate 

day-specific extinction using day-specific f(RH) values and the reconstructed mass data for 

comparison with the direct measurements and the model estimates. Whether and to what extend this 

more detailed investigation into light extinction is pursued would depend upon the availability of 

Optional Task 14 authorization. 

As part of the CMAQ operational evaluation, model outputs will be compared statistically 

and graphically to observational data obtained from the IMPROVE, SEARCH, CASTNet, EPAFRM, EPA-STN, and other monitoring networks. These monitoring data will be obtained from 

AIRS, VIEWS, and other appropriate organizations. These comparisons will likely include: 

> Daily monthly, seasonal and annual averages for SO2, SO4, NO3, EC, OC, PM2.5, and 

PM10, taking care to exclude periods of sampling interference in the observational data. 

We will look for systematic biases between the model results and IMPROVE 

observations, and if biases are found, identify possible sources of error in the model 

inputs. 

> Hourly, high resolution PM species and gaseous species concentrations at sites where 

available (e.g., SEARCH, AIRS and EPA-Supersites). 

> At sites with contrasting aerosol mass loadings, analysis of the temporal behavior of the 

major scattering and absorbing aerosol constituents along with the visibility trends, to 

establish correlations. 

 The optional Phase II CMAQ diagnostic model evaluations may entail several components, 

many of which can be identified presently. Of course, the actual diagnostic analyses to be 

performed and the scope of such analyses can only be determined once the initial Task 4a model 

testing is underway. These diagnostics analyses – to be funded through Optional Task 14 (Enhanced 

Model Performance Evaluation)-- will need to be carefully defined and rank-ordered in terms of 

their priority to ensure that they can be accommodated within available resources and schedule. 

Among the diagnostic model evaluation analyses that could be considered are: 

> Evaluate seasonal trends in observations of organic and inorganic aerosol precursors and 

their effects on PM composition and visibility, and evaluate the ability of the model to 

capture these seasonal trends. 

> Evaluate how well the model simulates various physicochemical processes by: 

(a) examining observed and modeled correlations between various species pairs, and 

 

performance for the major PM components was deemed more important in terms of evaluating 

alternative model science configurations. 

TN Regional Haze SIP Appendix C.1 - 135
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-18 

(b) comparing model-predicted ratios of various species (individual or families) with 

observations to evaluate gas/particle partitioning (e.g., nitrate/total nitrate, SO4/SOx). 

> Investigate the performance of the model at selected observational sites characterized by 

different chemical regimes that may be encountered either spatially or during different 

seasons to help identify any inadequacies in the model and to provide a better 

understanding of conditions under which model inferences may be weak. 

> Compare hourly 24-hour average, and episode averages of the PM constituents for the 

2002 Phase I VISTAS annual modeling period as well as the 2003 episodes across all 

sites. 

> Create scatter plots of modeled vs. observed data and hourly and 24-hour averages by 

site and subregion to help identify any site-specific biases. 

> Create time series plots of predicted and observed concentrations as appropriate. 

> Evaluate for total sulfur (SO2 + SO4), nitrate (HNO3 + NO3) and ammonia (NH3 + NH4). 

> Compare observed versus modeled mass fractions of PM constituents at various sites that 

are characterized by their proximity or remoteness relative to sources, or by specific 

meteorological conditions (e.g., frontal passage, stagnation, precipitation); these will 

enable identification of trends in the model of over- or under-prediction of specific PM 

constituents under these conditions. 

 > Calculate the measured and predicted relative abundance of key PM components and 

compare with EPA guideline recommendations and emergent alternative science 

recommendations (e.g., removing the soil component from the calculations, use of 

alternative relative importance equations [i.e., Boylan, 2004]). 

> Pay particular close attention to the model performance at the Class I areas for SO4, NO3, 

EC, OC, IP and CM on the 20% best and 20% worst days to document whether certain of 

these days should be eliminated from the visibility projections due to inadequate model 

performance. 

The suite of statistical metrics and graphical tools identified in the previous section for the 

core operational evaluation efforts under Task 4a/4b would likely also be used to diagnose 

performance problems with the CMAQ simulations should they exist and to highlight differences 

between model runs. Experience in ozone/PM modeling is the best basis upon which to identify 

obviously flawed simulation results. Efforts to improve the CMAQ model’s base case performance 

will be made, where necessary, warranted (i.e., to reduce the discrepancies between model estimates 

and observations), and consistent with the project resources and schedule; however, these model 

performance improvements efforts must be based on sound scientific principles. “Curve-fitting” 

exercises will be avoided. 

TN Regional Haze SIP Appendix C.1 - 136
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-19 

 7.6.2 Assessment of Aloft Gas-Phase and Aerosol Species 

 A substantial number of aircraft flights were conducted during 2002 over the Midwest and 

Eastern U.S. Should VISTAS elect to fund the optional aloft model performance evaluation, we will 

endeavor to obtain this information and use it in a scientific performance evaluations of aloft gasphase and aerosol species (see section 7.3.2). 

7.7 Performance Goals and Benchmarks 

 Establishment of performance goals and benchmarks for regulatory modeling is a necessary 

but difficult activity. Here, performance goals refer to targets that we believe a good performing 

model should achieve, where as performance benchmarks are based on historical model performance 

measures for the best performing simulations. Performance goals are necessary in order to provide 

consistency in model applications and expectations across the country and to provide standardization 

in how much weight may be accorded modeling study results in the decision-making process. It is a 

problematic activity, though, because many areas present unique challenges (e.g., Houston, San 

Joaquin Valley, Los Angeles) and no one set of performance goals is likely to fit all needs. Equally 

concerning is the very real danger that modeling studies will be truncated when the ‘statistics look 

right’ before full assessment of the model’s reliability is made. This has the potential from breeding 

built-in compensating errors (Reynolds et al., 1996) as modelers strive to get good statistics as 

opposed to searching for the explanations for poor performance and then rectifying them. A 

NARSTO review of more than two-dozen urban-scale ozone SIP applications found this tendency to 

be all too prevalent in the regulatory modeling of the 1990s. (Roth et al, 1997). 

Nearly 15 years ago, research sponsored by the California Air Resources Board (Tesche et 

al., 1990) led to the agency’s adoption of three performance goals for 1-hour ozone modeling in the 

state: 

> Unpaired (in time and space) peak prediction accuracy (≤ ± 20%); 

> Mean normalized bias in hourly averaged concentrations (≤ ± 15%); and 

> Mean normalized gross error in hourly concentrations (≤ 35%). 

These performance goals for 1-hr ozone concentrations were adapted from previous surveys of 

several dozen urban-scale photochemical grid modeling studies (principally in California) focusing 

on ozone episodes of 1 to at most 3 days in duration. A surprising number of these studies did not 

include biogenic VOC emissions in the inventory under the then prevailing belief that biogenics 

were a negligibly small source category compared to automobile emissions. Most of the studies 

(Tesche, 1985, 1988; Tesche et al., 1985; 1990) comprising the data base from which the California 

ozone performance goals were derived entailed hourly ozone concentrations well above background 

levels (~40-50 ppb). As a result, it was common practice to use a “cutoff values” ranging between 

40 ppb to 60ppb to eliminate prediction-observations pairs that would cause these bias and error 

residual statistics to become extraordinarily large when measured concentrations were low.) 

Accordingly, normalized statistics such as bias and error proved to be suitable in most applications 

since the observed concentrations were generally high. These three California ozone model 

performance goals were adopted by EPA (1991) as part of the nationwide photochemical modeling 

guidelines and have been heavily used since. 

TN Regional Haze SIP Appendix C.1 - 137
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-20 

However, when these evaluation metrics and goals were later adapted to PM and PM species, 

difficulties arose because performance statistics that divide by low concentration observations 

become much less useful. Indeed, some PM species may approach zero (e.g., NO3). In time, this 

has led to the introduction of the fractional and normalized mean bias and error metrics in addition to 

the mean normalized metrics and related performance expectations based on these alternative 

measures. 

While the 1-hr metrics and goals still have value in interpreting ozone and some gas-phase 

species performance, it has been necessary to develop new performance metrics and goals for fine 

particulates. EPA’s PM guidance document (EPA, 2001) guidance document identifies particulate 

matter components of interest to include: SO4 and/or S, NH4, NO3, mass associated with SO4, mass 

associated with NO3, EC, OC, IP, and mass of individual constituents of inorganic primary 

particulate matter (i.e., IP). Gaseous pollutants of interest include ozone, HNO3, NO2, PAN, NH3, 

NOy, SO2, CO, and H2O2. In addition, EPA guidance identifies several potentially useful statistical 

measures including: (a) accuracy of spatially averaged concentrations near a monitor, (b) fractional 

bias in means and standard deviations of predictions and observations, (c) normalized bias, (d) 

normalized gross error, (e) unpaired comparisons between predicted and observed peak 

concentrations. (Interested readers are referred to the EPA guidance document on the details of 

these metrics including mathematical formulae and implementation methods.) 

As with ozone in the 1980s, actual experience with PM models has led to the development of 

the current performance expectations for these models. For example, PM10 SIP model performance 

goals for mean normalized gross error of < 30% for southern California (SCAQMD, 1997; 2003) 

and < 50% for Phoenix (ENVIRON, 1998) have been used. As correctly pointed out by Seigneur 

and co-workers (2003), the current ability of regional PM models to predicting regional PM and 

visibility is an area of research with improvements needed for characterizing meteorology and 

emissions as well as PM models themselves. To this list we would add the need for improvements 

in model evaluation methodologies as well. 

When EPA’s draft guidance was developed nearly four (4) years ago, an interim set of fine 

particulate modeling performance goals were suggested for aggregated mean normalized gross error 

and mean normalized bias as follows: 

Pollutant Gross Error Normalized Bias 

PM2.5 ~30-50% ~"10% 

Sulfate ~30-50% ~"20-30% 

Nitrate ~20-70% ~"15-50% 

EC ~15-60% NA 

OC ~40-50% ~"38% 

Because regional-scale fine particulate and regional haze modeling is an evolving science, and 

considerable practical application and performance testing has transpired in the intervening years 

since these goals were postulated, we consider them general guidelines. As part of the Phase II 

model evaluation effort, we will compare the EPA goals in the 2001 guidance document with the 

results of the CMAQ/CAMx Phase I modeling, together with other recent PM modeling studies 

(e.g., WRAP, MRPO) to advance a proposed set of performance goals most relevant to the VISTAS 

TN Regional Haze SIP Appendix C.1 - 138
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-21 

study. Results of the Phase I model evaluation together with recommendations from science 

workshops (e.g., EPA’s PM Model Performance Evaluation Workshop in February 2004) and 

recently published scientific studies (e.g., Boylan, 2004) will be used to provide support to these 

recommendations. 

We regard the above goals as simply general interim guidance. Certainly, more information 

on likely performance expectations will be available once the Phase I CMAQ/CAMx ozone and PM 

performance evaluation and sensitivity results are fully analyzed. In the VISTAS Phase II 

modeling, we will generate the model performance statistics listed in Table 7-4 and make 

comparisons with EPA’s interim goals as one means for: 

> Establishing a benchmark on the annual 2002 VISTAS modeling episode, and 

> Guiding the interpretation of the CMAQ modeling results. 

Equally as useful, we will endeavour to compare the results of the Phase II multi-species evaluations 

of CMAQ with the Phase I results for the Jan ’02, Jul ’99 and Jul ’01 episodes as well as modeling 

results from other studies (e.g., BRAVO, EPA, CRC, etc.), other RPOs, and the modeling research 

community in general. This will allow judgments about the adequacy of the Phase II modeling 

results to be placed in the most current perspective possible. 

7-8. Diagnostic and Sensitivity Testing 

Rarely does a modeling team find that the first simulation satisfactorily meets all (or even 

most) model performance expectations. Indeed, our experience has been that initial simulations that 

‘look very good’, usually do so as the result of compensating errors. The norm is to engage in a 

logical, documented process of model performance improvement wherein a variety of diagnostic 

probing tools and sensitivity testing methods are used to identify, analyze, and then attempt to 

remove the causes of inadequate model performance. This is invariably the most technically 

challenging and time consuming phase of a modeling study. We anticipate that the annual CMAQ 

model base case simulations will present some performance challenges that may necessitate focused 

diagnostic and sensitivity testing in order for them to be resolved. Hopefully, these diagnostic and/or 

sensitivity tests can be adequately carried out within the resources and schedule of Tasks 4a/4b. If 

not, then it may be necessary to draw upon the Optional Task 14 (Enhanced Model Performance 

Evaluation) and/or Optional Task 15 (Contingency) resources to conduct the necessary work. 

Where practical, diagnostic or sensitivity analyses, if needed, could be performed on selected 

episodes within the annual cycle, thereby avoiding the time-consuming task of running CMAQ for 

the fully 2002 period. Below we identify the types of diagnostic and sensitivity testing methods that 

might be employed in diagnosing inadequate model performance and devising appropriate methods 

for improving the model response. 

 7.8.1 Traditional Sensitivity Testing 

Model sensitivity experiments are useful in three distinct phases or ‘levels’ of an air quality 

modeling study and all will be used as appropriate in the VISTAS Phase II modeling with CMAQ. 

These levels are: 

TN Regional Haze SIP Appendix C.1 - 139
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-22 

> Level I. Model algorithm evaluation and configuration testing; 

> Level II. Model performance testing, uncertainty analysis and compensatory error 

diagnosis, and 

> Level III. Investigation of model output response (e.g., ozone, aerosol, deposition) to 

changes in precursors as part of emissions control scenario analyses. 

Most of the Level I sensitivity tests with CMAQ have already been completed in the Phase I 

configuration and diagnostic analyses. However, given that open community nature of CMAQ and 

the frequent science updates to the model and supporting data abases, it is possible that some 

additional configuration sensitivity testing will be necessary in the early months of Phase II 

Potential Level I sensitivity runs would be carried out at one or more of the Team’s three modeling 

centers. 

Potential Level II sensitivity analyses might be helpful in accomplishing the following tasks: 

> To reveal internal inconsistencies in the model; 

> To provide a basis for compensatory error analysis; 

> To reveal the parameters (or inputs) that dominate (or do not dominate) the model’s 

operation; 

 > To reveal propagation of errors through the model; and 

 > To provide guidance for model refinement and data collection programs. 

At this time, it is not possible to identify one or more Level II sensitivity runs that might be needed 

to establish a reliable annual 2002 CMAQ base case. The merits of performing Level II sensitivity 

testing will depend upon whether performance problems are encountered in Tasks 4a/4b. Also, the 

number of tests possible, should performance difficulties arise, will be limited by the available 

schedule and Phase II resources under Optional Task 14 (Extended Model Performance Evaluation). 

 Thus, at this juncture, one cannot be overly prescriptive on the number and emphasis of sensitivity 

runs that may ultimately be desirable in Phase II. However, from past experience with CMAQ and 

other models, experience it is possible to identify examples of sensitivity runs could be useful in 

model performance improvement exercises with the annual 2002 CMAQ simulation. These include: 

> Modified biogenic emissions estimates; 

 > Modified on-road motor vehicle emissions; 

 > Modified air quality model vertical grid structure; 

 > Modified boundary conditions; 

 > Modified fire emissions; 

 > Modified EGU emissions; 

 > Modified ammonia emission estimates. 

 > Modified aerosol/N2O5/HNO3 chemistry; and 

 > Modified NH3 and HNO3 deposition velocities. 

Note that in a few cases (e.g., vertical grid structure, NH4 emissions estimates), some sensitivity 

experimentation has already been carried out in Phase I with the Jan ’02, Jul ’01 and Jul ’99 

episodes. To the extent that this Phase I information can help guide the Phase II diagnostics 

analyses, we will capitalize on this earlier work. 

TN Regional Haze SIP Appendix C.1 - 140
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-23 

If necessary, Process Analysis extraction outputs can be included in these Level II diagnostic 

sensitivity simulations in order to provide insight into why the model responds in a particular way to 

each input modification. Again, the number, complexity, and importance of these types of 

traditional sensitivity simulations can only be determined once the initial CMAQ annual 2002 

simulation(s) are executed. 

 Level III sensitivity analyses have two main purposes. First, they facilitate the emissions 

control scenario identification and evaluation processes. Today, four complimentary sensitivity 

“tools” can be used in regional photochemical models depending upon the platform being used. 

These methods include: (a) traditional or ‘brute force’ testing, (b) the direct decoupled method 

(DDM), (c) Ozone Source Apportionment Technology (OSAT) and PM Source Apportionment 

Technology (PSAT), and (d) Process Analysis (PA). Each method has its strong points and they will 

be employed in Phase II where needed. The second purpose of Level III sensitivity analyses is to 

help quantify the estimated reliability of the air quality model in simulating the atmosphere’s 

response to significant emissions changes. This important model evaluation need is addressed in 

further detail in section 7.9 below. 

 Based on experience in other regional studies, examples of Level III monthly or annual 

sensitivity runs for Phase II might include: 

> Ozone, sulfate, nitrate, ammonium and other aerosol sensitivities to SO2 emissions; 

> Ozone, sulfate, nitrate, ammonium and other aerosol sensitivities to elevated point 

source NOx emissions; 

> Ozone, sulfate, nitrate, ammonium and other aerosol sensitivities to ground level 

NOx emissions; and 

> Sulfate, nitrate, ammonium and other aerosol sensitivities to ammonia. 

 

 Of course, traditional ‘brute force’ sensitivity experiments is just one way of quantifying 

these or other Level III sensitivities. Other methods that can be applies include DDM, OSAT, or 

PSAT simulations. 

 The need to perform sensitivity experimentation (Levels I, II, or III) will depend on the 

outcome of the Task 4a/4b operational performance evaluations. If such a need arises, the ability to 

actually carry out selected sensitivity and/or diagnostic experiments will hinge on the availability of 

Optional Task 14 and/or 15 resources and sufficient time to carry out the analyses. Clearly, 

selection of the specific analysis method will depend upon the nature of the technical question(s) 

being addressed at the time. Note that as part of VISTAS Phase II modeling, Georgia Institute of 

Technology will be performing emissions sensitivities with CMAQ using the Phase I databases. 

 7.8.2 Diagnostic Tests 

A rich variety of diagnostic probing tools are available for investigating model performance 

issues and devising appropriate means for improving the model and/or its inputs. Previously, in 

section 7.4.4 we introduced the suite of ‘probing tools’ available for use in the CMAQ and CAMx 

modeling system for use in Phase II. Where the need exists (i.e., if performance problems are 

encountered) and assuming VISTAS elects to fund the use of the probing tool applications (under 

Optional Task 14 or elsewhere), these techniques could be employed as appropriate to assist in the 

TN Regional Haze SIP Appendix C.1 - 141
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-24 

model performance improvement efforts associated with the annual 2002 CMAQ basecase 

development. Here we describe an additional diagnostic method – indicator species and species 

ratios -- that is potentially useful not only in model performance improvement activities but also in 

judging the models reliability in estimating the impacts on air quality from future emissions. This 

method involves the use of so-called ‘indicator species’ and species ratios. If, during the conduct of 

Phase II, we determine that application of indicator species and species ratio techniques would be 

beneficial to the study (and if existing project resources allow), we will discuss with the TAWG the 

merits of including this additional probing tool as part of Optional Task 14. 

Beginning in the mid 1990s, considerable interest arose in the calculation of indicator species 

and species ratios as a means of diagnosing photochemical model performance and in assessing 

model credibility in estimating the effects of emissions changes. Major contributions to the 

development and refinement of this general diagnostic method over the past decade have been made 

many scientists including Milford et al., (1994), Sillman (1995, 1999), Sillman et al., (1997), 

Blanchard (2000), Blanchard and Fairley (2001), and Arnold et al., (2003). Indeed, a recent 

evaluation of CMAQ using indicator species ratios such as O3/NOx, NOz/NOy (a measure of 

chemical aging), and O3/NOz (a measure of the ozone production efficiency per NOx converted), 

showed not only good agreement with measurements (Arnold et al., 2003) but also convincingly 

demonstrated the utility of the method for diagnosing model performance in a variety of ways. 

 Traditionally, indicator species analyses have focused on ozone and its precursor and product 

species. However the method is equally applicable to PM species and species ratios given sufficient 

measurement data for comparisons. With some of the high-resolutions monitoring data available 

from the SEARCH program and the EPA Supersites, it is indeed feasible to compute relevant 

indicator species and ratios for PM and its component species. For example, Ansara and Pandis 

(1998) demonstrated how indicator species ratios could be applied to show how the modeled mass of 

PM might respond to sulfate, nitrate and ammonia emissions-related reductions.

7.9 Corroborative and Weight of Evidence Modeling Analyses 

This section identifies additional modeling analyses that might be worth pursuing under 

Optional Task 14 (Extended Model Performance Evaluation) to add strength to the core model 

evaluation efforts already planned as part of Tasks 4a and 4b. 

 7.9.1 Corroborative Models 

Noteworthy in EPA’s new ozone, PM, and regional haze guidance documents is the 

encouragement of the use of alternative modeling methods to corroborate the performance findings 

and control strategy response of the primary air quality simulation model. This endorsement of the 

use of corroborative methodologies, stems from the common understanding that no single 

photochemical modeling system can be expected to provide exact predictions of the observed ozone 

and PM species concentrations in a region the size of VISTAS, especially over time scales spanning 

1-hr to 1 year. Although the photochemical/PM models identified in EPA’s PM/regional haze 

guidance document possess many up-to-date science and computational features, there still can be 

important differences in modeled gas-phase and aerosol predictions when alternative models are 

exercised with identical inputs. 

TN Regional Haze SIP Appendix C.1 - 142
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-25 

As we discovered in the Phase I CMAQ/CAMx inter-comparisons, the general levels of 

difference revealed between the two model’s ozone and PM predictions is typical of what one 

encounters when inter-comparing alternative state-of-science regional models. These differences 

provide some insight into the current limits of predictability and reproducibility of today’s best 

photochemical/PM models. In light of these understandable differences in modeling results between 

state-of-science “One Atmosphere” models, not only is the issue of model selection for VISTAS 

Phase II critical6, but the procedures for the selected model (CMAQ in this instance) and interpreting 

its output are important as well. Thus, recognizing the uncertainty that attends even the most 

sophisticated models, the EPA’s draft PM/regional haze modeling guidance explicitly addresses the 

issue of modeling uncertainty by recommending that alternative models (photochemical and 

observation-based) be considered in the attainment demonstration ‘weight of evidence’ analyses. 

Indeed, the Relative Reduction Factor (RRF) approach in the formal attainment demonstration 

process is designed to address the fact that no model is fully capable of giving precise predictions. 

In fact the RRF approach was developed in response to the findings of many modelers (as we have 

also demonstrated in Phase I) that alternative, comparable models give somewhat divergent ozone 

and secondary aerosol predictions. 

Mindful of EPA’s endorsement of corroborative modeling methods and the rigorous use of 

‘weight of evidence’ investigations, we recommend that the most recent version of CAMx may be 

used in supportive, corroborative role to CMAQ in Phase II . Among other things, this will permit us 

to more explicitly identify the expected range of model uncertainty and to corroborate the general 

effectiveness of the Phase II CMAQ regional haze control strategies. Other corroborative modeling 

methods such as the CMAQ-AIM and CMAQ-MADRID should also be considered. However, as 

these models are derivatives of CMAQ they would not provide as robust independent corroboration 

as CAMx. 

Thus, we recommend CMAQ as the primary photochemical model for use in VISTAS Phase 

II and CAMx as a supportive model to be used for corroboration and are in the identification of 

compensation errors in the MM5/SMOKE/CMAQ modeling system. The use of CAMx is under 

Optional Task 11 in the Phase II SOW that is currently not funded (see Appendix A). Additional 

corroborative models should also be considered. 

 7.9.2 Weight of Evidence Analyses 

 EPA’s guidance recommends three general types of ‘weight of evidence’ analyses in support 

of the attainment demonstration: (a) use of air quality model output, (b) examination of air quality 

and emissions trends, and (c) the use of corroborative modeling such as observation-based (OBM) or 

observation-driven (OBD) models. We will consider the use of one or more methods in conducting 

the Phase II CMAQ modeling because it could significantly strengthen the credibility and reliability 

of the modeling available to the states for their subsequent use. The exact details of the ‘weight of 

evidence’ analyses must wait until Phase II evolves further. It is premature to prescribe which, if 

any of the WOE analyses would be performed since the model’s level of performance with the 2002 

episode is obviously not known at this time and the time and remaining project resources available 

to support WOE analyses is unknown as well. Nonetheless, we outline below our thoughts 

 

6 Model selection is a key issue that all RPOs have addressed and some such as WRAP and VISTAS have chosen 

CMAQ. Others, such as the MRPO have adopted CAMx instead. Still other RPO’s have yet to decide between these 

two or another model. Clearly, there does not appear to be any one ‘right’ selection at the present time. 

TN Regional Haze SIP Appendix C.1 - 143
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-26 

regarding what would likely be considered should the operational CMAQ model evaluation need to 

be bolstered with WOE analyses. 

Use of Air Quality Models. As just discussed, we recommend augmenting the CMAQ 

annual 2002 and episodic simulations with the use of CAMx to provide additional information on 

model uncertainty and sensitivity. More specifically, we propose to exercise CAMx on a somewhat 

limited basis to corroborate the key model performance evaluation results and emissions control 

findings of the primary model, CMAQ. Second, we propose to apply the DDM and OSAT/PSAT 

methods to develop corroborative information on source-receptor relationships and model 

sensitivities. These supplemental calculations would be performed with one or both models 

(CMAQ does not contain OSAT/PSAT presently) for one or more key episodes within the annual 

2002 cycle. The results of this additional modeling would be used directly in the ‘weight of 

evidence’ analyses to quantify the degree of modeling uncertainty and to corroborate 

appropriateness of the subsequent PM emissions reductions scenarios. 

Use of Emissions and Air Quality Trends. A limited scope emissions and trend analysis 

could be employed in Phase II to support the ‘weight of evidence’ determinations. However, 

traditionally, these types of analyses are performed by the lead agency’s own staff. With this 

expectation, we would coordinate our efforts with the VISTAS TAWG to develop a trends analysis 

supporting the future year applications of CMAQ. 

Use of Corroborative Observational Modeling. While regulatory modeling studies for 

ozone attainment demonstrations have traditionally relied upon photochemical models to evaluate 

ozone control strategies, there has recently been growing emphasis on the use of data-driven models 

to corroborate the findings of air quality models. As noted, EPA’s guidance now encourages the use 

of such observation-based or observation-driven models (OBMs/ODMs). As part of Phase II, we 

will consider the merits of using these techniques as supportive weight of evidence. While the 

OBD/OBM models cannot predict future year air quality levels, they do provide useful corroborative 

information on the extent to which specific subregions may be VOC-limited or NOx-limited, for 

example, or where controls on ammonia or SO2 emissions might be most influential in reducing 

PM2.5. Information of this type, together with results of DDM and traditional ‘brute-force’ 

sensitivity simulations, can be extremely helpful in postulating emissions control scenarios since it 

helps focus on which pollutant(s) to control. 

7.10 Assessing Model Reliability in Estimating the Effects of Emissions Changes 

 EPA identifies three methods (EPA, 2001, pg. 228) potentially useful in quantifying a 

model’s reliability in predicting air quality response to changes in model inputs, e.g., emissions. 

These include: 

> Examination of conditions for which substantial changes in (accurately estimated) 

emissions occur; 

> Retrospective modeling, that is, modeling before and after historical significant 

changes in emissions to assess whether the observed air pollution changes are 

adequately simulated; and 

 

TN Regional Haze SIP Appendix C.1 - 144
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-27 

3⁄4 Use of predicted and observed ratios of ‘chemical indicator species’. 

We note that in some urban-scale analyses, the use of weekday/weekend information has been 

helpful in assessing the model’s response to emissions changes. However, we suspect that this 

approach would not prove feasible with an annual episode over geographic domain as large as 

VISTAS. 

The first two methods have actually been considered for over 15 years and were the subject 

of intensive investigations in the early 1990s in Southern California in studies sponsored by the 

South Coast Air Quality Management District (Tesche, 1991) and the American Petroleum Institute 

(Reynolds et al., 1996). To date, neither method has proven useful largely because of the great 

difficulty in developing historical emissions inventories of sufficient quality to make such an 

analysis credible and the difficulties in removing the influences of different meteorological 

conditions such that the modeling signal reflects only the model’s response to emissions changes. It 

is difficult enough to construct reliable emissions inventories using today’s modeling technology let 

alone construct retrospective inventories 5-10 years ago prior to the implementation of significant 

emissions control programs or major land use changes. The use of indicator species, however, offers 

some promise. 

However, recent analytical and numerical modeling studies have demonstrated how the use of 

ambient data and indicator species ratios can be used to corroborate the future year control strategy 

estimates of Eulerian air quality models. Blanchard et al., (1999), for example used data from 

environmental (i.e., smog) chambers and photochemical models to devise a method for evaluating 

the 1-hr ozone predictions of models due to changes in precursor NOx and VOC emissions. 

Reynolds et al., (2003) followed up this analysis, augmented with process analysis, to assess the 

reliability of SAQM photochemical model estimate of 8-hr ozone to precursor emissions cutbacks. 

With respect to secondary aerosol PM, the recent CMAQ evaluation by Arnold et al. (2003) clearly 

demonstrated how the use of indicator species analysis could be use to develop insight into the 

expected reliability and adequacy of a photochemical/PM model for simulating the effects of 

emissions control scenarios. These researchers used three indicator ratios (or diagnostic ‘probes’) to 

quantify the model’s response to input changes: 

 > The ozone response surface probe [O3/NOx]; 

 > The chemical aging probe [NOz/NOy ]; and 

 > The ozone production efficiency probe [O3/NOz ]. 

 By closely examining CMAQ’s response to key input changes, properly focused in time and 

spatial location, Arnold et al., (2003) were able to conclude that the photochemical processing in 

CMAQ was substantially similar to that in the atmosphere 

 Thus, the extension of these techniques to address CMAQ predictions for secondary aerosols 

will doubtless be quite challenging, but the use of indicator species (e.g., ammonia or HNO3

limitation for nitrate particle formation) and species ratios appears to offer, at this time, the only real 

opportunity to quantify the expected reliability of the air quality model to correctly simulate the 

effects of emissions changes. In the Phase II CMAQ model evaluation, we will remain alert to 

opportunities to extend the indicator species ratio analyses to the problem of fine particulate and 

regional haze. This is one area where technical collaboration between the Emissions and Air Quality 

TN Regional Haze SIP Appendix C.1 - 145
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-28 

Modeling team and the VISTAS TAWG can be especially fruitful in terms of identifying and testing 

emergent methods for challenging the model’s ability to correctly simulate the effects of future year 

emissions changes. Finally, we note that this is truly a current research area and as such falls outside 

the scope of the Phase II modeling effort. However, given its importance, we will remain alert to 

opportunities to utilize newly available methods should this prove feasible within the Phase II 

resources and schedule. 

Table 7-1. Ground-Level Ambient Data Monitoring Networks and Stations Available in VISTAS 12 

Km Domain for CY-2002. 

Monitoring 

Network 

Chemical Species Measured Sampling 

Frequency; 

Duration 

Approximate 

Number of 

Monitors 

IMPROVE Speciated PM2.5 and PM10 1 in 3 days; 24 hr 38 

CASTNET Speciated PM2.5, Ozone Hourly, Weekly; 

1 hr, Week 

74 

SEARCH 24.hr PM25 (FRM Mass, OC, BC, 

SO4, NO3, NH4, Elem.); 24-hr PM 

coarse (SO4, NO3, NH4, elements); 

Hourly PM2.5 (Mass, SO4, NO3, 

NH4, EC, TC); and Hourly gases 

(O3, NO, NO2, NOy, HNO3, SO2, 

CO) 

Daily, Hourly; 8 

NADP WSO4, WNO3, WNH4 Weekly 100 

EPA-FRM Only total fine mass (PM2.5) 1 in 3 days; 24 hr 313 (?)

EPA-SPEC Speciated PM2.5 Varies; Varies 43 (?)

AIRS/AQS CO, NO, NO2, NOx, O3 Hourly; Hourly 6,407 

Table 7-2. Aircraft Sampling Programs Performed in the VISTAS 12 Km Domain for CY-2002.

Aircraft Program Meteorological Parameters & 

Chemical Species Measured 

Sampling 

Program & 

Flight Duration 

Approximate 

Number of 

Flights; Days; 

Aircraft 

University of 

Maryland (UMD); 

Univ. Research 

Foundation (URF) 

Meteorology: WS, WD, Temp, 

RH, 

Air Quality: O3, NO, NO2, CO, 

SO2, aerosol absorption, 

aerosol scattering. 

23 May to 3 Oct; 

Typically 3 hrs 

54 flights, 

54 days, 

1 aircraft 

Midwest RPO & 

Wisconsin DNR

Meteorology: WS, WD, Temp, 

RH, dew point, pressure 

Air Quality: O3, NO, NO2, NOx, 

NOy, speciated VOCs, 

carbonyls, HNO3, NH3, Hg, 

SO4, OC, EC, PM2.5, light 

scattering (Neph), visibility 

pictures. 

1 June to 22 Nov; 

Typically 3-5 hrs 

133 flights; 

29 days; 

2 aircraft 

(WDNR and 

Jacko Aircraft) 

TN Regional Haze SIP Appendix C.1 - 146
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-30 

Table 7-3. Federal Mandated Class I Areas in the VISTAS States.

State Acreage Federal Land Manager Public Law 

Alabama

Sipsey Wilderness Area 12,646 USDA-FS 93-622 

Arkansas 

Caney Creek Wilderness Area 7,344 USDA-FS 93-622 

Upper Buffalo Wilderness Area 9,912 USDA-FS 93-622 

Florida

Chassahowitzka Wilderness Area 23,360 USDI-FWS 94-557 

Everglades NP 1,397,429 USDI-NPS 73-267 

St. Marks Wilderness Area 17,745 USDI-FWS 93-632 

Georgia

Cohotta Wilderness Area 33,776 USDA-FS 93-622 

Okefenokee Wilderness Area 343,850 USDI-FWS 93-429 

Wolf Island Wilderness Area 5,126 USDI-FWS 93-632 

Kentucky 

Mammoth Cave NP 51,303 USDI-NPS 69-283 

Louisiana 

Breton Wilderness Area 5,000+ USDI-FWS 93-632 

 

North Carolina

Great Smoky Mountains NP{1} 273,551 USDI-NPS 69-268 

Joyce Kilmer-Slickrock Wilderness Area{2} 10,201 USDA-FS 93-622 

Linville Gorge Wilderness Area 7,575 USDA-FS 88-577 

Shining Rock Wilderness Area 13,350 USDA-FS 88-577 

Swanquarter Wilderness Area 9,000 USDI-FWS 94-557 

{1}Great Smoky Mountains National Park, 514,758 acres overall, of which 273,551 acres are in North Carolina, and 241,207 

acres are in Tennessee. 

{2}Joyce Kilmer-Slickrock Wilderness, 14,033 acres overall, of which 10,201 acres are in North Carolina, and 3,832 acres are 

in Tennessee. 

South Carolina

Cape Romain Wilderness 28,000 USDI-FWS 93-632 

Tennessee

Great Smoky Mountains NP{1} 241,207 USDI-NPS 69-268 

Joyce Kilmer-Slickrock Wilderness{2} 3,832 USDA-FS 93-622 

{1} Great Smoky Mountains National Park, 514,758 acres overall, of which 273,551 acres are in North Carolina, and 241,207 

acres are in Tennessee. 

{2} Joyce Kilmer Slickrock Wilderness, 14,033 acres overall, of which 10,201 acres are in North Carolina, and 3,832 acres are 

in Tennessee. 

TN Regional Haze SIP Appendix C.1 - 147
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-31 

Virginia

James River Face Wilderness 8,703 USDA-FS 93-622 

Shenandoah NP 190,535 USDI-NPS 69-268 

West Virginia

Dolly Sods Wilderness 10,215 USDA-FS 93-622 

Otter Creek Wilderness 20,000 USDA-FS 93-622 

Table 7-4. Core Statistical Measures to be used in the VISTAS Phase II Air Quality Model 

Evaluation with Ground-Level Data (see ENVIEON, 2003b,d for details).

Statistical 

Measure 

Shorthand 

Notation 

Mathematical 

Expression Notes 

Accuracy of 

paired peak (Ap) 

Paired_Peak 

peak

peak

O

− OP Ppeak = paired (in 

both time 

 and 

space) peak 

 prediction 

Coefficient of 

determination (r

2

) 

Coef_Determ 

∑ ∑

∑

= =

=

− −





 



 −−

N

i

N

i

i i

N

i

i i

OOPP

OOPP

1 1

2 2

2

1

)()(

))((

Pi = prediction at 

time and 

 location i; 

Oi = observation 

at time 

 and location 

i; 

P = arithmetic 

average of Pi, 

i=1,2,..., N; 

O = arithmetic 

average of Oi, 

i=1,2,...,N 

Normalized Mean 

Error (NME) 

Norm_Mean_Err 

∑

∑

=

=

−

N

i

i

N

i

ii

O

OP

1

1

Reported as % 

Root Mean 

Square Error 

(RMSE) 

Rt_Mean_Sqr_Err 

( )

2

1

1

1 2





 



 ∑ − =

N

i

OP ii N

Reported as % 

Fractional Gross 

Error (FE) 

Frac_Gross_Err 

∑= +

− N

i ii

ii

OP

OP

N 1

2 Reported as % 

TN Regional Haze SIP Appendix C.1 - 148
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 7.doc 7-32 

Mean Absolute 

Gross Error 

(MAGE) 

Mean_Abs_G_Err 

∑=

−

N

i

OP ii N 1

1

Mean Normalized 

Gross Error 

(MNGE) 

Mean_Norm_G_Err

∑=

− N

i i

ii

O

OP

N 1

1 Reported as % 

Mean Bias (MB) Mean_Bias 

∑( ) =

−

N

i

OP ii N 1

1 Reported as 

concentration 

(e.g., μg/m3

) 

Mean Normalized 

Bias (MNB) 

Mean_Norm_Bias ( ) ∑=

− N

i i

ii

O

OP

N 1

1 Reported as % 

Mean 

Fractionalized 

Bias (Fractional 

Bias, MFB) 

Mean_Fract_Bias 

∑= 















+

− N

i ii

ii

OP

OP

N 1

2 Reported as % 

Normalized Mean 

Bias (NMB)

Norm_Mean_Bias 

∑

∑

=

=

−

N

i

i

N

i

ii

O

OP

1

1

)(

Reported as % 

Bias Factor (BF) Bias Factor 

1

1 N

i

i i

P

N O =

      ∑

Reported as 

BF:1 or 1: BF or 

in fractional 

notation (BF/1 or 

1/BF). 

TN Regional Haze SIP Appendix C.1 - 149
 May 2004

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 8.doc 8-1

8.0 VISIBILITY ASSESSMENT AND COMPARISON 

 

 This chapter provides a summary of how the modeling results will be used to satisfy the 

requirements of the Regional Haze Rule (RHR) as part of the Section 308 visibility State 

Implementation Plans (SIPs) and Tribal Implementation Plans (TIPs) due in 2007. Note that the 

VISTAS Phase II modeling results will also likely be used in some capacity as the regional 

component in some State’s fine particulate (PM2.5) and 8-hour ozone SIPs that are also due in 

2007. However, how the modeling results will be used to address PM2.5 and 8-hour ozone issues is 

not discussed in this chapter and will be addressed by the pertinent States at a later time. The 

Phase II modeling results generated in 2004 will likely not be the final basis for the Section 308 

SIPs as the results will be updated and refined. However, it is important to start planning how the 

modeling results will be used to project visibility changes, optimize the presentation of results and 

identify any potential pitfalls early on. 

8.1 Regional Haze Rule Requirements 

The Regional Haze Rule (RHR) regulations were published by EPA in 1999 (Federal 

Register 35769, July 1, 1999) and are designed to address the requirements of Section 169A and 

169B of the Clean Air Act (CAA). The CAA and RHR call for the protection of visibility at the 

156 “mandatory Federal Class I areas.”1

 Section 169A of the Clean Air Act Amendments 

(CAAA) established a national visibility goal to remedy existing impairment due to air pollution at 

the Class I areas. This is accomplished by defining a visibility goal of “natural conditions” to be 

achieved at each Class I area by 2064. The RHR requires States with Class I areas to develop SIPs 

that include reasonable progress goals for improving visibility in each Class I area and emission 

reduction measures to meet those goals. For the initial control strategy SIPs due in 2007, States 

are required to adopt progress goals for improving visibility from baseline conditions. The 2000-

2004 five-year period is used to define baseline conditions and the first future progress period is 

2018, that has been interpreted as the 2013-2017 or 2014-2018 five year periods. A State is 

required to set progress goals for each Class I area in the State for two visibility metrics: 

• Provide for an improvement in visibility for the most impaired (i.e., 20% worst) 

visibility days; and 

 

• Ensure no degradation in visibility for the least impaired (i.e., 20% best) visibility 

days. 

The reasonable progress goals must provide for a rate of improvement sufficient to attain “natural 

conditions” by 2064, or justify any alternative rate. States are to define controls to meet progress 

goals every 10 years, starting in 2018, that defines progress periods ending in 2018, 2028, 2038, 

2048, 2058 and finally 2064. States will determine whether they are meeting their goals by 

comparing visibility conditions from one five-year period to another (e.g., 2000-2004 to 2013-

2017). As stated in 40 CFR 51.308 (d) (1), baseline visibility conditions, progress goals, and 

 

1

 Areas designated as mandatory Class I areas are those National Parks exceeding 6,000 acres, wilderness areas 

and national memorial parks exceeding 5,000 acres, and all international parks that were in existence on August 7, 

1977. Since then some new Class I areas have been defined and some existing Class I areas have been expanded. 

However, these areas are not included as part of the 156 “Federal mandated Class I areas” for which the RHR 

applies. In this document the term “Class I area” refers to the 156 “Federal mandated” Class I Areas. 

TN Regional Haze SIP Appendix C.1 - 150
 May 2004

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 8.doc 8-2

changes in visibility must be expressed in terms of deciview (dv) units. The deciview unit of 

visibility impairment is derived from light extinction (bext) as follows: 

dv = 10 ln (bext/10) 

 Where light extinction (bext) is expressed in terms of inverse megameters (Mm-1 = 10-6 m

-1). 

Section 8.2.1.1 below discusses how bext is calculated using the observed fine particulate 

concentrations from the IMPROVE monitors and Section 8.2.1.2 below defines how the modeling 

results will be mapped to the IMPROVE measurements for the purpose of projecting visibility 

improvements. 

8.2 EPA Guidance for Complying with Regional Haze Rule 

EPA has published three guidance documents that relate to how modeling results should be 

used to demonstrate compliance with the RHR: 

“Guidance for Demonstrating Attainment of Air Quality Goals for PM 2.5 and Regional 

Haze, Draft 2.1 (EPA, 2001). 

“Guidance for Tracking Progress Under the Regional Haze Rule” (EPA, 2003a). 

“Guidance for Estimating Natural Visibility Conditions Under the Regional Haze Rule” 

(EPA, 2003b). 

The first EPA modeling guidance document listed above (EPA, 2001) discusses the use of 

modeling results to demonstrate progress toward the RHR goals. This is a draft document that will 

be updated during 2004. When the draft modeling guidance is updated that may necessitate an 

update of this Modeling Protocol to keep it consistent with current EPA guidance. The second 

document (EPA, 2003a) focuses on monitored visibility and how to define the visibility baseline 

and how to track visibility goals. The third EPA guidance document discusses procedures for 

defining “natural conditions” for a Class I area that is the visibility goal in 2064. In the discussion 

below we are assuming that other VISTAS participants would define the “natural conditions” for 

each VISTAS Class I area. In this Chapter we address the use of the modeled and monitored fine 

particulate concentrations at the Class I areas for projecting visibility changes from the current to 

future years. 

VISTAS is currently evaluating EPA’s assumptions for projecting visibility improvements 

in their guidance documents (EPA, 2001; 2003a,b). If technically justified, VISTAS may make 

modifications to EPA’s procedures based on their analysis. For example, recent measurements 

suggest that, at least for the VISTAS region, sulfate is not always fully neutralized by ammonium 

and that ammonium bisulfate and sulfuric acid can exist, especially in the summer. Thus, the EPA 

approach assuming that sulfate is always completely neutralized by ammonium may have to be 

updated. The current EPA approach also does not account for sea salt that can be a contributor to 

visibility impairment, especially at coastal sites. VISTAS is also evaluating EPA’s default “natural 

conditions” and its appropriateness for the VISTAS region. Below we describe EPA’s approach 

for projecting visibility improvements, but be aware they may be modified by VISTAS at a later 

date. 

TN Regional Haze SIP Appendix C.1 - 151
 May 2004

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 8.doc 8-3

8.2.1 Procedures for Projecting Visibility Changes 

EPA guidance recommends using the model in a relative sense to project future-year 

visibility conditions (EPA, 2001; 2003a). This is done through the use of Relative Reduction 

Factors (RRFs) that are defined as the ratio of the future-year to the current-year modeling results. 

The RRFs are applied to the baseline visibility conditions to project future-year visibility. The 

major features of EPA’s recommended visibility projections are as follows (EPA, 2003a,b): 

• Monitored data should be used to define current air quality. 

• Monitored concentrations of PM10 are divided into six major components, the first 

five of which are assumed to be PM2.5 and the sixth is PM2.5-10. 

3⁄4 SO4 (sulfate); 

3⁄4 NO3 (particulate nitrate); 

3⁄4 OC (organic carbon); 

3⁄4 EC (elemental carbon); 

3⁄4 OF (other fine particulate); and 

3⁄4 CM (coarse matter). 

• Models are used in a relative sense to develop relative reduction factors (RRFs) 

between future and current predicted concentrations of each component. 

• Component-specific relative reduction factors are multiplied by current monitored 

values to estimate future component concentrations. 

• Estimates of future component concentrations are consolidated to provide an 

estimate of future air quality, which can be related to a goal for regional haze. 

• Future estimated air quality is compared with the goal to see if the simulated control 

strategy results in the goal being met. 

• It is acceptable to assume that all measured sulfate is in the form of ammonium 

sulfate [(NH4)2SO4] and all particulate nitrate is in the form of ammonium nitrate 

[NH4NO3]. 

In order to facilitate tracking visibility progress, three important visibility parameters are required 

for each Class I area: 

Baseline Conditions: Baseline Conditions represent visibility for the 20% best and 20% 

worst days for the initial five-year baseline period of the regional haze program. Baseline 

Conditions are calculated using monitor data collected during the 2000-2004 five-year 

period. 

Current Conditions: Current Conditions for the best and worst visibility days are calculated 

from the latest five-year average based on the latest five-years of monitored data (which is 

currently 1998-2002, but expected to be 1999-2003 by the time the initial future-year Phase 

II visibility projections will be made in late 2004).. 

TN Regional Haze SIP Appendix C.1 - 152
 May 2004

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 8.doc 8-4

Natural Conditions: Estimates of natural visibility conditions for the 20% best and 20% 

worst days at a Class I area that is the goal of the RHR is 2064 that has been defined as 

visibility conditions that would be experienced in the absence of human-caused 

impairment” (EPA, 2003b). 

8.2.1.1 Calculation of Baseline Conditions 

Baseline and Current Conditions for Class I areas are calculated using fine and coarse 

particulate concentration measured at Interagency Monitoring of Protected Visual Environments 

(IMPROVE) monitors. Currently, each Class I area in the VISTAS domain has an associated 

IMPROVE PM monitor. The IMPROVE monitors do not directly measure visibility, but instead 

measure speciated fine particulate (PM2.5) and total PM2.5 and PM10 mass concentrations from 

which visibility is obtained through the IMPROVE reconstructed mass extinction equation. 

Visibility conditions are estimated starting with the IMPROVE reconstructed 24-hour 

average particulate matter (PM) mass measurements for six PM species: 

• Sulfate [SO4]; 

• Particulate Nitrate [NO3]; 

• Organic Matter [OM]; 

• Elemental Carbon [EC]; 

• Other Fine Particulate [Soil]; and 

• Coarse Matter [CM]. 

The IMPROVE monitors do not directly monitor some of these species so assumptions are 

made as to how the IMPROVE measurements can be adjusted and combined to obtain these six 

components of light extinction. For example, sulfate and particulate nitrate are assumed to be 

completely neutralized by ammonium and are assume to occur solely in the fine particulate mode 

(that is any coarse mode sulfate and nitrate in the real atmosphere may be present in the CM 

IMPROVE measurement). Concentrations for the above six components of light extinction in the 

IMPROVE reconstructed mass (RCM) extinction equation are obtained from the IMPROVE 

measured species as follows: 

Table 8-1. Definition of IMPROVE Reconstructed Mass (RCM) Species from Measured 

IMPROVE Species. 

IMPROVE RCM IMPROVE Measured Species 

SO4 1.375 x ( 3 x S) 

NO3 1.29 x NO3_ 

OM 1.4*OC1 + 1.4*OC2 + 1.4*OC3 + 1.4*OC4 + 1.4*OP 

EC EC1 + EC2 + EC3 – OP 

Soil 2.2*AL + 2.49*SI + 1.63*CA + 2.42*FE + 1.94*TI 

CM MT – MF 

Where: 

• S is elemental sulfur as determined from proton induced x-ray emissions (PIXE) 

analysis of the IMPROVE Module A that is multiplied by 3 to account the presence of 

oxygen, if S is missing then the sulfate (SO4) measured by ion chromatography 

analysis of the Module B is used to replace (3 x S). It is assumed to be completely 

neutralized by ammonium (1.375 x SO4). 

TN Regional Haze SIP Appendix C.1 - 153
 May 2004

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 8.doc 8-5

• NO3_ is the particulate nitrate measured by ion chromatography analysis of the Module 

B. It is assumed to be completely neutralized by ammonium (1.29 x). 

• The IMPROVE Organic Carbon (OC) measurements are multiplied by 1.4 to obtain 

Organic Matter (OM) to adjust the OC mass for other elements assumed to be 

associated with OC. 

• Elemental Carbon (EC), which is also referred to as Light Absorbing Carbon (LAC) is 

determined by TOR analysis and is the sum of EC fractions minus the pyrolized 

fraction. 

• Soil is determined as a sum of the mass of those elements (measured by PIXE) 

predominately associated with soil, whose mass is adjusted for oxygen in the common 

compounds. 

• MT and MF are total PM10 and PM2.5 mass, respectively. 

Associated with each PM species is an extinction coefficient that converts concentrations (in 

g/m3

) to light extinction (in inverse mega meters, Mm-1). Sulfate and nitrate are hygroscopic so 

relative humidity adjustment factors [f(RH)] are used to modify the extinction coefficients that 

increase the particle’s extinction efficiency with increasing RH to account for the particles taking 

on water and having higher light scattering properties. Note that some Organic Matter (OM) 

compounds may also have hygroscopic properties, but the IMPROVE reconstructed mass 

extinction equations assume OM is non-hygroscopic. 

BSulfate = 3 x f(RH) x [SO4] 

bNitrate = 3 x f(RH) x [NO3] 

bEC = 10 x [EC] 

bOM = 4 x [OM] 

bSoil = 1 x [Soil] 

bCM = 0.6 x [CM] 

Monthly average f(RH) factors are used as recommended in EPA’s guidance (EPA, 2003a). 

These values have been recently updated (SAIC, 2003) and are available at: 

ftp://ftp.saic.com/raleigh/RegionalHaze_2002FRHcurve/fRH_analysis/. 

The total light extinction (bext) is assumed to be the sum of the light extinction due to the 

six PM species listed above plus Rayleigh (blue sky) background (bRay) that is assumed to be 10 

Mm-1. 

 Bext = bRay + bSulfate + bNitrate + bEC +bOM + bSoil + bCM

The total light extinction (bext) in Mm-1 is related to visual range (VR) in km using the 

following relationship: 

 VR = 3912 / bext

The Regional Haze Rule requires that visibility be expressed in terms of deciview (dv) that 

uses natural logarithms of the extinction as follows: 

 dv = 10 ln(bext/10) 

TN Regional Haze SIP Appendix C.1 - 154
 May 2004

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 8.doc 8-6

 The equations above using data from the associated IMPROVE monitor are used to 

estimate the daily average visibility at each Class I area. For each year from the 2000-2004 

baseline these daily average visibility values, in terms of deciview (dv), are then ranked from 

highest to lowest. The 20% worst days visibility for each given year is given as the average 

visibility across the 20% highest visibility days in deciview. Similarly, the 20% best days are 

given as the average visibility across the 20% lowest visibility days in terms of deciview. The 

Baseline Conditions is the average of the 20% best and 20% worst days across the five-years of 

2000-2004. The Current Conditions is the average of the 20% best and 20% worst days based on 

the latest five-years of available data, which would be 1999-2003 for the VISTAS Phase II 

analysis being conducted in late 2004. 

8.2.1.2 Mapping of Modeling Results to the IMPROVE Measurements

As noted above, to project future-year visibility at Class I areas the modeling results are 

used in a relative sense to scale current observed visibility for the 20% best and 20% worst 

visibility days using RRFs that are the ratio of modeling results for the future-year to current-year. 

This scaling is done separately for each of the six components of light extinction in the IMPROVE 

reconstructed mass extinction equations. The modeled species do not necessarily exactly match up 

with the IMPROVE reconstructed mass species, thus assumptions must be made to map the 

modeled species to the IMPROVE reconstructed mass species for the purpose of projecting 

visibility improvements. For example, the models explicitly simulates ammonium and sulfate may 

or may not be fully neutralized in the model, whereas the IMPROVE reconstructed mass equations 

assume sulfate is fully neutralized by ammonium. For the CMAQ Version 4.3 model that was 

used in VISTAS Phase I modeling, the mapping of modeled species to IMPROVE reconstructed 

mass (RCM) species listed in Table 8-2 is proposed to be used in the VISTAS Phase II modeling. 

Table 8-2. Proposed Mapping of CMAQ V4.3 Modeled Species Concentrations to IMPROVE 

Reconstructed Mass (RCM) Species. 

IMPROVE RCM CMAQ V4.3 Species 

SO4 1.375 x (ASO4J + ASO4I) 

NO3 1.29 x (ANO3J + ANO3I) 

EC AECJ + AECI 

OM AORGAJ + AORGAI + AORGPAJ + AORGPAI + AORGBJ + AORGBI 

Soil A25J + A25I 

CM ACORS + ASEAS + ASOIL 

If a different model is used (e.g., CAMx or CMAQ-AIM) or updates to CMAQ change its 

species definitions, then the species mappings would have to be defined specific for that model. 

Note that in the above species mapping it is assumed that all of the CMAQ estimated sulfate and 

particulate nitrate are in the fine mode. If a fully sectional model was used (e.g., CMAQ-AIM or 

CAMx), then a decision would have to be made whether to map the coarse mode sulfate and 

nitrate to the IMPROVE fine SO4 and NO3 species (as implicitly assumed for CMAQ) using 

mappings in Table 8-2) or to include it in the IMPROVE CM species. In fact this is also an issue 

for the CMAQ V4.3 modal approach in which the lognormal size distribution for sulfate and 

nitrate in the accumulation mode (i.e., ASO4J and ANO3J) may have some mass greater than 2.5 

:m, however the current usual convention for CMAQ is to assume all the sulfate and nitrate are in 

the fine mode. 

TN Regional Haze SIP Appendix C.1 - 155
 May 2004

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 8.doc 8-7

8.2.1.3 Using Modeling Results to Project Changes in Visibility 

Modeling results are used in a relative fashion to project future-year visibility using relative 

reduction factors (RRFs). RRFs are expressed of the ratio of the modeling results for the futureyear to the results of the base year and are Class I area and PM species specific. RRFs are applied 

to the Baseline Conditions observed PM species to project future-year PM levels from which 

visibility can be assessed using the IMPROVE RCM extinction equations listed above. EPA has 

identified the following six steps to project future-year visibility for the 20% best and 20% worst 

days (EPA, 2001;2003a): 

1. For each Class I area visibility is ranked using IMPROVE reconstructed PM mass (RCM) 

extinction equation for each year that comprises the five-year Baseline Conditions (i.e., 

2000-2004). 

2. Calculate the arithmetic average of the 20% best and 20% worst visibility days, in deciview 

(dv), for each year of the five-year baseline period and then calculate the five-year average 

of the 20% best and 20% worst days. Document which days during the five-year baseline 

period comprise the 20% best and 20% worst observed visibility days. 

3. Use an air quality model to simulate the baseline period emissions and future-year 

emissions. Extract 24-hour average PM species concentrations “near” each Class I area. 

Calculate the average PM species concentration estimates in the current and future-year 

simulation across the 20% best and 20% worst observed visibility days for each year in the 

five-year baseline period. Average across the five-years the average PM species 

concentrations for the 20% best and 20% worst days and for each year in the five-year 

baseline period. Calculate Class I area and best/worst 20% days RRFs for each of the six 

PM components as the ratio of the five-year average estimated PM species concentration in 

the future-year to current year. 

4. Multiply the 20% best/worst days and PM species dependent RRFs by the observed 24-

hour PM species concentrations for each day from the 20% best/worst days to obtain 

future-year daily average PM species concentrations for each day from the 20% best and 

20% worst visibility days from the five-year baseline period. 

5. Using the future-year estimated 24-hour PM species concentrations for the best/worst 20% 

observed days, calculate extinction using the IMPROVE reconstructed mass extinction 

equation and daily deciview, perform annual averaging of the visibility (dv) estimates for 

the 20% best/worst days for each year and then obtain the five-year average visibility (dv) 

for the future-year. 

When selecting model estimated PM species concentrations “near” the monitor, EPA (2001) 

recommends taking a spatial average of PM concentrations across a grid cell resolution dependent 

NX by NY array of cells centered on the grid containing the monitor. For the VISTAS Phase II 36 

km modeling, just the model estimates for the grid cell containing the monitor will be used (i.e., 

NX=NY=1). For the VISTAS Phase II 12 km modeling, EPA recommends that NX=NY=3, that is 

the model estimated PM species concentrations are averaged across 9 grid cells centered on the 

IMPROVE monitor (EPA, 2001). 

TN Regional Haze SIP Appendix C.1 - 156
 May 2004

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 8.doc 8-8

8.2.1.4. Exclusion of Days from 20% Best/Worst Conditions

As discussed in Chapter 7, the model performance evaluation will pay particular close 

attention to the model’s PM performance at each Class I area for the 20% Best and 20% Worst 

days. If the model performance is extremely poor for one or more of these days at a Class I area, it 

may be desirable to exclude that day when calculating the RRFs for visibility projections. For 

example, if a day is one of the Worst 20% days at a Class I area and the model estimates clean 

background concentrations then it would be inappropriate to use the modeling results, even in a 

relative fashion through the RRFs, to project future-year visibility. The modeling Team would 

identify such suspect days and work with the VISTAS TAWG on whether they should be excluded 

from the visibility projections on a case-by-case basis. 

8.3 VISTAS Phase II Future-Year Modeling 

The VISTAS Phase II future-year modeling will use the 2002 MM5 meteorological 

conditions. That is, we will assume that the meteorological conditions for the future-year are the 

same as for 2002. This will allow for the comparison of the changes in visibility at Class I areas 

from the current (2002) to future-year due to changes in emissions. This means that the effects of 

climate change and climatic variations will not be accounted for. Several other decisions 

concerning the future-year to be modeled, model(s) to be used and modifications to the model 

inputs to reflect future years do need to be made such as: 

8.3.1 Future-Year to be Modeled 

Visibility projects are needed for the 20% best and 20% worst days from the five-year 

baseline period of 2000-2004 to 2018, which has been interpreted as the five-year future-year 

periods of 2013-2017 or 2014-2018. The EPA guidance documents are inconsistent on what 

future-year should be used for the assessment. The 2001 modeling guidance says that the mid-year 

from the 2013-2017 five-year projection period should be used, which would be 2015 (EPA, 2001, 

pg. 221). The 2003 guidance for tracking progress doesn’t discuss the future-year that should be 

modeled, but does mention that goals should be compared across 5-year planning periods “(e.g., 

2000-2004 to 2013-2017) and progress should be measured as improvement from 2004 to “2018” 

(EPA, 2003, pg. 1-6). VISTAS and other RPOs will be looking to EPA for providing more 

definitive guidance as to which future-year should be used for modeling visibility progress and this 

Modeling Protocol will be updated when such information is available. Thus, in the discussion 

below we just refer to the “future-year” with the actual year to be determined during 2004. There 

may also be some benefits to projecting visibility improvements for an interim future-year between 

2002 and 2018 to help track progress to the 2018 visibility targets. 

8.3.2 Future-Year Emissions 

The future-year emissions will be provided to the emissions and air quality modeling team 

(ENVIRON/Alpine/UCR) by the VISTAS emissions projection contractor (MacTec). They would 

be processed into the gridded speciated hourly three-dimensional emissions inputs for 

photochemical grid modeling using the SMOKE emissions model and the procedures discussed in 

Section 5.2. The same biogenic emissions as used in the 2002 Base Case modeling will be used 

for the future-year modeling. This assumes that the same land use and biomass distribution as 

used in the 2002 Base Case emissions would exist in the future-year emission scenarios. The 

TN Regional Haze SIP Appendix C.1 - 157
 May 2004

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 8.doc 8-9

effects of urban sprawl, increased agricultural, deforestation, locations of new roads, etc. between 

the current (2002) and future-year would not be accounted for. The same Typical Year fire 

emissions that were used in the 2002 Typical Year emissions scenario would be used in the futureyear. Future-year Typical Year emissions for Electrical Generating Units (EGUs) would also be 

used for the future-year scenarios, however for EGUs these would not be identical to the 2002 

Typical Year EGU emissions. 

8.3.3 Future-Year Initial and Boundary Conditions 

The same Initial Conditions (ICs) as used in the 2002 Base Case would be used in the 

future-year modeling. Because a ~15 day spin up period is being used, ICs will have minimal if 

any influence on the model estimated concentrations. 

As discussed in Section 5.3, the source of the boundary conditions (BCs) to be used along 

the lateral edges of the continental US national RPO 36 km modeling domain for current (2002) 

and future-year modeling is being studied by VISTAS. The VISTAS Phase I modeling analyzed 

several different types of boundary conditions, including the CMAQ default values and seasonal, 

monthly and 3-hour average values estimated by the GEOS-CHEM global model for a 2001 

annual simulation. 

The effects of using time varying (3-hourly) BCs produced by the 2001 GEOS-CHEM 

global model simulation versus the seasonal GEO-CHEM BCs were analyzed using the CMAQ 

model and the VISTAS Phase I July 2001 episode. These results support the notion that a 2002 

GEOS-CHEM simulation would be beneficial to support regional air quality modeling of the 2002 

calendar year. Accordingly, VISTAS is pursuing performing a 2002 GEOS-CHEM simulation, 

whose results would be used to define BCs for the CMAQ revised basecase simulation in the fall 

of 2004. 

Initially, BCs based on seasonal average results from the GEOS-CHEM 2001 simulation 

will be used for the VISTAS Phase II CMAQ modeling of the initial 2002 base case. If a 2002 

GEOS-CHEM simulation is ready in time, then the Phase II modeling 2002 BCs would be updated 

with that data. If a 2002 GEOS-CHEM simulation is performed, there is the possibility that 

GEOS-CHEM would also be run for the future-year emissions scenario in which case the futureyear CMAQ BCs would be based on the future-year GEOS-CHEM simulation. In the absence of a 

future-year emissions scenario GEOS-CHEM simulation, we recommend that the same CMAQ 

BCs be used in the future-year as used in the 2002 base year as any adjustments to them would be 

arbitrary and speculative. 

8.3.4 Other Future-Year Modeling Inputs 

All other future-year CMAQ modeling inputs would be identical to the 2002 Base Case 

simulation including meteorology, photolysis rates, geophysical, and other inputs. 

TN Regional Haze SIP Appendix C.1 - 158
 May 2004

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 8.doc 8-10

8.4 Presentation of Results 

There are various ways that the future-year modeling results can be presented to display the 

visibility projections and convey the key findings. Although the projection of visibility 

improvements at the Class I areas is a key element of the Regional Haze Rule (RHR), analyzing 

the results solely at the Class I areas is a very narrow view of them, thus we intend to present the 

future-year modeling results in other forms to convey a more complete picture. 

8.4.1 Projection of Visibility Improvements at Class I Areas 

The results of the visibility projections at each Class I area in and around the VISTAS 

domain would be displayed graphically as well as in tabular summaries. Spread sheets of the 

results would be generated ordered by state and provided to VISTAS. The visibility projections at 

each Class I area would be compared with the 2018 visibility progress goal calculated following 

EPA’s “Guidance for Tracking Progress Under the Regional Haze Rule (EPA, 2003a). 

Figure 8-1 displays a sample plot of a visibility progress goal prepared for the Grand 

Canyon National Park as part of the WRAP Section 309 SIP work effort. The modeled projected 

visibility could be plotted on the same figure as a graphical way of presenting the results. Shown 

as a straight line near the bottom of Figure 8-1 is the EPA default “natural conditions” for the 20% 

worst visibility days as the green line with green triangles that represents the visibility goal at 

Grand Canyon of 6.97 dv in 2064. The black diamonds on the left side of the plot are the “current 

conditions” that are based on IMPROVE observations for the 20% worst days for five-year periods 

ending in 1993 (i.e., 1989-1993) to 2001 (i.e., 1997-2001). The “current conditions” for the latest 

five years of data, which is 1997-2001 in this graph (the Section 309 SIP analysis was performed 

in 2003 before the 2002 IMPROVE data were available), is assumed as the “baseline conditions” 

(i.e., 2000-2004) that is the starting point in 2004 (12.00 dv) for the glide path or linear uniform 

rate of progress to natural conditions in 2064 (purple line with squares). In this example, the 

visibility progress goal in 2018 would be 10.83 dv so that a 1.17 dv reduction in visibility is 

needed by 2018 to meet the visibility progress target in 2018 at the Grand Canyon National Park 

(GMNP). Note that we would expect higher observed visibility levels for the 20% worst days at 

Class I areas in VISTAS due to higher density population and higher sulfate concentrations in the 

eastern US, but Figure 8-1 illustrates the types of graphs that can be utilized to display visibility 

progress goals combined with visibility progress projections. 

Tabular summaries could list for each Class I area, in terms of deciview, the observed 

Baseline Conditions, the 2018 Progress Goal, the difference (i.e., visibility progress target) and 

then the modeled difference for each future-year emissions scenario. Such a table would clearly 

state which future-year emission scenarios met the RHR 2018 Progress Goals at which Class I 

areas. 

Another useful display of the projected visibility improvements are stacked bar charts of 

extinction showing the contributions of each of the six major PM components to light extinction 

for the IMPROVE measured current period to the future-year period and the 20% best and 20% 

worst days. These side–by-side current to future-year extinction comparisons can graphically 

display which extinction components are getting better and which are getting worse. They can 

also be used to identify why the emission controls may not have had the desired effect. For 

TN Regional Haze SIP Appendix C.1 - 159
 May 2004

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 8.doc 8-11

example, an SO2 emissions control strategy may have had the desired effect in reducing sulfate 

concentrations but the visibility estimates remain relatively unchanged as the lower sulfate 

concentrations frees up ammonia that then forms particulate nitrate that compensates for the lower 

sulfate in the visibility equations. Such a chemical shift in the extinction budget would be readily 

apparent in the stacked bar charts. The most useful forms to convey the modeling results will 

evolve over the project. 

Figure 8-1. Example Calculation Of Visibility Progress Goals Using IMPROVE Data Collected 

At The Grand Canyon National Park For The 20% Worst Days Showing EPA Default “Natural 

Conditions” (Green Triangles) For The 20% Worst Days That Is The 2064 Goal (6.97 dv), 

“Current Conditions” Observed Visibility (dv) As Black Triangles Whose Latest Value (12.00 dv) 

Is Assumed As The Starting Point For The 2000-2004 Baseline (First Purple Square), And The 

Glide Path From 2004 Baseline To “Natural Conditions” In 2064 Assuming Linear Uniform Rate 

Of Progress. 

Uniform Rate of Reasonable Progress Glide Path

Grand Canyon NP - 20% Worst Days

12.0011.66 10.83 9.99 9.15 8.31 7.47 6.97

0

5

10

15

20

25

30

1993 1998 2003 2008 2013 2018 2023 2028 2033 2038 2043 2048 2053 2058 2063

Year

Haziness Index Unit:Deciviews 

Glide Path Natural Visibility Conditions Observation

TN Regional Haze SIP Appendix C.1 - 160
 May 2004

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 8.doc 8-12

8.4.2 Spatial Maps of Results 

Spatial maps of changes in individual species, total PM mass, extinction and deciview from 

the 2002 Typical Year to future-year simulations would be used to provide a big picture view of 

the changes in regional air quality and visibility from 2002 to the future–year. The display of 

hourly and 24-hour animations as well as longer-term averages (e.g., monthly, seasonal and 

annual) may provide more insight into the modeling results. 

8.4.3 Time Series Plots 

Time series plots of concentrations extinction and deciview at key receptor locations, such 

as Class I areas, may also provide additional insight into the modeling results. Hourly time series 

for selected periods and locations may provide insight into short-term changes, whereas changes in 

24-hour concentrations for the year provides information for all days, not just the ~40 days that 

constitute the 20% best/worst days of the year. 

8.4.4 Box Plots and Q-Q Plots 

Box and whisker plots of the projected extinction and deciview at Class I areas as well as 

absolute modeled extinction and deciview at all sites can be used to see changes in visibility across 

all concentrations, not just the 20% best/worst days. Quantile-Quantile plots of the changes in 

frequency distribution projected using RRFs and absolute results from the model would also 

provide an indication of the changes in visibility from the current to future-year across all days. 

TN Regional Haze SIP Appendix C.1 - 161
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 9.doc 9-1

9.0 DATA MANAGEMENT 

Data management and data security procedures are critical components of the VISTAS 

regional fine particulate and haze modeling. Very large data files are used in each component of 

the modeling process, including processing of the meteorology data, emissions processing, and 

visibility modeling with CMAQ. An annual simulation on the VISTAS 36-km domain requires 

approximately 2 Terabytes (Tb) of disk storage while an annual simulation on the VISTAS 12-

km domain requires over 3 Tb of disk storage. This chapter describes data management 

practices. 

For all critical files we will maintain backup copies either on tapes or redundant disk 

systems. In addition, because model simulations will be performed separately by ENVIRON, 

UCR and AG, each institution will maintain its own copy and backup of critical input and output 

files. Because there are differences in system configurations at each of the 3 modeling centers 

(ENVIRON, UCR, and Alpine Geophysics) the data backup and archiving are discussed 

separately for each center, below. 

CMAQ generates large output files of which most information is rarely used (for 

example, model output for layers other than the surface layer). Under Phase I, we extracted layer 

1 data for analysis and then deleted the 3-dimensional CMAQ instantaneous concentration 

CONC output files. However, we permanently saved the script, executables and input data so 

that 3-D files can be regenerated if needed. During Phase I a new version of CMAQ was 

released that outputs hourly average layer 1 concentrations for selected species (ACONC) in 

addition to the 3-D instantaneous concentration CONC files. For Phase II we propose to output 

and archive all species on the layer 1 ACONC file and extract and archive 3-D CONC files just 

for the last hour of each month so that monthly sensitivity tests could be performed. In the event 

the Optional Task 14 is funded to evaluate CMAQ estimates aloft, we will save the 3-D CONC 

files for periods with aircraft flights. We do not plan redundant archiving for model output files 

(including output from MCIP and SMOKE) because these files can be regenerated by repeating 

model simulations, and this is approach more efficient and more cost effective than redundant 

archiving. 

 To promote efficient, reliable communication among project participants, the modeling 

team has created 4 different listservs to aid in dissemination of information and as a primary 

means for distributing emissions and air quality modeling information to the VISTAS TAWG. 

The listservs are: 

• vistas-all@cert.ucr.edu: general project information for all interested persons. 

• vistas-modeling@cert.ucr.edu: private list for the VISTAS project management team and 

modeling contractors. 

• vistas-emissions@cert.ucr.edu: list for sharing information on emissions processing and 

QA. 

• vistas-met@cert.ucr.edu: list for sharing information on meteorology modeling and 

processing of MM5 data using MCIP. 

TN Regional Haze SIP Appendix C.1 - 162
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 9.doc 9-2

These separate listservs are aimed at providing better organization of communications and 

allowing for detailed discussions of specific topics such as emissions QA. 

9.1 Project Website 

For the VISTAS Phase I modeling effort, a project webpage at UCR has been used to 

publish and disseminate all project results. The webpage address is: www.cert.ucr.edu/vistas and 

it includes sections on model input data, emissions QA, visibility modeling results, and project 

reports and presentations. Because the Phase I webpage already contains results for a very large 

number of model simulations, we propose to establish a new, parallel webpage for Phase II 

using the address www.cert.ucr.edu/aqm/vistas2 that will be identical in design to the Phase I 

webpage. The advantages of establishing a new webpage include the following: 

• Separating Phase I and Phase II results will avoid confusion in presenting or interpreting. 

Results. 

• The Phase II results will be simpler and more accessible. 

• All Phase I results will continue to be available at the previously published web page 

addresses. 

 

As in Phase I, the project webpage will be the primary mechanism for communicating results of 

model simulations and analysis to funding agencies and other interested parties. 

9.2 Data Transfer 

Data transfer among the 3 modeling centers and between other VISTAS participants or 

contractors will be accomplished using a combination of email, ftp downloads and portable disk 

drives depending on the size of the data transfer. For data files smaller than a few MB email 

typically works well and is most efficient. For data files of less than about 500 MB file transfer 

protocol (ftp) is typically the fastest and most efficient method. ENVIRON, UCR and AG each 

maintain webpages and ftp pages that can be used for exchanging data. In addition, each 

modeling center has several portable disk drives with both USB2 and firewire interfaces that can 

be FedEx among project participants to exchange large data sets. Portable disk drives range in 

size from 80 to 300 GB and are adequate for all large files data transfers. The approach 

described here has been used throughout the VISTAS Phase I project and has proven to be 

economical and efficient. 

9.3 Data Backup and Archiving 

Data backup and archiving will be performed at each of the modeling centers. Copies of 

critical project data will be maintained at each modeling center to provide redundant backup of 

key project data. In addition, each modeling center will perform backups of key project data to 

tape or redundant disk storage systems. Data storage and back up resources at each modeling 

center are described next. 

ENVIRON 

Over 10,000 Gigabytes (>10 Terabytes, Tb) of disk storage are available to the 

UNIX/Linux workstations. All of the workstations are networked together and are accessible 

TN Regional Haze SIP Appendix C.1 - 163
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 9.doc 9-3

from each employee’s desktop PC. All workstations have CD-ROM drives and can access DLT, 

4mm DAT and 8mm Exabyte tape drives for data backup and data transfer. ENVIRON can also 

create CDs (CD-R and CD-RW) and DVDs (DVD+ and DVD-) for data backup and distribution. 

For Phase II modeling, most CMAQ simulations would be performed on a new 9 node Beowulf 

Linux Cluster that included one master node and 8 processing nodes. Each node consists of two 

AMD Athlon 2600+ processors. The master node has 2 Gb of memory and is connected to a 2.8 

Tb RAID disk system. Each secondary processing node includes 1 Gb of memory. The 

ENVIRON Novato computing center also includes approximately 10 dual processor Linux 

workstations with processing speeds of 1700+ to 3000+. Three older Unix workstations are also 

available, SUN, DEC and SGI. The Linux computer systems are located in their own room with 

their own dedicated air conditioning (AC) system. The room includes a temperature sensitive 

power shut off device that will shut off the power to all computers in case the AC breaks down 

so that catastrophic failure due to too high temperatures does not occur. Backups are made on 

IDE disk drives that are removed from the computer and stored on a shelf to protect against 

power surges destroying the backup data. 

UC Riverside 

 Data storage systems at UCR include more than 22 TB of disk space configured as 

RAID5 disk systems. All computers and disk systems are connected using high speed Gigabit 

Ethernet for efficient simulation and analysis of large datasets. To provide maximum data 

security the systems are located behind the UCR firewall and an additional firewall internally 

within the laboratory. A separate system is used for the project websites and ftp site to allow 

project data to be accessed thought the UCR T1 internet connection. The data backup/archiving 

system include 8mm tape drives and DLT and Super DLT auto loading cartridge system capable 

of performing unattended archive/backups of over 1 Tb (uncompressed). Key disk systems have 

hot-swappable hard drives with stand-by spare drives and redundant power supplies. Copies of 

critical project data are backed up to tape and to a redundant RAID5 disk system. The compute 

clusters and disk systems are located in a locked, secure room with a dedicated climate control 

system and with backup air conditioning. The laboratory has a full time systems administrator to 

perform system backups, maintenance and updates and Dr. Tonnesen’s group includes a second 

full time systems administrator who performs weekly backups of critical data to tape.

Alpine Geophysics 

 Alpine Geophysics' computing facilities consist of SUN Microsystems SPARCstation 

computers and a very powerful array of 19 multiprocessor Linux-based workstations. Disk 

storage systems include 12 Tb of aggregate disk space and over 8 Tb of SCSI and IDE Raid-5 

protected space. All client data is stored on at least one RAID-5 protected disk array. To further 

protect client data, two of our main servers backup their disk drives on a weekly basis to a 

second server that is physically disconnected from the power supply when not doing an active 

transfer. This way a catastrophic power failure will not compromise the ability for Alpine 

Geophysics to deliver. 

TN Regional Haze SIP Appendix C.1 - 164
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 10.doc 10-1 

10.0 DOCUMENTATION 

 This section describes the documentation that be provided during VISTAS Phase II and the 

potential for modifications to this Modeling Protocol and QA Plan that might become necessary as 

this phase of the study unfolds. 

10.1 Planned Documentation 

Documentation associated with the emissions and air quality modeling performed during 

Phase II will include all relevant input data bases and scripts associated with the pre- and postprocessing associated with model input development, model application, sensitivity and diagnostic 

analyses, and performance evaluations. Interim reports describing the methodologies and results of 

the model performance evaluation, model intercomparison, and visibility assessment will be 

provided. As shown in Appendix A, the interim reports for many of the subtasks will consist of 

PowerPoint presentations and/or results posted to the project website. For the major tasks, however, 

we do intend to document the work activities in more formal reports. Such planned reports for the 

base effort include: 

• Phase II Modeling Protocol (this document) 

• Phase II Quality Assurance Project Plan (QAPP) 

• 2002 emissions modeling draft and final reports 

• Initial 2002 CMAQ modeling summary report 

• Revised 2002 CMAQ modeling summary report 

• Future-year visibility projections modeling report 

• Draft and final Executive Summary report 

• Optional Journal Article(s) 

In addition, copies of all scripts, reports and documentation provided to the VISTAS TAWG 

will be maintained on the project website. A final report summarizing all aspects of the project will 

also be provided. The list of deliverables for this project is described in a summary of the Phase II 

scope of work provided in Appendix A. 

 Reporting on each task in Phase II will consist of documentation of the data sources, 

methods, results, and findings. Individual task deliverables shall reflect any changes and revisions 

that occur over duration of Phase II. At the completion of Phase II, a draft final Executive 

Summary report will be prepared that details, documents and summarizes the results. This 

documentation will conform to the recommendations set forth in EPA’s "Draft Guidance for 

Demonstrating Attainment of Air Quality Goals for PM2.5 and Regional Haze (Jan ‘01)” and any 

subsequent versions. The final Executive Summary report will contain: (a) an executive summary 

abstract that provides a brief overview and summary of the modeling effort, emissions and air 

quality models used, model configuration, model performance evaluation overview and results, and 

rationale for the selected configuration, (b) technical detail covering all relevant aspects of the Phase 

II emissions and PM grid modeling, and (c) a discussion on data accessibility and availability for 

review by the public. The report will be provided in electronic form, (e.g., Word 2000 and pdf 

formats) and shall be submitted to the VISTAS Project Officer and also posted on the project web 

site. 

TN Regional Haze SIP Appendix C.1 - 165
 May 2004 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\Chapter 10.doc 10-2 

10.2 Procedures for Updating Modeling Protocol and QA Plan

One of the underlying realizations stemming from the Phase I activities was the awareness 

that the science of ‘One-Atmosphere’ PM/regional haze modeling is advancing very rapidly. Part of 

this stems from the parallel activities being carried out by the RPOs; some if it is due to other 

ongoing 8-hr ozone and PM modeling studies being performed by various states. In addition, EPA 

is in the process of revising its PM2.5 and regional haze guidance documents. Collectively, it is quite 

likely that there will be new opportunities to strengthen the modeling algorithms, input data sets, and 

evaluation procedures throughout the duration of Phase II. Moreover, the Phase II emissions and air 

quality modeling activities involve the collaboration with the TAWG as well as other VISTAS 

contractors who will provide the required emissions and meteorological inputs to the SMOKE and 

CMAQ models. Given the ongoing model refinement activities and the need for strong coordination 

with other VISTAS contractors, it may be necessary to modify certain aspects of this modeling 

protocol. In this event, modification will be made in consultation with the VISTAS TAWG and the 

revised protocol will be submitted to the workgroup for approval. 

TN Regional Haze SIP Appendix C.1 - 166
 May 2003 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\References.doc R-1 

REFERENCES 

Alpine Geophysics, 2003a. “VISTAS Emissions and Air Quality Modeling – Phase I Subtask 3c1 Report: MM5 Meteorology QA/QC Summary Report for Episode 1: 2-20 January 

2002.” Alpine Geophysics LLC, ENVIRON International Corporation and University of 

California at Riverside. Ft. Wright, Kentucky (available at: 

http://pah.cert.ucr.edu/vistas/docs.shtml). September 10. 

Alpine Geophysics, 2003b. “VISTAS Emissions and Air Quality Modeling – Phase I Subtask 3c1 Report: MM5 Meteorology QA/QC Summary Report for Episode 1: 13-22 July 1999.” 

Alpine Geophysics LLC, ENVIRON International Corporation and University of 

California at Riverside. Ft. Wright, Kentucky (available at: 

http://pah.cert.ucr.edu/vistas/docs.shtml). September 15. 

Alpine Geophysics, 2003c. “VISTAS Emissions and Air Quality Modeling – Phase I Subtask 3c1 Report: MM5 Meteorology QA/QC Summary Report for Episode 1: 11-27 July 2001.” 

Alpine Geophysics LLC, ENVIRON International Corporation and University of 

California at Riverside. Ft. Wright, Kentucky (available at: 

http://pah.cert.ucr.edu/vistas/docs.shtml). September 16. 

Anthes, R.A. 1977. “A Cumulus Parameterization Scheme Utilizing a One-Dimensional Cloud 

Model”, Mon. Wea. Rev., Vol. 105, pp. 270-286. 

Anthes, R.A. and T.T. Warner. 1978. “The Development of Mesoscale Models Suitable for Air 

Pollution and Other Mesometeorological Studies”, Mon. Wea. Rev., Vol. 106, pp, 1045-

1078. 

Arnold, J.R.; R.L. Dennis, and G.S. Tonnesen. 1998. Advanced techniques for evaluating 

Eulerian air quality models: background and methodology. In: Preprints of the 10th Joint 

Conference on the Applications of Air Pollution Meteorology with the Air & Waste 

Management Association, January 11-16, 1998, Phoenix, Arizona. American 

Meteorological Society, Boston, Massachusetts, paper no. 1.1, pp. 1-5. 

Arnold, J.R., R.L. Dennis, and G.S. Tonnesen. 2003. “Diagnostic Evaluation of Numerical Air 

Quality Models with Specialized Ambient Observations: Testing the Community 

Multiscale Air Quality Modeling System (CMAQ) at Selected SOS 95 Ground Sites”, 

Atmos. Environ., Vol. 37, pp. 1185-1198. 

ARS. 2002a. Analysis of IMPROVE and SEARCH Speciated PM2.5 Data. Air Resource Specialists, 

Inc. Report to VISTAS, August 28, 2002. http://www.vistas-sesarm.org. 

ARS. 2002b. Analysis of Alternative Methods. Air Resource Specialists, Inc. Report to VISTAS, 

August 28, 2002. http://www.vistas-sesarm.org.

ARS. 2002c. Review of Selected Factors Responsible for High and Low Extinction. Air Resource 

Specialists, Inc. Report to VISTAS, October 14, 2002. http://www.vistas-sesarm.org.

TN Regional Haze SIP Appendix C.1 - 167
 May 2003 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\References.doc R-2 

Barchet, W.R., and R.L. Dennis. 1990. “ANAPAP Model Evaluation, Volume 1: Protocol”, 

prepared for the U.S. Environmental Protection Agency, prepared by Battelle Pacific 

Northwest Laboratories, Richland, WA. 

Boylan, J. W., 2004. “Calculationg Statistics: Concentration Related Performance Goals”, paper 

presented at the EPA PM Model Performance Workshop, Chapel Hill, NC. 11 February. 

Brewer, P., S. Holman and J. Hornback. 2003. “Vistas Analyses of Regional Haze in the 

Southeastern United States.” Presented at Annual Air and Waste Management Association 

Meeting and Exhibition, San Diego, CA. June. 

Byun, D.W., and J.K.S. Ching. 1999. “Science Algorithms of the EPA Models-3 Community 

Multiscale Air Quality (CMAQ) Modeling System”, EPA/600/R-99/030. 

Carter, W.P.L. 1999. Documentation of the SAPRC-99 Chemical Mechanism for VOC 

Reactivity Assessment, Draft report to the California Air Resources Board, Contracts 92-

329 and 95-308, 9/13/99. 

Chang, J.S., R.A. Brost, I.S.A. Isakson, S. Madronic, P. Middleton, W.R. Stockwell and C.J. 

Walcek. 1987. A three-dimensional Eulerian acid deposition model: Physical concepts 

and formulation. J. Geophys. Res., 92, 14681-14700. 

Chang, J. S., et al. 1997. “The SARMAP Air Quality Model", prepared for the California Air 

Resources Board, prepared by the State University of New York, Albany, New York. 

Coella, P., and P. L., Woodward. 1984. “The Piecewise Parabolic Method (PPM) for GasDynamical Simulations”, J. Comp. Physics, Vol. 54, pp. 174-201. 

Dennis, R. L., et al. 1990. "Evaluation of Regional Acid Deposition Models", State-ofScience/Technology Report No. 5, National Acid Precipitation Assessment Program, 

Washington, D.C. 

Dentener, F. J., and P. J. Crutzen. 1993. Reaction of N2O5 on tropospheric aerosols: Impact on 

global distributions of NOx, O3, and OH, J. Geophys. Res., 98, 7149-7163. 

Donahue, N.M., M.K. Dubey, R. Mohrschladt, K.L. Demerjian, and J.G. Anderson. 1997. 

High-pressure flow study of the reactions OH+NOx → HONOx: Errors in the falloff 

region. J. Geophys. Res. 102, 6159-6168. 

Dudhia, J. 1993. "A Non-hydrostatic Version of the Penn State/NCAR Mesoscale Model: 

Validation Tests and Simulation of an Atlantic Cyclone and Cold Front", Mon. Wea. Rev., 

Vol. 121. pp. 1493-1513. 

Edgerton, E. S., Sources of PM2.5 Carbon in the Southeast US, presentation at the MARAMA 

Science Meeting, Baltimore, MD, January 22, 2003. http://www.marama.org.

TN Regional Haze SIP Appendix C.1 - 168
 May 2003 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\References.doc R-3 

ENVIRON, Alpine Geophysics, UC Riverside and UC Davis. 2003a. “VISTAS Emissions and 

Air Quality Modeling – Phase I Task 4a/b Report: Review of Model Sensitivity 

Simulations and Recommendations of Initial CMAQ Model Configurations and Sensitivity 

Tests.” Revised Draft July 25. 

ENVIRON, Alpine Geophysics , UC Riverside and UC Davis. 2003b. “VISTAS Emissions and 

Air Quality Modeling – Phase I Task 3 Report: Review and Assessment of Available 

Ambient Air Quality Data to Support Modeling and Modeling Performance Evaluation for 

the Three VISTAS Phase I Episodes.” Revised Draft July 22. 

ENVIRON. 2002. “User’s Guide Comprehensive Air Quality Model With Extensions (CAMx) 

Version 3.10.” ENVIRON International Corporation, Novato, California (available at 

www.camx.com) April. 

ENVIRON. 2003a. “Development of an Advanced Photochemical Model for Particulate Matter: 

PMCAMx.” ENVIRON International Corporation, Novato, CA. Prepared for 

Coordinating Research Council, Inc. Project A-30 (available at www.crcao.com). 

ENVIRON, 2003b. “VISTAS Emissions and Air Quality Modeling – Phase I Task 2 Report: 

Recommended Model Configurations and Evaluation Methodology for Phase I Modeling.” 

 Prepared by ENVIRON International Corporation, Alpine Geophysics, LLC and 

University of California at Riverside. Novato, California. (available at: 

http://pah.cert.ucr.edu/vistas/docs.shtml). August 4. 

ENVIRON, 2003c. “VISTAS Emissions and Air Quality Modeling – Phase I Task 3 Report: 

Review and Assessment of Availablre Ambient Air Quality Modeling and Performance 

Evaluation for the Three VISTAS Phase I Episoides.” Prepared by ENVIRON 

International Corporation, Alpine Geophysics, LLC and University of California at 

Riverside. Novato, California. (available at: http://pah.cert.ucr.edu/vistas/docs.shtml). 

July 22. 

ENVIRON, 2003d. “VISTAS Emissions and Air Quality Modeling – Phase I Task 4a/b Report: 

Review of Model Sensitivity Simulations and Recommendations of Initial CMAQ Model 

Configurations and Sensitivity Tests.” Prepared by ENVIRON International Corporation, 

Alpine Geophysics, LLC and University of California at Riverside. Novato, California. 

(available at: http://pah.cert.ucr.edu/vistas/docs.shtml). July 25. 

EPA, 1991. "Guidance for Regulatory Application of the Urban Airshed Model (UAM), "Office 

of Air Quality Planning and Standards, U.S. Environmental Protection Agency, Research 

Triangle Park, N.C. 

EPA, 1999. “Draft Guidance on the Use of Models and Other Analyses in Attainment 

Demonstrations for the 8-hr Ozone NAAQS”. Draft (May 1999), U.S. Environmental 

Protection Agency, Office of Air Quality Planning and Standards, Research Triangle 

Park, N.C 

TN Regional Haze SIP Appendix C.1 - 169
 May 2003 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\References.doc R-4 

EPA. 2000. “Guidance for Demonstrating Attainment of Air Quality Goals for PM2.5 and 

Regional Haze”, Preliminary Draft Report, U.S. Environmental Protection Agency, 

Research Triangle Park, NC. 

EPA. 2001. “Guidance for Demonstrating Attainment of Air Quality Goals for PM2.5 and 

Regional Haze”, Draft Report, U.S. Environmental Protection Agency, Research Triangle 

Park, NC. 

EPA. 2003a. “Guidance for Tracking Progress Under the Regional Haze Rule”, U.S. 

Environmental Protection Agency, Research Triangle Park, NC. 

EPA. 2003b. “Guidance for Estimating Natural Visibility Conditions Under the Regional Haze 

Rule”, U.S. Environmental Protection Agency, Research Triangle Park, NC.

Fahey, K.M., and S.N. Pandis. 2001. Optimizing model performance: variable size reduction in 

cloud chemistry modeling. Atmos. Environ. Vol. 35, pp. 4471-4478. 

FLAG. 2000. Federal Land Manager’s Air Quality Related Values Workgroup (FLAG) Phase I 

Report. (http://www2.nature.nps.gov/ard/flagfiee/AL.doc). 

Gery, M. W., G.Z. Whitten, J.P. Killus, and M.C. Dodge. 1989. A photochemical mechanism 

for urban and regional-scale computer modeling. J. Geophys. Res. 94, 12925-12956. 

Gipson, G, and J. Young. 1999. Process Analysis, in Science Algorithms of the EPA Models-3 

Community Multiscale Air Quality (CMAQ) Modeling System, Chapter 16, U.S. EPA, 

Office of Research and Development, Washington, DC 20460. 

Grell, G. A., J. Dudhia, and D. R. Stauffer. 1994. "A Description of the Fifth Generation Penn 

State/NCAR Mesoscale Model (MM5). NCAR Tech. Note, NCAR TN-398-STR, 138 pp. 

Hanna, S.R., A.G. Russell, J. Wilkinson, and J. Vukovich. 2002. Review of BEIS3 

Formulation and Consequences Relative to Air Quality Standards: Estimation of 

Uncertainties in BEIS3 Emissions Outputs. EPRI Technical Report 1005159, EPRI, 3412 

Hillview Ave., Palo Alto, CA 94304. 

Hanna, S.R., A.G. Russell, J. Wilkinson, and J. Vukovich. 2003. Review of BEIS3 Formulation 

and Consequences Relative to Air Quality Standards: Estimation of Effects of 

Uncertainties in BEIS3 Emissions on Uncertainties in Ozone Predictions by Chemical 

Transport Models, EPRI, Palo Alto, CA: 2002 (report is currently under review by 

EPRI). 

Hanna, S. R. 1994. "Mesoscale Meteorological Model Evaluation Techniques with Emphasis on 

Needs of Air Quality Models", in Mesoscale Modeling of the Atmosphere, R. A. Pielke and 

R. P. Pierce (Eds.), American Meteorological Society, Boston, MA.

Hermann, H. et al., 2000. manuscript submitted to J. Geophys. Res.

TN Regional Haze SIP Appendix C.1 - 170
 May 2003 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\References.doc R-5 

IPCC. 2001. "Climate Change 2001: The Scientific Basis -- Contribution of Working Group I to 

the Third Assessment Report of IPCC." Intergovernmental Panel on Climate Change. 

ISBN: 0521014956. 

Jacob, D. J. 1999. Heterogeneous chemistry and tropospheric ozone. Atmos. Environ., special 

issue, 1999 NARSTO Assessment. 

Khasibatla, P. et al. 1997. Impact of inert organic nitrate formation on ground-level ozone in a 

regional air quality model using the Carbon Bond Mechanism 4. Geophys. Res. Letters, 

24, 3205-3208. 

Liu, G., C. Hogrefe, and S.T. Rao. 2003. “Evaluating the Performance of Regional-Scale 

Meteorological Models: Effects of Clouds Simulation on Temperature Prediction”, Atmos. 

Environ., Vol. 36, pp. 1691-1705. 

Matthias-Maser, S. and R. Jaenicke. 1995. "The size distribution of primary biological aerosol 

particles with radii >0.2 mm in an urban-rural influenced region." Atmos. Res., 39, 279-

286. 

McNally, D.E., and T.W. Tesche. 1994. "MAPS2.3 User's Guide", Alpine Geophysics, LLC, 

Golden, CO. 

McNally, D. E. 1997. “Development of Methodology for Mapping MM5 Fields onto Arbitrary 

Eulerian Photochemical Air Quality Simulation Models (PAQSM)”, Alpine Geophysics, 

LLC, Arvada, CO. 

Odman, M. T.; Boylan, J.W.; Wilkinson, J.G.; and Russell, A.G. SAMI Air Quality Modeling Final 

Report. Southern Appalachian Mountains Initiative. 2002. http://www.saminet.org.

Olerud, D. 2003a. “Evaluation Methodologies for Meteorological Modeling in Support of 

VISTAS (Visibility Improvement – State and Tribal Association): Task 1 Deliverables” 

(Draft) Baron Advanced Meteorological Systems, LLC, Research Triangle Park, NC. 

(available at: http://www.baronams.com/projects/VISTAS/#reports). April 16. 

Olerud, D and A. Sims. 2004. “Episode Selection from a Meteorological Perspective for 

VISTAS (Visibility Improvement – State and Tribal Association): Task 3a Deliverable.” 

Baron Advanced Meteorological Systems, LLC, Research Triangle Park, NC. (available 

at: http://www.baronams.com/projects/VISTAS/#reports). February 9. 

Olerud, D. 2003c. “Protocol for Annual MM5 Modeling of VISTAS (Visibility Improvement – 

State and Tribal Association): Task 7c Deliverable.” Baron Advanced Meteorological 

Systems, LLC, Research Triangle Park, NC. (available at: 

http://www.baronams.com/projects/VISTAS/#reports). March 27. 

Olerud, D. 2003d. “MM5 Sensitivity Modeling in Support of VISTAS (Visibility Improvement 

– State and Tribal Association): Task 2e Deliverable” (Draft) Baron Advanced 

Meteorological Systems, LLC, Research Triangle Park, NC. 4 December 2003 

TN Regional Haze SIP Appendix C.1 - 171
 May 2003 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\References.doc R-6 

Olerud, D. 2003e. “VISTAS Meteorological Modeling Sensitivity Results”, PowerPoint 

presentation at the 29-30 September 2003 VISTAS Technical Review Meeting (Research 

Triangle Park, NC), prepared by Baron Advanced Meteorological Systems, LLC, 

Research Triangle Park, NC. 29 September 2003 

Pielke, R. A., and M. Uliasz. 1998. “Use of Meteorological Models as Input to Regional and 

Mesoscale Air Quality Models C Limitations and Strengths”, Atmos. Environ. Vol. 32, 

No. 8, pp. 1455-1466. 

Pilinis, C. and J. H. Seinfeld. 1987. “Continued Development of a General Equilibrium Model 

for Inorganic Multicomponent Atmospheric Aerosols”, Atmos. Environ. Vol.21, pp.2453. 

Pilinis, C., K.P. Capaldo, A. Nenes, and S.N. Pandis. 2000. “MADM - A new multicomponent 

aerosol dynamics model.” Aerosol Sci. Tech., Vol. 32(5), pp. 482-502. 2000. 

Russell, A.G., and R.L. Dennis. 2000. “NARSTO Critical Review of Photochemical Models 

and Modeling”, Atmos. Environ., Vol. 34, No. 12-14, pp. 2283-2324. 

Schere, K.L., and R.A. Wayland. 1989. "EPA Regional Oxidant Model (ROM2.0): Evaluation 

on 1980 NEROS Data Bases", EPA/600/3-80/057, Atmospheric Research and Exposure 

Assessment Laboratory, Research Triangle Park, NC. 

Seaman, N.L. 1995. “Status of Meteorological Pre-Processors for Air Quality Modeling”, 

International Conf. On Particulate Matter, Air and Waste Mgt. Assn., Pittsburgh, PA. 

Seaman, N.L. 1995. “Status of Meteorological Pre-Processors for Air Quality Modeling”, 

International Conf. On Particulate Matter, Air and Waste Mgt. Assn., Pittsburgh, PA. 

Seaman, N.L. 1996. “Study of Meteorological Variables Needed in Air-Quality Modeling’, 

Annual Progress Report prepared for the Coordinating Research Council (CRC), Interim 

Report, Project A-11, prepared by the Department of Meteorology, Penn State University, 

State College, PA. 

Seaman, N.L. 2000. “Meteorological Modeling for Air Quality Assessments”, Atmos. Environ., 

Vol. 34, No. 12-14, 2231-2260. 

Seaman, N. L., and D. R. Stauffer. 1996. "SARMAP Meteorological Model Final Report", 

prepared for the San Joaquin Valleywide Air Pollution Study Agency, prepared by the 

Department of Meteorology, Pennsylvania State University, University Park, PA. 

Seaman, N.L., and S.A. Michelson. 1998. “Mesoscale Meteorological Structure of a HighOzone Episode During the 1995 NARSTO-Northeast Study”, J. Appl. Meteo., 

(submitted). 

Seaman, N.L., D.R. Stauffer, and T.W. Tesche. 1992. "The SARMAP Meteorological Model: 

A Four-Dimensional Data Assimilation Technique Used to Simulate Mesobeta-Scale 

Meteorology During a High-Ozone Episode in California", International Specialty 

Conference on Tropospheric Ozone Nonattainment and Design Value Issues, U.S. 

TN Regional Haze SIP Appendix C.1 - 172
 May 2003 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\References.doc R-7 

EPA/AWMA, 27-30 October, Boston, MA. 

Seaman, N.L., D.R. Stauffer, and L.M. Lario. 1995. "A MultiScale Four-Dimensional Data 

Assimilation System Applied to the San Joaquin Valley During SARMAP. Part I: 

Modeling Design and Basic Performance Characteristics", J. Appl. Meteo., Vol. 34, pp. 

1739-1761. 

Seaman, N.L., D.R. Stauffer, and D.E. McNally. 1996a. "Application of the MM5-FDDA 

Meteorological Model to the Southern California SCAQS-1997 Domain: Preliminary Test 

Using the SCAQS August 1987 Case", Ninth Joint Conference on Applications of Air 

Pollution Meteorology, American Meteorological Society, 28 January-2 February, 

Atlanta, GA. 

Seinfeld, J.H. and S.N. Pandis. 1998. Atmospheric Chemistry and Physics: From Air Pollution 

to Global Change. Wiley, New York. 

Seigneur, C., et al. 2000. “Guidance for the Performance Evaluation of Three-Dimensional Air 

Quality Modeling Systems for Particulate Matter and Visibility”, J. Air & Wast Manage. 

Assoc. Vol. 50, pp. 588-599. 

Simoneit, B.R.T., J.N. Cardoso and N. Robinson. 1990. "An assessment of the origin and 

composition of higher molecular weight organic matter in aerosols over Amazonia." 

Chemosphere, 21, 1285-1301. 

Stockwell, W.R., P. Middleton, J.S. Chang, and X. Tang. 1990. The second generation 

Regional Acid Deposition Model chemical mechanism for regional air quality modeling. J. 

Geophys. Res., 95, 16,343-16,367. 

Stockwell, W.R., P. Middleton, J.S. Chang, and X. Tang, X. 1997. The second generation 

Regional Atmospheric Chemistry Mechanism regional air quality modeling. J. Geophys. 

Res., 102. 

Strader, R., F.W. Lurmann and S.N. Pandis. 1999. “Evaluation of secondary organic aerosol 

formation in winter.” Atmos. Environ. Vol. 33, pp. 4849-4864. 

Tang, Y. 2002, “A Case Study of Nesting Simulation for the Southern Oxidants Study 1999 at 

Nashville”, Atmos. Environ., Vol. 36, pp. 1691-1705. 

Tesche, T. W., et al., 1990. "Improved Treatment of Procedures for Evaluating Photochemical 

Models", prepared for the California Air Resources Board, prepared by Alpine 

Geophysics, Crested Butte, CO. Contract No. A832-103. 

Tesche, T.W., and D.E. McNally. 1993a. “Operational Evaluation of the CAL-RAMS 

Meteorological Model for LMOS Episode 1: 26-28 June 1991", prepared for the Lake 

Michigan Air Directors Consortium, prepared by Alpine Geophysics, LLC, Crested Butte, 

CO. 

TN Regional Haze SIP Appendix C.1 - 173
 May 2003 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\References.doc R-8 

Tesche, T.W., and D.E. McNally. 1993b. "Operational Evaluation of the CAL-RAMS 

Meteorological Model for LMOS Episode 2: 17-19 July 1991", prepared for the Lake 

Michigan Air Directors Consortium, prepared by Alpine Geophysics, LLC, Crested Butte, 

CO. 

Tesche, T.W., and D.E. McNally. 1993c. "Operational Evaluation of the CAL-RAMS 

Meteorological Model for LMOS Episode 3: 25-26 August 1991", prepared for the Lake 

Michigan Air Directors Consortium, prepared by Alpine Geophysics, LLC, Crested Butte, 

CO. 

Tesche, T.W., and D.E. McNally. 1993d. "Operational Evaluation of the CAL-RAMS 

Meteorological Model for LMOS Episode 4: 20-21 June 1991", prepared for the Lake 

Michigan Air Directors Consortium, prepared by Alpine Geophysics, LLC, Crested Butte, 

CO. 

Tesche, T.W. and D.E. McNally. 1996a. “Superregional Ozone Modeling and Analysis Study 

C Phase I: Work Element 3: Assessment of the OTAG Data Sets -- Task 2 Technical 

Memorandum: Review of the OTAG Meteorological Inputs and Outputs”, prepared for the 

Midwest Ozone Group, prepared by Alpine Geophysics, LLC, Ft. Wright, KY. 

Tesche, T.W. and D.E. McNally. 1996b. “Superregional Ozone Modeling and Analysis Study 

C Phase II: Work Element 5 Technical Report: Comparative Evaluation of the MM5 and 

RAMS Models for the July 1991 OTAG Episode”, prepared for the Midwest Ozone 

Group, prepared by Alpine Geophysics, LLC, Ft. Wright, KY. 

Tesche, T.W., and D.E. McNally. 1997a. “Superregional Ozone Modeling and Analysis Study 

C Final Report: Assessment of the Reliability of the OTAG Modeling System”, prepared 

for the Midwest Ozone Group, prepared by Alpine Geophysics, LLC, Ft. Wright, KY. 

Tesche, T.W., and D.E. McNally. 1997b. “Methodology for Climatological Analysis of MultiYear Aerometric Data to Support the Development of the Breton Conceptual Model and 

Field Program Design”, prepared for Walk Heydel Environmental, prepared by Alpine 

Geophysics, LLC, Ft. Wright, KY. 

Tesche, T.W., and D.E. McNally. 1997c. “The Use of the San Joaquin Valley Meteorological 

Model in Preparation of a Field Program in the South Coast Air Basin and Surrounding 

Regions of Southern California: Volume I: Final MM5 Evaluation for the 3-6 August 1990 

SARMAP Episode”, prepared for the California Air Resources Board, prepared by Alpine 

Geophysics, LLC, Ft. Wright, KY. 

Tesche, T.W. and D.E. McNally. 1999. “Comparative Evaluation of the MM5 and RAMS3c 

Prognostic Meteorological Models Over the Midwestern U.S. for Two 1999 LMOS 

Intensive Measurement Episodes”, prepared for the Coordinating Research Council, 

Project A-25 draft Final Report, prepared by Alpine Geophysics, LLC, Ft. Wright, KY. 

Tesche, T.W., and D.E. McNally. 2001. “Evaluation of CAMx and Models-3/CMAQ Over the 

Lower Lake Michigan Region with Inputs from the RAMS3c and MM5 Models”, 

TN Regional Haze SIP Appendix C.1 - 174
 May 2003 

G:\VISTAS Modeling\Task6 Modeling Protocol\Draft#3\References.doc R-9 

prepared for the Coordinating Research Council, prepared by Alpine Geophysics, LLC, 

Ft. Wright, KY. 

Tesche, T.W., and H.E. Jeffries. 2002. “Findings From BCCA-AG Science Studies”, prepared 

for the Business Coalition for Clean Air- Appeals Group, (2002), prepared by Alpine 

Geophysics, LLC, Ft. Wright, KY. 

Timin, B. 2002. “PM2.5 and Regional Haze Modeling Guidance”. Prepared by the U.S. 

EPA/OAQPS. April 24. 

Timin, B. 2004. “PM2.5 Model Performance Evaluation: Purpose and Goals”. Presented at PM 

Model Evaluation Workshop, Chapel Hill, NC. February. 

Tonnesen, G.S. 1999. Effects of uncertainty in the reaction of the hydroxyl radical with nitrogen 

dioxide on model simulated ozone control strategies. Atmos. Environ. 33:1587-1598. 

Tonnesen, G.S. and R.L. Dennis. 2000. Analysis of radical propagation efficiency to assess 

ozone sensitivity to hydrocarbons and NOx. Part 1: Local indicators of odd oxygen 

production sensitivity. J. Geophys. Res., 105, 9213–9225. 

Watson, J.G. and J.C. Chow. 1999. Reconciling Urban Fugitive Dust Emissions Inventory and 

Ambient Source Contribution Estimates: Summary of Current Knowledge and Needed 

Research. Document No. 61110.4D2, Desert Research Institute, Reno NV, September 3. 

Zaveri, R.A and L.K. Peters. 1999. A new lumped structure photochemical mechanism for 

large scale applications, J. Geophys. Res., 30,387-30,415 

Zheng,, M.; Cass, G.R.; Schauer, J.J.; Edgerton, E.S. Environ. Sci. Tech. 2002, 36, 2361-2371.

TN Regional Haze SIP Appendix C.1 - 175
 

Appendix A 

Summary of VISTAS Phase II 

Emissions and Air Quality Modeling 

Scope of Work (SOW) 

(Updated May 6, 2004)

TN Regional Haze SIP Appendix C.1 - 176
 

VISTAS Phase II 2004 Atmospheric Modeling Scope of Work 

ENVIRON/Alpine/UCR 

Draft Version#5: May 6, 2004 

INTRODUCTION 

ENVIRON International Corporation, Alpine Geophysics (Alpine) and the University of California 

at Riverside (UCR) propose to perform the VISTAS Phase II emissions and air quality modeling 

as described in the Scope of Work (SOW) given below. As was seen in the VISTAS Phase I 

modeling, the SOW will likely be refined and modified during the course of the project and the 

ENVIRON/Alpine/UCR team is flexible in achieving the ultimate objectives and requirements 

needed by VISTAS. In putting together the VISTAS Phase II SOW, the ENVIRON/Alpine/UCR 

team is dependent on receiving data and information from other participants. The following are 

our assumed dates for deliverables that we expect to receive key data sources in developing 

our SOW. 

• January 31, 2004 – Initial 2002 baseline emissions for VISTAS states delivered by 

VISTAS contractors. (Achieved February 9, 2004.) 

 

• January 31, 2004 – First 8 months of 2002 MM5 modeling results delivered by VISTAS 

contractors. (Achieved February 20, 2004) 

• March 30, 2004 – Last 4 months of 2002 MM5 modeling results. 

• April 15, 2004 -- Initial 2018 Future-Year VISTASA States emissions for one scenario 

and 2 of the Phase I episodes (Jan ’02 and Jul ’01) from VISTAS contractor. Second 

2018 scenario available April 30, 2004. 

• September 15, 2004 – Revised 2002 baseline emissions for VISTAS states. 

• September 15, 2004 – Revised 2002 base year using typical year EGU and fire 

inventory for VISTAS states. 

• September 15, 2004 – 2002 base year inventory and typical year EGU and fire inventory 

for non-VISTAS states, Mexico, Canada as available from non-VISTAS sources 

• October 15, 2004 – Revised Future-Year emissions for VISTAS states, three future year 

scenarios and the 2002 annual period. 

• October 31, 2004 – MM5 modeling results for ~60 episode days from 2003 (or other 

years as appropriate) for optional episode modeling task. 

Key deliverables for the modeling team are as follows: 

• Draft modeling protocol by March 5, 2004, final protocol 2 weeks after VISTAS review 

comments received. 

• Analysis of GEOS-CHEM 2001 hourly and monthly outputs for CMAQ initial and 

boundary conditions (IC/BC) and recommendation whether to use 2002 GEOS-CHEM 

TN Regional Haze SIP Appendix C.1 - 177
 

for IC/BC in CMAQ revised 2002 modeling in fall 2004 – due March 2004. 

• 2002 initial inventory for Non-VISTAS States, Mexico, Canada by March 15, 2004. 

• 2018 initial emissions estimates for Non-VISTAS States, Canada, Mexico and 2 Phase I 

episodes (Jan 2002 and Jul 2001) by April 15, 2004. 

• 2018 CMAQ-ready emissions from SMOKE for 2 Phase I episodes (July 2001 and Jan 

2002) for one scenario by May 15, 2004 and the other scenario by June 15, 2004. 

• 2002 CMAQ-ready emissions from SMOKE for 2002 annual period by May 31, 2004 

• Quality Assurance Project Plan (QAPP) for Phase II modeling (June 2004) 

• CMAQ model performance evaluation for annual 2002 runs using initial 2002 inventory 

by August 30, 2004 

• Collection of available 2002 revised inventory and typical year EGU and fire for nonVISTAS states, Canada, Mexico by September 15, 2004 

• Future year revised inventory for non-VISTAS states, Canada, Mexico by October 15, 

2004 for 3 scenarios 

• 2002 revised CMAQ-ready SMOKE outputs using revised 2002 inventories by October 

15, 2004 

• 2002 revised plus typical EGU and fire CMAQ-ready SMOKE outputs by October 31, 

2004 

• 2002 revised CMAQ annual run and model performance evaluation by December 1, 

2004 

• 2002 revised plus typical EGU and fire CMAQ annual run and model performance 

evaluation by December 31, 2004 

• Future year CMAQ-ready SMOKE outputs using revised inventories for first scenario by 

November 15, 2004, for second and third by December 15, 2004 

• Future year revised emissions CMAQ annual modeling runs for 3 scenarios due by 

January 31, 2005. 

• Optional task CMAQ-ready SMOKE outputs for 2003 episodes after January 1, 2005 

Table 1 summarizes the task structure, task leader, schedule and deliverables for the VISTAS 

Phase II proposed SOW. Figure 1 displays the current proposed schedule for the VISTAS 

Phase II modeling based on the above assumptions on data deliverables to the Modeling Team. 

This schedule will likely be revised and updated as more refined delivery dates are developed.

TN Regional Haze SIP Appendix C.1 - 178
 

Figure 1. Current proposed schedule for the VISTAS Phase II modeling (May 6, 2004).

TN Regional Haze SIP Appendix C.1 - 179
Table 1. Summary of proposed VISTAS Phase II Tasks, Task Managers, Deliverables and Costs (shaded tasks are currently not funded, May 6, 

2004). 

Work Element Manager Deliverables (Date) Task 1: Management Ralph Morris • Monthly Conference Calls (2004) 

• 3 Meetings with 2 People in Southeast US (2004) • First Draft Modeling Protocol (03/05/04) • Second Draft Modeling Protocol (03/30/04) • Updated Modeling Protocol (as needed) • QAPP (June 2004) • Draft Final Executive Summary Report (01/15/05) • Final Executive Summary Report (01/31/05) 

Task 2: IC/BCs Analysis Gail Tonnesen • Initial analysis of the need for 3-hr vs. seasonal BCs for CMAQ 

modeling (02/12/04). 

• 2002 IC/BC recommendations PowerPoint presentation, which 

includes monthly average vs. 3-hr BC CMAQ comparisons for July 2001 episode (March 2004) 

Subtask3a: Air Quality Gatekeeper 

Gail Tonnesen • 2002 Air Quality MPE database, QA/QC and PowerPoint 

presentation (04/30/04) 

• 2002 initial CMAQ IC/BC inputs (04/30/04) • Review of 2002 special study (e.g., aircraft data) data 

availability that would be used in Optional Task 14 Enhanced model evaluation (05/30/04) 

Subtask 3b: Meteorological Gatekeeper 

Dennis McNally • First eight months 2002 MM5 QA/QC Gatekeeper PowerPoint 

presentation (03/26/04) 

• CMAQ-ready meteorological inputs for first six months of 2002 

(03/31/04) 

• Last four months 2002 MM5 QA/QC Gatekeeper PowerPoint 

presentation (04/15/04) 

• CMAQ-ready meteorological inputs for first six months of 2002 

(04/15/04) 

Subtask 3c: Emissions Gatekeeper 

Gregory Stella • Recommendation for Mexico and Canadian Emissions 

(02/29/04) 

• VISTAS States emissions QA/QC Gatekeeper PowerPoint 

presentation on initial 2002 emissions (03/31/04) 

• VISTAS States emissions QA/QC Gatekee

per PowerPoint 

TN Regional Haze SIP Appendix C.1 - 180
Work Element Manager Deliverables (Date) 

presentation on initial future year emissions (5/15/04) 

• VISTAS States emissions QA/QC Gatekeeper PowerPoint 

presentation on revised 2002 and future year emissions (10/15/04) 

• Initial 2002 Base Case emissions inventory for non-VISTAS 

states (03/15/04) 

• Revised 2002 emissions inventory for non-VISTAS States 

(09/15/04) 

• Initial future-year inventory for non-VISTAS states (05/15/04) • Three future-year emission inventories for non-VISTAS states 

(09/15/04) 

Subtask 3d: Emissions Modeling 

Cyndi Loomis • 2002 CMAQ-ready emission input using initial emissions for all 

of 2002 (05/30/04) 

• Initial 2002 emissions PowerPoint presentation (05/30/04) • 2002 CMAQ-ready emission inputs using revised and typical 

emissions (10/15/04) 

• Revised and typical 2002 emissions modeling PowerPoint 

presentation (10/31/04) 

• Future-year CMAQ-ready emissions for two scenarios using 

revised emissions (11/15/04 and 12/15/04) 

• Draft emissions summary report (11/30/04) • Final Phase II emissions modeling report (12/31/04) 

Subtask 3e: Future-Year Emissions Modeling for 2 VISTAS Phase I Episodes 

Cyndi Loomis • CMAQ-ready initial future-year emission inputs for two VISTAS 

Phase I episodes (05/15/04) 

• CMAQ-ready future-year emission inputs for second scenario 

and two VISTAS Phase I episodes (06/15/04) 

• Summary PowerPoint presentation on VISTAS Phase I episode 

future-year emissions (06/15/04) 

Subtask 3f: Interim Future-Year Modeling for 2002 

Cyndi Loomis • CMAQ-ready future-year emissions for interim future-year and 

2002 meteorology (12/15/04) 

• Interim future-year emissions summary PowerPoint presentation 

(12/30/04) 

Subtask 4a: Initial 2002 CMAQ Modeling and MPE 

T.W. Tesche • PowerPoint presentation of the initial CMAQ model performance 

for 2002 annual period (06/30/04) 

TN Regional Haze SIP Appendix C.1 - 181
Work Element Manager Deliverables (Date) 

• Summary report of CMAQ initial 2002 model performance 

evaluation and sensitivity analysis including recommendations for “best” configuration (08/30/04) 

Subtask 4b: Revised 2002 CMAQ Modeling and MPE 

Ralph Morris • Power Point presentations on the revised CMAQ base case 

simulation and sensitivity analysis (11/30/04) 

• Draft Report of revised CMAQ 2002 Base Case simulation and 

MPE (12/15/04) 

• Report on model sensitivity tests (01/15/05) 

Subtask 4c: Typical year 2002 CMAQ Modeling 

Ralph Morris • PowerPoint presentation on 2002 typical year CMAQ modeling 

(12/31/04) 

Subtask 4d: Future-Year CMAQ Modeling for Two Scenarios 

Gail Tonnesen • Draft Report on future-year base case modeling for two 

scenarios using 2002 and episodic databases (01/31/05) 

Subtask 4e. Interim Future-Year CMAQ Modeling 

Gail Tonnesen • PowerPoint presentation of Interim Future-Year modeling 

results (01/31/05) 

Optional Task 5: 2003 Episodes 

• Starts after completion of future-year emissions modeling (starts 

~01/01/05) 

Subtask 5a: Episodic Emissions Gatekeeper and Modeling 

Cyndi Loomis • Emissions Gatekeeper summary PowerPoint presentation for 

revised episodic modeling emissions (2005) 

• Revised Episodic CMAQ-ready revised 2003 Base Case and 

Typical Year emission inputs and summary (2005) 

• Future-year CMAQ-ready episodic inputs for three scenarios 

and summary report on the episodic emissions modeling (2005) 

Subtask 5b: Episodic Meteorological Gatekeeper 

Dennis McNally • CMAQ MCIP meteorological modeling inputs (2005) 

• MM5 Gatekeeper evaluation and QA/QC PowerPoint 

presentation (2005) 

Subtask 5c: Episodic AQ Data Gatekeeper 

Gail Tonnesen • MPE air quality data database for episodic modeling periods 

(2005) 

• AQ data summary presentation for episode periods (2005) • CMAQ IC/BC inputs for “2003” episode periods (2005) 

Subtask 5d: Episodic Emissions for non-VISTAS states 

Greg Stella • Revised “2003” Base Case, Typical Year and Future-Year 

emissions for episodic modeling (2005) 

TN Regional Haze SIP Appendix C.1 - 182
Work Element Manager Deliverables (Date) 

Subtask 5e. Episodic Base Case and Typical Year CMAQ Modeling 

Gail Tonnesen • CMAQ Base Case and MPE and Typical Year modeling report 

for episode periods (2005) 

Subtask 5f: Episodic Future-Year Modeling for Three scenarios 

Ralph Morris • Summary Report on future-year base case modeling for three 

scenarios using the episodic databases (2005) 

Subtask 6a: VISTAS Modeling Website 

Gail Tonnesen • VISTAS modeling website will be maintained and updated 

continuously during 2004. 

Subtask 6b: Data Transfer, Storage and Retrieval 

Gail Tonnesen • Performed continuously during 2004 

o Archive (2004) o Data transfer drive (12/31/04) 

Optional Task 7 Evaluate Relative Reduction Response using CB-IV vs. SAPRC for 2 Phase I episodes 

Gail Tonnesen • PowerPoint presentation documenting the emissions reduction 

sensitivity tests for the January 2002 and July 2001 episodes using CB4 and SAPRC99 chemistry (06/30/04) 

Optional Task 8: Emissions Sensitivity Modeling using Phase I episodes 

Ralph Morris • PowerPoint presentation and model output files for the emission 

reduction sensitivity simulations using winter and summer Phase I episodes (TBD, ~09/30/04) 

Optional Task 9: Additional Sensitivity using VISTAS Phase I Episodes 

T.W. Tesche • Summary report of additional Phase I sensitivity tests (TBD, 

09/30/04???) 

Optional Task 10: Journal Paper on VISTAS Phase I Modeling 

Gail Tonnesen • Draft Journal paper (05/31/04) 

• Draft final Journal paper submitted to Journal; (06/30/04) • Final Manuscript (TBD, 10/31/04???) 

Optional Task 11: Corroborate Analysis using Alternative Models 

Ralph Morris • Summary presentation of corroborative modeling analysis using 

alternative models (01/31/05) 

Optional Task 12: Natural Conditions Modeling 

Gail Tonnesen • Natural conditions summary presentation (TBD???) 

Optional Task 13: 2002 GEOS-CHEM BCs 

Gail Tonnesen • 2002 CMAQ BCs based on 2002 GEOS-CHEM simulation. 

TN Regional Haze SIP Appendix C.1 - 183
Work Element Manager Deliverables (Date) 

Optional Task 14: Enhanced Model Performance Evaluation 

T.W. Tesche • Summary presentation of the enhanced model performance 

evaluation using initial database (08/31/04) 

• Summary presentation of the enhanced model performance 

evaluation using the revised base case results (11/30/04) 

Optional Task 14: Contingency 

Ralph Morris • To be determined. 

TN Regional Haze SIP Appendix C.1 - 184