GUIDELINE FOR REGULATORY APPLICATION OF THE URBAN AIRSHED MODEL

U.S. ENVIRONMENTAL PROTECTION AGENCY Office of Air Quality Planning and Standards Technical Support Division Source Receptor Analysis BranchResearch Triangle Park, NC 27711

July 1991DISCLAIMER

This report has been reviewed by the Office of Air Quality Planning and Standards, U.S. Environmental Protection Agency, and has been approved for publication. Any mention of trade names or commercial products is not intended to constitute endorsement or recommendation for use.CONTENTS

Page

DISCLAIMER . . . . . . . . . . . . . . . . . . . . . . . . . . ii

TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

ABBREVIATIONS. . . . . . . . . . . . . . . . . . . . . . . . .vii

ACKNOWLEDGEMENTS . . . . . . . . . . . . . . . . . . . . . . ix90

1 INTRODUCTION. . . . . . . . . . . . . . . . . . . . . . . 1

2 MODELING PROTOCOL . . . . . . . . . . . . . . . . . . . . 5

2.1 Protocol Development Process . . . . . . . . . . . . 6 2.2 Contents of Protocol Document. . . . . . . . . . . . 8

3 DOMAIN AND DATA BASE ISSUES . . . . . . . . . . . . . . . 11

3.1 Episode Selection. . . . . . . . . . . . . . . . . . 11 3.2 Size of the Modeling Domain. . . . . . . . . . . . . 15 3.3 Horizontal Grid Cell Size. . . . . . . . . . . . . . 16 3.4 Number of Vertical Layers. . . . . . . . . . . . . . 17 3.5 Meteorological Data. . . . . . . . . . . . . . . . . 19

3.5.1 Wind fields. . . . . . . . . . . . . . . . . 19 3.5.2 Data needs for wind field development . . . . . . . . . . . . . . . . 23 3.5.3 Mixing heights . . . . . . . . . . . . . . . 24 3.5.4 Clear-sky assumption for photolysis rate calculations . . . . . . . . . . . . . . . . 25

3.6 Air Quality. . . . . . . . . . . . . . . . . . . . . 26

3.6.1 Initial and boundary conditions. . . . . . . 26 3.6.2 Performance Evaluation Data. . . . . . . . . 31

3.7 Emissions. . . . . . . . . . . . . . . . . . . . . . 32

3.7.1 VOC speciation . . . . . . . . . . . . . . . 35 3.7.2 Spatial gridding of area sources . . . . . . 35 3.7.3 Mobile sources . . . . . . . . . . . . . . . 36 3.7.4 Episode-specific adjustments . . . . . . . . 373.7.5 Biogenic emissions . . . . . . . . . . . . . 38 3.7.6 Point-source and plume-rise cutoff levels. . 39 3.7.7 Consistency with national inventories. . . . 40

CONTENTS (Continued)

Page

4 DATA QUALITY ASSURANCE AND MODEL DIAGNOSTIC ANALYSES. . . 41

4.1 Step 1: Quality Assurance Testing of Component Fields. . . . . . . . . . . . . . . . . . . 43 4.2 Step 2: Diagnostic Testing of the Base Case Meteorological episodes . . . . . . . . . . 44 4.3 Step 3: Additional Base Meteorological Episode Sensitivity Testing . . . . . . . . . . . . 46

5 MODEL PERFORMANCE EVALUATION . . . . . . . . . . . . . . . . 49

5.1 Performance Measures . . . . . . . . . . . . . . . . 49

5.1.1 Graphical performance procedures . . . . . . 51 5.1.2 Statistical performance measures . . . . . . 53

5.2 Assessing Model Performance Results. . . . . . . . . 55

6 ATTAINMENT DEMONSTRATION. . . . . . . . . . . . . . . . . 59

6.1 Developing Attainment-Year Model Inputs. . . . . . . 59 6.2 Construction of Attainment Year Emission Control Strategies . . . . . . . . . . . . . . . . . . . . . 60 6.3 Performing Attainment-Year Simulations to Assess Various Control Strategies . . . . . . . . . . . . . 62 6.4 Using Modeling Results in the Attainment Demonstration. . . . . . . . . . . . . . . . . . . . 63 6.5 Exceptions to Guidance Document. . . . . . . . . . . 64

REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . 65

APPENDIX A RECOMMENDED MODELING PROTOCOL CONTENTS . . . . . . 69

APPENDIX B IDENTIFICATION OF METEOROLOGICAL REGIMES CORRESPONDING WITH HIGH OBSERVED OZONE . . . . . . 77APPENDIX C PERFORMANCE MEASURE FORMULATIONS . . . . . . . . . 81TABLES

Number

1 Example Table of Contents for Protocol Document . . . . . . . . . . . . . . . . . . . .9

2 Default Boundary Condition Concentrations for Carbon Bond-IV Species . . . . . . . . . . . 28ABBREVIATIONS

AIRS Aerometric Information Retrieval System BEIS Biogenic Emissions Inventory System CAAA Clean Air Act Amendments CARB California Air Resources Board CMSA Consolidated Metropolitan Statistical Area CO Carbon Monoxide CSC Computer Sciences Corporation DWM Diagnostic Wind Model (UAM preprocessor program) EKMA Empirical Kinetic Modeling Approach EPA U.S. Environmental Protection Agency EPS Emissions Preprocessor System for the UAM GMISS Gridded Model Information Support System MSA Metropolitan Statistical Area NAAQS National Ambient Air Quality Standard(s) NO Nitric Oxide NOx Nitrogen oxides NO2 Nitrogen Dioxide NSR New source review NTIS National Technical Information Service NWS National Weather Service OAQPS EPA Office of Air Quality Planning and Standards OMS EPA Office of Mobile Sources ORD EPA Office of Research and Development PWD Predominant wind direction RACT Reasonably available control technology RFP Reasonable further progress (a type of tracking required under Section 182 of the CAAA) ROM EPA Regional Oxidant Model SAI Systems Applications InternationalABBREVIATIONS (Continued)

SCC Source Category Code SCRAM BBS EPA Support Center for Regulatory Air Models Bulletin Board System SIP State Implementation Plan UAM Urban Airshed Model UV Ultraviolet VOC Volatile Organic Compound VMT Vehicle Miles TraveledACKNOWLEDGEMENTS

Mr. Dennis C. Doll is the principal author of this document. Dr. Richard D. Scheffe, Dr. Edwin L. Meyer, and Mr. Shao-Hang Chu also contributed significantly to the guidance document's content. Mrs. Cynthia J. Baines was responsible for typing and assembling the manuscript.

A draft version of this guidance document has been subject to public review and comment through inclusion in Docket A-88-04 for public comment on the Fifth Conference on Air Quality Modeling held in March 1991. In addition, the authors would like to acknowledge the efforts of a working group set up to review and advise them regarding the issues addressed in the guidance document. While all the recommendations do not necessarily reflect the views of each working group member nor their organizations, members' free exchange of ideas and the generous amount of time they spent in reviewing various drafts added materially to the guidance document's content. Working group members (in alphabetical order) included Dr. C.S. Burton (Systems Applications International Incorporated (SAI, Inc.)), Mr. C. Durrenberger (Texas Air Control Board), Mr. G. Gipson (U.S. Environmental Protection Agency (EPA), Office of Research and Development (ORD)), Mr. T. Helms (EPA, Office of Air Quality Planning and Standards), Dr. H. Jeffries (University of North Carolina), Mr. M. Koerber (Lake Michigan Air Directors Consortium), Dr. C. Liu (South Coast Air Quality Management District), Mr. T. McGuire (California Air Resources Board), Mr. R. Morris (SAI, Inc.), Dr. S. T. Rao (New York Department of Environmental Conservation), Dr. P. Roth (private consultant), Mr. K. Schere (EPA, ORD), Dr. T. Tesche (private consultant), Mr. J. Vimont (U.S. National Park Service), Mr. D. Wackter (Connecticut Department of Environmental Protection), and Dr. S. Ziman (Chevron USA, Inc.).

Finally, the authors would like to thank Ms. Jeanne R. Eichinger (CSC) for her thorough review and many helpful editorial
comments.CHAPTER 1 INTRODUCTION

The Clean Air Act Amendments (CAAA) of 1990 require ozone nonattainment areas designated as extreme, severe, serious, or multi-State moderate to demonstrate attainment of the ozone National Ambient Air Quality Standard (NAAQS) through photochemical grid modeling or any other analytical method determined by the Administrator to be at least as effective. The Environmental Protection Agency (EPA) has adopted the Urban Airshed Model (UAM) as the guideline model for photochemical grid modeling applications involving entire urban areas. Procedures described in this guidance document are intended to satisfy the CAAA attainment demonstration requirements, foster technical credibility, and promote consistency among UAM regulatory applications.

This guidance document provides recommendations and general procedural guidance for exercising the UAM (described in References 1-5) in regulatory applications. However, methodologies and procedures discussed in this guidance document generally apply for other urban-scale photochemical grid models as well. Acceptance criteria for alternative models is beyond the scope of this guidance document. Use of alternative urban-scale photochemical grid models as well as regional-scale photochemical grid models other than the EPA Regional Oxidant Model (ROM) must be addressed on a case-by-case basis through the EPA Regional Offices.

The UAM source code is maintained and distributed by the Source Receptor Analysis Branch, Technical Support Division of the EPA Office of Air Quality Planni ng and Standards (OAQPS). Users will be informed of modifications or enhancements to the UAM through the Support Center for Regulatory Air Models Bulletin Board System (SCRAM BBS). Additionally, the UAM source code, user's guide, and test case data base are available from the National Technical Information Service (NTIS)(703-737-4600). The NTIS document numbers are noted in the reference list.

This guidance document provides recommendations and procedures for conducting an ozone analysis with the UAM for ozone attainment demonstrations. Some of the recommendations and procedures described were adopted from the California Air Resources Board (CARB) Technical Guidance Document: Photochemical Modeling6, and will be referenced as such throughout the text.Steps needed to conduct an urban-scale photochemical modeling study using the UAM typically consist of the following:

1. Establish a protocol for the modeling study.

2. Compile air quality, meteorological, and emissions data to develop UAM input files for each meteorological episode to be used in the attainment demonstration model simulations.

3. Execute the UAM for each meteorological episode.

4. Conduct diagnostic analyses on each meteorological episode simulation. The principal purpose of diagnostic analyses is to ensure that the model properly characterizes physical and chemical phenomena (e.g., wind fields, spatial and temporal emission patterns) instrumental in leading to observed ozone concentrations. The visible product is enhanced model performance (i.e., better spatial and temporal agreement with observed data). Diagnostic model simulations uncover potential model input data gaps that, when corrected, may lead to improved model performance.

5. Exercise the UAM for each meteorological episode and conduct a series of performance measures to determine overall model performance in replicating observed ozone concentrations and patterns. Model performance evalua- tion should also be done for ozone precursors (e.g., NO, NO2) if suitable monitoring data are available.

6. For each meteorological episode, estimate emissions and air quality for the projected attainment year required under the CAAA. Perform model simulations for each episode to determine whether the ozone NAAQS can be met in the attainment year.

7. If the model simulations for the attainment year do not show attainment for each modeled episode, develop emission control measures on selected source categories (for example, volatile organic compound (VOC) and/or NOx controls on selected source categories, alternative fuel scenarios, etc.).

8. Perform model simulations for the emission control measures to demonstrate attainment of the ozone NAAQS foreach meteorological episode. If the control measures do not show attainment, repeat steps 7 and 8 as an iterative process until attainment is shown for each modeled episode.

These steps are addressed in subsequent chapters of this guidance document as follows:

Chapter 2: Modeling Protocol

Chapter 3: Domain and Data Base Issues



### ! Episode selection



### ! Domain selection/grid spatial allocation



### ! Meteorological/air quality data



### ! Emission inventories

Chapter 4: Data Quality Assurance and Model Diagnostic Analyses

Chapter 5: Model Performance Evaluation

Chapter 6: Attainment DemonstrationCHAPTER 2 MODELING PROTOCOL

Regulatory application of the UAM potentially affects a broad spectrum of society. The UAM modeling domains may encompass multiple geopolitical boundaries (counties, cities, and States) with a potentially large regulated community. Therefore, the development of a Modeling Protocol is required. This Protocol is necessary to (1) promote technical credibility, (2) encourage the participation of all interested parties, (3) provide for consensus building among all interested parties concerning modeling issues, and (4) provide documentation for technical decisions made in applying the model as well as the procedures followed in reaching these decisions.

The Protocol should detail and formalize procedures for conducting all phases of the modeling study, such as (1) describing the background and objectives for the study, (2) creating a schedule and organizational structure for the study, (3) developing the input data, (4) conducting diagnostic and model performance evaluations, (5) interpreting modeling results, (6) describing procedures for using the model to demonstrate whether proposed strategies are sufficient to attain the ozone NAAQS, and (7) producing documentation and data analyses that must be submitted for EPA Regional Office review and approval.

All issues concerning the modeling study must be thoroughly addressed during the Protocol development. Modifications to the Protocol as the study progresses should not be needed unless significant, unforeseen procedural and/or technical issues are encountered. All parties involved in the study should agree to Protocol modifications through the Modeling Policy Oversight Committee (see below). It is especially important that the State/ local agencies and EPA Regional Office(s) overseeing the study concur on Protocol modifications.

2.1 Protocol Development Process

Ordinarily, the State agency responsible for developing the ozone State Implementation Plan (SIP) is also the lead agency responsible for developing the Modeling Protocol. For domains encompassing parts of more than one State, the responsible State agencies need to develop the Modeling Protocol jointly. The Protocol should describe the modeling policy and technical objectives of the study. This will require input from various EPA and State/local personnel dealing with regulatory policy issues andfrom others with modeling expertise. It is likely that Modeling Policy Oversight and Technical Committees will be needed for addressing these issues. The composition and responsibilities of the Committees should be defined in the Modeling Protocol.

Responsibilities of the Modeling Policy Oversight Committee may be, at a minimum, to set the objectives of the study, set the schedule, determine resource needs, and implement any modifications to the Protocol as the modeling study proceeds. The Committee should include representatives from the appropriate EPA Regional Office(s), State/local agencies, the regulated community, andpublic interest groups. It is important that appropriate policyoriented personnel be identified for membership on the Committee.

Responsibilities of the Technical Committee may be, at a minimum, to develop the Protocol's technical specifications concerning emission inventories, meteorological data, air quality data, data quality assurance, development of emission control strategies, model diagnostic analyses, model performance evaluation procedures, and interpretation of model results. The Technical Committee should include appropriate technically-oriented members from the EPA Regional Office(s), State/local agencies, the regulated community, and public interest groups.

For some areas, regional modeling is being planned to establish initial and boundary conditions for urban-area modeling attainment demonstrations. The urban-area Modeling Protocol development should be coordinated with the regional Modeling Protocol. Some members of the urban-area Policy Oversight and Technical Committees would probably also be members of the regional Policy Oversight and Technical Committees.

The Modeling Protocol must be submitted to the appropriate EPA Regional Modeling Contact for review and approval. The EPA Regional Modeling Contact should be a member of the Policy Oversight and/or Technical Committees so that rapid review and approval of the Protocol is assured.

Recommendations A Protocol Document is required for each UAM application used for an ozone attainment demonstration. This Protocol should describe the methods and procedures to be used for conducting the photochemical modeling study.

Additionally, it is recommended that both a Policy Oversight Committee and a Technical Committee be established to developthe Modeling Protocol. The composition and responsibilities of the Committees should be defined in the Protocol.

The Modeling Protocol and any modifications to it should be agreed upon by all parties involved in the study, through the Policy Oversight Committee. It is especially important that the State/local agency participants and EPA Regional Office(s) overseeing the modeling study concur on any Protocol modifications. Protocol modifications should be documented for subsequent public review.

For some nonattainment areas, regional modeling is being planned to provide initial and boundary conditions as well as other inputs for the urban-area modeling attainment demonstrations. Procedures for coordinating the development of the urban-area Modeling Protocol with the regional Modeling Protocol should be clearly described.

It is especially important that close technical coordination be maintained during the Protocol development among nearby urban-area domains within a regional modeling domain. Procedures should be established for coordinating the Modeling Protocols among these areas, and these coordination procedures should be clearly specified in each nonattainment area's Modeling Protocol.

The Modeling Protocol must be submitted to the appropriate EPA Regional Modeling Contact for review and approval.

2.2 Contents of Protocol Document

The recommended contents of the Protocol Document (Table 1) are patterned after those described in a CARB Technical Guidance Document.6

Recommendations

It is recommended that, at a minimum, the components listed in Table 1 be included in the Protocol Document for each attainment demonstration modeling study. A description of each component is presented in Appendix A.TABLE 1

EXAMPLE TABLE OF CONTENTS FOR PROTOCOL DOCUMENT

1. UAM Modeling Study Design Background and Objectives Schedules Deliverables Management Structure/Technical Committees Participating Organizations Relationship to Regional Modeling Protocols Relationship to Other Urban-Area Modeling Protocols Relationship to Planning/Strategy Groups

2. Domain and Data Base Issues Applicable Preprocessor Programs (e.g., ROM-UAM Interface System) Data Bases:



### ! Air quality



### ! Meteorology Base Meteorological Episode Selection Modeling Domain Horizontal Grid Resolution Number of Vertical Layers Emission Inventory Specification of Initial and Boundary Conditions Wind Field Specification Mixing Depths Sources of Other Input Data

3. Diagnostic Analyses Quality Assurance Tests of Input Components Diagnostic Tests of Base Case Simulation Test Results/Input Modifications

4. Model Performance Evaluation Performance Evaluation Tests

5. Attainment Demonstrations Identification of Attainment-Year Mandated Control Measures Methodologies for Generating Control Strategy Emission Inventories Procedures for Attainment Demonstration6. Submittal Procedures Data Analyses Review Documentation Review and ApprovalCHAPTER 3

DOMAIN AND DATA BASE ISSUES

Described in this chapter are the following topics: episode selection, domain selection, meteorological data, air quality data, and emissions inventories. Choices made in each topic area are often interrelated. Accordingly, decisions concerning a particular topic area probably will be based on consideration of several areas. In several topic areas, recommendations are made concerning minimum requirements for data availability and modeling resolution. To reduce uncertainties in modeling inputs and outputs, users are encouraged to exceed these minimum recommendations whenever possible.

3.1 Episode Selection

A major component of the Modeling Protocol is selection of meteorological episodes. In general, episode selection involves a review (described below) of several multiday periods during which high ozone was monitored. At least 1 day is chosen as the day of primary interest for each selected episode. Model simulations typically begin at least one day prior to the day of primary interest. This minimizes the effects of assumed initial conditions on predicted concentrations for the critical day. The length of a modeled episode is generally a minimum of 48 hours, and the last day in this period--the day of primary interest--is referred to as the "primary day."

Episodes that have a high probability of covering different sets of meteorological conditions corresponding with high ozone concentrations should be modeled. Clearly, a trained meteorologist familiar with local and regional weather patterns should be consulted in the selection process. Conditions resulting in distinctly different source-receptor configurations should be the prime consideration in distinguishing different meteorological regimes. Generally, conditions reflecting both poorly defined wind flow (stagnation) and better defined flow (transport) will need to be included. It is important to coordinate episode selection with those responsible for Modeling Protocols in nearby domains, particularly when observed exceedances may result from "overwhelming transport."7

The following approach is recommended for selecting episode days for use in modeling:1. Identify the meteorological regimes associated with high- ozone episodes. The procedure recommended for identifying meteorological regimes is described in Appendix B.

2. Select candidate episode days for modeling from the period from 1987 to the present time. Place each candidate episode day in the appropriate meteorological regime (see Appendix B).

3. Rank each candidate episode day within each regime according to the magnitude of the peak observed ozone concentration (ranked highest are days with the highest observed daily maximum ozone from among all sites in or near a nonattainment Consolidated Metropolitan Statis- tical Area/Metropolitan Statistical Area [CMSA/MSA]).

4. Select the episode days for modeling from among the three highest ranked episode days from each meteorological regime. In choosing from among the top-ranked episode days, consider the availability and quality of air quality and meteorological data bases, the availability of supporting regional modeling analyses, the number of monitors recording daily maximum ozone concentrations greater than 0.12 ppm (i.e., pervasiveness), number of hours for which ozone in excess of 0.12 ppm is observed, frequency with which the observed meteorological condi- tions correspond with observed exceedances, and model performance (discussed in Chapter 5). For example, the top-ranked episode day within a meteorological regime may have only routine air quality and meteorological data bases available for use in the modeling. The third- highest day, however, may have occurred during an intensive field study, so that a more comprehensive data base is available. Thus, the third-highest day may be more desirable for modeling than the top-ranked day. As another example, the three highest-ranked episode days may have air quality and meteorological data bases of similar quality and quantity, and the number of monitoring sites recording daily maximum ozone greater than 0.12 ppm may also be similar. If model performance on the initially chosen day is questionable, the Technical Committee may wish to consider a second- or third-ranked day from the three highest-ranked days for a regime. The day with the overall best model performance may be selected as the primary day formodeling in the attainment demonstration. Note that a more comprehensive model performance evaluation may be needed for the selected day, as described in Chapter 5.

5. At least 1 day should be modeled from each identified meteorological regime. Further, a minimum of 3 days from among all meteorological regimes should be modeled for the attainment demonstration (e.g., three meteorological regimes each containing 1 primary episode day, or two meteorological regimes with at least 2 primary days from one of those regimes). Using the model results in the attainment demonstration is described in Section 6.4.

States may want to consider a technique other than the one outlined in steps 1-5 for selecting modeling episodes. Any such techniques should be described in the Modeling Protocol and approved by the appropriate EPA Regional Office.

Consideration of several meteorological regimes that correspond to observed daily maximum ozone levels above 0.12 ppm is important, because certain emission control strategies that are effective in reducing peak ozone under some meteorological conditions may be less so under others. The goal is to develop strategies that are robust with respect to effectiveness over most scenarios.

Recommendations It is recommended that episodes for modeling be selected from the period from 1987 to the present time. Selected episodes should represent different meteorological regimes observed to correspond with ozone >0.12 ppm (as described above). When selecting episodes, both stagnation and transport conditions should be examined. A minimum of 3 primary episode days should be simulated.

Primary episode days falling within each meteorological regime are ranked according to the highest observed daily maximum ozone concentration measured within or near the nonattainment CMSA/MSA. Episodes may be chosen to include any of the three top-ranked days in each regime. In addition to considering the magnitude of the highest observed daily maximum ozone concentration in making this choice, data availability and quality, model performance, availability of regional modeling analyses, pervasiveness, frequency with which observed meteorological conditions coincide with exceedances, and duration of observations >0.12 ppm may be considered.Other techniques for selecting episodes should be described in the Protocol Document and approved by the appropriate EPA Regional Office.

3.2 Size of the Modeling Domain

The size and location of the modeling domain define the data requirements for the modeling. In selecting a modeling domain, consideration should be given to (1) typical wind patterns, (2) the location of major area and point emission sources, (3) the location of air quality monitors and important receptor locations, and (4) the need to mitigate effects of uncertainty in upwind boundary conditions. Generally, the domain should be set as large as feasible in order to reduce the dependence of predictions on uncertain boundary concentrations and to provide flexibility in simulating different meteorological episodes. It is generally much easier to subsequently reduce the size of a modeled area than it is to subsequently increase it.

Once UAM input data for a sufficiently large domain have been assimilated and processed, the size of the modeling domain can be reduced for modeling purposes by specifying domain boundary values in the UAM. Procedures for reducing the size of the domain are described in Reference 2. This could save resources in simulating modeling episodes in which light or poorly defined wind fields result in a smaller domain being adequate. In contrast, expanding domain dimensions would require reconstructing most of the UAM input files.

Recommendations It is recommended that the domain's downwind boundaries be sufficiently far from the CMSA/MSA that is the principal focus of the modeling study to ensure that emissions from the CMSA/MSA occurring on the primary day for each selected episode remain within the domain until 8:00 p.m. on that day. The extent of the upwind boundaries will depend on the proximity of large upwind source areas and the adequacy of techniques7 used to characterize incoming precursor concentrations. Large upwind emission source areas should be included in the modeling domain to the extent practicable. Also, if large uncertainty is anticipated for domain boundary conditions, the upwind boundaries should be located at a distance sufficient to minimize boundary effects on the model predictions in the center of the domain. Sensitivity analyses described in Section 4.3 assist in determining the effects of boundary conditions on predicted values.3.3 Horizontal Grid Cell Size

The horizontal dimension of each model grid square is based upon (1) the sensitivity of predicted concentrations to horizontal grid size, (2) the resolution of observed meteorological and air quality data and/or estimated emissions data, and (3) limitations imposed by other considerations such as a required minimum domain size. Generally, large grid square dimensions result in smoothing of the emission gradients, wind fields, and spatially varying mixing heights, which in turn leads to a smoothing of the predicted concentration field. Also, larger grid cell dimensions reduce both computer storage space and computational time.

The following should be considered when selecting the horizontal grid cell size:

1. The grid cells should be small enough to reflect emission gradients and densities in urban areas, particularly those resulting from large point sources and major terrain or water features that may affect air flow

2. Sensitivity studies conducted by the EPA suggest that peak ozone predictions may increase as grid size decreases

3. Practical limitations on the grid cell size are the resolution of the emission inventories and the density of meteorological and air quality monitoring networks

Previous modeling studies have used horizontal grid cell sizes generally in the range of 2 x 2 km to 8 x 8 km. A grid size of 5 x 5 km has generally been compatible with computer resource requirements and emission inventory development.

Recommendations It is recommended that the size of the horizontal grid cells should not be greater than 5 x 5 km. Grid cell sizes coarser than this should be justified and should, at a minimum, address items 1-3 above. Smaller grid cell sizes are encouraged because they allow more accurate gridding of area and mobile sources. Additionally, emissions from major point sources are better characterized by smaller grid cell sizes. However, grid cell sizes smaller than 2 x 2 km are not recommended because of potential model formulation inconsistencies for those grid sizes.3.4 Number of Vertical Layers

In specifying the number of vertical layers, issues analogous to those raised for horizontal grid cell dimensions must be addressed. Again, a compromise is generally needed between the number of vertical layers and the adequacy of available data bases and computer resources. It is important that sources with tall stacks or sources having plumes with high buoyancy be assigned toan appropriate altitude. Pollutants in elevated, buoyant, pointsource plumes often have effective release heights in layers well above the surface. Increased vertical resolution allows more accurate representation of the vertical layer at which these emissions interact with emissions occurring closer to the ground. Also, increased vertical resolution minimizes dilution that may result from placing emissions into artificially large vertical layers. Finally, increased vertical resolution improves the simulation of when and where plumes are mixed to ground level. Simulation of the chemistry between individual plumes and the environment can be greatly affected by how well the model simulates mixing of these plumes with the ambient air.

Previous applications of the UAM have generally used four or five vertical layers, with two layers between the surface and the morning mixing height (diffusion break in the UAM) and three layers between the mixing height and the top of the modeling domain. Sensitivity studies suggest that using fewer than three layersabove the mixing height may artificially dilute elevated pointsource plumes, which may cause the model to underpredict nearsurface ozone and precursor concentrations.

Users of the UAM should consider specifying greater detail for the horizontal and vertical grid cell size than the minimum recommended in this guidance document. This is encouraged particularly in modeling domains containing complex terrain or land/water interfaces. Wind field models can typically produce wind fields for many more vertical layers than the minimum number given here.3 The number of vertical layers considered in the UAM is more likely to be constrained by the time-consuming calculations needed to simulate atmospheric chemistry.

Recommendations Based on previous model applications, it is recommended that a minimum of five vertical layers be used in the modeling study, with at least three layers above the morning mixing height (diffusion break in the UAM). Additionally, it is recommended that the top of the modeling domain (region top inthe UAM) be specified above the mixing height by at least the depth of one upper-layer cell. This can be done by setting the region top value equal to the maximum mixing depth plus the minimum depth of the upper-layer cells.

Previous applications have typically used 50 m for the minimum depth of the vertical layers below the diffusion break and 100 or 150 m for the vertical layers above the diffusion break. It is recommended that 50 m be used as the minimum thickness for layers below the diffusion break and 100 m as the minimum thickness for layers above the diffusion break.

3.5 Meteorological Data

The availability of meteorological data varies widely among prospective modeling domains. Also, there are a variety of techniques available for developing wind fields, temperature fields, and mixing heights. Although high resolution and confidence for all meteorological data are desirable, time and resource constraints force a compromise between desirable and acceptable methods. Historically, measured meteorological data have been interpolated for most UAM applications. More recently, diagnostic and prognostic meteorological modeling techniques have been explored as possible means to develop input fields (particularly wind fields) for air quality models.

Wind fields and mixing heights are two of the most important meteorological inputs that significantly affect photochemical model predictions. Methodologies and recommendations for determining these inputs are described below.

3.5.1 Wind fields

Methodologies to construct wind fields for the UAM applications have historically fallen into three categories:6

1. Objective analyses that interpolate observed surface and aloft data throughout the modeling domain

2. Diagnostic wind models in which physical constraints are used in conjunction with objective analyses to determine the wind field

3. Prognostic models based on numerical solution of the governing equations for mass, momentum, energy, and moisture conservation along with numerical solutions forthermodynamic processes

More recently, an additional methodology has been developed in areas where the EPA ROM has been applied. Computer software has been developed to map a ROM diagnostic gridded wind field into a nested UAM domain.5

Objective analysis - These procedures generally involve straightforward interpolative techniques. They have the advantage of being relatively simple and inexpensive to use. The primary disadvantages are that these analyses contain limited physical concepts, and results are highly dependent upon the temporal and spatial resolution of the observed values. Thus, in domains containing sparse observational data or complex topography, results may be unsatisfactory.

Diagnostic wind models - These models improve mass consistencyfor the flow fields. This may be addressed through parameterizations for terrain blocking effects and upslope and downslope flows, as in the UAM Diagnostic Wind Model.3 Diagnostic models generallyrequire minimal computer resources and can produce a threedimensional wind field. However, diagnostic models need representative observational data to generate features such as land and sea breezes.

Prognostic models - These models simulate relevant atmospheric physical processes while requiring minimal observational data. Prognostic models require a specification of the synoptic-scale flow. Reliability of these approaches is usually enhanced if sufficient observations are available to "nudge" solutions closer to observations. Since these models can simulate temperature fields in addition to the wind field, it is possible to determine stabilities and mixing heights, thus eliminating the need to generate these from sparse observational data. Another significant advantage is that interdependencies of various meteorological inputs with one another are considered in prognostic models. A major disadvantage is the extensive computational resources needed to run a prognostic model. Additionally, the availability of evaluated models and expertise needed to apply them for general application with photochemical grid models is limited.

The ROM-UAM Interface System - This system can develop a UAM gridded wind field from a diagnosti cally derived wind field used in the ROM. Such a ROM-derived wind field can be applied for a UAM domain that is nested within a ROM domain, provided ROM data are available for identical episode periods. Use of ROM data has theadvantage of being easy to implement and also provides a consistency between ROM model predictions used to specify UAM boundary conditions and the corresponding wind fields. The ROM data are based on an approximately 18 x 18 km horizontal grid cell size. Thus, one disadvantage is that ROM gridded wind fields may not sufficiently describe detailed features such as land/sea circulations. A more finely resolved wind field may be obtained by using the ROM-gridded winds as the initial wind field for the UAM's Diagnostic Wind Model (DWM) preprocessor (see Reference 3). This provides a means for mass consistency when using the ROM data as boundary conditions in conjunction with another wind model.

The selection of a specific technique for generating the domain wind field depends largely on (1) availability of concurrent ROM diagnostic wind fields, (2) the spatial and temporal resolution of surface and upper-air observations, (3) available modeling expertise in applying alternative meteorological models, and (4) available computer resources. However, some guidelines on preferences for generating the wind fields are as follows.

Recommendations The ROM-UAM Interface System should be used to derive the UAM gridded wind fields when the UAM domain is nested within a ROM domain for concurrent time periods and ROM predictions are used to derive the hourly UAM boundary conditions. If it is judged by the Technical Committee (and identified in the Protocol) that a wind field derived from the UAM DWM is more representative of the domain-scale flow, then this wind field may be used in lieu of the ROM diagnostic wind field. To minimize mass inconsistency problems, the ROM-gridded winds may be used as the initial wind field in the DWM (see Reference 3) when generating the UAM gridded wind field.

For cases in which concurrent ROM applications are unavailable, it is recommended that the DWM be used to generate the UAM gridded wind fields. The use of other techniques for deriving the wind field, such as prognostic wind models or other objective techniques, may be employed on a case-by-case basis, subject to approval from the appropriate EPA Regional Office.

3.5.2 Data needs for wind field development

The development of a wind field for each modeling episode depends upon ground-level and elevated wind observation data. Itis preferred that a surface-based monitoring network report wind speed and direction as hourly averages, because an hour is the time period commensurate with most UAM concentration output analyses. The surface monitoring network should be broad and dense so that diagnostic models (if that is the technique chosen) can depict major features of the wind field. Data representing vertical profiles of wind speed and direction are required in order to establish upper-level wind fields. Preferably, data should provide adequate spatial (horizontal) and temporal resolution. Results of UAM applications are often criticized because of inadequate meteorological data, and lack of sufficient meteorological data often prevents definitive diagnostic analyses. Thus, the need for adequate meteorological data cannot be overstated.

Time constraints imposed by the 1990 CAAA will probably preclude consideration of new meteorological monitoring stations. Thus, it is likely that the base case to be used in the attainment demonstration will be from an historical episode for which model performance has been deemed acceptable.

Recommendations

Meteorological data routinely available for a UAM modeling demonstration usually consist of National Weather Service (NWS) hourly surface and upper-air observations (for winds aloft). If these data are the only data available for use in a modeling demonstration, they may have to suffice. However, the NWS data consist of observations made over very short periods rather than hourly averaged values. An assumption that wind velocity measured over a very short period persists unaltered over an hour may lead to an overestimate of transport. Therefore, whenever possible, hourly averaged meteorological data (e.g., from an intensive field study) should be used. Additional meteorological data may be available from other sources in the domain (e.g., an on-site meteorological monitoring program at an industrial facility). These data may be used to supplement the NWS data, provided the data have been adequately quality assured. Additionally, the EPA guideline entitled On-Site Meteorological Program Guidance for Regulatory Modeling Application9 should be consulted to assess whether the supplementary data reflect proper siting of meteorological instruments and appropriate data reduction procedures.

In planning a special field study to provide a more spatially and temporally dense meteorological data base, the number ofsurface meteorological monitoring stations should be sufficient to describe the predominant wind flow features within the modeling domain. An experienced meteorologist familiar with local climatic patterns should be consulted concerning the location and suitability of the surface meteorological stations. Vertical sounders or profilers are highly encouraged in a special field study to resolve winds aloft and mixing heights. Any special field study and monitoring program should be planned in consultation with the appropriate EPA Regional Office before implementing the study.

3.5.3 Mixing heights

Predictions from the UAM have been shown to be sensitive to the mixing height field.6 Therefore, the temporal variations in the mixing height field over the UAM domain should be described as realistically as possible. The UAM modeling system contains a methodology for deriving mixing heights (diffusion break in the UAM) based on surface temperatures, vertical sounding measurements of temperature, and cloud cover (see Reference 2). However, because of the diversity of techniques and data bases that may be available on a case-by-case basis, we cannot recommend a specific procedure for deriving the mixing height field in all cases.

Recommendations It is recommended that, at a minimum, the techniques described in Reference 2 be used in establishing the mixing height field in the domain.

The choice of upper-air stations to be used in the mixing height calculations should be based on prevailing wind fields and location of the upper-air stations relative to the UAM domain. If there are no upper-air stations within the domain, stations outside the domain may need to be used. A trained meteorologist should be consulted on the selection of upper- air stations for use in determining mixing heights.

The techniques for generating the mixing height field should be described in the Protocol Document. Techniques other than that described in Reference 2 should be documented and justified.

3.5.4 Clear-sky assumption for photolysis rate calculations

For regulatory UAM applications, clear-sky conditions have typically been assumed for photolysis rate calculations in theMETSCALARS processor. The UAM's current structure does not allow for spatial variation in cloud cover, so the choice is either uniformly clear or a uniform cloud cover based on a mean cloud cover over the domain. Use of mean cloud cover could significantly understate reaction activity in "clear" patches of the domain. Potentially, this could be a more serious error than assumingclear-sky conditions and simulating an overall excess of "domainwide" insolation. Additionally, the ROM-UAM Interface System IMETSCL processor assumes clear-sky conditions for photolysis rate calculations.

Recommendations For applications involving the current regulatory version of the UAM, it is recommended that clear-sky conditions be assumed for calculating photolytic rate constants in the METSCALARS processor.

3.6 Air Quality

Ambient air quality data are generally used for two purposes: (1) to specify initial- and boundary-condition concentrations, and (2) to assess the model performance for each meteorological episode to be used in the attainment demonstration. These topics are addressed in the following two subsections.

3.6.1 Initial and boundary conditions

Three general approaches for specifying boundary conditionsfor UAM simulations are as follows: (1) use objective/interpolation techniques with a sufficient amount of measured data (i.e., data from an intensive field program), (2) use default background values and expand the upwind modeling domain and simulation period to mitigate uncertainties due to paucity of measurements, and (3) use regional-scale model predictions of ozone and precursor concentrations. Initial conditions for UAM simulations are handled in one of two ways: (1) use regional-scale model predictions to derive initial conditions, and/or (2) begin the UAM simulation sufficiently far in advance of the primary day to eliminate sensitivity of results to arbitrary assumptions regarding initial conditions.

Clearly, the nature of case-specific applications will determine what approaches should be taken for establishing initialand boundary conditions for particular domains. Ideally, the preferred technique would be based on an intensive field program with regional-scale modeling used to fill in spatial and temporal gaps. This approach is seldom feasible, however, particularly for historical episodes. Presented below are recommendations for implementing each of the three techniques identified above for deriving boundary conditions, including discussion of theadvantages and disadvantages of each technique. Default boundarycondition values for ozone and precursor concentrations are also provided. Finally, recommendations are provided on the approach most likely to be feasible for specifying the initial and boundary conditions for modeling historical episodes in most locations.

Use of measured data - All sources of air quality data for a particular modeling domain should be evaluated for applicability in establishing initial and boundary conditions. Unfortunately, most ongoing monitoring programs have been designed (understandably so) with a receptor-based orientation. While available monitoring data are useful for evaluating model performance, they usually are not adequate for establishing initial and boundary concentrations.

Recommendations To develop initial and boundary conditions, it is recommended that one or more monitoring stations be sited upwind of the central urban area along prevailing wind trajectories that give rise to ozone exceedances.

The sampling and analysis program should provide data to calculate hourly values for ozone, NO, NO2, and speciated hydrocarbons.

At the inflow boundaries, air quality data at the surface and aloft should be used whenever available to specify the boundary conditions. Default values (Table 2) may be used where necessary.

Use of default values - Some urban areas may lack adequate data suitable for establishing initial and boundary conditions. Section 3.2 on domain selection and Chapter 4 on diagnostic analyses recommend constructing domains and simulation periods large enough to minimize the sensitivity of inner core and downwind concentrations to assumed initial and boundary conditions.TABLE 2

DEFAULT BOUNDARY CONDITION CONCENTRATIONS FOR CARBON-BOND-IV SPECIES (SEE REFERENCES 1 AND 7)

Species Species Name Concentration (ppbC)

OLE Olefins 0.60 PAR Paraffins 14.94 TOL Toluene 1.26 XYL Xylene 0.78 FORM Formaldehyde 2.1 ALD2 Higher Aldehydes 1.11 ETH Ethene 1.02 CRES Cresol,Higher Phenols 0.01 MGLY Methyl Glyoxal 0.01 OPEN Aromatic ring fragment 0.01 acid PNA Peroxynitric acid 0.01 NXOY Total nitrogen 0.01 compounds PAN Peroxyacyl nitrate 0.01 HONO Nitrous acid 0.01 H2O2 Hydrogen peroxide 0.01 HNO3 Nitric acid 0.01 MEOH Methanol 0.1 ETOH Ethanol 0.1 O3 Ozone 40.0 (ppb) NO2 Nitrogen Dioxide 2.0 (ppb) NO Nitric oxide 0.0 (ppb) CO Carbon monoxide 350.0 (ppb) ISOP Isoprene 0.1 (ppb)Initial- and boundary-condition concentrations are influencedby large- and small-scale weather patterns and emissions distributions that are unique to each modeling domain. Thus, casespecific attributes should be used in estimating these concentrations whenever feasible. For example, boundary concentrations of hydrocarbons, particularly those species (or intermediate products) emitted from vegetation, are likely to be higher in urban areas surrounded by dense vegetation than in areas surrounded by sparse vegetation.

Recommendations It is recommended that use of default values to establish boundary conditions be limited to areas surrounded by large expanses of low-density anthropogenic emissions. Accordingly, the modeling domain may need to envelop rural areas.

Those choosing to use default values should plan to perform diagnostic/sensitivity simulations (see Chapter 4) to evaluate the sensitivity of domain-interior model predictions to the boundary conditions.

Table 2 lists the recommended default boundary values for the chemical species used in the model. Use of default boundary values under regional transport conditions should be closely evaluated. When using default values, the boundary of the domain should extend as far upwind as practicable.

To diminish dependence on arbitrary specification of initial conditions, a simulation should begin at least 1 day prior to the primary day.

Use of regional model concentration predictions - Output from regional-scale models such as the EPA ROM provides estimates of initial and boundary conditions (as well as certain meteorological inputs) for urban-scale models. This is especially important under regional transport conditions. The ROM-UAM Interface System referred to in Section 3.5.1 can use ROM concentration predictions to develop UAM input files of initial and boundary conditions. This interfacing software should be used for UAM domains nested within more extensive ROM domains. Using the ROM is the recommended approach for generating boundary conditions. It is the most technically defensible approach for estimating future boundary conditions for the attainment year.

Certain considerations arise when using interfacing methods. First, selection of historical episodes is limited to those thathave been modeled on a regional scale. Second, there may be inconsistencies in mass conservation when applying ROM-derived initial and boundary conditions in conjunction with wind fields not derived from the ROM wind field (see Section 3.5.1). The combinations of concentrations and wind velocities produced by the ROM-UAM Interface System represent mass fluxes passing through the urban-scale modeling domain. In cases where ROM-derived initial and boundary conditions are applied without ROM-generated wind fields, locally developed wind fields may need to be evaluated for mass consistency throughout the urban-scale domain. Methods for addressing this problem will need to be chosen on a case-by-case basis. A general procedure for enhancing mass consistency is described in Section 3.5.1. Additionally, initial and boundary conditions derived from the ROM data should be compared with corresponding monitoring data wherever available. This will ensure that the ROM wind fields adequately represent the transport of ozone and precursors into the domain region.

Recommendations It is recommended that, whenever feasible, the ROM-UAM Interface System be applied to derive the initial and boundary conditions for the episode(s) being modeled. If the Interface System is used to derive the initial and boundary conditions, it is also recommended that it be used to derive the UAM gridded wind field, unless there is sufficient justification that other techniques for deriving the wind field are more
accurate.

In cases for which ROM predictions are not available, it is generally recommended that measured data be used to establish the initial and boundary conditions, provided the Technical Committee identified in the Protocol determines the data are adequate. If measured data are not adequate, the default values may be used. To diminish sensitivity of results to assumed initial conditions, simulations should begin 1 day prior to each primary day.

3.6.2 Performance Evaluation Data

Air quality data are needed to diagnose problems in setting up model applications and assessing model performance for the meteorological episodes being considered in the attainment demonstration. A lean air quality data base may introduce significant uncertainties in characterizing model performance.

Under Title I, Section 182 of the CAAA of 1990, the EPA isrequired to develop regulations for enhanced monitoring of ozone and ozone precursors in serious, severe, and extreme ozone nonattainment areas. When promulgated, these regulations will specify criteria for network design, monitor siting, monitoring methods, operating schedule, quality assurance, and data submittal.11 The enhanced ozone monitoring system is designed to provide a more comprehensive data base for model input and to improve model performance evaluation.

Recommendations It is recommended that the data base used in the attainment demonstration modeling meet the requirements for the enhanced ozone monitoring system to be promulgated by the EPA. However, the EPA recognizes that some historical episodes that will be used in the attainment demonstration modeling for the November 1994 ozone SIP submittals may have data bases that would not meet the requirements for an enhanced ozone monitoring system. Under these conditions, the data bases should be scrutinized in detail by the Technical Committee to help ensure that model performance that appears to be acceptable has not actually resulted from compensating errors in the data bases. Additional diagnostic analyses may be necessary for lean data bases from historical episodes.

If it is determined that the existing air quality monitoring program does not meet the requirements for the enhanced ozone monitoring system, responsible regulatory agencies should begin planning for development of an enhanced ozone monitoring system for potential future modeling studies.

3.7 Emissions

The credibility of UAM applications is directly tied to formulating the best possible emission inputs. Model performance may hinge on how well emissions are estimated. Also, in the attainment demonstration, modeling results are used to determine emission scenarios that lead to improved air quality levels consistent with the NAAQS. A faulty emission inventory could lead to erroneous conclusions about the extent of needed controls and, in some cases, errors in judgment about the need to control certain classes of precursors (e.g., NOx).

Developing photochemical model emission input data is the most intensive task of model applications, and requires consideration of many issues. The source of the UAM modeling emission inventory will be the 1990 SIP nonattainment base year inventory requiredunder the CAAA of 1990 for all ozone nonattainment areas. A further discussion of the 1990 base year inventory is contained in Emission Inventory Requirements for Ozone State Implementation Plans.12 It is important to note that the 1990 modeling inventory will not be identical to the 1990 nonattainment area inventory required for reasonable further progress (RFP) tracking under Section 182 of the CAAA. For example, the modeling inventory will probably have to cover a larger geographical area than that required for the nonattainment area inventory. The discussion of modeling domain and boundary-condition issues in Sections 3.2 and 3.6 makes it clear that the modeling inventory must encompass a larger area than the nonattainment MSA. A complete description of relationships between the modeling inventory and the nonattainment area inventory is provided in Procedures for the Preparation of Emission Inventories for Volatile Organic Compounds, Volume II: Emission Inventory Requirements for Photochemical Air Quality Simulation Models (Revised).13 Additional guidance for developing the modeling emission inventory is found in Reference 4.

For use in regulatory applications of the UAM, the 1990 modeling inventory will have to undergo several adjustments. First, the inventory needs to be adjusted to be consistent with meteorological conditions during each selected episode (i.e., "1990 day-specific emissions"). Second, the resulting "1990 day-specific emissions" should be adjusted to reflect control programs and activity levels prevailing during the year(s) of selected episodes.For example, if a selected episode occurred in 1988, the "1990 dayspecific emissions" would be further adjusted to reflect controls and activity levels prevailing in 1988. This latter adjustment is needed to provide an estimate of emissions most suitable for evaluating performance of the UAM.

As noted in Chapter 1, once the UAM's performance has been evaluated and the model has been determined to perform satisfactorily, it is used to derive control strategies to attainthe NAAQS. This requires another adjustment to the "1990 dayspecific emissions" described above. This adjustment entails use of growth factors, ongoing control programs and retirement rates for obsolete sources of emissions to project "1990 day-specific emissions" to the years by which the CAAA specify that the NAAQS must be attained. Reference 14 describes the appropriate methodology for making emission projections. The resulting "attainment year modeling inventory" is used as a starting point from which to construct "strategy inventories." A "strategy inventory" is obtained by superimposing additional control measures on sources of emissions in the "attainment year modelinginventory."

In summary, a 1990 modeling inventory is first adjusted to evaluate UAM performance. The 1990 modeling inventory is then readjusted to reflect emissions most likely at the time the CAAA require attainment of the NAAQS.

Two emission files drive the UAM, a file of emissions that are injected into the first, surface-based vertical layer, and an elevated point source file of emissions that are injected intovertical layers above ground level. The UAM Emissions Preprocessing System (EPS)4 reads county-level area- and point-source files and performs four major functions: (1) area sources and point sources are allocated to grid cells; (2) temporal profiles are assigned to source categories; (3) hydrocarbon speciation profiles are assigned to source categories, and (4) point sources with effective plume heights greater than a prescribed cutoff level are assigned to the elevated point source file and the remaining point sources are assigned to the surface-layer emissions file.

Addressed below are the following issues that arise in developing emission input data: (1) use of speciation profiles, (2) use of surrogate factors to grid area sources, (3) treatment of mobile sources and top/down versus bottom/up approaches, (4) episodic adjustment of inventories to day-specific modeling inputs, (5) treatment of biogenic emissions, (6) cutoff levels for NOx point sources, and (7) consistency with national inventories.

3.7.1 VOC speciation

The EPA provides "default" nationwide VOC speciation profiles for various source category codes (SCCs).13 Use of local speciation information, especially for major emitters, is preferable tonational default profiles. If feasible, major VOC point- and areasource categories should be surveyed to determine appropriate VOC composition profiles. In many cases, both the quantity and the composition of emissions change as process operations are modified. To the extent feasible, this should be accounted for when deriving local speciation profiles and in simulating control strategies. The emissions inventory guidance document13 provides details on developing local speciation profiles.

Most current-year applications are likely to rely on existing default data for speciating mobile-source emissions. Projected future-year mobile-source emissions files may be based on differentformulations of gasoline and use of alternative fuels. Speciation guidance for these fuels will be provided by the EPA Office of Mobile Sources (OMS) through the appropriate EPA Regional Office.

Recommendations It is recommended that local speciation profiles for point- source and area-source categories be used whenever feasible. The Technical Committee should determine the appropriateness of using local or national default speciation profiles. Profiles used in the modeling demonstration must be documented, and any changes assumed in profiles as the result of control strategies must be identified and justified.

3.7.2 Spatial gridding of area sources

Area-source emission data, including motor vehicle emission data, are often supplied on a county basis. Spatial allocation of county-level emission estimates to grid cells is performed for each identified area-source category; the allocation requires use of surrogate distribution factors such as population distribution, land use, and road type. The UAM EPS4 contains a program that uses gridded surrogate factors to allocate county-level emissions data to the grid cell size of the modeling domain.

Recommendations It is recommended that the emission inventory guidance document13 be consulted for alternative surrogate factor choices and sources of information for assimilating surrogate data. The EPA is currently developing a utility to provide gridded surrogate data. States will be notified of the availability of gridded surrogate data through the EPA Regional Offices.

3.7.3 Mobile sources

Development of gridded, time-variant mobile-source inputs raises several concerns and often represents the largest fraction of effort when assimilating mobile-source emissions inputs. Mobile-source emissions have been compiled from original data orfrom existing county-level emissions.13 Developing gridded mobilesource emissions from original data requires aggregating sub-gridcell-level components. This may require exercising transportation models that produce inputs for the mobile-source emissions model (i.e., the latest EPA MOBILE model), and then performing the necessary spatial allocations to grid cells and temporal distribution over every hour. This practice is far fromstandardized. Also, in certain areas, execution of transportation models is restricted by lack of appropriate traffic count and speed data.

The emission inventory guidance document13 provides direction for developing mobile-source inputs from original data (referred to as a bottom/up method) or from an existing county-level inventory (referred to as a top/down method).

Recommendations Bottom/up methods are the preferred approach for estimating vehicle activity levels and emission factors because these methods have potential for resolving variations in speed and vehicle miles traveled (VMT) among different grids over hourly time slices. Bottom/up approaches are most appropriate for addressing the inner urban core of modeling domains. Peripheral, less dense traffic areas can be treated with top/down methods. Exceptions to these recommendations should be considered by the Technical Committee on a case-by-case basis. Justification for more extensive use of top/down methods should be sought in discussions with the appropriate EPA Regional Office.

3.7.4 Episode-specific adjustments

Several source categories of VOC emissions are sensitive to meteorological conditions. Thus, it is important for modeling inventories to reflect episode-specific meteorological conditions.13 For example, biogenic emissions, mobile-source evaporative emissions, and solvent categories will need to reflect specific modeling days. In addition, known episode-specific events such as changes in process operations for point sources affect emissions rates and should be reflected in the episode modeling inventory.

Recommendations Mobile-source emissions should be adjusted for episode- specific temperatures. This is done by running the latest EPA MOBILE model using episode-specific maximum and minimum temperatures. Chapter 7 of the emission inventory guidance document13 describes the procedures for deriving episode- specific mobile-source emissions using the latest MOBILE model. Use of models other than the latest EPA MOBILE model should be reviewed by the Technical Committee on a case-by- case basis, and is subject to approval by the EPA Regional Office.Biogenic emissions must reflect episode-specific conditions (see Section 3.7.5).

If available, episode-specific operating rates for point sources are preferable for estimating temporal point-source emissions. Procedures for temporally adjusting point and area sources are also provided in the emission inventory guidance document.13

3.7.5 Biogenic emissions

Biogenic emissions can be a significant portion of the overall VOC emission inventory for a given domain, particularly in areas of high vegetative density. The EPA provides the Biogenic Emissions Inventory System (BEIS), which can develop day-specific, hourly, gridded, speciated inputs (see Reference 4), and also provides a national data base of land use distributions with this system. Spatial variability is limited to the county level (i.e., emissions are evenly spread throughout the grids within a specific county).

The EPA is currently modifying the BEIS to allow users to input user-derived and possibly more up-to-date land use distribution data. Users will be advised of the expected delivery date of the modified processor via the SCRAM BBS and EPA Regional Offices.

Recommendations Biogenic emissions must be included in the emission inventory developed for each model simulation (i.e., base case and control strategy). The biogenic emission processor (BEIS) that is part of the EPA Emissions Preprocessor System4 should be used to derive the inventory. Use of alternative land use factors in the BEIS should be described and documented in the Protocol Document.

Also, methods other than the BEIS may be considered for deriving the biogenic emissions. These methods must be described in the Protocol Document along with justification for using them.

3.7.6 Point-source and plume-rise cutoff levels

Guidance for initiating ozone/CO SIP emission inventories pursuant to the 1990 CAAA15 specifies point-source cut-off levels of 10 tons/yr and 100 tons/yr for VOCs and NOx, respectively. Any source may be treated as a point source as long as stack data arespecified that allow derivation of effective plume height, and source location is provided.4 In some cases, the Technical Committee may wish to treat selected smaller sources as point

sources.

The UAM EPS4 requires the specification of a plume-rise cutoff level for delineating elevated point sources from area sources. If the plume rise that the EPS calculates for a given point source is below the user-specified level, then the point-source emissions are placed in the area-source emissions file. If the plume rise is above the level, the emissions are treated as coming from elevated point sources and are then placed within the appropriate UAM vertical layer.

Recommendations Point-source cutoff levels of 10 tons/yr for VOCs and no greater than 100 tons/yr for NOx are recommended for inclusion in the modeling emission inventory. Point sources must have the stack data needed to calculate effective plume height, so that the heights at which emissions are injected into the modeling system can be determined.

The Technical Committee may consider using a lower plume-rise cutoff level, particularly in areas where there may be a high density of point sources. Additionally, the CAAA specifies "major source" definitions that have lower cutoff limits for purposes such as application of reasonably available control technology (RACT), new source review (NSR) and creation of Emission Statements.15 The Technical Committee may consider using these lower cutoff limits in the modeling inventory. The Technical Committee should specify the plume-rise cutoff level to be used in delineating point-source and area-source emissions, and the level should be identified in the Protocol Document.

3.7.7 Consistency with national inventories

Comparisons should be made between the modeling inventory and the 1990 SIP and RFP tracking emission inventories reported in the EPA Aerometric Information Retrieval System (AIRS).16 Although these inventories will not be identical, such a check can be considered part of the quality assurance process. Major inconsistences should be noted and documented. It is especially important that those planning to use ROM-derived air quality data in the model simulations follow applicable guidance/regulations for reporting statewide emissions data to AIRS. These nationalinventories are used in the ROM modeling. As noted previously, using the ROM is the preferred procedure for estimating UAM boundary conditions and meteorological inputs. Attainment demonstrations will be less consistent if the ROM and the UAM use significantly different emissions data bases.

Recommendations For an acceptable attainment demonstration, documentation should be provided that shows that the modeling emission inventory is consistent with the emission inventory being reported in AIRS in accordance with applicable guidance and regulations.16CHAPTER 4 DATA QUALITY ASSURANCE AND MODEL DIAGNOSTIC ANALYSES

This chapter provides general guidance for quality assurance testing of component data input fields and diagnostic testing of base case episodes. These analyses are designed to establish and improve reliability of the input data and proper functioning of the model.

Although the UAM has been evaluated on a number of historical data bases, measures of model behavior with respect to observed data are necessary for new applications. Model developers and users perform diagnostic tests to uncover potential input data gaps that, when corrected, may lead to improved treatment of model processes. Regulators need some indication that the model captures the key features of the base meteorological episodes being applied in the model simulations in order to have confidence in a model's ability to predict future ozone (1) after applying projected growth and planned emission controls and (2) after applying alternative emission control strategies.

Important prerequisites for a model performance evaluation (see Chapter 5) are (1) quality assurance testing of model inputs and (2) diagnostic testing of the base meteorological episode simulation to ensure that the model is functioning properly and that apparently accurate model results are being obtained for the right reasons. For example, quality assurance testing of input data helps to ensure that apparently good model results have not resulted from compensating errors in input data. An excellent compilation of model performance evaluation techniques, including diagnostic tests and related issues, is contained in Reference 17. This reference serves as the basis for this chapter and for the model performance evaluation described in Chapter 5. Various graphical and numerical measures described below are treated in detail in Reference 17.

Two useful graphical displays for both quality assurance and diagnostic testing are mapping and time-series plots.

Mapping is a two- or three-dimensional spatial display of values illustrated with various contouring and tiling methods. These displays may depict political boundaries and monitoring site locations as well. Mapping capability is a multipurpose tool applicable for all forms of gridded data, such as future-year emission control strategy results and most input data fields (e.g., gridded wind fields, temperatures, and emission densities). Point-source locations may also be depicted to ensure that they are properly located. Spatial displays of predicted and observed ozone patterns are particularly useful as part of a model performance evaluation.

Time-series plots display hourly measured and predicted ozonevalues for specific locations such as monitoring sites. Timeseries plots provide an overview of the temporal performance of the model predictions. Comparison of time-series plots across multiple monitoring sites provides an indication of spatial response. Even though measured VOC or NOx species data may be limited, it may still be useful to plot time-series plots for some of these species, particularly for cases where ozone predictions do not meet expectations. Such plots may provide insights to the ozone prediction patterns and also to data base inconsistencies requiring further investigation.

The following sections describe recommended steps for conducting diagnostic testing of each base case meteorological episode simulation.

4.1 Step 1: Quality Assurance Testing of Component Fields

Starting with initial, quality-assured data, input data are developed for use in various UAM preprocessors. The first stage of diagnostic testing should focus on assessing the accuracy of major UAM input fields produced by the UAM preprocessors. Generally, the testing is qualitative in nature and based on comparing visual displays of preprocessor outputs with patterns exhibited by the observed data. Prior to conducting a base case meteorological episode simulation, individual air quality, meteorological, and emissions fields should be reviewed for consistency and obvious omission errors. Both spatial and temporal characteristics of the data should be evaluated. These checks may be only cursory, but errors uncovered by this component testing might be extremely difficult to diagnose later in the modeling process, when errors could arise from any subset of the data inputs. Examples of component testing include the following:

Air Quality: Check for correct order of magnitude, especially when using background values; assure reasonable speciation

Emissions: Plot various source types by grid cell and review major source locations with local emissions patterns; check major highwayroutes; generally, look for obvious omission errors; plot VOCs, NOx and CO by grid cell and cross-check with source distribution for logical patterns, such as high NOx levels associated with major power plants

Meteorology: Plot surface and elevated wind vectors and compare with monitoring stations and weather maps for consistent patterns; compare mixing height fields with sounding data; check temperature fields

In quality assurance testing of component input fields, the emphasis is on capturing rather large errors before performing model simulations.

Recommendations It is recommended that quality assurance testing of the air quality, emissions, and meteorological data input files be conducted before proceeding to diagnostic testing of the base case meteorological episodes. At a minimum, emissions data should be quality assured by looking at emission distribution maps and known source locations and emission strengths.

4.2 Step 2: Diagnostic Testing of the Base Case Meteorological episodes

After confidence has been achieved in producing UAM input fields, the UAM should be exercised for each base case meteorological episode. The initial run is termed a diagnostic simulation because review of initial base case simulations usually uncovers additional input errors requiring correction before an acceptable set of base case inputs can be derived. During this stage of the process, emphasis is placed on assessing the model's ability to correctly depict plume orientation and the timing of observed ozone maxima. Accordingly, visual methods such as mapping and time-series plotting, using measured data as reference marks, may be used to assess model behavior.

Recommendations To aid in interpreting simulation results, it is recommended that predicted and observed ozone concentration maps be constructed for each base meteorological episode simulation.

Concentration maps present spatial information on the structure of the ozone plume.Maps at 1- or 2-hour intervals should be constructed over the periods of most interest. While a typical period might be defined as early morning to late afternoon for the day of highest ozone, it is also useful to look at most time intervals under recirculation, stagnation, and transport conditions.

Consideration should also be given to constructing a map that depicts the highest predicted daily maximum ozone value for each grid cell. Examples of various mapping techniques are described in Reference 17.

It is also recommended that the predicted concentrations used in the time-series plots be consistent with the method for deriving predicted concentrations for the model performance evaluation described in Chapter 5. This method is based on Reference 17 and produces a weighted average using bilinear interpolation of the predictions from the four grid cells nearest to the monitor location.

Other methods for deriving predicted concentrations for time- series comparisons may be judged appropriate by the Technical Committee; some suggestions are contained in Chapter 5. These methods should be described in the Modeling Protocol.

If suitable data are available, time-series plots should be developed for NO and NO2, and for VOC species at selected locations, particularly for cases in which ozone time-series or mapping results are not consistent with expectations.

Comparisons of ozone precursors should be done for concentration levels above the detection limits for the monitoring equipment.4.3 Step 3: Additional Base Meteorological Episode Sensitivity Testing

In addition to running the base meteorological episode diagnostic simulation, other episode diagnostic simulations that perturb levels of emissions, initial and boundary conditions, and meteorological inputs may provide valuable information for identifying critical input areas and ensuring proper domain and episode selection. The following sampling of simulations, which are equivalent to sensitivity tests on major model inputs, illustrate the utility of this exercise.

1. Zero emissions - To indicate levels of sensitivity to emissions, all emissions are set to zero and the resulting predicted concentrations are compared with the base meteorological episode predictions that include emissions. A lack of substantial sensitivity may indicate a need to reexamine the selection of episodes or domains. Variations can be performed by zeroing out emission subsets, such as biogenic emissions, mobile- source emissions, and individual source categories.

2. Zero boundary concentrations - Inflow concentrations at the lateral boundaries and top of the modeling domain are reduced to zero or low background levels. Sensitivity of concentrations in the inner core and downwind portions of the modeling domain provide a measure of the boundary conditions' influence. This simulation can identify transport-dominated episodes and provide assurance that the upwind extent of the domain is adequate for episodes where intradomain emissions dominate. In minimum trans- port conditions, the second- and third-day concentrations (inner core and downwind locations) should be relatively insensitive to boundary-condition concentration changes.

3. Mixing height and wind speed variations - Much uncertainty is associated with mixing heights and wind speeds, and simulated concentrations are often sensitive to these inputs. Simulations that test the sensitivity of model estimates to variations in wind speed and/or mixing height provide bounds on some of the uncertainty resulting from these parameters. Large sensitivity may suggest that future model applications will need improvement in the meteorological data bases. Also, large sensitivity may suggest a need to consider alternative wind field generation techniques.Certain numerical measures, which are recommended in the discussion of model performance evaluation in Chapter 5, are also useful diagnostic tools. For example, consistent underpredictions usually produce bias values less than zero. This phenomenon could be due to various factors, such as overstatement of wind speeds or mixing heights, or low emission estimates. Modelers are encouraged to use numerical as well as graphical techniques in the diagnostic
process.

The diagnostic analyses described in this chapter are considered to be a starting point for a specific modeling study. Diagnostic tests discussed in Reference 17 should be consideredwhenever possible. Also, the EPA is developing a UAM Postprocessing System18 to assist in diagnostic testing of the base meteorological episodes. Availability of this software will be announced through the SCRAM BBS.

Recommendations Diagnostic testing of the model should begin with quality assurance testing on input data files (Section 4.1). Diagnostic testing of each base meteorological episode should follow (Section 4.2). Additional diagnostic sensitivity tests for the base episode should also be considered (Section 4.3), including using zero emissions and/or zero boundary condi- tions, and varying mixing height and wind speed estimates.

Agreement should be obtained among members of the Technical Committee concerning input field modifications arising from the quality assurance testing. These modifications should be based on scientific or physical reasoning and not just on what will improve model performance. All changes to the data that result from the diagnostic testing should be documented and justified.

In addition, all diagnostic steps should be documented to avoid misinterpretation of model performance results. After confidence is gained that the simulation is based on reasonable interpretations of observed data, and model concentration fields generally track (both spatially and temporally) known urban-scale plumes, a performance evaluation based on numerical measures is conducted for each base meteorological episode (see Chapter 5).CHAPTER 5 MODEL PERFORMANCE EVALUATION

At some point in the modeling process, an assessment of model performance is required. Specifying rigid rejection/acceptance criteria has not been supported by model developers nor by decision makers participating in previous modeling efforts. Instead, performance measures derived from previous photochemical modelapplications may provide a reasonable benchmark for model performance. Also, graphical procedures reveal qualitative relationships between predicted and observed concentrations that can be used in model performance evaluation.

Poor performance may necessitate (1) delaying model applications until further diagnostic testing and quality assurance checks are reflected in the input data base, or (2) selecting another meteorological episode for modeling. However, this is not a valid reason for delaying SIP attainment demonstration submittals beyond the dates required in the 1990 CAAA. Also, cases where good model performance is shown should be reviewed as well, because compensating errors can induce spurious agreement among observed and predicted values.

5.1 Performance Measures

This section describes recommended graphical and statistical performance measures for ozone predictions. These measures should be applied for modeling results beginning on the second day of the modeled episode. As described in Section 3.1, the first day is eliminated to mitigate the effects of specifying initial conditions arbitrarily. Performance measures should also be considered for ozone precursors wherever possible, based on availability of monitored data. Obvious problems exist in comparing model predictions with observed values. The UAM output represents volumetric (e.g., 25 km3), 1-hour average concentrations, but air quality data represent point locations with various sampling periods. This "incommensurability" may lead to considerable uncertainty in the comparisons, especially for precursor species that are not buffered chemically and may have been sampled at locations not representative of areawide concentrations.

As part of the UAM Postprocessing System, the EPA is currently developing a model performance utility that will contain the performance measures listed below. Users will be able to access this utility for model performance evaluation testing. This utility is expected to be available in late 1991. Model users willbe advised on its availability through the EPA SCRAM BBS.

The measures used in the performance evaluation should include both qualitative (e.g., graphical) and quantitative (e.g.,statistical) analyses. Statistical measures may provide a meaningful test of model performance for dense monitoring networks, such as those for special field studies. However, for some routine monitoring networks where coverage may be sparse, statistical measures may provide a distorted view of model performance, especially for paired values.

Reference 17 provides detailed descriptions of graphical and statistical measures available for assessing the performance of photochemical grid models. The Technical Committee should consult this reference when formulating model performance evaluationmethods, and may want to use it for developing additional performance evaluation procedures other than those recommended in this Guidance Document.

5.1.1 Graphical performance procedures

Graphical displays can provide important information onqualitative relationships between predicted and observed concentrations. At a minimum, the following graphical displays should be developed for each meteorological episode: time-series plots, ground-level isopleths, quantile-quantile plots, and scatterplots of predictions and observations.

Time-series plots - The time-series plot, developed for each monitoring station in the modeling domain, depicts the hourly predicted and observed concentrations for the simulation period. The time series reveals the model's ability to reproduce the peak prediction, the presence of any significant bias within the diurnal cycle, and a comparison of the timing of the predicted and observed ozone maxima.17

Ground-level isopleths or tile maps - Ground-level isopleths or tile maps display the spatial distribution of predicted concentrations at a selected hour. Isopleths of predicted daily maxima may also be constructed. The isopleths provide information on the magnitude and location of predicted pollutant "plumes." Superimposing observed hourly or daily maximum concentrations on the predicted isopleths reveals information on the spatial alignment of predicted and observed plumes. Subregional biases of predictions versus observations may result from spatial misalignments.Scatterplots of predictions and observations - Scatterplots depict the extent of bias and error in the ensemble of hourly prediction-observation pairs. Bias is indicated by the systematic positioning of data points above or below the perfect correlation line. The dispersion of points is a measure of the error in thesimulation. The scatterplot also reveals outlier predictionobservation pairs.

Quantile-quantile plots - Quantile-quantile plots compare the frequency distributions of rank-ordered observed and rank-orderedpredicted concentrations. The observed and predicted concentrations are sorted from highest to lowest then plotted on an x-y plot. This graphically depicts any model bias over the frequency distribution.

"Paired" predictions of daily maxima - In attainment demonstrations, particular interest is focused on daily maximum ozone concentrations. One test that may provide insight into model performance is to consider model predictions occurring within ±1 hour of the observed daily maxima at each monitoring site in the nine grid squares surrounding and including the monitor. The "prediction," for purposes of this pairing, would be the one that agrees most closely with the observed daily maximum for each site. This method may be useful for sparse meteorological and air quality networks, because it recognizes that both the inputs and air quality observations have some attendant uncertainty. Resulting comparisons can be superimposed on a map depicting emissions and monitors to help assess model performance.

Recommendations At a minimum, the following graphical displays are recommended in the evaluation of each meteorological episode:

Time-series plots of predicted and observed hourly ozone values should be constructed for each simulation period for each monitoring station where data are available.

Ground-level isopleths or tile maps of the spatial distribu- tion of predicted concentrations should be constructed for selected hours. Also, ground-level isopleths or tile maps of the daily ozone maxima should be constructed. The correspond- ing observed concentrations should be superimposed on the predicted concentration isopleths to analyze spatial plume patterns and ozone magnitudes.

Scatterplots should be constructed for all hourly prediction-observation pairs for each simulation; quantile-quantile plots are also recommended for each simulation.

The development of additional graphical displays, such as the paired predictions of daily maxima, is encouraged. The graphical displays to be used in the model performance evaluation should be described in the Protocol.

5.1.2 Statistical performance measures

Statistical performance measures can provide meaningful measures of model accuracy for dense monitoring networks, such as those for special field studies. However, statistical measures may give a distorted view of model performance in cases of routine monitoring networks, where coverage may be sparse. The Technical Committee should evaluate the adequacy of the existing monitoring network for conducting statistical tests for performance evaluation.

Recommendations It is recommended that, at a minimum, the following mathematical formulations be applied as measures for model performance evaluation. These formulations are detailed in Appendix C.

Unpaired highest-prediction accuracy - This measure quantifies the difference between the highest observed value and highest predicted value over all hours and monitoring stations.

Normalized bias test - This test measures the model's ability to replicate observed patterns during the times of day when available monitoring and modeled data are most likely to represent similar spatial scales.

Gross error of all pairs above 60 ppb - In conjunction with bias measurements, this metric provides an overall assessment of base case performance and can be used as a reference to other modeling applications. Gross error can be interpreted as precision.

Additional measures may include the following:

Average station peak prediction accuracy - This is a measure of peak performance at all monitor sites, using pairings based on time and space.Bias of all pairs above 60 ppb - This bias metric measures the overall degree to which model predictions overestimate or underestimate observed values. Note, however, that a zero bias for several observation-prediction pairs can be caused by a canceling effect of overprediction and underprediction in different subregions.

Bias of all station peaks - For this metric, bias calculations are performed on observation-prediction pairs associated with peak ozone values for each monitoring station. This metric provides information on the model's ability to replicate peak ozone observations.

Fractional bias for peak concentration - Fractional bias is calculated for both the mean and the standard deviation of peak predicted and observed values. This metric provides additional information on the model's ability to replicate peak ozone observations.

5.2 Assessing Model Performance Results

Both graphical and statistical performance measures should be used for the performance evaluation. However, although the recommended statistical measures should be applied for all meteorological episodes and monitoring networks, caution is suggested for interpreting these measures in cases of sparse monitoring network coverage. The Technical Committee should consider the monitoring network design in interpreting statistical

measures.

In assessing model simulation results for the performance evaluation, there is no rigid criterion for model acceptance or rejection (i.e., no pass/fail test). Reference 17 states that, based on past photochemical model evaluations, this type of modeling "generally produces peak (unpaired) prediction accuracy, overall bias, and gross error statistics in the approximate ranges of ±15-20 percent, ±5-15 percent, and 30-35 percent, respectively." In general, performance results that fall within these ranges would be acceptable. However, caution is urged in using these ranges asthe sole basis for determining the acceptability of model performance. These ranges were derived from past model performanceevaluations with varying densities of air quality and meteorological monitoring networks and corresponding variations in the qualityand quantity of aerometric model input data. In some cases, they reflect use of earlier versions of the UAM. Thus, these ranges should be used in conjunction with the graphical procedures to assess overall model performance.

If statistical results are worse than the above ranges and graphical analyses also indicate poor model performance, users should consider choosing an alternative meteorological episode for modeling. Performance evaluations should be done on other candidate episodes to identify those that might result in better model performance.

If statistical results are worse than the above ranges for any of the three statistics, but graphical analyses generally indicateacceptable model performance, simulation results used for attainment demonstration should be applied with caution. Users may consider conducting performance evaluations on other candidate episodes to identify any that might yield improved model performance.

A minimum of 3 primary episode days is required for use in the model simulations for attainment demonstration (Section 6.4). If fewer than 3 primary episode days can be identified that have acceptable model performance for the attainment demonstration, the responsible regulatory agencies are strongly encouraged to take steps that will improve model performance for any future attainment demonstrations. For serious and above nonattainment areas, thismay require short, intensive field studies to supplement installation of the enhanced monitoring network required under the CAAA of 1990.

Recommendations It is recommended that the model performance for each meteorological episode be assessed as follows:

1. The graphical performance procedures specified in Section 5.1.1 should be conducted for each meteorological episode. To assess model performance, the Technical Committee should analyze the time-series plots, ground-level isopleths, quantile-quantile plots, and scatterplots. Use of "paired" predictions of daily maxima may also be considered.

2. The statistical performance measures specified in Section 5.1.2 should also be derived and evaluated for each meteoro- logical episode. When interpreting these measures, the monitoring network density and design should be considered.Caution is urged when interpreting the statistical measures for a sparse monitoring network.

It is recommended that the statistical performance measures be compared with the following ranges:



### ! Unpaired highest prediction accuracy: ±15-20 percent



### ! Normalized bias: ±5-15 percent



### ! Gross error of all pairs >60 ppb: 30-35 percent

If all of these statistical measures are within the ranges shown, and the graphical performance procedures also are interpreted to yield acceptable results, then the model is judged to be performing acceptably.

If any of the statistical measures are worse than the above ranges, or the graphical procedures are interpreted to yield unacceptable performance, users should consider choosing an alternative highly ranked meteorological episode for the attainment demonstration. Performance evaluations should be conducted on a prospective alternative episode to determine whether it yields improved model performance.

Additional model performance measures are encouraged. These should be described in the Modeling Protocol.CHAPTER 6 ATTAINMENT DEMONSTRATION

This chapter provides guidance on using modeling simulations for attainment demonstrations. The primary reason for conducting photochemical modeling is to demonstrate the effectiveness of alternative control strategies in attaining the NAAQS for ozone throughout the modeling domain. This demonstration consists of four main parts: (1) developing attainment-year modeling emission inventories, (2) developing alternative-control strategy emission inventories, (3) performing model simulations for the attainment year with and without alternative control strategies, and (4) comparing attainment year and control strategy simulation results with the ozone NAAQS. Attainment year and control strategy simulations are conducted for each selected meteorological episode (see Section 3.1).

6.1 Developing Attainment-Year Model Inputs

The attainment-year modeling inventory must be derived from the 1990 SIP nonattainment base year inventory, adjusted for episode-specific meteorology, and then projected to the attainmentyear. Also, to the extent possible, initial- and boundarycondition ozone and precursor concentrations must be projected to the attainment year. The attainment year is determined by the nonattainment area designation and the attainment dates specified in the 1990 CAAA. Projections of emission inventories reflect the net effect of mandated controls and growth projections for various source categories. Guidance for projecting inventories is available in Procedures for Preparing Emissions Projections.14 The most direct method for projecting initial- and boundary-condition precursor concentrations is by applying ROM simulation results for which the UAM domain is nested within the ROM domain (see Chapter 3). In the absence of available ROM data, the projection of ozone precursor concentrations used for initial conditions typically has been done by linear scaling based on emission changes projected to take place from the 1990 base year to the future year. For initial ozone concentrations, there is little basis for doing anything other than assuming initial ozone remains constant. In the absence of regional modeling results or better information, the guidance in Reference 7 for specifying future boundary conditions may be followed.

Recommendations It is recommended that the EPA guidance document entitled Procedures for Preparing Emissions Projections14 be consultedfor developing attainment-year inventories. The guidance document provides procedures for projecting point-source, area-source, mobile-source, and biogenic emissions, and addresses projections of spatial, temporal, and chemical composition changes between the 1990 SIP inventory and the attainment-year inventory.

Also, if regional modeling predictions for the attainment year are available, it is recommended that these be used to derive the attainment-year initial and boundary conditions for the attainment-year model simulations (see Chapter 3).

6.2 Construction of Attainment Year Emission Control Strategies

Many possible attainment-year emission control strategies can be set up and simulated. Eventually, a modeling analysis must be submitted for approval as the basis of a SIP demonstration. The effectiveness of a given set of control measures in reducing ozone (and perhaps other pollutants) is a major factor in selecting the final emission control strategy.

Prior studies have typically used a progression of control strategy scenarios in the modeling to ascertain an effective strategy for attainment. A suggested logical progression is the following:

1. Simulate the CAAA and other mandated control measures for the attainment year to determine if these measures are sufficient to demonstrate attainment of the ozone NAAQS.

2. If mandated controls are insufficient to demonstrate attainment, superimpose a series of additional, across- the-board reductions in VOCs-only, VOCs-plus-NOx, and NOx-only strategies, relative to the mandatory CAAA controls, to identify a suitable emission-reduction target range.

3. Once an approximate target range is ascertained in steps 1 and 2, simulate control strategies that reflect source-specific or source-category-specific control measures and that realize the approximate emission reductions identified as sufficient to reduce daily maximum ozone to 0.12 ppm or less.

4. Adjust the strategy chosen in step 3 until it is sufficient to demonstrate attainment of the NAAQS, asdescribed in Section 6.4. Adjustments may be needed in VOC controls, or NOx controls, or both.

Recommendations The procedures for deriving control strategies for evaluation in the attainment demonstration must be specified in the Modeling Protocol.

Emission control strategies for linked urban-area modeling domains (e.g., northeastern U.S. Corridor) should be coordi- nated among State agencies having lead responsibility for respective domains to ensure consistency among the domains.

6.3 Performing Attainment-Year Simulations to Assess Various Control Strategies

Many graphical display and numerical procedures are available for illustrating the effects of alternative emission control strategies on predicted concentrations of ozone and other species. For example, the emission levels in the control strategies are often compared with the attainment-year base emissions. Also of interest are comparisons with the inventory derived for purposes of model performance evaluations and corresponding base-case UAM results. Difference maps are extremely useful for illustrating changes in daily maximum ozone predictions throughout the modeling domain.

Recommendations The focus of any ozone attainment demonstration is on the daily maximum 1-hour concentration predicted at each location in the modeling domain. However, it is recommended that responsible parties broaden the scope of an attainment demon- stration to examine the impact on other important metrics, such as different concentration averaging times, population exposure, subdomain and temporal impacts, effects on other pollutant species, and other important measures that are sensitive to emission control strategies.

For deriving initial and boundary conditions for a particular urban-area domain, using appropriate regional model predic- tions that reflect control measures applied in other urban- area domains within the regional modeling domain is recommended.6.4 Using Modeling Results in the Attainment Demonstration

As described in Section 3.1, at least 3 primary episode days should be modeled for the attainment demonstration. Also, a minimum of 1 primary day should be modeled from each identified meteorological regime. Therefore, for example, if there are three meteorological regimes, at least 1 primary episode day from each regime should be modeled; if there are only two meteorological regimes, at least 2 primary episode days should be modeled from one of the regimes and at least 1 primary episode day modeled from the other regime. Note that the episodes simulated would generally be at least 48 hours long (i.e., the first day would be an initial modeling day and the second day would be the primary episode day). This would count as simulation of 1 primary episode day.

To demonstrate attainment of the ozone NAAQS, there should be no predicted daily maximum ozone concentrations greater than 0.12 ppm anywhere in the modeling domain for each primary episode day modeled. Alternative methods for demonstrating attainment mustbe approved by the appropriate EPA Regional Office on a case-bycase basis.

The attainment test described in the preceding paragraph is consistent with the flexibility allowed in the choice of episode days (see Section 3.1) and reflects concerns over the difficulty of accurately estimating emissions inputs to the model.

Recommendations To demonstrate attainment of the ozone NAAQS, there should be no predicted daily maximum ozone concentrations greater than 0.12 ppm anywhere in the modeling domain for each primary episode day modeled. At least 3 primary episode days should be modeled.

States may opt to conduct more comprehensive statistical testing of the modeling results for the attainment demonstra- tion. Any alternative methods for attainment demonstration must be approved by the appropriate EPA Regional Office on a case-by-case basis. Any optional methods should be agreed upon during the development of the Modeling Protocol.

6.5 Exceptions to Guidance Document

It is not possible in a general guidance document like this to anticipate all contingencies associated with developing anattainment demonstration study. The Modeling Policy Oversight and Technical Committees responsible for a specific modeling study may propose an alternative photochemical modeling approach provided that (1) the Modeling Protocol requires consensus on the proposed alternative approach within the Technical Committee, and (2) justification for the proposed approach is documented. Application of any alternative photochemical modeling approach must first receive concurrence in writing from the responsible EPA Regional Office(s).REFERENCES

1. Morris, R.E., T.C. Myers and J.L. Haney, 1990. User's Guide for the Urban Airshed Model, Volume I: User's Manual for UAM (CB-IV), EPA-450/4-90-007A, U.S. Environmental Protection Agency, Research Triangle Park, NC (NTIS No.: PB91-131227).

2. Morris, R.E., T.C. Myers, E.L. Carr, M.C. Causley, S.G. Douglas and J.L. Haney, 1990. User's Guide for the Urban Airshed Model, Volume II: User's Manual for the UAM (CB-IV) Modeling System, EPA-450/4-90-007B, U.S. Environmental Protection Agency, Research Triangle Park, NC (NTIS No.: PB91- 131235).

3. Douglas, S.G., R.C. Kessler and E.L. Carr, 1990. User's Guide for the Urban Airshed Model, Volume III: User's Manual for the Diagnostic Wind Model, EPA-450/4-90-007C, U.S. Environmental Protection Agency, Research Triangle Park, NC (NTIS No.: PB91- 131243).

4. Causley, M.C., J.L. Fieber, M. Jimenez and L. Gardner, 1990. User's Guide for the Urban Airshed Model, Volume IV: User's Manual for the Emissions Preprocessor System, EPA-450/4-90- 007D, U.S. Environmental Protection Agency, Research Triangle Park, NC (NTIS No.: PB91-131250).

5. Tang, R.T., S.C. Gerry, J.S. Newsom, A.R. Van Meter, R.A. Wayland, J.M. Godowitch and K.L. Schere, 1990. User's Guide for the Urban Airshed Model, Volume V: Description and Operation of the ROM-UAM Interface Program System, EPA-450/4- 90-007E, U.S. Environmental Protection Agency, Research Triangle Park, NC (NTIS No.: PB91-131268).

6. DaMassa, J., 1990. Technical Guidance Document: Photochemical Modeling. California Air Resources Board, Technical Support Division, Sacramento, CA.

7. U.S. Environmental Protection Agency, 1991. Criteria for Assessing the Role of Transported Ozone/Precursors in Ozone Nonattainment Areas, EPA-450/4-91-015, Office of Air Quality Planning and Standards, Re search Triangle Park, NC (NTIS No.: PB91-195958). 8. U.S. Environmental Protection Agency, 1991. Gridded Model Information Support System (GMISS) User's Guide, Volume II: UAM Subsystem, EPA-450/4-91-009, Office of Air Quality Planning and Standards, Res earch Triangle Park, NC (NTIS No.:PB91-206268).

9. U.S. Environmental Protection Agency, 1987. On-Site Meteoro- logical Program Guidance for Regulatory Modeling Applications, EPA-450/4-87-013, Office of Air Quality Planning and Stan- dards, Research Triangle Park, NC (NTIS No.: PB87-227542).

10. U.S. Environmental Protection Agency, 1989. Procedures for Applying City-Specific EK MA, EPA-450/4-89-012, Office of Air Quality Planning and Standards, Research Triangle Park, NC (NTIS No.: PB90-256777).

11. U.S. Environmental Protection Agency, 1991. Enhanced Ozone Monitoring Regulations: Proposed Rule (Draft), 40 CFR Part 58, Office of Air Quality Planning and Standards, Research Triangle Park, NC.

12. U.S. Environmental Protection Agency, 1991. Emission Inven- tory Requirements for Ozone State Implementation Plans, EPA- 450/4-91-010, Office of Air Quality Planning and Standards, Research Triangle Park, NC.

13. U.S. Environmental Protection Agency, 1991. Procedures for the Preparation of Emission Inventories for Volatile Organic Compounds, Volume II: Emission Inventory Requirements for Photochemical Air Quality Simulation Models, EPA-450/4-91-014, Office of Air Quality Planning and Standards, Research Triangle Park, NC (NTIS No.: PB91-216176).

14. U.S. Environmental Protection Agency, 1991. Procedures for Preparing Emissions Projections, Office of Air Quality Planning and Standards, Research Triangle Park, NC (Draft).

15. U.S. Environmental Protection Agency, 1991. Guidance for Initiating Ozone/CO SIP Emission Inventories Pursuant to the 1990 Clean Air Act Amendments, Office of Air Quality Planning and Standards, Research Triangle Park, NC.

16. U.S. Environmental Protection Agency, 1989. Aerometric Information Retrieval System (AIRS), Volume I. Office of Air Quality Planning and Standards, Research Triangle Park, NC.

17. Tesche, T.W., P. Georgopoulos, F.L. Lurmann and P.M. Roth, 1990. Improvement of Procedures for Evaluating Photochemical Models. Draft Final Report. California Air Resources Board, Sacramento, CA.18. U.S. Environmental Protection Agency, 1991. Urban Airshed Model Postprocessing System User's Guide, Office of Air Quality Planning and Standards, Research Triangle Park, NC (Draft).APPENDIX A RECOMMENDED MODELING PROTOCOL CONTENTS

Table 1 of Chapter 2 lists recommended contents for a ModelingProtocol. This appendix gives a general description of each component, to aid in the development of the Protocol. As stated in Chapter 2, the contents presented here are adopted from the CARB Technical Guidance Document: Photochemical Modeling.6

UAM MODELING STUDY DESIGN

Background and Objectives

The Protocol Document should describe the policy and technical objectives of the study and pertinent background information such as the legislative mandate under which the study is being done.

Schedule

Development of a complete schedule for all phases of the project is needed. The critical paths and deadlines should be identified and discussed, as should a schedule for addressing critical issues that require special attention, such as air quality and meteorological data preparation and quality assurance, episode selection, and emission inventory preparation and quality

assurance.

Deliverables

A list of the interim and final deliverables for the modeling study should be specified.

Modeling Policy Oversight/Technical Committees

The composition and responsibilities of the Modeling Policy Oversight and Technical Committees should be specified to the extent possible. Meeting frequency and circumstances for convening a meeting should be identified. Because technical conflicts may arise, a resolution process for handling them should be included.

Participating Organizations

The organizations that are sponsoring the modeling study andthose that may contribute to it should be identified.

Relationship to Regional Modeling Protocols

Procedures for coordinating development of the urban-area Modeling Protocol with the regional Modeling Protocol should be described. This would include a description of control strategies, emission inventories, projection years, modeling episodes, etc. The coordination of urban-area Modeling Policy and Technical Committees with their regional counterparts should be described.

Relationship to Other Urban Area Modeling Protocols

In some cases, such as the Northeast U.S., nonattainment MSA/CMSAs required to do attainment demonstrations may be linked to other nonattainment MSA/CMSAs. It is important that procedures be established for coordinating the Modeling Protocols among these areas, and that these procedures be clearly specified in each nonattainment area Modeling Protocol. It is likely that Modeling Policy Oversight and Technical Committees will include some joint membership among the nonattainment areas.

Relationship to Planning/Strategy Groups

Key planning agencies and others responsible for emission projections or other model inputs should be identified, and the means by which these groups interact to obtain realistic growth projections and control strategies should be discussed.

DOMAIN AND DATA BASE ISSUES

Preprocessor Programs

The preprocessor programs to be used in constructing any of the model input fields should be identified and described.

Data Bases

The proposed air quality and meteorological data bases should be described. The completeness of the data base, techniques for filling in missing data, and quality assurance procedures should be discussed.

Base Meteorological Episode Selection

The episode selection criteria should be detailed, includingthe methodology to group candidate episodes into meteorological regimes. How the episodes will be used in the modeling study should also be described.

Modeling Domain

The Protocol should describe the criteria for selecting the size and location of the modeling domain. This would include a description of the MSA/CMSA area size, locations of major sources outside the MSA/CMSA that may affect it, sensitivity analyses that may be conducted to assess boundary effects on domain predictions, relationship of domain size to the episodes selected for use in the modeling study, etc.

Horizontal Grid Resolution

The Protocol should describe the horizontal grid resolution to be applied to the modeling domain. If a resolution coarser than 5 x 5 km is chosen, justification for this choice should be provided.

Number of Vertical Layers

The Protocol should specify the number of vertical layers to be used in the UAM simulations. If a layering scheme other than the one recommended in Chapter 3 is chosen, justification for using the alternative layering should be given.

Emission Inventory

The assumptions, methodologies, and appropriate guidance references to be used in constructing the modeling emission inventory should be described. Quality assurance procedures should also be described.

Specification of Initial and Boundary Conditions

The techniques to be used to specify the initial and boundary conditions for the base meteorological episodes and the attainmentyear should be described. The assumptions to be used in forecasting attainment-year conditions should be documented. If a nested grid approach is used (e.g., using predictions from the ROM through the ROM/UAM Interface System), the details for implementation should be described (see Chapter 3).Wind Field Specification

The proposed techniques for specifying the wind fields shouldbe described. The procedures to be used to determine the representativeness of the simulated wind fields should be technically justified and documented (see Chapter 3).

Mixing Heights

The techniques to be used for deriving the mixing height for the modeling domain should be described.

Sources of Other Input Data

The Protocol Document should describe the data and techniques to be used to specify other input data, such as cloud cover, water vapor, UV radiation, surface temperature, terrain, and land use and surface characteristics.

QUALITY ASSURANCE AND DIAGNOSTIC ANALYSES

Quality Assurance Tests of Input Components

The specific quality assurance tests to be used on the data input fields should be described (see Chapter 4).

Diagnostic Tests of Base Case Simulation

The specific diagnostic tests to be used for the base-case meteorological episode simulations should be described. Asdiscussed in Chapter 4, these should include, at a minimum, timeseries plots, observed and predicted ozone maps, zero emissions and zero boundary conditions tests, and tests on the mixing height variations and wind fields. Additional diagnostic tests are encouraged and should be described in the Protocol.

MODEL PERFORMANCE EVALUATION

Performance Evaluation Tests

The graphical, statistical, and other measures to be used in the model performance evaluation should be specified. At aminimum, the tests recommended in Chapter 5 should be included. Additional measures may also be considered and should be described if they are to be used.

ATTAINMENT DEMONSTRATIONS

Identification of Attainment-Year Mandated Control Measures

The Protocol Document should include a description of the 1990 CAAA control measures and other measures mandated to be implemented by the attainment year.

Methodologies for Generating Control Strategy Emission Inventories

The procedures for deriving alternative-control-strategyemission scenarios to meet the study objectives should be described. A description of how the control scenarios would relate to applicable control strategies for areas adjacent to the modeling domain (particularly upwind areas) should be included.

Procedures for Attainment Demonstration

Procedures for using the model simulation results in demonstrating attainment of the ozone NAAQS should be included.

SUBMITTAL PROCEDURES

The documentation and analyses that will be submitted for EPA Regional Office review should be described. Also, any documentation other than the Modeling Protocol requiring EPA Regional Office approval should be described.APPENDIX B

IDENTIFICATION OF METEOROLOGICAL REGIMES CORRESPONDING WITH HIGH OBSERVED OZONE

The following is a procedure that may be used to assist in selecting modeling episodes. Other techniques may be considered on a case-by-case basis; they should be described in the Modeling Protocol and approved by the a ppropriate EPA Regional Office.

Identification of meteorological regimes for a given area under review begins with constructing a climatological windrose of high ozone days. The windrose is constructed by first selecting all days from the period 1987 to present during which at least one ozone monitor within the area recorded an exceedance of the ozone NAAQS or some other cutoff level (e.g., 100 ppb). Additional years of data are encouraged in constructing the climatological windrose (e.g., 1980-1991). Next, for each exceedance day, calculate the morning (i.e., 7:00 a.m. - 10:00 a.m.) resultant wind velocity. Then group the resultant wind velocities for all of the exceedance days into eight compass directions plus calm, to establish a climatic windrose of high-ozone days for the area under review. Calm winds are defined as those with speeds less than 1.5 m/s and referred to as the null wind direction. The windrose will include nine bins (0-8); place the wind directions corresponding to the eight compass points into bins 1-8, and the calm or null wind direction into bin 0. The bins with frequencies significantly higher than the average frequency for all bins should be defined as the "predominant wind directions" (PWD). Next, compare the morning (i.e., 7:00 a.m. - 10:00 a.m.) resultant wind velocity for each exceedance day during 1987-89 and more recent years with the climatic windrose of high-ozone days. Categorize exceedance days with wind directions corresponding to previously identified climatic PWD's as belonging to that PWD. Lump all other exceedance days occurring during 1987-89 and later into a category called "other." Rank each exceedance day within each PWD category and within the "other" category according to its areawide daily maximum ozone observation. Within each category, the day with the highest areawide daily maximum concentration is ranked first.

After the steps described in the two preceding paragraphs are completed, meteorological regimes can be defined for use in the attainment demonstration test described in Section 6.4. This may be done as follows:1. Choose the two PWD's which contain the highest areawide daily maximum ozone values from 1987 to the most recent year with data available. These represent two of the meteorological regimes to consider in the attainment
test.

2. The third "meteorological regime" to be considered in the attainment test is comprised of all exceedance days previously categorized as "other" plus those belonging to any PWD not chosen in step 1.

3. Identify the top 3-ranked exceedance days from within each of the three meteorological regimes identified in steps 1 and 2. These days are candidates for modeling in the attainment test. Final choice from among these candidates is based on criteria identified in Section 3.1.

4. It may happen that one or more of the meteorological regimes identified in step 1 contains fewer than 3 exceedance days. If this occurs, exceedance days included within PWD's which have been lumped in the third meteorological regime (see step 2) may be added to one or both of the first two regimes. If this proves necessary, selection of days to supplement those in one or both of the first two regimes needs to be decided on a case-by- case basis keeping in mind the goal of this exercise: to provide a choice of exceedance days reflecting high ozone concentrations with meteorological conditions which frequently coincide with observed exceedances.APPENDIX C PERFORMANCE MEASURE FORMULATIONS

RECOMMENDED PERFORMANCE MEASURES

1. Unpaired Highest-Prediction Accuracy (Au)

where

Au = unpaired highest-prediction accuracy (quantifies the difference between the magnitude of the highest 1-hour observed value and the highest 1-hour predicted value

co(.,.) = maximum 1-hour observed concentration over all hours and monitoring stations

cp(.,.) = maximum 1-hour predicted concentration over all hours and surface grid squares

2. Normalized Bias Test (D*)

where

D* = normalized bias obtained from all hourly prediction-observation pairs

N = number of monitoring stations

Hi = number of hourly prediction-observation pairs for monitoring station i

NT = total number of station-hours

co(i,j) = observed value at monitoring station i for hour j

cp(i,j) = predicted value at monitoring station i for hour j

3. Gross Error of All Pairs >60 ppb (Ed*)where

Ed* = normalized gross error for all hourly prediction-observation pairs for hourly observed values >60 ppb

NT = total number of station hours (defined previously)

N = number of monitoring stations

Hi = number of hourly prediction-observation pairs for monitoring station i

co(i,j) = observed value >60 ppb at monitoring station i for hour j

cp(i,j) = predicted value at monitoring station i for hour j

OTHER SUGGESTED PERFORMANCE MEASURES

_ 1. Average Station Peak Prediction Accuracy

where

_ A = mean paired peak prediction accuracies averaged over all monitoring stations

N = number of monitoring stations

Co(i,ti) = peak observed value at monitoring station i for hour ti

Cp(i,ti) = predicted value at monitoring station i for hour ti

ti = hour of peak observed value at monitoring station i2. Bias of All Pairs >60 ppb(D60)

where

D60 = non-normalized bias from all hourly prediction-observation pairs for observed values >60 ppb

NT = total number of station-hours (defined previously)

N = number of monitoring stations

Hi = number of hourly prediction-observation pairs for monitoring station i

co(i,j) = observed value >60 ppb at monitoring station i for hour j

cp(i,j) = predicted value at monitoring station i for hour j

3. Bias of All Station Peaks (Dpeak)

where

Dpeak = non-normalized bias from all prediction- observation pairs for peak observed values at all monitoring stations

N = number of monitoring stations

co(i,ti) = peak observed value at monitoring station i for hour ti)

cp(i,ti) = predicted value at monitoring station i for hour ti

ti = hour of peak observed value at monitoring station i

4. Fractional Bias for Peak ConcentrationThe fractional bias is calculated for both the mean and standard deviation of peak ozone values, as follows:

where

Fm = fractional bias of means

Fs = fractional bias of standard deviation

mo = mean maximum observed concentration

mp = mean peak predicted concentration

so = standard deviation of peak observed concentrations

sp = standard deviation of peak predicted concentrations

The means and standard deviations of predicted and observed concentrations are determined by each of two methods:

Peak station values:

co(i,.) = maximum observed concentration at monitoring station i across all hours

cp(i,.) = maximum predicted concentration at monitoring station i across all hours

where i = 1,...,N monitoring stations

Peak hourly values:

co(.,j) = maximum observed concentration at hour j across all monitoring stations

cp(.,j) = maximum predicted concentration at hour j across all monitoring stations

where j = 1,...,H hours

The fractional bias of the mean and standard deviation varies from -2 to +2. Negative values indicate overprediction and positive values indicate underprediction.