9-1 9.0 QUALITY ASSURANCE

9.1 Objective of the QA Program

A quality assurance (QA) program is needed to ensure that products produced in the BART exemption or BART control applicat ions satisfy the regulatory objec tives of the program. The scope of the QA program for a particular application should be defined in the protocol. Common elements include the configuration, data ba se development, setup and execu tion of the CALPUFF air quality model and processing of modeling results to determin e compliance with visibili ty thresholds. In most if not all screening cases (see Chapter 6), states or source operators will use the CALMET datasets provided by CENRAP. Applications involving sour ce-specific or alternat ive modeling that use different datasets, modeling options or tools will need to ensure th at an appropriate QA program is defined in their protocols. More extensive qua lity assurance will be required in these latter applications. It is the re sponsibility of the state ensure that an adequa te QA program, defined in the protocol, is imple mented faithfully.

The CALPUFF modeling system contains features to facilitate quality assurance. These include the automatic production of “QA” files fo r various datasets, incl uding geophysical fields, sources and receptors, and imbedded tracking of mode l options and switches w ithin the output files from the major modules. The CALPUFF Graphical Us er Interface system (GUI) allows these QA files to be displayed graphically. In addition, a software management system is available to track version and level numbers associated each program and utili ty. This information reproduced in all of the output files, creating an aud it trail of software versions and major model options.

The BART modeling process involves multiple or ganizations. As mentioned earlier, the states have overall responsibility for the process and may al so execute some or all of the modeling. CENRAP is contributing general guidance via these gui delines and has funded the development of meteorological fields for use in screening applicati ons. BART-eligible source operators will need to provide process information and emissions data for use in the analyses. Furthermore, ‘subject to BART’ sources will need to be actively involved in control technology decisions and assessments. Finally, some or all of the modeling steps may be car ried out by contractors on behalf of a state or a

source operator.

Each of these organizations has a responsibili ty to ensure that it is providing correct information to others and to evaluate the quality of any analyses it is performing, whether with data of its own or from others. This chapter provides ge neral guidance and information on those aspects of quality assurance that are specific to the CALPU FF modeling effort, regardle ss of which organization is carrying out the effort. The focus is on the CALP UFF screening analyses de scribed in Chapter 6. More comprehensive QA is needed for the unique aspects of the CALPUFF source-specific or the use

of alternative models described in Chapters 7 and 8, respectively.

9.2 Quality Assurance Procedures

Chapter 6 recommends procedures for conducti ng CALPUFF screening modeling to determine the whether a particular source or group of sources is subject to BART controls. For initial applications, the regional 6 km resolution CALMET da ta files developed for CE NRAP are available to modelers for the screening assessments. The devel opment of these CALMET datasets was subject to a QA program. Thus, the amount of effort for modelers perf orming QA of these pre-defined9-2 meteorological fields is less than that required when developing sour ce-specific meteorological fields

at finer scale and/or ove r tailored domains.

9.2.1 Input Data

The input data required by the model depends on the application. At a minimum, source data is required by CALPUFF (see Section 6.3.4) along with specifi cation of the model configuration (choices of options and switches). Most of the modeli ng option choices are specified or recommended by regulatory guidance and default values (see Appendi x A through Appendix D). If an application uses

CENRAP’s CALMET files and the CALPUFF screeni ng configuration recommended in Chapter 6, the quality assurance of input data will be straightforw ard. More detailed steps ar e needed for the setup of modeling files for CALPUFF source-specific applica tions using domains and meteorological fields that are not already developed.

The basic procedures that will apply to all CALPUFF model applications include a confirmation of the source data, including units, ve rification of the correct source and receptor locations, including datum and projec tion, confirmation of the switch se lections relative to modeling guidance, checks of the program switches and file names for the various processing steps, and confirmation of the use of the proper version and level of each model program. An independent modeler should review the CALPUFF input files to confirm the switches and data entry in the model input files are correct. The inde pendent reviewer should also exer cise the CALPUFF modeling system

for one or a few corroboration simu lations (e.g., run of the worst case event) as a confirmation check.

The protocol should stipulate that a set of DVD s be created that contain all of the data and program files needed to reproduce th e results presented in the final re port. The model list files from each step should be included on the project DVD. Th is information allows independent checking and confirmation of the modeling process and f acilitates archival of the study results.

9.2.2 MM5 and CALMET Applications

The CALMET meteorological data sets are available on external USB2 or IEEE 1394 (Firewire) hard drives in a format ready for use with CALPUFF. The QA steps used in the development of the CENRAP common datasets ar e described in the docum entation attending the distribution to CENRAP. While these QA steps n eed not be repeated, some testing should be performed to show that the data ar e indeed suitable for the application for which it is being used. This

is discussed in more detail below.

Appendix A contains a list of recommended CALMET switch settings for screening and source-specific modeling. Except as modified in a source-specific protocol , these configurations should be used in setting up the CALMET simu lations. The CALMET model obtains the switch settings from an ASCII control f ile with a default name of CALMET. INP. Whether the model is run using a GUI or from the control line in DOS or Linux, it is essential that the control file be reviewed as part of the CALMET QA analysis. The CALMET GUI retains all of the input descriptive information that is part of the standard CALPUFF.INP file stru cture. This includes the default value for each variable, a text descripti on of the variable, the m eaning of each variable option, the units of the

variable and inter-relatio nships among variables indi cating if/when the variable is used. If CALMET is set up and run from the command line or if othe r user interfaces are employed, this QA step may

become somewhat more difficult. It is essential nonetheless.9-3 Part of the CALPUFF modeling system’s built-in QA capability is a variable tracking system that retains the control file inputs for CALMET and CALPUFF in the output files create by the models.

This information includes the Version and Level numbers of the processor codes and main model codes used in the simulations as well as the control files from the main models (CALMET and CALPUFF). The information from the prepro cessing steps and the CA LMET and CALPUFF model simulations is all carried forward and saved in the CALPUFF/postprocessor output files so that the final concentration/flux files contain a history of the model options and switch settings. This allows a user or reviewing agency to confir m the switch settings provided in a control file with that actually used in the model simulations. An optional switc h in the CALPOST processor creates a complete listing of the QA data. This step requires access to the output CALPUFF concentration and/or flux files, which are normally practical to store on CD s or DVDs and to provide a part of the project

CD/DVD set.

9.2.3 CALPUFF Applications

The minimum source data required by CALPUFF includes:

- Number of sources - Locations (e.g., UTM coordinates, UTM zone and datum) - Stack heights above the ground

- Stack diameters

- Exit velocities - Exit temperatures - Emission rates (SO 2 , H 2 SO 4 , NO

x and PM 10 ).

There are additional requirements for building dimension information (building width, length, height and corner locations) for short stacks that are less than Good Engineering Practice (GEP) height. This information is used in providing effective structure dimensions fo r building downwash calculations. The requirement to conduct build ing downwash modeling may be waived by individual States where the pertinent transport distance to the nearest Class I area is large.

The source coordinates must be expressed in the coordinate system used to define the CALMET and CALPUFF modeling domains. For CENRAP screening applications, a Lambert Conformal Conic (LCC) coordinate system is use d. The required paramete rs include two matching parallels, latitude/l ongitude of the projection origin, coordina te datum, and false Easting and Northing (if used) of the projection origi n. For source-specific CALMET/CALP UFF domains, either an LCC or UTM projection may be used. The CALPUFF user interface includes softwa re (e.g., COORDS) to compute to/from latitude/longitude, LCC and UTM coordinates for a large number of datum. In addition, the CALVIEW graphics f eature allows the use of georeferenced satellite or aerial photographs to be used as base maps to confirm s ource locations. Links to sources of suitable base maps can be found on the CALPUFF data site ( www.src.com ) in the section on “Aerial Photos”. The CALPUFF QA files provide the source coordinates used by the model in a CALVIEW-ready format for plotting. Many errors are found in source coordinates and relate d projection/datum parameters, so

confirmation of the source location is an important part of the model QA.

The PM 10 emissions should be broken into filterable and condensable components. The

filterable PM 10 emissions should be speciated into elem ental carbon (EC), fine particulate matter (PMF) less than 2.5 μm diameter and coarse partic ulate matter (PMC) between 2.5 to 10 μm diameter. The condensable emissions should be divided into a primary H 2 SO 4 component and organic and non-9-4 H 2 SO 4 inorganic components. (The sources of this in formation, such as source-specific data and AP42 tables, are not addressed in this protocol.) The speciation of the PM 10 emissions is important because the light extinction efficiency varies by more than a factor of 16 for different species, from 0.6 for coarse particulate matter to 10.0 for elemental carbon. Plus, hygros copic aerosols are subject to a humidity growth factor (f(RH)), which can increase the light scatteri ng from dry conditions by more than an order of magnitude in highly humid conditi ons. Thus, careful QA of the source and emissions data is an important component of the modeling.

The locations of the Class I area receptors may be obtained from the National Park Service (NPS) receptor dataset. Although the latitude and longitude of each re ceptor point is provided, it is necessary to ensure that the proper UTM or LCC c oordinates have been computed for computational domain selected. In particular, the datum of the NPS conversion software is not specified, so it is recommended that coordinates be checked using the CALPUFF GUI’s COORDS software or another comparable coordinate translation software package that recognizes various datums.

Most of the CALPUFF input variables contain default values. Appendix B contains a list of recommended CALPUFF switch settings for CENR AP screening applications. CALPUFF is configured using the ASCII “c ontrol file”, CALPUFF.INP file . As with CALMET, whether

CALPUFF is exercised GUI or from the command line in a DOS, Linux, or Unix window, it is

essential that the control file be reviewed manua lly as part of the QA an alysis. The CALPUFF GUI retains all of the input desc riptive information that is part of the standard CALPUFF.INP file structure. This includes the default value for each variable, a te xt description of the vari able, the meaning of each variable option, the units of the variable and inte r-relationships among variables indicating if/when the

variable is used.

9.2.4 Application of CALPOST and POSTUTIL

CALPOST is run separately for each Class I ar ea in order to obtain th e necessary visibility statistics for evaluating compliance with the BART screening and refined m odeling thresholds. The inputs to CALPOST involve selecti on of the visibility method (Met hod 6), entry of Class I areaspecific data for computing background extincti on and monthly relative humidity factors for hygroscopic aerosols. CALPOST c ontains a receptor screening that allow subsets of a receptor network modeling in CALPUFF to be selected for pr ocessing in a given CALP OST run. This is how receptors within a single Class I ar ea are selected for processing from a CALPUFF output file that may contain receptors from several Clas s I areas. CALPOST contains optio ns for creating plot files that will help in the confirmation that th e proper receptor subset is extracted.

The CALPOST output file contains a listing of the highest visibility impact each day of the model simulation over all receptors included in CALPOST analysis. Receptors will normally be

selected in each CALPOST run so that each CALPOS T run represents the impacts at a single Class I area. For a screening assessment, the peak value of the change in extinction is presented near the end of the CALPOST output file. For a source-specific studies where the 98 th percentile value (8 th highest day) is used for comparison against the BART th reshold of 0.5 deciviews, the standard CALPOST output must be additionally processed. One may either obtain special-purpose processing software (see, for example CDPHE, 2005) or perform the pro cessing manually. That is, one may import the results of the CALPOST table into a sorting program su ch as a spreadsheet to rank the daily change in extinction values in order to identify the 98 th percentile or 8 th highest day visibility increment.9-5 CALPOST inputs that require pa rticular QA review include:

> Visibility technique (Method 6);

> Monthly Class I-specific rela tive humidity factors for Method 6;

> Background light extinction values;

> Inclusion of all appropriate species from modeled sources (e.g., sulfate, nitrate, organics, (as SOA), coarse and fine particulate matter and elemental carbon;

> Appropriate species names for coarse PM used;

> Extinction efficiencies for each species;

> Appropriate Raylei gh scattering term (10 Mm -1 for screening modeling but Class I area specific value for source-specific modeling); and

> Screen to select appropriate Clas s I receptors for each CALPOST simulation.

The CALPOST program produces plot files compatib le with CALVIEW that allow confirmation of receptor locations that is useful in evaluating the receptor screening step.

POSTUTIL allows the user to sum the cont ributions of sources from different CALPUFF

simulations into a total concentratio n file. In addition, it contains op tions to scale the concentrations from different modeled species (e.g., different partic le sizes) into species- depe ndent size distributions for the particulate matter. For example, PM is often simulated with unit emission rates for each particle size category and, in the POSTUTIL stage, the contributions of each size category based on the species being considered (e.g., elemental carbon, coarse particulate matter, etc) is combined to form the species concentrations for inpu t into CALPOST. This process, although simple, requires a careful review of the weighting factors for each source.

POSTUTIL also allows a repart itioning of nitric acid and nitrat e to account for the effects of ammonia limiting conditions. The four-step pro cedure for applying the AL M method is summarized in Chapter 7 and described in detail in Appendix A of the VISTAS protocol 1 . If source-specific modeling is performed using different sources of ammonia data or di fferent techniques, the protocol should provide justification for all assumptions and a QA plan specific for the application and the data

bases used.

9.2.5 QA Issues for Alternative Source -Specific or Alternative Modeling

The level of QA required for application of so urce-specific or altern ative modeling protocols will be substantially higher than for the use of datasets that has already been subject to a QA procedure. For example, source-specific CALP UFF protocols may includ e the use of on-site meteorological datasets, the use of higher resolution prognostic me teorological (e.g., MM5) datasets, alternative visibility calculations, different extincti on coefficients, or other changes to the screening

1 “Comments on the Computation of Nitrate Using the Ammonia Limiting Method in CALPUFF”, (Scire et al., 2005)9-6 approach. Source-specific must therefore include the development of a QA pl an to properly evaluate the data used in the modeling.

The critical CALMET input parameters depe nd on the mode in which the model is run (observations mode, hybrid mode or no-observ ations mode), and the location and spatial representativeness of any observational data. In a source specific protocol involving the development of a meteorological dataset, the el ements of the QA process include preparation of wind rose (using observed, MM5 and CALMET-derived data ), including examination of the data as a function of season and time of day (e.g., 4am, 10am, 4pm wind roses), time series an alyses, and presentation of 2-D vector plots illustrating terrain effects/sea breeze circul ation or other features of the flow expected to occur within the domain. For example, 2-D vect or plots produced during light wind speed stable conditions (e.g., early morning such as 4am) ar e good for assessing the performance of the CALMET model configuration and switches in reproducing terr ain effects because these conditions are likely to maximize the terrain impacts in the model. Seas onal wind roses at 4am, 10am and 4pm would be expected to show the development of seas breeze circulations that may be important for certain applications. Customization of the QA process for the individual source-specific domain based on the availability of data and the physic al processes expected to be impor tant at that location should be conducted as part of the source- specific QA plan development.

If source-specific CALPUFF simulations i nvolving the ALM are conducted, the procedure summarized in Chapter 7 should be used. This include s an evaluation of the performance of the model in reproducing observed CASTNet or IMPROVE sulfat e and nitrate concentrations at measurements sites within the source-specific modeling domain. Th e use of alternative amm onia concentration data (e.g., one-atmosphere model output rather than deri ved ammonia based on aerosol measurements) will require a review of the CMAQ or CAMx model performance resu lts developed for CENRAP by the regional modeling contractor (Morris et al., 2005d)

9.3 Quality Assessment Summary

Each BART application requires a modeling repor t that includes a discussion of the datasets, modeling assumptions and a general discussion of limitations and uncertainties of the modeling

exercise. Where CALPUFF screeni ng analyses are performed followe d by source-specific modeling, it will be possible to assess the relative conservatism of the screening approach. This may be helpful in assessing the impact of uncertainties in screening sources from BART controls. The reliability of source-specific CALPUFF modeling is expected to be somewhat better in predicting changes in visibility impacts due to BA RT controls than in predicting absolute visibility values. This is because errors and uncertainties in defining meteorological conditions, real-world atmospheric chemistry, and plume transport and dispersion rates may be diminished somewhat sin ce the errors are included in both the base and sensitivity simulation cases.