WRAP IMPROVE Data Substitutions 04/03/07

In the Western Regional Air Partnership (WRA P) states, data substitution was performed for nine IMPROVE monitoring si tes to achieve RHR data completeness, or to fully populate 2002, WRAP’s selected modeling y ear. These data substitutions included estimating missing species from other on-site measurements and appr opriately scaling data collected at selected donor sites which had favorable long-term comparisons. This document outlines the data substitution methods used at these sites.

Data Completeness Requirements

Regional Haze Rule (RHR) guidance outli nes IMPROVE aerosol data completeness requirements including the following conditions:

 Individual samples must contai n all species required for the calculation of light extinction (sulfate, nitrate, organic carbon, elemental carbon, soil, coarse mass, and, for the new IMPROVE algorithm, chloride or chlorine)

 Individual seasons must contain at least 50% of all possible daily samples

 Individual years must contain at le ast 75% of all possible daily samples

 Individual years must not contain more than 10 consecutive missing daily samples

 The baseline period (2000-04) must contain at least 3 complete years of data

Further details can be found in the RHR guidance document for tracking progress: http://www.epa.gov/ttn/oarpg/ t1/m emoranda/rh_tpurhr_gd.pdf

.

Routine Data Substitutions

RHR guidelines provide provisions to fill in missing data under specific circumstances. There are currently two methods routinely used in preparing the RHR data set to substitute data for missing samples:

 The use of a surrogate in the data set:

 Total sulfate is generally de termined as 3 times the sulfur measured on the A module filter. If sulfur is missing, the sulfur measurement from th e B module filter is used to calculate sulfate.

 For the new IMPROVE algorithm, sea salt is calculated from chloride measured on the B module filter. If chloride is mi ssing or below detection limit, the chlorine measurement from the A module filter is used to calculate sea salt.

 The application of “patching” missing data described by the RHR guidance:

1 Missing s amples not substituted using a surrogate as described above can be patched, or replaced, by a seasonal average if the pa tching exercise passes a series of tests outlined in the guidance document.

Once these methods have been applied to th e data, the resulting complete years are eligible for use in calculation of baseline c onditions and tracking pr ogress under the Regional Haze Rule. These methods have be en applied to all IMPROVE data.

Sites Not Meeting Data Completeness Requirements

After routine data subs titutions were made, some WRAP sites still failed to meet data completeness requirements for the baseline period. These sites are listed in Table 1. Sites were candidates for substitution for two reasons:

 The sites had fewer than 3 complete years of data, thus RHR visibility metrics for the baseline period could not be calculated.

 The sites had at least 3 years of complete data, but were missi ng 2002, the year selected for regional modeling. If this year is missing, then the worst 20% visibility days from 2002 cannot be determined, and the relative response factors (RRFs), which are used to predict visibility metrics in 2018, cannot be calculated.

Sites that did not meet data completeness requirements were not necessary for submittal of State Implementation Plans (SIPs) are indicated with an asterisk (*) in Table 1. Additional data substitutions for these sites have not been applied.

Table 1 WRAP Sites Failing RHR Data Completeness Requirements

State Site <3 years Missing 2002
BALD1 X X INGA1* X X AZ TONT1 X KAIS1 X X RAFA1 X X SEQU1 X CA

TRIN1 X FLAT1* X X FOPE1* X X GLAC1 X MT

NOCH1* X X UT CAPI1 X X WA NOCA1 X * Indicates additional substitution is not required for a SIP.

2Additional Data Substitutions

This section outlines the WRAP methods fo r additional data substitutions designed to address the problems at sites list ed in Table 1. Similar methods have been used at IMPROVE sites with incomplete data records in other RPOs.

Figure 1 presents a flow chart of the WRAP data substitution methods. The starting data set was the RHR IMPROVE data using the “New IMPROVE Algorithm,” updated March 2006, ( http://vista.cira.colostate.edu/vi ews/W eb/IMPROVE/SummaryData.aspx ). This data set includes the routine surrogate a nd patched data substitutions a llowed by RHR guidance. Note that only years deemed incomplete under RHR guidance were candidates for additional data substitutions. Years deemed complete were not changed, even thought there may have been missing samples during those years.

The first of the additional substitution met hods used organic hydrogen as a surrogate for organic carbon, and resultant orga nic carbon as a surrogate for elem ental carbon. If the carbon data substitution was not sufficient to complete th e required years, measured mass for individual species from nearby IMPROVE sites with favor able long-term comparisons were scaled appropriately and used as surrogates. IMPROVE donor sites were selected in consultation with individual states. These methods are described in detail below.

Figure 1

Flow Chart of Data Substitution Methods Used

Start with IMPROVE RHR dataset (daily_budgets_nia_20060306.csv)

3 complete years (including 2002)?

Check year for completeness Substitute OC based on Organic H, and EC based on OC (use raw data linear regression statistics) For each missing year, check each incomplete day

Final Data Set Substitute missing species from a nearby donor site (use raw data linear regression statistics) yes

Check year for completeness

no yes

yes no no

Year remains incomplete

3All substitu tions were made using quarterly specific Kendall-Theil linear regressions statistics. These statistics were chosen becaus e they are more resistan t to outliers than the standard linear least squares statistics. Kendall-Theil slopes and intercepts were used to calculate substituted values from surrogates.

1. Carbon Substitutions

The first substitution method relied on using a surrogate for carbon mass measurements when the C module data is not av ailable. Hydrogen (H) is meas ured on the A module filter, and is assumed to be primarily associated with or ganic carbon and inorganic compounds such as ammonium sulfate. Therefore, organic carbon (OC) can be estimate d using the historical comparison between estimated organic H and OC. Organic H is estimated by subtracting the portion of H that is assumed to be associated with the inorganic compounds from the total H (Org_H = H – 0.24*S).

Figure 2 presents a sample comparison fo r data collected at the Tonto National Monument site in Arizona during the second qu arter between 2000-04 for OC and organic H. Once OC has been estimated using this method, el emental carbon (EC) mass is determined using long-term comparisons between OC and EC at th e site. Statistics were calculated and applied quarterly to account for seasonal variations.

Figure 2

Comparison of OC and Estimated Organic H, and EC and OC at Tonto Na tional Monument, AZ Using Second Quarter Raw OC and Organic H Data, 2000-04

Q2

r2 = 0.9

0 2 4 6 8 10

0.0 0.2 0.4 0.6 0.8 1.0 org_H (μg/m3)OC (μg/m3) Q2

r2 = 0.82

0.0 0.2 0.4 0.6 0.8 1.0

0 2 4 6 8 10 OC (μg/m3)EC (μg/m3)

42. Donor Site Substitutions

In the W RAP, the carbon data substitution me thods were not sufficient to complete the required years. A second method involved iden tification of another nearby IMPROVE site which had favorable long-term comparisons and similar regional characteristics to be used as a donor site. Candidate sites were identified, and final donor sites for surrogate mass were selected in consultation with states.

Figure 3 presents a sample inter-site mass comparison by species for data collected during the second quarter, 2000-04, between the T onto National Monument site and the Sierra Ancha site in Arizona. Component specific correl ations were calculated and applied quarterly. Note that only species missing in a given samp le were substituted based on donor site data. Species collected at the site under investigation were never replaced with data from a donor site.

5Figure 3

Com parison of Aerosol Species Mass Between Tonto National Monument, AZ (y-axis) and Sierra Ancha, AZ (x-axis) Using Second Quarter Raw Data, 2000-04

Q2

r2 = 0.84

0 2 4 6 8 10

0 2 4 6 8 10 AmmSO 4 (μg/m3)AmmSO4(μg/m3) Q2

r2 = 0.61

0 1 2 3 4 5

012345 AmmNO 3 (μg/m3)AmmNO3(μg/m3)

Q2

r2 = 0.02

0 2 4 6 8 10

0 2 4 6 8 10 OC (μg/m3)OC (μg/m3) Q2

r2 = 0.01

0.0 0.2 0.4 0.6 0.8 1.0

0.0 0.2 0.4 0.6 0.8 1.0 LAC (μg/m3)LAC (μg/m3)

Q2

r2 = 0.67

0 2 4 6 8 10

0 2 4 6 8 10 Soil (μg/m3)Soil (μg/m3) Q2

r2 = 0.59

0 10 20 30 40 50

0 10 20 30 40 50 CM (μg/m3)CM (μg/m3)

62. Data Comple teness Following Substitutions

Table 2 indicates which years required some degree of substitution, where a 2 indicates a substituted year, a 1 indicates the year was al ready complete under RHR guidelines, and dashes indicate the year did not meet RHR guidelines and no additional substitutions were made. The table also lists sites that we re selected as donor sites.

The minimum data requirement of 3 comple te years (including 2002) was met for each site, and additional substitutions beyond these requi rements were made on a case by case basis in consultation with individual states. For exampl e, at the KAIS1 site, substitutions were made only for the 2002 year even though substituted data (from the YOS E1 donor site) was available for other years. In this case, the years 2000 and 2001 had less than 50% of the original RHR data. In contrast, additional s ubstitutions were applied for all incomplete years (2000-2002) at the RAFA1 site. For the RAFA1 site, the orig inal RHR data was more substantial (73-86% available) and substitutions had less of an impact on the worst days distributions.

Table 2 Data Completeness at WRAP Sites Following Data Substitution

State Site <3 years Missing 2002 Donor 2000 2001 2002 2003 2004

BALD1 X X TONT1 -- 2 2 1 1 AZ TONT1 X SIAN1 -- 1 2 1 1 KAIS1 X X YOSE1 -- -- 2 1 1 RAFA1 X X PINN1 2 2 2 1 1 SEQU1 X DOME1 1 1 2 2 1 CA

TRIN1 X LAVO1 -- 1 2 1 1 MT GLAC1 X FLAT1 1 1 2 2 1 UT CAPI1 X X CANY1 2 2 2 1 1 WA NOCA1 X SNPA1 -- 1 1 2 2 -- indicates an incomplete year with no substitutions made 1 indicates a complete RHR year 2 indicates a year is considered complete with some substituted values

7Availability and Archival of Data Sets

A dedicated page on the VIEW S database will act as the repository of all site-specific substitute data sets: http://vista.cira.colostate.edu/view s/web/docum ents/substitutedata.aspx

. Table 3 presents a key to the substituted data file s. All materials prepared in the data substitution work (descriptive narrative, tables of regression statistics, graphics, etc.) will be posted on this site for review by states, tribes, and other data users. This information will also be made accessible through the TSS.

Table 3 Key to Substituted Data Files

Column Header Description site IMPROVE site code
year month day QUARTER 1 = Jan. – Mar., 2 = Apr.-Jun., 3 = Jul. – Sept., 4 = Oct. – Dec. date Group 10 = One of the 20% best visibility days; 90 = One of the 20% worst visibility days good_year 0 = incomplete year, 1 = complete RH R year, 2 = complete year with substitutions ss_rayleigh Site specific Rayleigh value (clean air extinction) fsrh f(RH) value for small sulfate, nitrate and organic mass flrh f(RH) value for large sulfate, nitrate and organic mass fssrh f(RH) value for sea salt mass Sea_Salt Sea salt mass (μg/m3) Soil Soil Mass (μg/m3) Amm_NO3 Ammonium n itrate mass (μg/m3) OMC Organic mass by carbon (μg/m3) LAC Light absorbing carbon (aka EC/Elemental Carbon) (μg/m3) CM Coarse mass (μg/m3) Amm_SO4 Ammonium su lfate mass (μg/m3) Large_OMC Large organic mass (μg/m3) Small_OMC Small organic mass (μg/m3) Large_Amm_SO4 Large ammoni um sulfate mass (μg/m3) Small_Amm_SO4 Small ammoni um sulfate mass (μg/m3) Large_Amm_NO3 Large ammonium nitrate mass (μg/m3) Small_Amm_NO3 Small ammoni um nitrate mass (μg/m3) EAmm_SO4 Extinction due to ammonium sulfate (Mm-1) EAmm_NO3 Extinction due to ammonium nitrate (Mm-1) EOMC Extinction due to organic carbon mass (Mm-1) ELAC Extinction due to light absorbing carbon mass (Mm-1) ESoil Extinction due to soil mass (Mm-1) ECM Extinction due to coarse mass (Mm-1) ESea_Salt Extinction due to sea salt mass (Mm-1) RBext Reconstructed aerosol extinction (Mm-1) TBext Reconstructed total extinction (Mm-1) OC_SUB1 OC substituted using OC vs. organic H correlations EC_SUB1 EC substituted using EC vs. OC correlations (NH4)2SO4_SUB2 Ammonium sulfate substituted using site donor correlations (NH4)NO3_SUB2 Ammonium nitrate substituted using site donor correlations OM_SUB2 Organic mass substituted using site donor correlations EC_SUB2 Elemental carbon (aka light absorbing ca rbon) substituted using site donor correlations Soil_SUB2 Soil substituted us ing site donor correlations CM_SUB2 Coarse mass substituted using site donor correlations SeaSalt_SUB2 Sea salt substituted using site donor correlations
89