1 

TSRIA Document 2.3: 

ATTACHMENT B 

INVALIDATION OF ELEVATED COBURN ROAD SO2 POINTS 

USING NEW EPA STANDARDS 1

1.0 INTRODUCTION 

On February 27, 2014, EPA provided TSRIA with a handout outlining a new “weight of 

evidence” standard from 40 C.F.R. Part 58, promulgated in early 2013, and a portion of the SO2

Data Validation Template found in Appendix D of Volume II of EPA’s Quality Assurance 

Handbook for Air Pollution Measurement Systems, issued in May 2013 (EPA Handout). The 

validation template lists validation criteria, with two falling into the “Critical Criteria” category: 

1) “One Point QC Check” – commonly referred to as a precision check – must be performed 

every two weeks, and the analyzer response should not differ from the known concentration by 

more than ± 10%. 

2) Zero/span check – must also be performed every two weeks. The allowable error for the span 

point is ± 10%, while the zero point must be in error by no more than ± 1.5 ppb. 

According to the EPA Handout, “observations that do not meet each and every criterion on the 

critical criteria table should be invalidated unless there are compelling reason and justification 

for not doing so.” 

2.0 RE-EVALUATION OF ELEVATED COBURN ROAD SO2 POINTS 

Based on the EPA Handout, TSRIA asked Bison Engineering to conduct a new review of the 

elevated Coburn Road SO2 data points for 2009-2013, using only failure to satisfy the Critical 

Criteria from the EPA Handout as a basis for invalidation. 

Based on this review, Bison Engineering identified a total of five data points that must be 

considered invalid based on the EPA Handout, as shown in Table 1: 

Table 1 – Summary of Invalid Coburn Road Data Points

Date 

SO2

Conc. 

(ppb) Reason(s) for Invalidation 

Feb 10, 2010 91 Error for preceding zero check on 2/9/10 was + 3 ppb 

Error for following zero check on 2/17/10 was + 2 ppb 

July 8, 2010 111 Error for preceding zero check on 6/28/10 was + 7 ppb 

Error for following zero check on 7/12/10 was + 5 ppb 

July 9, 2010 92 Error for preceding zero check on 6/28/10 was + 7 ppb 

Error for following zero check on 7/12/10 was + 5 ppb 

Oct 22, 2010 89 Interval between preceding (10/4/10) and following (10/25/10) 

zero/span/precision checks was 3 weeks 

 

1 Steve Heck, Bison Engineering Inc. March 7, 2014.
2 

Jan 16, 2013 51 Error for following zero check on 1/18/13 was + 1.6 ppb 

Of particular note is that these data points include four of the five highest daily maximum 

concentrations measured in 2010. 

3.0 IMPLICATIONS

Exclusion of the above data points significantly affects the 99th-percentile daily maximum value 

in 2010, since four of the five highest daily maximum concentrations measured in 2010 are 

invalid. In descending order, the four highest values in 2010 now become: 

Dec 24: 101 ppb 

Oct 3: 87 ppb 

Dec 21: 86 ppb 

Sep 28: 83 ppb 

Thus, exclusion of the four 2010 data points in Table 1 reduces the 99th-percentile value from 91 

ppb to 83 ppb. This also has a significant effect on the three-year design value calculations used 

by EPA to justify a proposed Non-Attainment designation, whether based on a 2009-2011 or 

2010-2012 evaluation period: 

1) Design value for 2009-2011 using all data: 79.00 ppb 

2) Design value for 2009-2011 excluding four 2010 data points: 76.33 ppb 

3) Design value for 2010-2012 using all data: 78.33 ppb 

4) Design value for 2010-2012 excluding four 2010 data points: 75.67 ppb 

The results summarized above show that the distinction between the 2009-2011 and 2010-2012 

design value periods is significant. Originally, EPA allowed use of data from 2012, but chose to 

base the Designation on 2009-2011 because it claimed that use of 2012 (as part of a 2010-2012 

calculation) made no difference to the designation. See EPA RTC at 52. 

Based on analysis using the New Standards, invalidation of data that do not meet Critical Criteria 

results in a 2010-2012 design value that is technically only 0.18 ppb above the 75 ppb standard 

(because a value of 75.49 ppb would be rounded down to 75 under EPA’s rules). This difference 

is well below the 1 ppb resolution specification of the SO2 analyzer used to make the Coburn 

Road measurements. 

A lesser effect on the 99th-percentile value is noted for 2013: when the single value of 51 ppb is 

invalidated, the 99th-percentile value changes from 48 ppb to 46 ppb. 

4.0 ADDITIONAL DATA QUALITY CONCERNS

Based on the EPA Handout and the ZSP Check problems identified above, the “Weight of 

Evidence” standard suggests that relevant air quality data behavior and trends that relate to the 

validity of data with quality assurance concerns must be examined. 
3 

After reviewing the raw AQS data download files for 2009-2012, Bison Engineering has 

identified three additional data points that should be considered suspect because 1) they occurred 

in hours immediately preceding or following an Automated Zero/Span (AZS) check; and 2) they 

are inconsistent with surrounding ambient data. They are summarized in Table 2, and discussed 

below. 

Table 2: Summary of Suspect Coburn Road Data Values 

September 22, 2009 March 11, 2012 September 13, 2012 

Hour SO2 ppb Hour SO2 ppb Hour SO2 ppb 

06 30 06 9 01 7 

07 24 07 3 02 12 

08 18 08 8 03 7 

09 107 09 27 04 41 

10 AZS 10 93 05 AZS

11 2 11 AZS 06 129

 12 0 07 35 

 08 16 

AZS checks are automated analyzer calibration checks that normally occur with no site operator 

present. They are normally performed every 25 hours and are to be accomplished within one 

discrete hour. The AZS routine is programmed into the site’s datalogger; it instructs the analyzer 

to stop sampling ambient air, and instead sample calibration gas and zero air generated by the 

site’s calibration system. After the AZS is completed, the analyzer is instructed to resume 

sampling ambient air. This is accomplished by opening and closing solenoid valves controlling 

different analyzer sampling ports. On occasion these solenoid valves can leak, or may fail to 

open or close entirely. 

It is statistically odd that three of the seven highest daily maximum one-hour concentrations from 

2009 – 2012 (or 42.9%) occurred in hours immediately preceding or following an AZS check, 

whereas only ~10% of all hours immediately precede or follow such a check. Because the time 

of occurrence of AZS checks advances by one hour per day to eliminate temporal data bias 

effects, only ~10% of these seven values (perhaps one value) should fall in an hour adjacent to 

an AZS based on random chance. 

While the seeming preference of high values to occur at hours adjacent to AZS checks may be 

purely coincidental, the data trends themselves are also concerning, and raise at least the 

possibility of a leaking or ineffective solenoid – or a timing error – so that some portion of the 

high-concentration calibration gas may have been introduced into the ambient air sampling 

pathway, thereby affecting a reported data value: 

 On September 22, 2009 the concentrations for hours 6 through 8 were between 18 and 

30 ppb. In hour 9 - the hour preceding the AZS – the concentration suddenly jumped 

to 107 ppb. Yet in hour 11 the concentration was only 2 ppb. 
4 

 On March 11, 2012 the concentration during hours 6 through 9 ranged between 9 and 

27 ppb. In hour 10 – the hour preceding the AZS – the concentration suddenly jumped 

to 93 ppb. Yet, in hour 11 the concentration had dropped to zero. 

 On September 13, 2012 concentrations from hours 1 through 3 ranged from 7 to 12 

ppb. In hour 4 – immediately preceding the AZS – the concentration suddenly jumped 

to 41 ppb. In hour 6 the recorded concentration jumped to 129 ppb, and then dropped 

sharply to 35 ppb in the following hour. The concentration at hour 6 is a remarkable 

outlier because it is so much higher than other values observed during similar upwind 

emissions. It is highly improbable that such a value would occur in the fashion 

described above. 

Effect on Design Value Calculations

Exclusion of any one of these inconsistent SO2 values, in addition to the clearly invalid data 

points discussed in Sections 2.0 and 3.0, further reduces the calculated design values as shown 

below: 

 If only the September 22, 2009 value of 107 ppb is excluded, the 99th-percentile value 

for 2009 drops from 72 ppb to 69 ppb. The 2009-2011 design value correspondingly 

drops from 76.33 ppb to 75.33 ppb – showing compliance. 

 If only the March 11, 2012 value of 93 ppb is excluded, the 99th-percentile value for 

2012 drops from 70 ppb to 55 ppb. Accordingly, the 2010-2012 design value drops 

from 75.67 ppb to 70.67 ppb – well below the 75 ppb standard. 

 If only the September 13, 2012 value of 129 ppb is excluded, the 99th-percentile value 

for 2012 also drops from 70 ppb to 55 ppb and the design value again becomes 70.67 

ppb. 

 If both of the anomalous 2012 values are excluded, the 99th -percentile value for 2012 

drops to 54 ppb and the design value drops to 70.33 ppb. 

Excluding both the 2009 and 2012 values discussed above has no cumulative effect, since both 

of those years cannot be part of the same 3-year design value calculation.