1 

TSRIA Document 2.1: 

Applying the New Standards in EPA’s Handout Results 

in a Design Value Below the SO2 NAAQS 

At the February 27, 2014, meeting with TSRIA, EPA distributed a handout with data 

quality standards (EPA Handout or the Handout)1 that EPA claimed validated the design value of 

79 ppb upon which it based the nonattainment designation for a portion of Yellowstone County 

(the Designation). However, correctly applying the standards from the EPA Handout results in a 

design value for the Coburn Road monitor of 612

 and thus invalidates the Designation. 

The EPA Handout outlines a “weight of evidence” standard for data quality assurance 

(QA). The Handout also sets forth a validation template with monitoring QA standards for 

Critical Criteria and Operational Criteria (New Standards). EPA stated that these New Standards 

governed its analysis of the Coburn Road monitoring data and documented QA failures. EPA 

asserted that when the New Standards were applied to the 2010 data, Critical Criteria QA 

requirements were met and thus rendered other identified QA failures insufficient to invalidate 

the data from use for a NAAQS Designation.3

However, there are two critical errors in EPA’s application of the New Standards and 

reasoning. 

 First, EPA inaccurately applied the New Standards and thus failed to discern that 

there are failures in the Critical Criteria QA that invalidate several critical data points. 

 Second, EPA has not met the burden of proof set forth in the New Standards that 

require data subject to identified Critical Criteria and Operational Criteria QA failures 

to be invalidated unless EPA provides other compelling evidence that demonstrates 

that the data are valid.4

 

 

1 See TSRIA Document 2.4: EPA Meeting Handout. The EPA Handout contains data quality standards that were 

not in effect when the data were gathered and on which Petitioners did not have the opportunity to comment. 

However, application of these new standards, upon which it was impracticable to raise objections during the public 

comment period, actually contradicts the principal grounds on which the Designation is based and therefore is of 

central relevance to the outcome of the rule. See 42 U.S.C. § 7607(d)(7)(B) (standard for petition for 

reconsideration). EPA quietly incorporated the new “weight of evidence” standard into Appendix A of 40 C.F.R. 

Part 58 as part of revisions to the PM2.5 NAAQS, which only became effective in March of 2013. See EPA, National 

Ambient Air Quality Standards for Particulate Matter, 78 Fed. Reg. 3086 (Jan. 15, 2013). The validation template 

and associated text on the Handout are from the EPA Quality Assurance Handbook for Air Pollution Measurement 

Systems (EPA QA Handbook) § 17.3.3, issued in May of 2013. 

2 See TSRIA Document 2.2: Attachment A for an explanation of the new design value calculation. 3

 TSRIA does not concede that the New Standards may be legitimately applied to data gathered and quality assured 

before the standards were promulgated and on which TSRIA could not practicably comment. EPA’s use of the New 

Standards denied TSRIA and other stakeholders the opportunity to comment on whether EPA appropriately applied 

the New Standards and whether the New Standards themselves are appropriate for the data in question, which were 

collected and submitted before the New Standards were effective. 

4

 Previous data quality standards establish a similar burden of proof for data with QA failures. See, e.g., EPA 

Quality Assurance Handbook for Air Pollution Measurement Systems (EPA QA Handbook) § 17.3.3, issued in 

December of 2008. 
2 

Correct application of the New Standards to the Coburn Road monitoring data with documented 

QA failures invalidates numerous data points, bringing the design value below the SO2 NAAQS 

and nullifying the Designation.5 

A. When correctly applied the New Standards invalidate data with Critical Criteria 

failures. 

As more fully explained in Attachment B, EPA failed to fully apply the New Standards 

before using them to justify reliance on data with documented Critical Criteria QA failures. Page 

two of the EPA Handout provides the minimum QA requirements for the Critical Criteria 

“Zero/Span” and “One-Point QC” checks that EPA relied on to dismiss other QA failures. 

According to the EPA Handout, for the zero/span portion of these Critical Criteria checks, the 

analyzer’s zero response must not exceed +/- 1.5 ppb, and the zero/span check must be 

conducted every two weeks. However, four of the critical high readings in 2010 fail to meet 

these criteria, as explained in Attachment B.6 EPA emphasizes in its own Handout that 

“observations that do not meet each and every criterion on the critical criteria table should be 

invalidated unless there are compelling reason and justification for not doing so.”7 Invalidating 

these high readings from 2010 and using data from 2012, as is allowed by EPA,8

 brings the 

design value to within 0.18 ppb of the 75 ppb standard, an amount well below the 1 ppb 

resolution specification of the Coburn Road monitor. See Attachment B for a more thorough and 

technical analysis. 

B. EPA must also invalidate data with Operational Criteria failures because EPA 

has not demonstrated otherwise. 

EPA’s articulated grounds for reliance on the 2010 data, despite numerous documented 

QA failures, do not satisfy its obligation to demonstrate that data are not “suspect” before relying 

on them in a regulatory context, as set forth in both the New Standards and previous standards.9 

Because EPA has not met this burden, the data in Chart 5 with QA failures10 must be invalidated. 

In addition to the documented Critical Criteria failures set forth in the previous section and 

Attachment B—which render data invalid until proven otherwise11—the EPA Handout 

emphasizes that QA failures for “Operational Criteria” also render data “suspect unless other 

quality control information demonstrates otherwise.”12 The “Operational Criteria” portion of the 

 

5

 Regardless of which QA standards are applied, the burden is on EPA to justify using data that do not meet 

Montana QAPP or EPA QA Handbook standards. See, e.g., 40 C.F.R. § 58, App. A, § 1 (prior to March 18, 2013); 

EPA QA Handbook December 2008, § 17.3.3. TSRIA continues to assert that the 1996 Montana QAPP was the 

legal document in effect when the data used for the Designation were gathered and submitted. Thus the MDEQ 

QAPP contains “applicable standards specifications” that must be complied with per 40 C.F.R. § 58, App. A § 2.1. 

If the applicable standards specifications of the MDEQ QAPP are not complied with, the burden is on EPA to show 

the data meet the “more stringent” QA standards for use in a NAAQS designation. 6 See TSRIA Document 2.3: Attachment B, which also explains that the zero/span check invalidates another reading 

in 2013 and identifies other data of concern in the hours just preceding or following automated zero/span checks. 

7

 EPA Handout, quoted from EPA QA Handbook, 2013 § 17.3.3 (emphasis added by EPA). 

8 See EPA, RTC at 52. 9

 EPA Handout/EPA QA Handbook, 2013 § 17.3.3 (Validation Template for Operational Evaluations). 

10 See TSRIA Presentation to EPA on February 27, 2014, Chart 5 (revised 2/24/14). 11 Id. (Validation Template for Critical Criteria). 12 Id. (Validation Template for Operational Evaluations) (emphasis added). 
3 

data validation table in the EPA Handout is an abridged version. The full table also includes 

criteria for calibration results and calibration gas standards, in addition to other data quality 

indicators. These other Operational Criteria, especially relating to instrument calibration,13 are 

important QA measures that must reasonably be part of the “more stringent” QA standards14

necessary for data supporting a NAAQS designation. 

Because this is a NAAQS designation, EPA must apply “more stringent” QA standards to 

the data points with documented Operational Criteria failures. Since EPA has provided no other 

consistently credible quality control information that proves the validity, EPA must invalidate 

the data points with documented QA failures. As outlined in Chart 5, many of the high readings 

at Coburn Road suffer from numerous Operational Criteria failures. EPA has not made the 

demonstration necessary to rely on these data in the face of the QA failings, while TSRIA has 

provided significant data and analyses that demonstrate the data are unreliable, inconsistent, 

unrepresentative, and invalid. In addition, neither EPA nor MDEQ have adequately investigated, 

mitigated, or justified the reasons for not meeting these criteria as required by the EPA 

Handout.15 In total, the Critical Criteria and Operational Criteria QA failures invalidate enough 

data that the resulting design value of 61 does not support the Designation. 

C. EPA’s “moving target” assumptions do not support use of the data with QA 

failures. 

Because of the “more stringent” QA standard required for a NAAQS designation,16 the 

QA failures that have been documented place a high burden on EPA to demonstrate why it will 

not invalidate these data. EPA has never presented credible data quality control information that 

indicates the QA failures should not invalidate the data. To the contrary, since rejecting the State 

of Montana’s initial designation and repeated requests to avoid using the unrepresentative 2010 

data, EPA has relied on a “moving target”17 of unsupported assumptions: 

 First, EPA assumed the 2010 data with documented QA failures were 

representative.18 

o TSRIA has demonstrated both that the 2010 data suffer from an unusually 

high number of QA failures in comparison to subsequent years and that the 

2010 data are unrepresentative of emission trends, existing SO2 controls, and 

the current air quality. 

 

13 See TSRIA Document 2.5: Full 2013 Validation Template. 14 EPA, Quality Assurance Handbook for Air Pollution Measurement Systems, Vol. II (Ambient Air Quality 

Monitoring Program), EPA-454/B-13-003, May 2013, at § 1.3.1. 

15 EPA Handout/EPA QA Handbook, 2013 § 17.3.3 (Validation Template for Operational Evaluations and Critical 

Criteria). 

16 EPA QA Handbook § 1.3.1. 

17 See North Carolina v. EPA, 531 F.3d 896, 927 (D.C. Cir. 2008) (granting Minnesota Power’s petition because 

EPA failed to adequately address claims of technical errors and used a “moving target” for data determinations in its 

response to comments and rejection of the petition for reconsideration). 18 See EPA, TSD at 10. 
4 

 Second, EPA relied on audits which it claimed outweighed any QA failures.19

o TSRIA demonstrated that the audits also suffer from QA failures, which EPA 

should have further investigated before relying on them.20

 Third, EPA relied on Critical Criteria and a new “weight of evidence” standard which 

it claimed outweighed all other QA failures. 

o TSRIA has identified four critical data points in 2010 that fail to meet the 

Critical Criteria, and EPA has failed to meet the weight of evidence standard 

and the Critical Criteria and Operational Criteria standards in the EPA 

Handout to support use of data with QA failures.21

The record before EPA demonstrates that the QA failures during 2010 and the unreliable nature 

of the monitoring data do not provide an adequate foundation for a NAAQS designation. EPA, 

consistent with regulatory requirements, should invalidate the data and repeal the Designation. 

 

19 See EPA, RTC at 46. 20 See TSRIA Presentation titled EPA Additional Information Requests, Revised Chart 5; Attachment to Comments 

from Montana Chemical & Sulphur Company at 14-25 Docket EPA-HQ-OAR-2012-0233-0298; TSRIA Comments 

at 13-16, Docket EPA-HQ-OAR-2012-0233-0285. 

21 See Attachment B and discussion above.