RMB Consulting & Research, Inc. 

5104 Bur Oak Circle Phone (919) 510-5102 

Raleigh, North Carolina 27612 Fax (919) 510-5104

M E M O R A N D U M

TO: Steffan Johnson, EPA 

FROM: Stephen Norfleet 

DATE: April 5, 2016 

SUBJECT: SES Presentation on EPA’s Proposed Revisions to the Reporting Requirements 

for Performance Test Data under 40 CFR Parts 60 and 63 

As requested, here is a copy of the presentation that I made at the recent Stationary Source 

Sampling & Analysis for Air Pollutants Conference that is organized by the Source Evaluation 

Society (SES). As you are aware, given that you serve as one of SES's current board members, 

SES is an organization that was formed to advance the science of emission source testing and to 

foster the professional development of individuals involved with source testing. Accordingly, 

SES's membership is largely made up of stack testers and includes federal, state and local agency 

personnel who are involved with assessing emission test data. 

My presentation focused on two sets of rule revisions that the Agency has proposed: 1) the 

proposed changes to §60.8(f) and §63.7(g) that were part of a set of proposed "Revisions to Test 

Methods, Performance Specifications, and Testing Regulations for Air Emission Sources," which 

were published on September 8, 2015, and 2) the proposed "Electronic Reporting and 

Recordkeeping Requirements for New Source Performance Standards (NSPS)," which were 

published on March 20, 2015. The proposed changes to §60.8(f) and §63.7(g) would identify 

specific information to be included in performance test reports required under 40 CFR Parts 60 

and 63 regardless of whether the reports are "submitted electronically or in paper format." The 

proposed electronic reporting provisions would require existing sources under many NSPS 

subparts to begin electronically reporting data via the Electronic Reporting Tool (ERT) and/or 

the Compliance and Emissions Data Reporting Interface (CEDRI). 

Although I do have some concerns with the proposed §60.8(f) and §63.7(g) revisions (as 

reflected in RMB's comments, which were included as part of Hunton &Williams LLP's 

comments submitted on behalf of the Utility Air Regulatory Group), my presentation mostly just 

outlined those proposed requirements. With respect to the proposed NSPS electronic reporting, I 

focused on the ERT reporting requirements, highlighted significant issues with the ERT, and 

proposed an alternative reporting approach. As discussed in my presentation, the alternative 

would provide the benefits of electronic reporting, reduce the burden on sources and testers, 

permit EPA to readily apply consistent reporting for different methods/sources, and allow the 

data to be more easily used by agencies and others are interested in reviewing the information. 
2 

ERT Issues 

Given the audience and the direct impact that the proposed revisions would have on the way 

source testers report performance test data, the presentation elicited significant interest. Many 

expressed frustration regarding the time consuming and cumbersome task entering test data using 

the ERT, even when they use the templates for transferring some of the data into the Access 

database. Some discussion involved differences in the amount of data that testers actually "fillin" in the ERT, with it being apparent that some try to complete all the fields while other leave 

portions blank. This issue is exacerbated by the fact that there is no regulatory basis for some of 

the fields within the ERT and by questions over what information is necessary to submit in terms 

of the test results. 

These issues, as well as other problems, pose serious questions about the choice of using the 

ERT as a vehicle for electronic reporting. For years, the EPA has touted the ERT as an all-inone tool that "replaces the time-intensive manual preparation and transcription of stationary 

source emissions test plans and reports currently performed by contractors for emissions sources 

and the time-intensive manual quality assurance evaluations and documentation performed by 

State agencies."1

 But, these claims are betrayed by the fact that sources, testers, and states have 

not adopted the ERT. In spite of EPA expending resources towards many ERT revisions, testers 

have not migrated to used the tool beyond where it has been required (e.g., for ICR tests and for 

newer Subpart Da units). This real-life feedback on the purported usefulness of the ERT is 

echoed by its not being used by agencies that review test data. In fact, none of the state or local 

agency personnel that attended the conference indicated that their agencies used the reported 

ERT data in any way, suggesting that the effort to produce ERT files may result in relatively 

useless artifacts rather than reaching the potential benefits that electronic reporting offers. 

The ERT is also ill-suited to fill the goals of EPA's "Next Generation Compliance" Initiative. 

The tool does not meet the objective of making environmental reporting more efficient, and it 

certainly does not "expand transparency by making information more accessible to the public."2

 

I would suggest that the awkward tool and Access database structure actually creates a roadblock 

to potential use by the public. If anything, the ERT makes the data more opaque and less 

accessible than the hard-copy stack test reports (or in the electronic arena, PDF versions of the 

full stack test report), which the ERT seems intended to replace. 

But, these complaints against the ERT are not new. The Agency has received significant adverse 

comments highlighting problems with the ERT, including comments from state agencies, not 

only in response to this current rulemaking but also in response to the initial Advanced Notice of 

Proposed Rulemaking (ANPR) in 2009 that preceded these revisions as well in response to other 

proposed rules that have required the use of the ERT. In light of these comments, EPA's 

steadfastness in proposing the ERT as the basis for electronic reporting instead of proposing 

other options is perplexing. While I understand that EPA has expended resources toward the 

development of the ERT, it is simply not a good choice for electronic reporting. 

Part of the problem with the ERT is that it is not a format. Proposing rule language stating that 

"performance test data must be submitted in a file format generated through the use of the EPA’s 

 

1

https://www3.epa.gov/ttnchie1/ert/ 

2

 https://www.epa.gov/compliance/next-generation-compliance
3 

ERT" does not define the data that must be reported. Not only does the ERT contain fields for 

data for which there seems to be no regulatory basis, but the Agency might change the ERT at 

any time. In the rulemaking, EPA suggests that it might add new methods to the ERT at any 

time (and that sources would need to stay abreast of such changes to know when it should report 

such information using the ERT) but, since the format is not defined, potential changes would 

not necessarily be confined to new methods. In other words, EPA seems to be proposing 

reporting requirements that are completely outside the normal regulatory review and comment 

process. 

EPA needs to define an actual format. Sources and emission testers already have spreadsheets 

and tools that they have personally verified over time. Defining the reporting in terms of an 

actual format would allow the reporting to be more easily automated and integrated using the 

tools they already have. 

Alternative XML Reporting Format 

Given these issues, I am pleased to hear that EPA is considering developing an XML reporting 

alternative to the ERT. But, I would suggest that developing a such an XML format (rather than 

requiring the ERT) is where the Agency should be focusing its effort. Proposing electronic 

reporting prior to the development of such a format (and prior to the proposed changes to 

§60.8(f) and §63.7(g)) was a premature action. 

 

If EPA wants to reduce the burden of reporting while still providing the information needed to 

assess compliance, the Agency should consider how to best to achieve its goals. I would suggest 

a balanced approach. Some data may be useful to have in a numeric format, but other 

information is better suited to be documented in a PDF (or equivalent) format. For example, the 

raw data sheets and example calculations that EPA has proposed to include in test reports do not 

readily lend themselves to a non-graphical format (particularly if the raw data is a hand entered 

form). Also, with a PDF (or equivalent) format, the information may be detailed in a way that 

provides greater context and allows for site-specific or test-specific information to be more easily 

documented. A combination of both PDF and numeric data, properly implemented, would allow 

the information to be reported electronically and can achieve the benefits of paper work 

reduction and accessibility. 

In my presentation, I outlined such an alternative reporting approach that was developed by the 

workgroup that RMB is coordinating to address reporting issues associated with the Mercury and 

Air Toxics Standards (MATS) Rule. While the approach developed by the workgoup was 

intended to address the specific needs of reporting under the MATS rule, it provides a template 

that could be used to report performance test data for any method or subpart. Under our 

proposed reporting approach, critical details would be reported in XML, but source would also 

be required to submit a full stack test report in PDF format for complete transparency and to 

document other details so that agency personnel can have ready access to that information in the 

full context of the report. 

 

As shown in the presentation slides, sources would report XML data detailing the test methods, 

times, and concentration data for each reference method run as well as the parameters used to 

convert the concentrations to the units of the emission standard (e.g., CO2 or O2 values used to 
4 

calculate the lb/MMBtu values), the applicable emission limit, process load, sample times and/or 

sample volumes, and various critical quality assurance (QA) parameters (e.g., for Methods 5, the 

QA parameters might include parameters the iso-kinetic rate, filter temperature, leak check 

results). Such values could be readily reviewed by agency personnel and might be validated as 

part of the submission process to ensure that compliance is achieved, to verify that the test was 

properly performed and to estimate the potential uncertainty of the results. Since one of EPA's 

goals with electronic reporting is to provide data for emission factor development, some process 

data might be reported (e.g., a electric utility source might report the electrical load or average 

heat input during the performance test). But, the specific process information, if it can be 

justified, would need to be defined for each type of source. Thus, although this information is 

not necessarily needed to assess compliance with the NSPS, EPA might need to define such 

process reporting in the individual subparts in order to provide a regulatory basis for the 

reporting. 

In comparison to the ERT, the alternative approach streamlines the reporting for sources and for 

the agencies reviewing the data by focusing on the data that is needed to establishing compliance 

and to develop emission factors. It would also provide for consistent reporting across different 

methods and sure types. Rather than having to apply patch over patch to the ERT and deferring 

electronic reporting until the methods are "supported by the ERT," the Agency could simply 

define the critical QA parameters for each method. 

The ERT requires much more data than is needed to verify compliance and develop emission 

factors. The ERT includes fields for entering traverse point-by-point details that would be better 

documented with other relevant test data in the full stack test report submitted as PDF file. 

Focusing on providing the critical values in a numeric format reduces the reporting burden and 

allows testers and sources to spend more time quality assuring the results. Adequate review time 

to allow sources to ensure the accuracy of the information is an important consideration since 

EPA is proposing these new electronic reporting requirements without extending the time when 

such reports are due. 

Some have suggested that states and local agencies need point-by-point data in order to 

recalculate the results. However, while occasional spot checks might be reasonable, requiring all 

sources to submit traverse point-by-point data in numeric format would seem to provide little 

benefit. After all, once you check a source tester's calculation spreadsheets, what are you really 

expecting to find the next time? And, if we are going to recalculate all the results from the pointby-point data, why has EPA proposed to include example calculations within the test reports? Is 

not the purpose of the example calculations to illustrate that those calculations are being 

performed properly? 

Simply checking calculations does not verify data quality or ensure that the testing was 

conducted properly or that the right values or assumptions were made when entering the data. A 

more meaningful assessment requires test observation and/or critically reviewing the report 

details and results. The point-by-point details will already be documented in the full stack test 

report for review and auditing purposes. Furthermore, if a state or local agency believes that it 

needs the numeric data, it has its own regulatory mechanisms to require sources within its 

jurisdiction to provide that data. In other words, putting aside the question of whether reporting 
5 

point-by-point numeric data is really necessary, the EPA does not need to create a requirement to 

provide this data to the state and local agencies because they already have regulatory avenues 

that they could use to obtain the data. 

Based on the discussion following the presentation, it is fair to say that there is considerable 

support within the stack test community for using an alternative electronic reporting format in 

lieu of the ERT. The Agency should also appreciate that this support comes from a group that 

has been directly involved with using the ERT. This support for a more streamlined and 

balanced approach is not from group of Luddites who are shying away from the concept of 

electronic reporting. SES even has an "ERT Improvement" committee. The problem is that the 

ERT is just not the right tool for the job. The Agency can do much better. 

Potential Rule Interaction 

As we discussed at the conference, I am also concerned with potential interaction between the 

two proposed rulemakings. As I admitted, I did not initially see this potential issue. From my 

perspective, the proposed changes to §60.8(f) and §63.7(g) would dictate what source testers 

would need to include in their reports and the proposed NSPS electronic reporting requirements 

would also require sources to submit test data using the ERT (or, hopefully, submit data via an 

alternative XML format). However, during recent conversations regarding MATS reporting, 

EPA personnel have made statements suggesting that the ERT would need to be revised to 

address additional information required by the proposed changes to §63.7(g). 

This statement did not make sense to me until I realized that they were equating the ERT with a 

test report. First, I do not believe that that this interpretation necessarily the most reasonable 

one. But, more importantly, the assumption that the ERT should serve as the required test report 

is a bad paradigm. The official test report should be the complete test report that includes all the 

information that the source and/or testers believe should be included to fully document the test 

results, quality assurance aspects, and other relevant details of the test. 

The ERT does include the ability to provide attachments, but I would argue that patchwork of 

various attachments specified by the ERT not only increases the burden of reporting but also 

makes cobbled-together ERT agglomeration more difficult to use than a complete and unified 

report that can provide such details in a meaningful context. Furthermore, while the series of 

ERT attachments have been described as a method for ensuring that the "proper" documentation 

is included in the report, the contents of the report will now be specified under the proposed 

changes to §60.8(f) and §63.7(g). Thus, the contrived piecemeal structure will no longer serve a 

purpose. 

[Note: If you select the "Report/Data Table" feature within the ERT, there is an option to export 

a "Full Test Report (excludes RATA's)" (sic). However, the option only regurgitates the data 

entered into the ERT and does not provide the same degree of detail and context that can be 

found in a typical stack test report. Again, I caution the Agency against the notion of 

considering the ERT as a replacement or surrogate for a true full stack test report.] 

As previously discussed, some test information is simply better suited to be included in a 

graphical format within the full stack test report in PDF (or equivalent) format rather than trying 
6 

to address all potential details within the ERT or an alternative XML format. If EPA believes 

that the revisions to §60.8(f) and §63.7(g) would have to be addressed by the proposed NSPS 

electronic reporting requirements, the Agency should revise the proposed rule language to 

resolve this poor construct (i.e., the rules should be revised to allow sources to meet the 

requirements by submitting a full stack test report in PDF format). 

Next Steps 

As I implored during my presentation, now is the time for EPA to carefully consider how best to 

achieve the potential benefits of electronic reporting. The Agency should not burden sources and 

testers with the legacy of the ERT just because it is something you have "on the shelf" if it is not 

the right tool for the job. EPA should slow down the rulemaking process and allow itself the 

time needed to rise to the potential of this task. In the interim, sources will continue successfully

documenting compliance using existing procedures. But, in the end, we can have a better system 

for submitting and reviewing performance test results if we do it right. 

Obviously, I advocate a more streamlined approach like the one we have proposed for MATS 

reporting, which would provide critical information in XML format while submitting all the 

other details within a PDF file of the full stack test report. This approach would allow the data to 

be reported consistently and, I would argue, simply for all methods. Notwithstanding, even if 

EPA believes that the point-by-point data are necessary, the Agency should not just blindly try to 

duplicate everything that is in the ERT within an XML format. The ERT includes a lot of 

detritus (e.g., calibration gas cylinder IDs, point-by-point times, equipment IDs, etc.) that is just 

not needed if the objective is simply to recalculate the results and that could be better addressed 

within the PDF of the full stack test report. RMB has significant experience helping sources 

with electronic reporting under the Acid Rain Program, which has required electronic reporting 

for over twenty years, and would be happy to help the Agency with the task of developing a 

viable XML format for reporting NSPS performance test data. 

Thank you for your consideration regarding these issues. Please feel free to contact me with any 

questions that you may have.