## RMB Consulting & Research, Inc.

5104 Bur Oak Circle Phone (919) 510-5102 Raleigh, North Carolina 27612 Fax (919) 510-510 4

M E M O R A N D U M

TO : Steffan Johnson, EPA

FROM : Stephen Norfleet

DATE : April 5, 2016

SUBJECT : SES Presentation on EPA’s Proposed Revisions t o the Reporting Requirements

for Performance Test Data under 40 CFR Parts 60 and 63

As requested, here is a copy of the presentation th at I made at the recent Stationary Source Sampling & Analysis for Air Pollutants Conference t hat is organized by the Source Evaluation Society (SES). As you are aware, given that you se rve as one of SES's current board members, SES is an organization that was formed to advance t he science of emission source testing and to foster the professional development of individuals involved with source testing. Accordingly, SES's membership is largely made up of stack tester s and includes federal, state and local agency personnel who are involved with assessing emission test data.

My presentation focused on two sets of rule revisio ns that the Agency has proposed: 1) the proposed changes to §60.8(f) and §63.7(g) that were part of a set of proposed "Revisions to Test Methods, Performance Specifications, and Testing Re gulations for Air Emission Sources," which were published on September 8, 2015, and 2) the pro posed "Electronic Reporting and Recordkeeping Requirements for New Source Performan ce Standards (NSPS)," which were published on March 20, 2015. The proposed changes to §60.8(f) and §63.7(g) would identify specific information to be included in performance test reports required under 40 CFR Parts 60 and 63 regardless of whether the reports are "submi tted electronically or in paper format." The proposed electronic reporting provisions would requ ire existing sources under many NSPS subparts to begin electronically reporting data via the Electronic Reporting Tool (ERT) and/or the Compliance and Emissions Data Reporting Interfa ce (CEDRI).

Although I do have some concerns with the proposed §60.8(f) and §63.7(g) revisions (as reflected in RMB's comments, which were included as part of Hunton &Williams LLP's comments submitted on behalf of the Utility Air Reg ulatory Group), my presentation mostly just outlined those proposed requirements. With respect to the proposed NSPS electronic reporting, I focused on the ERT reporting requirements, highligh ted significant issues with the ERT, and proposed an alternative reporting approach. As dis cussed in my presentation, the alternative would provide the benefits of electronic reporting, reduce the burden on sources and testers, permit EPA to readily apply consistent reporting fo r different methods/sources, and allow the data to be more easily used by agencies and others are interested in reviewing the information.2

ERT Issues Given the audience and the direct impact that the p roposed revisions would have on the way source testers report performance test data, the pr esentation elicited significant interest. Many expressed frustration regarding the time consuming and cumbersome task entering test data using the ERT, even when they use the templates for trans ferring some of the data into the Access

database. Some discussion involved differences in the amount of data that testers actually "fillin" in the ERT, with it being apparent that some tr y to complete all the fields while other leave portions blank. This issue is exacerbated by the f act that there is no regulatory basis for some of the fields within the ERT and by questions over wha t information is necessary to submit in terms

of the test results.

These issues, as well as other problems, pose serio us questions about the choice of using the ERT as a vehicle for electronic reporting. For yea rs, the EPA has touted the ERT as an all-inone tool that "replaces the time-intensive manual p reparation and transcription of stationary source emissions test plans and reports currently p erformed by contractors for emissions sources and the time-intensive manual quality assurance eva luations and documentation performed by State agencies." 1 But, these claims are betrayed by the fact that s ources, testers, and states have not adopted the ERT. In spite of EPA expending res ources towards many ERT revisions, testers have not migrated to used the tool beyond where it has been required (e.g., for ICR tests and for newer Subpart Da units). This real-life feedback o n the purported usefulness of the ERT is echoed by its not being used by agencies that revie w test data. In fact, none of the state or local agency personnel that attended the conference indic ated that their agencies used the reported ERT data in any way, suggesting that the effort to produce ERT files may result in relatively useless artifacts rather than reaching the potentia l benefits that electronic reporting offers.

The ERT is also ill-suited to fill the goals of EPA 's "Next Generation Compliance" Initiative. The tool does not meet the objective of making envi ronmental reporting more efficient, and it certainly does not "expand transparency by making i nformation more accessible to the public." 2
I would suggest that the awkward tool and Access da tabase structure actually creates a roadblock to potential use by the public. If anything, the E RT makes the data more opaque and less accessible than the hard-copy stack test reports (o r in the electronic arena, PDF versions of the full stack test report), which the ERT seems intend ed to replace.

But, these complaints against the ERT are not new. The Agency has received significant adverse comments highlighting problems with the ERT, includ ing comments from state agencies, not only in response to this current rulemaking but als o in response to the initial Advanced Notice of Proposed Rulemaking (ANPR) in 2009 that preceded th ese revisions as well in response to other proposed rules that have required the use of the ER T. In light of these comments, EPA's steadfastness in proposing the ERT as the basis for electronic reporting instead of proposing other options is perplexing. While I understand th at EPA has expended resources toward the development of the ERT, it is simply not a good cho ice for electronic reporting.

Part of the problem with the ERT is that it is not a format. Proposing rule language stating that "performance test data must be submitted in a file format generated through the use of the EPA’s

1 https://www3.epa.gov/ttnchie1/ert/

2 https://www.epa.gov/compliance/next-generation-com pliance3

ERT" does not define the data that must be reported . Not only does the ERT contain fields for data for which there seems to be no regulatory basi s, but the Agency might change the ERT at any time. In the rulemaking, EPA suggests that it might add new methods to the ERT at any time (and that sources would need to stay abreast o f such changes to know when it should report such information using the ERT) but, since the form at is not defined, potential changes would not necessarily be confined to new methods. In oth er words, EPA seems to be proposing reporting requirements that are completely outside the normal regulatory review and comment

process.

EPA needs to define an actual format. Sources and emission testers already have spreadsheets and tools that they have personally verified over t ime. Defining the reporting in terms of an actual format would allow the reporting to be more easily automated and integrated using the tools they already have.

Alternative XML Reporting Format Given these issues, I am pleased to hear that EPA i s considering developing an XML reporting alternative to the ERT. But, I would suggest that developing a such an XML format (rather than requiring the ERT) is where the Agency should be fo cusing its effort. Proposing electronic reporting prior to the development of such a format (and prior to the proposed changes to §60.8(f) and §63.7(g)) was a premature action.

If EPA wants to reduce the burden of reporting whil e still providing the information needed to assess compliance, the Agency should consider how t o best to achieve its goals. I would suggest a balanced approach. Some data may be useful to ha ve in a numeric format, but other

information is better suited to be documented in a PDF (or equivalent) format. For example, the raw data sheets and example calculations that EPA h as proposed to include in test reports do not readily lend themselves to a non-graphical format ( particularly if the raw data is a hand entered form). Also, with a PDF (or equivalent) format, th e information may be detailed in a way that provides greater context and allows for site-specif ic or test-specific information to be more easily

documented. A combination of both PDF and numeric data, properly implemented, would allow the information to be reported electronically and c an achieve the benefits of paper work reduction and accessibility.

In my presentation, I outlined such an alternative reporting approach that was developed by the workgroup that RMB is coordinating to address repor ting issues associated with the Mercury and Air Toxics Standards (MATS) Rule. While the approa ch developed by the workgoup was intended to address the specific needs of reporting under the MATS rule, it provides a template that could be used to report performance test data for any method or subpart. Under our proposed reporting approach, critical details would be reported in XML, but source would also be required to submit a full stack test report in P DF format for complete transparency and to document other details so that agency personnel can have ready access to that information in the full context of the report.

As shown in the presentation slides, sources would report XML data detailing the test methods, times, and concentration data for each reference me thod run as well as the parameters used to

convert the concentrations to the units of the emis sion standard (e.g., CO 2 or O 2 values used to4

calculate the lb/MMBtu values), the applicable emis sion limit, process load, sample times and/or sample volumes, and various critical quality assura nce (QA) parameters (e.g., for Methods 5, the QA parameters might include parameters the iso-kine tic rate, filter temperature, leak check results). Such values could be readily reviewed by agency personnel and might be validated as part of the submission process to ensure that compl iance is achieved, to verify that the test was properly performed and to estimate the potential un certainty of the results. Since one of EPA's goals with electronic reporting is to provide data for emission factor development, some process data might be reported (e.g., a electric utility so urce might report the electrical load or average heat input during the performance test). But, the specific process information, if it can be justified, would need to be defined for each type o f source. Thus, although this information is not necessarily needed to assess compliance with th e NSPS, EPA might need to define such process reporting in the individual subparts in ord er to provide a regulatory basis for the reporting.

In comparison to the ERT, the alternative approach streamlines the reporting for sources and for the agencies reviewing the data by focusing on the data that is needed to establishing compliance and to develop emission factors. It would also pro vide for consistent reporting across different methods and sure types. Rather than having to appl y patch over patch to the ERT and deferring electronic reporting until the methods are "support ed by the ERT," the Agency could simply define the critical QA parameters for each method.

The ERT requires much more data than is needed to v erify compliance and develop emission factors. The ERT includes fields for entering trav erse point-by-point details that would be better

documented with other relevant test data in the ful l stack test report submitted as PDF file. Focusing on providing the critical values in a nume ric format reduces the reporting burden and allows testers and sources to spend more time quali ty assuring the results. Adequate review time to allow sources to ensure the accuracy of the info rmation is an important consideration since EPA is proposing these new electronic reporting req uirements without extending the time when such reports are due.

Some have suggested that states and local agencies need point-by-point data in order to recalculate the results. However, while occasional spot checks might be reasonable, requiring all sources to submit traverse point-by-point data in n umeric format would seem to provide little benefit. After all, once you check a source tester 's calculation spreadsheets, what are you really expecting to find the next time? And, if we are go ing to recalculate all the results from the pointby-point data, why has EPA proposed to include exam ple calculations within the test reports? Is not the purpose of the example calculations to illu strate that those calculations are being performed properly?

Simply checking calculations does not verify data q uality or ensure that the testing was conducted properly or that the right values or assu mptions were made when entering the data. A more meaningful assessment requires test observatio n and/or critically reviewing the report details and results. The point-by-point details wi ll already be documented in the full stack test report for review and auditing purposes. Furthermo re, if a state or local agency believes that it needs the numeric data, it has its own regulatory m echanisms to require sources within its jurisdiction to provide that data. In other words, putting aside the question of whether reporting5

point-by-point numeric data is really necessary, th e EPA does not need to create a requirement to provide this data to the state and local agencies b ecause they already have regulatory avenues that they could use to obtain the data.

Based on the discussion following the presentation, it is fair to say that there is considerable support within the stack test community for using a n alternative electronic reporting format in lieu of the ERT. The Agency should also appreciate that this support comes from a group that has been directly involved with using the ERT. Thi s support for a more streamlined and balanced approach is not from group of Luddites who are shying away from the concept of electronic reporting. SES even has an "ERT Improve ment" committee. The problem is that the ERT is just not the right tool for the job. The Ag ency can do much better.

Potential Rule Interaction As we discussed at the conference, I am also concer ned with potential interaction between the two proposed rulemakings. As I admitted, I did not initially see this potential issue. From my perspective, the proposed changes to §60.8(f) and § 63.7(g) would dictate what source testers would need to include in their reports and the prop osed NSPS electronic reporting requirements would also require sources to submit test data usin g the ERT (or, hopefully, submit data via an alternative XML format). However, during recent co nversations regarding MATS reporting, EPA personnel have made statements suggesting that the ERT would need to be revised to address additional information required by the prop osed changes to §63.7(g).

This statement did not make sense to me until I rea lized that they were equating the ERT with a test report. First, I do not believe that that thi s interpretation necessarily the most reasonable one. But, more importantly, the assumption that th e ERT should serve as the required test report is a bad paradigm. The official test report should be the complete test report that includes all the

information that the source and/or testers believe should be included to fully document the test results, quality assurance aspects, and other relev ant details of the test.

The ERT does include the ability to provide attachm ents, but I would argue that patchwork of various attachments specified by the ERT not only i ncreases the burden of reporting but also makes cobbled-together ERT agglomeration more diffi cult to use than a complete and unified report that can provide such details in a meaningfu l context. Furthermore, while the series of

ERT attachments have been described as a method for ensuring that the "proper" documentation is included in the report, the contents of the repo rt will now be specified under the proposed changes to §60.8(f) and §63.7(g). Thus, the contri ved piecemeal structure will no longer serve a

purpose.

[Note: If you select the "Report/Data Table" featu re within the ERT, there is an option to export a "Full Test Report (excludes RATA's)" (sic). Howe ver, the option only regurgitates the data entered into the ERT and does not provide the same degree of detail and context that can be found in a typical stack test report. Again, I cau tion the Agency against the notion of considering the ERT as a replacement or surrogate f or a true full stack test report.]

As previously discussed, some test information is s imply better suited to be included in a graphical format within the full stack test report in PDF (or equivalent) format rather than trying6

to address all potential details within the ERT or an alternative XML format. If EPA believes that the revisions to §60.8(f) and §63.7(g) would h ave to be addressed by the proposed NSPS electronic reporting requirements, the Agency shoul d revise the proposed rule language to resolve this poor construct (i.e., the rules should be revised to allow sources to meet the requirements by submitting a full stack test report in PDF format).

Next Steps As I implored during my presentation, now is the ti me for EPA to carefully consider how best to achieve the potential benefits of electronic report ing. The Agency should not burden sources and testers with the legacy of the ERT just because it is something you have "on the shelf" if it is not the right tool for the job. EPA should slow down t he rulemaking process and allow itself the time needed to rise to the potential of this task. In the interim, sources will continue successfully documenting compliance using existing procedures. But, in the end, we can have a better system for submitting and reviewing performance test resul ts if we do it right.

Obviously, I advocate a more streamlined approach l ike the one we have proposed for MATS reporting, which would provide critical information in XML format while submitting all the

other details within a PDF file of the full stack t est report. This approach would allow the data to be reported consistently and, I would argue, simply for all methods. Notwithstanding, even if EPA believes that the point-by-point data are neces sary, the Agency should not just blindly try to duplicate everything that is in the ERT within an X ML format. The ERT includes a lot of detritus (e.g., calibration gas cylinder IDs, point -by-point times, equipment IDs, etc.) that is just not needed if the objective is simply to recalculat e the results and that could be better addressed within the PDF of the full stack test report. RMB has significant experience helping sources with electronic reporting under the Acid Rain Progr am, which has required electronic reporting for over twenty years, and would be happy to help t he Agency with the task of developing a viable XML format for reporting NSPS performance te st data.

Thank you for your consideration regarding these is sues. Please feel free to contact me with any questions that you may have.