### MEMORANDUM

TO: Project Record

FROM: EPA

DATE: 1 2 February 2013

SUBJECT : Quality Assurance Activities During the Review o f the Economic P ortion of

the Detailed Questionnaire for the Coalbed Methane (CBM) Extraction Sector and the Development of the Economic Database

EPA distributed the Detailed Questionnaire for the Coalbed Methane (CBM) Extraction

Sector 1 (Detailed Questionnaire) to 105 out of 251 CBM operators (U.S. EPA, 2009a)

. The Detailed Questionnaire comprises the engineering portion (Parts A and C) and the eco nomic portion (Part B)

. EPA received responses from 88 operators in hardcopy format, which EPA reviewed and compiled in several Detailed Questionnaire databases for su bsequent industry analyses. Table 1 summarizes the information requested in the Detailed Questionnaire.

Table 1 . Summary of Information Requested in the Detailed Questionnaire for the
CBM Extraction Sector

Part A: CBM Projects 2

Section 1

– Project
Identification Selected for Questionnaire and Certification

I dentifies the projects and wells associated with the operator and determines whether the operator responded to the S creener survey 3

Part B: Financial and Economic Information . (U.S. EPA, 2009b)

Section 1

– Contact
Information I dentifies a contact for the financial and economic portion of the questionnaire.

Section 2

– Firm

- level
Financial I ncludes firm

- level financial information at the lowest corporate level (i.e., the immediate owner or operator).

Section 3

– Project

- level Questions I ncludes project

- specific financial information from the operator who manages the day

- to

- day operations of the project. Part C: Produced Water Management System (PWMS) Information

Section 1

– General Operator Information I dentifies a contact for the technical portion of the questionnaire.

Section 2

– PWMS General I ncludes project

- specific information on the components and costs of the produced water management system that is used for the project.

1 E PA used the information collected from the CBM industry in the Screener Questionnaires to select a sample of operators and projects required to comple te the Detailed Questionnaire. The statistical sampling plan is located in
the CBI docket . (Batelle, 2009)

2 EPA identified the primary unit of analysis for the CBM industry as a “project,” defined as a well, group of wells, lease, group of leases, or recognized unit that is operated as an economic unit when making production decisions.

3 The Screener requested operators to verify small business status; CBM operations in 2008; and number of projects operated. For each project, the Screener requested for calendar year 2008 the number of wells, gas production, and produced water management methods.Memorandum February 12 , 2013 Page 2

Table 1 . Summary of Information Requested in the Detailed Questionnaire for the
CBM Extraction Sector

Section 3

– Detailed PWMS
and Treatment I ncludes project

- specific information related to the discharge and treatmen t practices of the produced water management system.

Section 4

– Produced Water Quality Data I ncludes water quality data submitted by the operator, which may include data for untreated produced water, treated produced water

, and outfall discharges.

This memorandum describes the review and database development activit ies for the
economic portion of the Detailed Questionnaire (Part B ) 4 . ERG, EPA’s contractor for the
economic portion of the CBM rulemaking, conducted the questionnaire review and database development according to the Quality Assurance Project Plan f or the Economic Portion of the
Coalbed Methane Detailed Questionnaires and Coalbed Methane Economic Analysis Task (E conomic QAPP) approved on March 21 , 2012 (U.S. EPA, 2012)

. ERG received a majority of
the hardcopies of the Detailed Questionnaire from EPA in March 2010 and the remainder of the hardcopies in May 2011 . EPA had completed some initial follow

- up with operat ors upon receipt of the questionnaires ; however, upon EPA direction, ERG did not complete follow

- up activities

. This memorandum discusses the steps ERG took to review the questionnaires and develop a
database.

Section 1 discusses the questionnaire revie w procedures, Section 2 dis cusses the development of the economic database, Section 3 discusses additional updates to the data , and
Section 4 discusses the data quality of the Detailed Questionnaire responses and database

.

1.0 R EVIEW OF THE E CONOMIC P ORTION O F THE D ETAILED Q UESTIONNAIRE (P ART B )

Table 2 summarizes EPA’s distribution and receipt of the Detailed Questionnaire

responses. Table 3 summarizes the quality control (QC) activities ERG performed during the
review

. The following subsections describe the specific procedures that ERG used to ensure quality during review or the e conomic portion of the Detailed Questionnaire.

4 The review of the engineering portion of the Detailed Questionnaire is described in a separate memorandum, Quality Assurance Activities During the Review of the E ngineering Portion of the Detailed Questionnaire for the Coalbed Methane (CBM) Extraction Sector and the De velopment of the Detailed Questionnaire Database s. (U.S. EPA, 2013 a )Memorandum February 12 , 2013 Page 3

Table 2. Summary of Responses to the E conomic Portion of the Detailed Questionnaire

Number of Detailed Questionnaires

Distributed to Industry 114

Indicated Out of Scope

– No CBM wells a 7

Partial or Non

- Respondent b 19

E conomic Portion (Part B ) Reviewed 88

a

– The count of out

- of

- scope questionnaires only includes those questionnaires
that indicated that they do not operate any CBM projects

. b

– The count of partial or non

- respondent questionnaires only includes those questionnaires that did not respond or only responded partially to all projects for which EPA requested information. Partial questionnair es were not reviewed.

Table 3. QC Statistics for Review and Data Entry of the Detailed Questionnaire Responses

Number of Detailed Questionnaires Percent Reviewed

Initial Technical Review 88 100%

Second Technical Review a 88 100%

Data Ent ry into Economic Database 88 100% Data Entry First Review b 8 8 100 % c Data Entry Second Review 80 91 % d

a

– Second technical review of the Detailed Questionnaire was conducted by a senior staff member familiar with the CBM Detailed Questionnaire, the CBM industry, and the issues operators had completing the questionnaire.
b

– Data entry review was conducted by a technical staff member familiar with the technical review (either as initial reviewer or second reviewer).

c

– ERG received responses for one Detailed Questionnaire in 2012 and reviewed and data entered the responses
into the database at a later time (with data entry first review). This was not part of the original 2010 review process; however, ERG implemented the same review procedures for this questionnaire.
d

– ERG data entered and second reviewed 80 of the Detailed Questionnaires in 2010 prior t o the end of the contract period. During the 2011 review, ERG determined the data entry of the responses provided a high level of accuracy and did not continue the data entry second review of the remaining questionnaires

.

1.1

E RG developed a n internal tracking spreadsheet (Figure 1 of Economic QAPP, Appendix A) to quickly identify issues with Part B of the Detailed Questionnaire and to keep track of the receipt, review, and transcription of the responses t o Part B of the questi onnaire. Key fields in

t he tracking database table include : Log

- in and Tracking Procedures

**•** Survey ID

**•** Company

**•** Survey Receipt (Date)

**•** Folder Prepared (Date)

**•** Form One Complete

**•** Financial Transcription RequiredMemorandum February 12 , 2013 Page 4

**•** Financial Transcription Complete

**•** Review 1 and Coding Complete

**•** Review 2 Complete

**•** Callback Required

**•** Callback Complete

**•** Data Entry Complete

**•** Initial Checks of Data Entry

**•** Review of Data Entry Checks

Each Detailed Questionnaire received by ERG was stored in a designated folder with a review packet (Appendix A.1 of the Economic QAPP) to doc ument the levels of review for e ach questionnaire

. ERG maintained separate file drawers for responses in each step of the review process. As one step was completed for a questionnaire, it was placed in the designated file drawer for the next step to furth er identify the questionnaires by stage of the process. Additionally, at the completion of a step, each folder was marked with a colored dot, which signified what stages the questionnaire had already passed through . Thus, any out

- of

- place
folders could be immediately recognized. As each questionnaire passed through each stage, the staff person who performed the specific task (e.g., First Reviewer) initialed a form appropriate to the stage of the review and coding process (held within the questionnaire folde r ) , and reported the completed stage to the review and coding supervisor, who updated the tracking spreadsheet

.

EPA gave respondents the option to claim answers to specific questions as CBI by marking question

- specific indicators in the Detailed Questionnaire or to claim all eligible data CBI by marking the “allCBI” box at the beginning of the Detailed Questionnaire. During the review of the Detailed Questionnaire, ERG handled all questionnai res as CBI, as specified in the
Economic QAPP.

1.2

ERG reviewed the responses to the economic portion of the Detailed Questionnaires. Review and Coding of Questionnaire Responses

ERG had conducted some initial follow

- up during the 2010 review of the quest ionnaires. When ERG continued review in 2011 under a separate contract and under EPA direction, we did not conduct any additional follow

- up.

An initial review determined whether all key sections and questions in the Detailed Questionnaire were complete d . Some Detailed Questionnaires were largely blank and were
counted as non

- responses. EPA offered to transcribe financial data for small operators if they included their tax forms and financial reports with the Detailed Questionnaire

. During the initial

re view, ERG determined whether financial transcription was necessary for each response (see
section 2.0 for further details)

.

After completing the financial transcriptions, ERG performed the first detailed review of
each questionnaire and organized respondent comments. To preserve the original hardcopies of the questionnaires, ERG created and marked up review copies of the Detai led Questionnaires according to the Instructions for Survey Tracking, Review and Coding (Appendix A in theMemorandum February 12 , 2013 Page 5

Economic QAPP). ERG reviewed the completeness and reasonableness of the questions and flagged discrepancies and documented any corrections in Form 6 of Appendix A of the Economic QAPP.

ERG conducted a second detailed review of all the Detailed Questionnaires

. ERG documented the issues found during second review on F orm 6 but did not complete any follow

up calls, as directed by EPA. The initial te chnical reviewer resolved any issues from second review and noted additional changes to the questionnaire on the review copy

, Form 6, and the respondent comment section of Part B

.

ERG also coded the responses during review to ensure they could easily b e entered into a
database. E RG used the following codes during review:

**•** Unknown (respondent wrote in unknown):

- 777

**•** Not applicable:

- 888

**•** Responded left a required response blank:

- 999

.

1.3

This section describes ERG’s d at a entry and proofing procedures

. Data Entry and Database Development

1.3.1 Data Entry and Initial Checks

ERG developed the economic database to house the questionnaire responses by constructing tables that followed the design of the Detailed Questionnaire. The CBM economic survey database is a confidential database that is located on EPA’s CBI docket (U.S. EPA, 2013b). The memorandum, Summary of Confidential Business Information Claims in Industry Responses to EPA’s CBM Screener Surveys and Detailed Questio nnaires and Industry Data
Available in the EPA Docket , describes EPA’s decision to maintain this database in the CBI docket (U.S. EPA, 2013c). The database dictionary provides a description of the tables and data
fields included in the economic database ( U .S. EPA, 2013 d ).

Because EPA received most of the responses to the Detailed Questionnaires in hardcopy, ERG manually entered all responses into the database, including all corrections, coding, and documented changes. Using the review copies of the Detail ed Questionnaires, E RG constructed
the e conomic database and transcribed the responses or E RG ’s corrections to the responses into
the economic database. ERG performed first review of 100% and second review of 90% of the questionnaires entered into the data base by visually comparing all the information contained in the database to the review copy. ERG internally tracked the data entry and data entry review in the survey tracking form.

After data entry was completed, ERG performed a series of quality contr ol and assurance
checks , including queries to check reporting consistency between questions and to check responses against publicly available data. See Section 4.0 for details of ERG’s quality control procedures

.Memorandum February 12 , 2013 Page 6

1.3.2 Data Entry Proofing

A reviewer (a person who had not entered the data) performed the first data entry review for 87 of the 88 responses (99%) 5 by visually comparing all the information contained in the database to the review copy . A second reviewer reviewed the changes that were made in the firs t data entry review for 80 of the 88 responses ( 90 %) against the review copy

. E RG documented
th e s e review s on Forms 4 and 5 in Appendix A of the Economic QAPP.

After E RG completed data entry , data checks, and data entry review of the Detailed Questionna ire s , E RG performed database clean

- up and standardization activities as needed to facilitate subsequent data analyses. Sections 3.0 and 4.0 describe these activities.

2.0 EPA R EVIEW AND P ROCESSING OF D ETAILED Q UESTIONNAIRE A TTACHMENTS FOR
P ART B

CBM operators may have included the following supplemental information in
attachments to Part B of their Detailed Questionnaires :

**•** Tax forms (generally, S

- Corp filings) ;

**•** Financial reports (income statements and balance sheets); and

**•** Production data, decline curves, or gas and water projections.

For respon dents requesting assistance with their financial information, ERG identified
relevant financial data in financial reports or tax forms and transcribed the data into Part B

- 2 of the Detailed Questionnaire. ERG transcribed financial information for 15 questionnaires. Because inclusion of financial reports or tax forms was only required from those requesting assistance, ERG was not able to fill in missing or incorrectly entered or calculated financial
information from most other questionnaire respondents . In a few cases, however, missing data or data that did not compute correctly could be checked against Security and Exchange Commission (SEC) submi ssions for publicly held firms.

When ERG found data to complete the firm level information for such firms

, ERG
transcribed the new numbers in to the Review copy and added the printout of the relevant pages from the annual report to the questionnaire fold er. Respondents provided financial a ttachments in hard copies so ERG manually transcribed relevant information onto the Review Copies of the responses. Before transcription, however, ERG used data from the attached financial reports
and/or tax forms to cal culate the specific financial metrics requested in the Detailed Questionnaire.

A senior staff member performed the calculations and initial transcription. An ERG analyst who did not perform the initial calculations and transcription reviewed the transc ription

5 One Detailed Questionnaire was considered a non

- response, because part of it was not received by ERG until after the quality control checks of the Economic Database were completed. Afte r this response was entered into the database, the data entry was checked, though not as a part of the formal data entry and review effort.Memorandum February 12 , 2013 Page 7

process verifying that all calculations were accurate and that the results had been correctly transcribed onto the review copies. If any changes appeared to be needed, the two analysts
conferred to ensure that the correct numbers were transcribed. Any changes made were checked by both analysts. Only when a final pass was made by both analysts with no changes needed was the questionnaire returned to the review and coding process.

If the respondent had provided gas and water production attachments in lieu of answering production questions

, ERG performed c alculations and transcriptions followed the same model
as that for financial data. The Detailed Questionnaire requested that respondents provide d ecline curve and/or projection attachments . ERG did not transcribe these materials but retained them in
the folder to assist in projection estimates.

3.0 A DDITIONAL U PDATES TO THE D ETAILED Q UESTIONNAIRE R ESPONSES

ERG incorporated additional information from public data sources into the economic
database. T his section discusses how ERG obtained this information and added it to the Detailed Questionnaire responses contained in the economic database.

3.1.1 Public Data Used to Fill in Missing Responses for the Detailed Questionnaire

Whenever a response appeared to be an outlier, or information provided in the response conflicted with information collected in the screener survey, ERG verified non

- financial key data elements such as gas produced, sold, and used; water produced; firm

- size; and acquisition and operatin g dates against publicly available databases such as operator websites, SEC filings, and state oil and gas commission websites. All corrections were documented on the Review Copies of the Survey, and in the comments section of Part B of the Detailed Questi onnaire, which were
data entered into the economic database. ERG downloaded copies of the information obtai ned
from SEC filings that were used to correct questionnaire responses and include d them in the operator file.

3.1.2 Verifying Net Revenues for Dischargi ng Projects

Based on 2008 revenues, costs, and other financ ial metrics, ERG estimated 2008 and 2010 Net Revenues to analyze the impacts of potential costs. ERG used the Producer Price Index ( P PI) for the Crude Petroleum and Natural Gas Extraction Industry to inflate financial data responses to the desired year and used the average over the questionnaire

- reported years 2006

- 2008 (in 2008 and 2010 dollars, respectively) to estimate firms ’ and projects ’ typical net

revenues

. ERG performed several additional data checks and modifications at this stage to ensure that gas production, water production, taxes, and other relevant estimates were reported with
consistent units across firms. Also some respondents did not report key el ements used to
calculate net revenues , or their responses were well outside the range of expected responses

. Where data were not reported or an extreme outlier was suspected , ERG imputed average wellhead prices or royalty, severance, and ad valorem tax rat es based on the responses of others operating in the same basin. ERG described all changes and additions to respondent

- provided
information in the comments field of the economic database

.Memorandum February 12 , 2013 Page 8

4.0 D ATA Q UALITY OF THE D ETAILED Q UESTIONNAIRES AND E CONOMICS D ATABAS E

ERG performed extensive checks on the responses to Part B of the Detailed Questionnaire to verify that the financial information collected and the economic database containing the information was complete, accurate, and reasonable. This section documents ERG’s review of the data quality of the detailed questionnaire responses and the economics
database.

4.1.1 Completeness

ERG reviewed 88 out of 88 Detailed Questionnaires (100%). ERG used Instructions for Survey Tracking, Review and Coding (Appendix A of Econ omic QAPP) to review question

specific checks for completeness and reasonableness. As directed by EPA, ERG did not continue
follow

- up activities during its 2012 review of the Detailed Questionnaire responses. Subsequent industry analyses using the economi c database included only the data available from the complete responses or from responses that ERG completed using a vailable information throughout the questionnaire

.

Table 4 summarizes ERG’s check on completeness for questions that were used in EPA’s industry analyses, including responses to questions on revenues, taxes, costs, and gas production

. Responses that were illegible, unknown or NA, or left blank (but required) were coded according to the instructions and not used in any subsequent EPA analyses.

Table 4. Completeness of Revenues, Costs, and Production Questions in the Detailed Questionnaire

Question

Number Question Description Number of Complete Responses Number of Expected Responses Percent Complete

B2

- 17e Total revenues 65 88 74

B3

- 50 Gas production operating costs 170 172 98

B3

- 59 Gas production 170 172 98

B3

- 62 Average project wellhead price 166 172 97

B3

- 65 Royalty rates 166 172 97

B3

- 66 Severance rates 166 172 97

B3

- 67 Other taxes 165 172 96

4.1.2 Reasonableness

ERG performed question

- specific consistency checks on the financial data used in EPA’s industry analyses. In general, the majority of the data provided seemed reasonable given ERG’s knowledge of the individual operators, how they answered their screeners, and how the CBM industry operates. ERG performed the following to ensure we reviewed in more depth any

po tentially unreasonable data and made corrections, or imputed data, where possible:Memorandum February 12 , 2013 Page 9

**•** Checking for outliers, determining which could be possible, and which were far beyond any reasonableness using the screener, publicly available information, or general kno wledge of usual conditions or circumstances in the industry (for example, knowledge that a 0.2 percent royalty rate is highly unlikely but that reporting .2 instead of a much more likely 20 percent might be a more reasonable circumstance);

**•** Checking for s ummations where items were supposed to sum, checking for units and order of magnitude issues (reporting $ instead of $1,000s, e.g.);

**•** Checking outliers against publicly available information; and

**•** Checking other information where general knowledge of the industry indicated the answers might have been in error against screener information or publicly
available information.

ERG recorded all corrections on the Review Copy of the Detailed Questionnaire and updated the data entry for each response in the econo mic database.

4.1.3 Accuracy

ERG conducted review of the initial data entry for all questionnaires into the economic
database and conducted a second review for 9 1 % of the questionnaires after initial checks and proofing changes were made to the database.

ERG also checked that responses reporting total revenues and costs for all CBM projects
in 2008 in section B

- 3 of the Detailed Questionnaire matched those reported for CBM revenues reported by the firm in Section B

- 2. ERG ran the financial data queries ite ratively to ensure that all errors that could be resolved had been resolved when fixes were made. Thus, ERG systematically verified the financial data for all discharging operators to identify any incorrect financial estimates reported as a part of the ini tial responses.Memorandum February 12 , 2013 Page 10

5.0 R EFERENCES

1. Batelle, 2009

. Coalbed Methane Study: Comparison of Original to Final Detailed Questionnaire Sample Design

. Batelle Memorial Institute. May 27, 2010. EPA

- HQ

- OW

2010

- 0824, DCN CBM00635.

2. U.S. EPA, 2009a

. Detailed Questionnaire for the Coalbed Methane (CBM) Extraction

Sector . U.S. Environmental Protection Agency. October 2009. EPA

- HQ

- OW

- 2006

- 0771,
DCN 10072

. 3. U.S. EPA, 2009b

. Co albed Methane Industry Screener. U.S. Environmental Protection Agency. 2009. EPA

- HQ

- OW

- 2006

- 0771, DCN 10068

. 4. U.S. EPA, 201 2

. Quality Assurance Project Plan for the Review a nd Data Entry of the

Economic Portion of the Coalbed Methane Questionnaire

. U.S. Environmental Protection Agency. March 21, 201 2

. EPA

- HQ

- OW

- 2010

- 0824, DCN CBM00 6 92

. 5. U.S. EPA, 2013a. Quality Assurance Activities During the Review of the Engineering Portion of the Detailed Questionnaire for the Coalbed Methane (CBM) Extraction Sector and the Development of the Detailed Questionnaire Databases. U.S. Environmental

Pro tection Agency. February 2013. EPA

- HQ

- OW

- 2010

- 0824, DCN CBM00663.

6. U.S. EPA, 2013 b

. CBM Economic Survey D atabase (Microsoft Access 2003 database)

. U.S. Environmental Protection Agency. February 2013. EPA

- HQ

- OW

- 2010

- 0824, DCN
CBM00656.

7. U.S. EPA, 2013c. Summary of Confidential Business Information Claims in Industry Responses to EPA’s CBM Screener Surveys and Detailed

. U.S. Environmental Protection Agency. February 2013. EPA

- HQ

- OW

- 2010

- 0824, DCN CBM00661.
8. U.S. EPA, 2013 d

. CBM Economic Survey Database D ic tionary

. U.S. Environmental Protection Agency. February 2013. EPA

- HQ

- OW

- 2010

- 0824, DCN CBM 00659.