# Quality Assurance Project Plan For the Economic Portion of the Coalbed Methane Detailed Questionnaires and Coalbed Methane Economic Analysis Task
(Element 1.A)

Prepared for:


## U.S. Environmental Protection Agency Engineering and Analysis Division O ffice of Water 1200 Pennsylvania Avenue, NW Washington, D.C. 20460

Prepared by:


## Eastern Research Group, Inc.

110 Hartwell Avenue Lexington, MA 02421

March 15 , 2012

EPA Contract No. 68

- C

- 0 2

- 095 Work Assignment 9

- 37QAPP

— CBM Detailed Questionnaire and Economic Analysis

ii History of the QAPP for the Economics Portion of the Coalbed Methane Project

Rev 0 Delivery Date November 2, 2010 Work Assignment 8

- 05 Scope CBM Detail ed Questionnaire (Part B)
Comments Received NA
Comments Addressed NA

Approval November 3, 2010

Rev 1 Delivery Date January 4, 2012 Work Assignment 9

- 37 Scope CBM Detailed Questionnaire (Part B) and Economic Analysis Task
Comments Received Febru ary 27, 2012
Comments Addressed March 15, 2012

ApprovalQAPP

— CBM Detailed Questionnaire and Economic Analysis

iii Summary of Changes Made in Rev. 1

Applicable Section Change QAPP Title Changed to reflect the broader scope of the QAPP Entire QAPP Minor editorial changes and reorganization for clarity and readability and updates to Work Assignment number and revision number Approvals Updated to reflect ERG and EPA personnel changes on project
Distribution Updated to reflect ERG and EPA personnel changes on project
1. Introduction and Background Upda ted to reflect new ERG WAM, a change in Engineering Contractor, expansion of scope, and extension of project history
2.1 Project/Task Organization Updated Project Organization figure to reflect change in Engineering Contractor and updated text to reflect new staffing and lines of communication and reorganized section for clarity
2.2 Problem Definition and Background Revised to reflect expanded scope and new tasks under the Economic Analysis Task
2.3 Project/Task Description Revised to reflect expanded s cope and new tasks under the Economic Analysis Task , change in Engineering Contractor
3.4 Data Handling and Custody Revised to reflect change in Engineering Contractor

3.5.2 Questionnaire Response Review and Followup ERG deleted a sentence to reflect th e fact that EPA no longer believes a letter informing operators of changes made to their data during data cleaning is required and made minor edits to reflect a change in initial callback procedures necessitated by a change in task priorities under Work As signment 8

- 05
3.5.3 Questionnaire Response Coding and Data Entry Revised to reflect more extensive proofing used to QA data entry

4.1 Assessments and Response Actions Revised to reflect expanded scope

4.2 Reports to Management Revised for compatibility with overall Work Assignment 9

- 37 QAPP
6. Addendum to the QAPP Added d etailed discussion of QA procedures associated with the expanded scope of the economic analysis activities, including the analysis of the detailed questionnaire data, replacement of mis sing data, methodology development, application of methodologies to data, and report writing tasks. Appendix A Deleted contact listQAPP

— CBM Detailed Questionnaire

iv CONTENTS (Element A.2)

1. I NTRODU CTION AND B ACKGROUND

................................ ................................ ..................... 1

2. P ROJECT M ANAGEMENT E LEMENTS

................................ ................................ .................... 1 2.1 Project/Task Organization (Element A.4)

................................ ............................... 1 2.2 Problem Definition and Background (Element A.5)

................................ .............. 3 2.3 Project/Task Description (Element A.6)

................................ ................................ . 4 2.3.1 Finalizing the Detailed Questionnaire

................................ ........................ 5 2.3.2 Providing Survey Assistance

................................ ................................ ...... 5 2.3.3 Receipt and Internal Tracking o f Questionnaires

................................ ....... 5 2.3.4 Developing a Survey Database and Data Dictionary

................................ .. 6 2.3.5 Reviewing, Transcribing, Coding, and Entering Respondent Data into Survey Database

................................ ................................ .................. 6 2.3.6 Following Up on Incomplete Surveys

................................ ........................ 6 2.3.7 Tabulating Data

................................ ................................ ........................... 7 2.4 Quality Objectives and Criteria (Element A.7)

................................ ....................... 7 2.4.1 Completeness

................................ ................................ .............................. 7 2.4.2 Accurac y

................................ ................................ ................................ ..... 7 2.4.3 Reasonableness

................................ ................................ ........................... 8 2.5 Special Training Requirements/Certification (Element A.8)

................................ .. 8 2.6 Documentation and Records (Element A.9)

................................ ........................... 8

3. D ATA G ENERATION AND A CQUISITION E LEMENTS

................................ .............................. 1 3.1 Sampl ing Process Design (Element B.1)

................................ ................................ 1 3.2 Sampling Methods (Element B.2)

................................ ................................ ........... 1 3.3 Completeness

................................ ................................ ................................ .......... 1 3.3.1 Accuracy and Reasonableness

................................ ................................ .... 2 3.4 Data Handling and Custody (Element B.3)

................................ ............................. 3 3.5 Data Management (Element B.10)

................................ ................................ .......... 3 3.5.1 Questionnaire Non

- Response Follow Up

................................ ................... 3 3.5.2 Questionnaire Response Review and Follow Up

................................ ........ 4 3.5.3 Questionnaire Response Coding and Data Entry

................................ ........ 5

4. A SSESSMENT AND O VERSIGHT E LEMENTS

................................ ................................ ........... 1 4.1 Assessments and Response Actions (Element C.1)

................................ ................ 1 4.2 Reports to Management (Element C.2)

................................ ................................ ... 2

5. R ECONCILIATION WITH U SER R EQUIREMENTS

................................ ................................ ..... 1

6. A DDENDUM C OVERING THE D ATA A NALYSIS AND U SE OF E XISTING D ATA IN T HE C OALBED M ETHANE E CONOMIC A NALYSIS

................................ ................................ ......... 1 6.1 Introduction

................................ ................................ ................................ ............. 1 6.2 Project Description and Objectives

................................ ................................ ......... 2 6.3 Data Needs, Sources, and Quality Criteria

................................ ............................. 3 6.4 Data Acquisition and Review

................................ ................................ ................. 7CONTENTS (Continued)

Page

v 6.4.1 Existing Data Tracking Database (Tracking Database) and Development of Detailed Data Tables

................................ ........................ 8 6.4.2 Development of Data Subsets

................................ ................................ ..... 9 6.5 Use of Existing Data and Quality Verification

................................ ..................... 10 6.5.1 Develop Methodologies and Evaluate Profile Information (Tasks 8.a, 8.c, 8.d, and 8.f)

................................ ................................ .................. 10 6.5.2 Calculate Impacts

................................ ................................ ...................... 12 6.5.3 Check Results of Impact Estimates

................................ ........................... 13
6.6 Project Reports (Deliverables)

................................ ................................ .............. 15

7. R EFERENCES

................................ ................................ ................................ ........................ 1

Appendix A: INSTRUCTIONS FOR SURVEY REVIEW & CODINGQAPP

— CBM Detailed Questionnaire

vi LIST OF TABLES

Page

1 QAPP D istribution

.............................................................................. . .. vii
6

- 1 Existing Data Collection Criteria............................................................... .6

- 6
6

- 2 Industry Profile and Updated Production and Financial Data Acceptance Criteria

........ 6

- 5
6

- 3 Criteria for Acceptance of Replacement Data for Missing or Erroneous Data

............... 6

- 6
6

- 4 Impact Estimate Quality Specification

................................ ................................ ............ 6

- 7
6

- 5 Summary of Procedures and Result s Quality Verification

................................ ............ 6

- 14
6

- 6 Key Products

................................ ................................ ................................ .................. 6

- 16

LIST OF FIGURES

Page

2

- 1 Project Organization (Element A.4)

................................ ................................ ................ 2

- 2

3

- 1 CBM Detailed Questionnaire Contact Record

................................ ................................ . 3

- 6QAPP

— CBM Detailed Questionnaire

vii DISTRIBUTION (Element A.3)

This Quality Assurance Project Plan (QAPP) was distributed to staff of the U.S. Environmental Protection Agency and Eastern Research Group, Inc. (Table 1)

. A copy of the document was provided to all Eastern Research Group involved in the project, includi ng those who joined the project after publication of the QAPP.

Table 1

. QAPP Distribution

Name Title Contact Information Mailing Address
U.S. Environmental Protection Agency Meghan Hessenauer Technical Project Officer (202) 566

- 1040 (phone) hessenauer.meghan@epa.gov

U.S. Environmental Protection Agency Office of Science and Technology 1200 Pennsylvania Avenue, NW (4303T) Washington, DC 20460 (202) 566

- 1053 (fax) Marion Kelly Quality Assurance Coordinator (202) 566

- 1045 (phone) kelly.marion@epa.gov

Marla Smith Work Assignment Manager (202) 566

- 1047 (phone) smith.marla@epa.gov Ahmar Siddiqui Alternate Work Assignment Manager (202) 566

- 1044 (phone) siddiqui.ah mar @epa.gov

James Covington, III Economics Task Manager (202) 566

- 1034 (phone) covington.james@epa.gov

Janet K. Goodwin Technology & Statistics Branch Chief (202) 566

- 1060 (phone) goodwin.janet@epa.gov

Eastern Research Group, Inc

Deborah Bartram ERG Program Manager (703) 633

- 1669 (phone) Deborah.Bartram@erg.com Eastern Research Group, Inc. 14555 Avion Parkway Suite 200 Chantilly, VA 20151

- 1102 (703) 263

- 7280 (fax) Mary Willett ERG Corporate Quality Assurance Manager and Project QA Coordinator (703) 633

- 1615 (phone) Mary.Willett@erg.com

Lori Weiss Work Assignment Manager (703) 633

- 1658 (phone Lori.Weiss @erg.com

Anne Jones Economics Task Manager (781) 674

- 7328 (phone) Anne.Jones@erg.com Eastern Research Group, Inc 110 Hartwell Avenue Lexington, MA 02421

- 3136 (781) 674

- 2851 (fax) Melanie Sands Environmental Policy Analyst (781)

- 674

- 7321 (phone) Melanie.Sands@erg.com

Christiana Marsden Economist (781) 674

- 7331 (phone) Christiana.Marsden @erg.comSection No. 1
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 1 of 2

1. I NTRODUCTION AND B ACKGROUND

The U.S. Environmental Protection Agency (EPA), acting through its Office of Science and Tech nology (OST), Engineering and Analysis Division (EAD), plans to revise the effluent limitations guidelines and standards (ELGs) for the Oil and Gas Point Source Category to control pollutant discharges from coalbed methane (CBM) produced water

. EPA’s decis ion to proceed with a rulemaking was announced in the final 2010 Effluent Guidelines Program Plan (76 FR 66286) . The decision was based, in part, upon data collected from an industry survey conducted in 2008. EPA is now planning to consider the economic d ata received in the questionnaires in its rulemaking for the CBM industry

.
EPA tasked Eastern Research Group, Inc. (ERG) with supporting the Agency in its Economic Analysis under Work Assignment 9

- 37 of Contract No. 68

- C

- 0 2

- 095

. Specifically, EPA tasked E RG with finalizing the development of the database for Part B (economic and financial data) of the detailed questionnaire that had been developed under earlier work assignments (Work Assignments 8

- 37, 8

- 05, 7

- 05, 6

- 05

, and 5

- 05) under the same contract. EP A has also tasked ERG with using the detailed questionnaire data, along with additional information, to analyze the potential impacts of produced water management practices on operators of CBM projects, CBM projects, production, and the wider economy.

The detailed questionnaire contains both technical (Pa rts A and C) and economic (Part B) sections

. Currently, support to EPA in finalizing and analyzing Parts A and C of the questionnaire under Task 9

- 37 is covered by the Quality Assurance Project Plan for the Review and Data Entry of the Engineering Portions of the Coalbed Methane Questionnaire ( 1 )

, the Quality Assurance Project Plan for the Coalbed Methane Existing Data Collection , dated August 25, 2011 ( 2 ) , and the Data Analysis Addendum to the Quality Assur ance Project Plan for the Coalbed Methane Existing Data Collection , dated November 30, 2011 ( 3 ). This document presents ERG’s quality assurance procedures for the review of Part B data, creation of a database to house the responses, economic analysis, and incorporation of secondary data into the analyses. The new activities for Work Assignment 9

- 37 are discussed as an addendum in Section 6.0. Quality assurance procedures related to survey design and statistical analyses are provided in the QAPP prepared by Battelle under a separate contract ( Quality Assurance Project Plan for WA 2

- 11: Statistical Support for Coalbed Methane Detailed Study; Contract No. EP

- C

- 05

- 030) ( 4 )

. Procedures for maintaining the CBM public record are provided in the Coalbed Methane Proj ect File Management Plan prepared under this contract (Contract No. 68

- C

- 0 2

- 095, Work Assignment 8

- 37 ; see Coalbed Methane Project File Management Plan, May, 2011 ). The Project File Management Plan is Appendix A of the approved technical QAPP (Contract No. 68

- C

- 02

- 095, Work Assignment 8

- 37; see Quality Assurance Project Plan for the Review and Data Entry of the Engineering Portions of the Coalbed Methane Questionnaire , dated July 13, 2011 (1).

This QAPP is responsive to applicable elements in EPA Requireme nts for Quality Assurance Project Plans QA/R

- 5 ( 5 )

. It is a project

- specific plan that describes the procedures that staff supporting EPA on the CBM Economic Analysis use to plan, implement, and assess project quality

.Section No. 1
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 2 of 2

ERG has organized this document acco rding to the elements outlined in the above

mentioned EPA guidance document

. Group A elements are addressed from the title page through Section 2.6 of the document, while applicable Group B elements are addressed in Section 3.0

. Group C elements are addres sed in Section 4.0, and applicable Group D elements are addressed in Section 5.0

. The addendum addressing the economic analysis and use of secondary data is provided in Section 6.0. References are provided in Section 7 .0.Section No. 2
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 1 of 9

2. P ROJECT M ANAGEMENT E LEMENTS

In th is section, ERG discusses its project management approach for the assignment. The discussion is organized according to the nine elements EPA identified for inclusion in this section:

A.1 Title and Approval Sheet A.2 Table of Contents A.3 Distribution Lis t A.4 Project/Task Organization A.5 Problem Definition/Background A.6 Project/Task Description A.7 Quality Objectives and Criteria A.8 Special Training/Certification A.9 Documents and Records

Elements A.1 to A.3 were provided earlier in this docume nt

. The remaining elements are discussed below.

2.1 Project/Task Organization (Element A.4)

Figure 2

- 1 presents the project organization for the CBM economics tasks

. The chart depicts the staff with program management and technical responsibilities and those w ith QA/QC roles

. It shows the relationship and lines of communication among all project participants, including those from EPA, as users of the data.

Deborah Bartram serves as ERG Program Manager overseeing all tasks performed under Contract No. 68

- C

- 0 2

- 09 5. Lori Weiss serves as the ERG Work Assignment Manager (WAM) for 9

- 37

, Coalbed Methane (CBM) Effluent Guidelines Regulatory Support , of which the CBM Economics Analysis Task is one subtask. Ms. Weiss will be the principal contact for the EPA WAM on projec t issues, deliverables, and schedules. Ms. Weiss will be responsible for:

**•** Ensuring that the quality of work, schedule, and budget me e t s the requirements of the EPA/EAD effluent guidelines program ;

**•** Providing technical direction to ERG staff and managing the daily activities on the project;

**•** Obtaining appropriate technical review of all ERG deliverables and ensuring deliverables conform to EPA’s technical review requirements; and

**•** Keeping the ERG Project QA Coordinator and the ERG Program Manager advised of ay quality problems that arise.

Ms. Mary Willett, will serve as Project QA Coordinator, responsible for ensuring that the requirements of this QAPP are implemented and documented. Ms. Willett also serves as ERG ’s overall Corporate QA Manager (QA M )

, and as su ch ensures that project

- specific QA/QCSection No. 2
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 2 of 9

programs are commensurate with project objectives and with ERG’s quality system. Ms. Willet has no other role on the CBM project and is independent from the CBM project team.

On this work assignment, Ms. Willett, fol lowing consultation with ERG’s Program Manager (PM), has the authority to suspend the performance of any activities determined to be deviating from the established quality management procedures, work plan, or project

- specific QAPPs until appropriate correc tive actions are instituted. S he can, at her discretion, conduct an audit of ERG’s QA activities during the period of performance of the assignment.

Figure 2

- 1. Project Organization (Element A.4) ERG Corporate QA Manager

ERG Project QA Coordinator

ERG Technical Reviewers To be assigned ERG Program Manag er

ERG Work Assignment

Manager

ERG Task Lead

ERG Staff EPA Project Officer

EPA Work Assignment

Manager

EPA QA Officer

Lines of Communication Reporting and AuthoritySection No. 2
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 3 of 9

Under Ms. Weiss , Anne Jones, Senior Economist and ERG Vice President, manages the CBM Economics Task, providing support to James Covington, III, EPA Economics Task Manager for the Economic Analysis Task

. Anne Jones is supported by other ERG staff as needed. The staff adhere to the QA guidelines outlined in this QAPP when providing support under the work assignment.

The ERG WAM is responsible for reviewing all deliverables before transmittals to EPA. This review ensures a consistent level of quality for the deliverables. In addition, it ensures consistency in data selection, definitions, and other aspects between the technical and economic support delivered under the work assignment. In addition, t he ERG project QA C oordinator

, ERG WAM, and ERG Economics Task Manager appoint Technical Reviewers as needed during the conduct of this assignment. The Technical Reviewers evaluate the quality of work products as they are being developed and once they are completed. The reviewers do not have any involvement in the products or processes for whi ch they were assigned QA/QC responsibilities. The Technical Reviewers could be internal or external to the ERG Team. They have economics or policy backgrounds, as well as experience in survey design, administration, and analysis. (For further discussion on technical review procedures, see Section 4.1 of this QAPP.)

2.2 Problem Definition and Background (Element A.5)

EPA developed and administered the CBM screener and detailed questionnaires during the Detailed Study to collect information to determine whether i t would be appropriate to revise the effluent guidelines for the Oil and Gas Point Source Category to control pollutant discharges from CBM produced water

. The data are now being used in economic analyses for rulemaking activities. The data collection for the detailed study was intended to be comprehensive so that an additional survey would not be needed if EPA were to go forward with regulating the industry

. Thus, the screener and detailed questionnaire databases were designed to provide the level of detai l and the data needed to support an ELG.

The purpose of the questionnaires was to gather industry information that would enable EPA to go forward with analyses necessary to support an ELG (in addition to providing information suitable for a detailed study report). The information gathered supports :

**•** Developing a profile of the CBM industry and its produced water management practices ;

**•** Determining the scope of the industry and any subcategories ;

**•** Determining whether technically and economically feasible con trol technologies or management strategies exist to reduce or eliminate the discharge of pollutants ; and

**•** Determining whether the current regulatory controls are adequate.

In addition to answering these questions, the questionnaires will allow EPA t o addre ss the key questions and project outcome described below

. EPA employed a two

- phase sample design

. In the first phase, EPA collected information about CBM projects from a census of CBM projects with more than two wells

. The responses to the screener questio nnaire used in thisSection No. 2
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 4 of 9

census served as the basis for EPA to select a sample of CBM projects to receive the detailed questionnaire

. (For further discussion on sample design, see Section 3.1 of this QAPP.)

The detailed questionnaire was designed to answer the following key questions:

**•** What pollutants are being discharged by CBM operations?

**•** What are the existing produced water management alternatives?

**•** How do produced water management alternatives vary by basin ?

**•** What are the operation and maintenance costs of th e control technologies?

**•** For what produced water characteristic is each management alternative feasible?

**•** What are the expected incremental pollutant removals attributable to each management alternative?

**•** What are the firm

- level baseline financial characteris tics?

**•** What are the historic and current capital and operating costs of CBM projects specific to each basin?

**•** What are the names, locations, and production histories of CBM extraction projects?

**•** What are the gas and produced water production rates over the l ife of each project?

**•** What is the economic impact (e.g., production losses, firm failures, impacts on small business) of each produced water management alternative?

**•** What is the status of current National Pollutant Discharge Elimination System (NPDES) and o ther operating permits?

**•** What are the non

- water

- quality impacts of each management alternative?

In terms of the survey implementation, data collection, and database development, t his QAPP specifically addresses implementation of Part B , Financial Informati on, of the detailed questionnaire

. Other aspects of the use of survey and other data and the economic analysis are addressed in the addendum in Section 6 .0.

2.3 Project/Task Description (Element A.6)

EPA received approval from the Office of Management and Budg et (OMB) for an industrial survey (screener and detailed questionnaires) under the Paperwork Reduction Act on February 29, 2009

. EPA published two Federal Register Notices announcing the data collection efforts (73 FR 4556, January 25, 2008; 73 FR 40575, J uly 15, 2008)

. ERG support of this survey consisted of:

**•** R evi sing and finaliz ing the detailed questionnaire;

**•** Responding to questions on Part B received on the CBM H elpline ;

**•** R eceiv ing copies of the detailed questionnaire and track ing these copies ;

**•** D evelo p ing a database dictionary and a database to house questionnaire data;

**•** R eview ing Part B of questionnaires to prepare them for data entry

, including transcrib ing financial data for some respondents from their submitted financial reports ; andSection No. 2
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 5 of 9

**•** Entering the d ata into the prepared database

.

ERG has completed the steps above . A brief description of these tasks is provided in the subsections that follow. The QA/QC activities associated with the tasks are discussed in Section 3 .0 of this QAPP.

Those tasks curre ntly underway or to be performed include the following:

**•** Conducting follow up on incomplete or problematic detailed questionnaires , if so directed by EPA;

**•** Summarizing questionnaire data;

**•** Identifying outliers and missing data and developing approaches for e stimating missing data;

**•** Continuing to clean questionnaire data as outliers are identified or due to follow

up ;

**•** Developing economic analysis methodologies and implementing the methodologies to calculate economic impacts ;

**•** Using survey data and existing data to create an industry economic profile ; and

**•** Developing draft and final documents supporting economic analyses.

A brief description of these tasks is provided in Section 6 .0 , with the exception of conducting follow up.

, which is described in Section 2.3.6

.

2.3.1 Finalizing the Detailed Questionnaire

ERG assisted EPA in finalizing the detailed questionnaire in October 2009 . There were no significant changes to the detailed questionnaire

–– only edits to improve the look and ease of

use

. Questions for the detailed questionnaire were designed to elicit information in the areas outlined in Section 2.2. This task has been complete d

.

2.3.2 Providing Survey Assistance

ERG assisted respondents on Part B questions on behalf of EPA during the detailed questionnaire response peri od , while maintaining communications throughout the active survey data collection period with EPA and other EPA support contractors. When ERG received a question from EPA , ERG responded either by email from the ERG CBM Helpline email address or contacted the respondent directly, depending on whether the questionnaire respondent ’s contact was made by email or by phone

. After contacting the respondent , ERG entered the question and response on a contact log sheet

. For those questions that required additional input from EPA, ERG noted that fact on the contact log sheet. This task has been complete d

.

2.3.3 Receipt and Internal Tracking of Questionnaires

ERG set up a survey tracking form and procedures to quickly identify issues with Part B portions of questionnaires a nd to ensure the expeditious routing of questionnaires through theSection No. 2
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 6 of 9

review, coding, and data entry phases (see Appendix A, which presents a portion of the ERG survey tracking form)

. This task has been completed.

2.3.4 Developing a Survey Database and Data Diction ary

Under Work Assignment 6

- 05, ERG developed a database for entering qu estionnaire responses for Part B of the questionnaire, as well as a data dictionary describing the data elements

. In Work Assignment 8

- 05, ERG made minor revisions to these items. In a ddition, ERG developed procedures to use for performing QA/QC of the data and preparing and printing data tabulations

. This task has been completed.

2.3.5 R eviewing, Transcribing, Coding, and Entering Respondent Data into Survey Database

For the detailed questio nnaire, ERG reviewed each survey for missing information that is needed prior to data entry

. ERG also identified those respondents requesting assistance with transcribing their financial information from tax forms and other financial reports into the quest ionnaire format

. If requested by EPA, ERG will telephone respondents whose surveys were missing information (see Section 2.3.6) . Following the review process, ERG coded responses on the hard

- copy questionnaires to ensure that the responses were in a form t hat can be easily entered into the detailed survey database

. Using forms developed in Access for this purpose, ERG then entered the data. These procedures have been developed so that complete, legible, permanent, and defensible records documenting all work performed are found within the hard copy questionnaires and in the database itself . An example of the detailed work required of ERG staff during the review, transcription, coding, callbacks and data entry process can be seen in Section 3.4.2 and Appendix A (Instructions for Survey Tracking, Review & Coding). This task has been completed.

CBM operators are able to identify responses containing confidential business information (CBI) on the questionnaires

. ERG is handl ing all CBI data in accordance with appl icable EPA procedures (see S ection 2.5).

2.3.6 Following Up on Incomplete Surveys

As ERG received the detailed questionnaires, they were reviewed for completeness (i.e., to ensure that responses to all questions were provided). A very few questionnaires were ide ntified as having insufficient data and would require callbacks prior to data entry

. Others, more nearly complete, were allowed to go through data entry before callbacks are made

. Procedures for contacting respondents are identified in S ection 3.4 of this QAPP

. ERG has develop ed documentation for the public record that summarize s how the questionnaires were reviewed, coded, and data entered . ERG will call respondents with incomplete surveys only under technical direction from EPA. If ERG calls respondents, we will document the call and

any revisions made to the database as a result of the call (see Appendix A)

. To date, ERG has not called any respondents for missing data

.Section No. 2
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 7 of 9

2.3.7 Tabulating Data

Once all the questionnaires were entered into the survey database, i n accordance with the procedures in Section 3.4, ERG conduct ed and continues to conduct accuracy checks and will follow up further on any problems identified in reviewing the hard

- copy questionnaires and data

, as requested by EPA

. ERG also is identifying w hether the database contains any outliers and will investigate them accordingly

. The criteria for making any changes to outliers in the database are discussed in Section 6 .0

.

2.4 Quality Objectives and Criteria (Element A.7)

The primary objective of this QAPP is to ensure that the data collected using the CBM detailed questionnaires will be of the quality necessary to support EPA in determining whether it would be appropriate to revise the Oil and Gas Point Source Category ELGs to control pollutant discharges f rom CBM produced water

. The tasks described earlier, in Section 2.3, involved the collection and use of data gathered using a detailed survey of selected CBM projects

. Questionnaire data collected directly from industry personnel are considered primary dat a. ERG defined the acceptance criteria (i.e., data specifications) it used to evaluate whether the data collected under the economic portions of the screener and detailed questionnaires would be suitable for EPA’s study needs

. The acceptance criteria were related to the completeness, accuracy, and reasonableness of questionnaire responses

.

2.4.1 Completeness

Completeness was defined as the number of valid responses received compared to the number of questionnaires distributed

. As described in P art B of the ICR S upporting Statement (Appendix B), EPA considered non

- responses to specific questions and non

- responses to the overall detailed questionnaires in developing the sampling design for the CBM industry

. Therefore, acceptable completeness was determined in terms of EPA’s statistical sampling plan

. ERG will work with EPA’s statisticians to determine the completeness required for specific questions or within the sample selection strata

. The specific methods ERG is us ing to improve completeness are described in Sect ion 3.4 of this QAPP.

2.4.2 Accuracy

Questionnaire accuracy pertains to :

**•** W hether the respondents provided correct answers to the questions asked;

**•** W hether the responses provided were interpreted correctly by the questionnaire sponsor; and

**•** W hether the response s entered into the questionnaire database were free from

error

.
ERG was not tasked with developing and implementing tools, such as conducting on

- site inspections and interviews, to measure elements 1 and 2

. The methods ERG is using to measureSection No. 2
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 8 of 9

element 3, and to increase the accuracy of responses, are described in Section 3.4 of this QAPP

. ERG’s goal for data entry accuracy was 100 percent error

- free

.

2.4.3 Reasonableness

To be suitable for use in EPA’s overall CBM Detailed Study, the data collected by the detai led questionnaires must reasonably reflect economic and financial realities and industry practices

. For example, a small project with 4 wells is highly unlikely to be producing billions of cubic feet of gas per year; likewise a CBM project would not have r eceived a wellhead price of gas in 2008 of $100/ million Btus (MMBtu) , which historically have ranged generally between $2 and $10/MMBtu in the U.S.). ERG’s goal is to ensure that all responses are reasonable. The methods ERG uses to evaluate and improve th e reasonableness of questionnaire responses are described in Section 6 .0 of this QAPP.

2.5 Special Training Requirements/Certification (Element A.8)

As noted previously, the owners and operators of CBM industries could indicate that certain responses on their questionnaires contained CBI

. ERG’s Document Control Officer (DCO) for CBI is working in conjunction with EPA’s DCO to regulate access to all CBI received by ERG under this work assignment

. Only authorized ERG employees are allowed access to the CBI

. Only the ERG DCO or an authorized employee can release CBI material.

Access to CBI by ERG employees is based on an employee’s need

- to

- know status

. Before access is authorized, an employee’s justification for access is documented and submitted to the ERG DCO

. Th is notice indicates the name of the employee, why access to CBI is necessary, and the Work Assignment number under which the employee use s CBI

. The ERG DCO informs the employee of his or her responsibilities for the security of CBI

. The employee is require d to familiarize himself or herself with the appropriate procedures for handling CBI and is required to pass a test of CBI handling procedures specific to ERG’s Lexington Office

.
ERG submitted a CBI security plan for the Lexington Office (where the CBM CB I financial materials are held under this work assignment) to EPA in 2006; this plan, which was approved by EPA at that time, remains active for all effluent guidelines

- related support.

2.6 Documentation and Records (Element A.9)

ERG developed and instituted document control mechanisms for reviewing, revising, and distributing QAPPs

. Each QAPP has a signed approval form, title page, table of contents, and
EPA

- approved document control format, which appears in the upper right corner of each page

. (See header at the top of each page of this QAPP for a specific example.) During the course of the project, ERG circulate s any revisions to the QAPP to technical and QA/QC project staff at ERG and EPA

.
To finalize and implement the detailed questionnaire and the Econo mic Analysis task

, ERG has collected information and data and will prepare summaries of those data (additional data collection that is currently ongoing is discussed in Section 6 .0 )

. These collected data andSection No. 2
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 9 of 9

summaries include questionnaire responses, helpl ine questions and responses, questionnaire tracking documentation, and summary memoranda related to the survey effort

. In addition,

survey responses have been entered into a database for subsequent analysis by EPA

. Memoranda and reports that will be genera ted under the Economic Analysis Task are discussed in more detail in Section 6 .0

. All technical data collected, generated, and analyzed on the project are handled and stored in a centralized file in ERG’s Lexington, Massachusetts, office, located in a room dedicated to CBI materials

. All documents collected or produced in the course of the project are stored in the centralized file or (if not CBI) in designated files held by the ERG Economics Task Manager or her designates

. The ERG Economics Task Manager de signated an economics project staff member to manage and maintain all CBI project files in the Lexington Office CBI room

. She also ensures that appropriate files are submitted to ERG Chantilly (where the ERG WAM

manages all rulemaking record compilations f or the CBM industry under Work Assignment 9

- 37 ) at periodic intervals throughout the assignment

.
ERG describes how it managed data obtained from the CBM detailed questionnaires in Section 3.4 of this QAPP under Element B.10, Data Management

. Computer file s containing the survey data are maintained by ERG employees on CDs or flash drives that are used (but not stored) on dedicated, non

- networked CBI computers as required by ERG’s Lexington Office CBI plan

. An ERG staff member designated as database manager prepares CD or flash drive backups of all survey data daily throughout the data entry process and whenever updates to the database

are made following the data entry process

.
As described in Section 2.5 above, materials designated as CBI are maintained und er restricted access and used only in accordance with ERG’s policies and procedures for CBI by staff who have r eceived CBI clearance, with oversight by the Lexington and Chantilly DCOs.

All quality system documents and records will be maintained for the li fe of the contract or until EPA request s that ERG destroy the records or return them to EPA

. Quality system records were filed according to guidance provided by the ERG Program Manager and Project QA Coordinator

.
Disposition of quality system documents an d records will be determined in accordance with contract requirements

.Section No. 3
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 1 of 7

3. D ATA G ENERATION AND A CQUISITION E LEMENTS

The detailed questionnaire involved collecting and reviewing data obtained from the owners and/or operators of CBM extraction projects

. Elemen ts B.1 to B.3 and Element B.10 below are relevant to this task

. ERG’s approach to each of these elements is described in the subsections that follow.

**•** Element B.1, Sampling Process Design

**•** Element B.2, Sampling Methods

**•** Element B.3, Sample Handling and Custod y

**•** Element B.4, Analytical Methods

**•** Element B.5, Quality Control

**•** Element B.6, Instrument/Equipment Testing, Inspection, and Maintenance

**•** Element B.7, Instrument/Equipment Calibration and Frequency

**•** Element B.8, Inspection/Acceptance of Supplies and Consumables

**•** Element B.9, Non

- direct Measurements

**•** Element B.10, Data Management

3.1 Sampling Process Design (Element B.1)

As discussed in Sections 1.0 and 2.3 of this QAPP, EPA conducted a detailed survey of a sampling of CBM projects identified with the screener questio nnaire

. Using a sample frame of 773 projects (screener surveys), EPA, with support from Battelle, selected 203 CBM projects to receive detailed questionnaires

. EPA selected the projects for sampling by basin, establishment size, and discharge method (i.e., direct or indirect discharge, and zero discharge)

. Additional details on the sampling process design have been provided in a memorandum prepared by Battelle under a separate contract ( EPA Contract No. EP

- 05

- 030, Work Assignment 4

- 11: Coalbed Methane (CBM) Study: Final Detailed Questionnaire Sample Design ) ( 6 )

. An additional 20 projects were selected on the basis of a judgment sampling methodology, as outlined in the memorandum titled Selected Projects for the San Juan Judgment Sample ( 7 ). This task has bee n completed.

3.2 Sampling Methods (Element B.2)

EPA distributed the detailed questionnaires under the authority of Section 308 of the Clean Water Act (Federal Water Pollution Control Act, 33 U.S.C. § 1318)

. All CBM project operators that received a detailed qu estionnaire were required by law to respond to it

. This task has been completed.

3.3 Completeness

As it relates to the sampling method, completeness was defined as the number of valid responses received compared to the number of questionnaires distributed

. B ecause responses are legally required, EPA assumed a relatively high response rate for the survey effort

. For the detailed questionnaire, EPA adjusted the estimated sample size assuming a non

- response rate of 10 percent (for more details, see EPA Contract No. EP

- 05

- 030, Work Assignment 4

- 11: CoalbedSection No. 3
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 2 of 7

Methane (CBM) Study: Final Detailed Questionnaire Sample Design ) ( 6 )

. This assumption of a 10 percent non

- response rate was based on EPA’s experience with a typical effluent guidelines questionnaire

. EPA will ad just the detailed questionnaire sample weights for any non

- response for the survey.

When developing the initial drafts of the detailed questionnaire EPA and ERG worked closely with applicable industry trade associations

. The CBM industry was

, therefore

, ge nerally well informed of the goals and objectives of EPA’s study effort and of EPA’s schedule for questionnaire distribution and receipt. This effort was intended, in part, to improve completeness.

Under a previous work assignment, ERG and EPA had develop ed contact names and mailing lists for CBM operators

. An accurate mailing list reduced the number of undeliverable questionnaires and improved completeness

.

3.3.1 Accuracy and Reasonableness

For the questionnaires to result in accurate information concerning th e CBM industry, respondents had to clearly comprehend the questions asked and provide responses that accurate ly reflect their operations. ERG and EPA then must interpret those responses correctly

. Under an earlier work assignment, ERG and EPA worked with i ndustry representatives to determine the best terminology to use when developing questions

. Industry representatives also made suggestions regarding the types of questions that would be clear to most respondents

. For example, EPA r espond ed to comments that small businesses might not be able to understand the form in which firm

- level data was being requested and insert ed a question allowing small businesses to provide tax forms or accounting reports in lieu of responding to many of the firm

level financial q uestions

. ERG transcribed such financial information into the questionnaire for respondents that requested this approach

. ERG and EPA also incorporated wording changes

, provided definitions for key terms, and incorporated other suggestions from industry so that draft questions would elicit the information needed to answer questions related to EPA’s overall CBM Economic Analysis Task

.
The entities listed below provide d comments on drafts of Part B of the detailed questionnaire , including the terminology use d, the definitions of key terms, and the availability of the data requested:

**•** Internal ERG Technical Reviewers ;

**•** EPA core workgroup members ;

**•** EPA extended workgroup members ;

**•** CBM industry and trade association personnel ;

**•** General public, including tribes, e nvironmental groups, and other stakeholders ; and

**•** The Office of Management and Budget (OMB).

These steps have been completed. The reasonableness of responses is currently being assessed

. This task is discussed in Section 6 .0

.Section No. 3
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 3 of 7

3.4 Data Handling and Custody (E lement B.3)

Under Contract No . 68

- C

- 0 2

- 095

, ERG Lexington prepared files and tracking spreadsheets using the survey identification number , and all communications regarding the

survey used the survey identification number to ensure the proper identification of all survey respondents and connect them to the records of those communications. ERG Lexington and ERG Chantilly use these identifiers in communications under Contract No. 68

- C

- 02

- 095, Task 09

- 37.

ERG established an electronic system to track each detai led questionnaire internally at the ERG Lexington office, from receipt of the copy through all stages of review, coding, data entry and callbacks

. The system included the unique survey identifier and the dates the questionnaires were received in Lexington, reviewed, coded, or assigned a callback requirement

. Appendix A shows the form used to track the surveys

.

To document reviewer mark

- ups, ERG Lexington used the photocopies of detailed questionnaires sent for this purpose (with originals held at ERG Chant illy ). These photocopies

are stored in ERG’s CBI room in files arranged by the unique survey identifier number and respondent name

. Any coding or revisions to data entered into these copies were made in non

black pen, color coordinated to an individual rev iewer, who documented the pen color by entering his/her signature on a reviewer cover sheet or by signing his/her initials in the review

copy

. In this way, ERG documented which reviewers made which changes, and changes can be distinguished from the origina l responses easily

. The review and coding tasks are complete. ERG

can provide EPA with pdfs on CD

- ROM or DVDs of the marked

- up photocopies for the public record (minus any data identified as CBI) , if requested

. ERG, under a separate task on this work assig nment, is maintaining the public record for the entire CBM survey effort

.

ERG managed the data entry process for Part B of the detailed questionnaire. For this

process, ERG reviewed, coded, and entered the Part B detailed questionnaire data into the previ ously constructed database

. This task is complete. ( ERG’s activities related to review and data entry of Parts A and C of the detailed questionnaire are being performed under other tasks under this work assignment and have a separate QAPP as noted in Secti on 1 .0

. ) ERG is adhering to EPA procedures for handling CBI when handling and transmitting questionnaires.

3.5 Data Management (Element B.10)

ERG has implemented and continues to implement the following QA activities to improve the completeness, accuracy, and reasonableness of responses to the detailed questionnaires received.

3.5.1 Questionnaire Non

- Response Follow Up

ERG Lexington will conduct follow

- up (e.g., telephone respondents ) , if requested by EPA, when respondents submitted surveys but fail ed to complete P art B of the questionnaires

. ERG Chantilly is responsible for follow

- up , if requested by EPA, when respondents did not

return any part of the survey . ERG has provided information to EPA detailing which respondentsSection No. 3
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 4 of 7

are refusing to provide firm

- level or proj ect

- level financial information for further follow

- up. EPA will decide which of several other means for collecting questionnaire data should be implemented , including, as necessary, additional follow

- up phone calls and letters, criminal fines, civil penalt ies, and other sanctions, as provided by law

. ERG also operated a helpline (through telephone or email systems) where CBM industry owners and operators could submit questions regarding the surveys and ERG coordinated with EPA to respond to these inquiries in a consistent manner

. The helpline task is complete. These activities helped to improve the completeness of responses on individual questionnaires (see Section 3. 3 ).

3.5.2 Questionnaire Response Review and Follow Up

ERG Lexington reviewed Part B of each receiv ed detailed questionnaire

. This initial review evaluate d completeness and reasonableness

. Certain questions if not completed, or if a large number of key questions were incomplete, were identified as trigger ing a potential need for

an initial callback prio r to data entry (for example, if no Part B was submitted) or for subsequent callbacks after data entry (if certain key information was missing) . As necessary

, and if requested by EPA , ERG will call respondents when questionnaires are found to be substantia lly incomplete

. After the initial review, ERG conduct ed a more detailed review

. This review identifie d :

**•** Missing or incomplete responses not necessitating callbacks prior to data entry during initial Part B review and tracking phase ;

**•** Inconsistent respons es (e.g., questions completed when a response directed that they be skipped, responses that contradicted other questionnaire responses) ; and

**•** Unreasonable responses (out of the permissible range).

ERG is also conducting a comprehensive evaluation following data entry

. Database queries are being used to identify inconsistent and/or unreasonable responses, as well as issues such as line items not summing to totals reported. These activities are ongoing (see Section 6 .0 for more details relating to this evalua tion )

. If any of these issues are identified, if so directed by EPA, ERG will call the respondent to obtain a complete, consistent response to each question

. ERG will document those communications and change the original questionnaire response based on the follow

- up communication. The communications will be logged, and the communication record will be stored in the survey file in ERG’s CBI

- secured room

. After review and follow

- up

are completed, ERG will report the number of valid responses provided for each questio n, so that EPA statisticians will be able to determine the accuracy of the estimates made using those

responses.

To further improve the completeness and accuracy of questionnaire responses in Part B, ERG and EPA provided assistance to questionnaire respondents by telephone and an email messaging system

. For the detailed survey effort, ERG also offered support to respondents in completing the detailed questionnaire

. For small businesses who request ed the help, ERG transcribed firm financial data from tax forms and accounting reports submitted by respondents into the questionnaires

.Section No. 3
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 5 of 7

Before the detailed questionnaires were mailed, ERG also developed FAQs that contained anticipated questions from questionnaire respondents and appropriate responses from ERG and EPA staff

. ERG routinely discussed with EPA the nature of questions being received and the answers being given to ensure that bias was not being introduced into the survey process

. ERG updated the FAQs as necessary and distributed the updated FAQs to appropriate project team members.

ERG has completed the documentation of the following with respect to the detailed questionnaires:

**•** R eceipt dates;

**•** S pecifics regarding follow

- ups to helpline and email inquiries; and

**•** F indings from initial and detailed r eview of completed questionnaires

.
ERG will document or continues to document the following with respect to the detailed questionnaires:

**•** N otes from any follow

- up inquiries with respondents and related edits made to questionnaire responses; and

**•** A ny chan ges made to questionnaires or the database

.

ERG used or will use a form (Figure 3

- 1 ) to document telephone conversations with respondents

. Those forms documenting any callbacks made to respondents that might be requested by EPA during the data analysis o r tabulation stages, will be stored in the applicable questionnaire file folder in the CBI room for the study

. Forms documenting Helpline calls are stored electronically on ERG’s network (no CBI helpline calls have been received at ERG )

. ERG will provide c opies of these records to EPA as requested. A more detailed discussion of the survey tracking, handling, review, coding, and callback procedures is provided in Appendix A

.

3.5.3 Questionnaire Response Coding and Data Entry

ERG developed a Microsoft Access data base in which responses to the detailed questionnaires were entered. Responses were entered using Access Forms developed specifically for the database structure to minimize data entry errors.

Before entering the data for Part B of the detailed questionnai re, ERG staff coded the completed detailed questionnaires to ensure that all responses were in the format required for data entry. (Questionnaire coding improves the accuracy of data entry.) A CBI

- cleared ERG employee with prior experience related to ques tionnaire data entry keyed information from the hard

- copy surveys into the database using the Access Forms. Each data entry staff member was required to briefly review their entries as they completed each form for error checking purposes. To further ensure that the database is accurate, a second CBI

- cleared ERG staff member independently reviewed all of the questionnaires for data entry accuracy. If data entry errors were identified, ERG staff logged the changes necessary in a form and made the change to th eSection No. 3
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 6 of 7

data. All changes were rechecked by another staff member. To further minimize the potential for keying or other errors, all data entry staff were selected based on their experience with using Microsoft Access. When errors in database entry are discovered during any reviews, ERG

correct ed the erroneous data fields. ERG staff not involved in the data entry corrections perform ed a 100 percent check to ensure that resolved errors were correctly re

- entered into the Microsoft Access database. Similar 100 percen t checks are also being or will be performed for any changes made to the database on the basis of reasonableness changes for follow up (see Section 6 .0 ) or callbacks to the survey respondents.

Project Name: CBM Economic Analysis Task
Project No:

Faci lity ID:

Facility Name:

Contact Name: Contact Phone No: Contact Email Address:

Date: Staff Name:

Time: AM PM

This document contains CBI: _____ Yes _____ No Type of Con tact:

Email Telephone

General Subject(s) Discussed:

CBI = Yes (Y) Question No. Topics Discussed

Figure 3

- 1

. CBM Detailed Questionnaire Contact Record

The completed database containing de tailed questionnaire Part B data is stored electronically on flash drives in ERG’s secure CBI files

.
After all responses are added to the database, ERG is executing a series of quer ies to determine whether any responses were missing, did not add to a repo rted total, were outliers, or were otherwise problematic for a given question for any questionnaire respondent

. ERG will follow up on any such identified discrepancies , if requested by EPA, by contacting respondents

.Section No. 3
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 7 of 7

Any changes made to the database to cor rect erroneous entries will be handled as discussed in Section 6 .0

.Section No. 4
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionna ire Page 1 of 3

4. A SSESSMENT AND O VERSIGHT E LEMENTS

This section describes the methods ERG used to assess the quality of work conducted for the questionnaire data collection tasks.

4.1 Assessments and Response Actions (Element C.1)

The ERG WAM

, ERG Economics Task Manager

, and ERG project QA C oordinator appoint Technical Reviewers as needed during the conduct of the work assignment

. The products that received a technical review included one or more of the follow ing:

**•** Revisions to the detailed questionnaire and

**•** Draft/final survey database and database dictionary

.
The products that will receiv e a technical review include:

**•** Formal memoranda, such as those outlining methodologies or methods for replacing missing or erroneous data in the Detailed Questionnaire ; and

**•** Any draft economic analysis report chapters or economic analysis report delivered to EPA

.
ERG will document all reviews of project deliverables in an electronic tracking spreadsheet. All deliverables wil l subsequently be reviewed by ERG’s Economics Task Lead and the ERG WAM

.

ERG performed a technical review of the documentation of follow

- up work on incomplete questionnaires. Technical reviews are critical evaluations of work performed

. ERG adhered to pro cedures for conducting and documenting the review, coding, and callback process as outlined in detail in Appendix A during the questionnaire review . Each Questionnaire Part B received two separate reviews by at least two different staff members. The first reviewer reviewed and code d the questionnaire and the second reviewer ensured that the reviewing and coding was complete, accurate, legible , and consistent with the reviews of other questionnaires

. ERG will continue to follow these procedures if EPA requir es addition al follow

- up with questionnaire respondents.

Following these two reviews, a sampling of questionnaires with identified data completion issues and all supporting documentation was reviewed by a Technical Reviewer (a staff member other than the re viewers performing the review and coding tasks) to gauge adherence to the procedures outlined in Appendix A and to ensure that all documentation was complete

. In general, technical reviews are conducted by professionals who are proficient in the work areas of interest, but who are not directly responsible for performance of the work

. Technical reviewers document the results of their reviews

. For this work assignment, the ERG Economics Task Manager is responsible for ensuring that technical reviews are condu cted by the appropriate personnel and are documented. These reviews have generally been completed,Section No. 4
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionna ire Page 2 of 3

although if any callbacks are made, additional review of callback documentation and data changes will also be performed.

ERG’s Project QA Coordinator assesse s the implementation of QA/QC procedures on this project as follows:

**•** Reviewed this QAPP for completeness and applicability and signed the signature page of the QAPP following successful review(s).

**•** Ensure project staff understand and follow the QC procedur es in this QAPP prior

to continuing work on the project

.

**•** Audit project files to verify that project staff are (1) using the QC procedures identified in this QAPP and (2) documenting the deliverable review process

. If not, the ERG QA Coordinator will direct the ERG WAM and/or ERG Economics Task Manager to implement corrective action and document that the action was taken

. Corrective action could include additional staff instruction or the development and use of additional measures to ensure the quality of da ta management and review.

4.2 Reports to Management (Element C.2)

The ERG WAM informs the EPA WAM of the progress of all work activities through the monthly progress reports required under Work Assignment 9

- 37 of Contract No. 68

- C

- 0 2

- 095.

ERG’s Technical Rev iewers and project QA Coordinator communicate any quality deficiencies identified as a result of reviews or audits to the ERG WAM in writing

. If quality objectives and criteria are not met, the ERG WAM is responsible for ensuring that the appropriate corre ctive actions are taken and reported to ERG’s QA team

. An example of an appropriate corrective action would be additional data entry review if data entry errors are identified

.
As required by Work Assignment 9

- 37

, ERG will electronically submit to the EPA WAM and PO a monthly progress report that documents the costs incurred and work performed during the previous accounting period and work planned for the current accounting period. Additionally, the ERG WAM will provide the EPA WAM with monthly reports on the status of quality assurance (QA) activities. These reports will be incorporated into our monthly technical progress reports which will be supplemented to include the following:

**•** Quality assurance activities performed during the reporting period as par t of the implementation of the work assignment;

**•** Identification of any problems encountered;

**•** Identification of any deviations from the QAPP; and

**•** Identification of problem resolution and/or corrective actions taken during the reporting period, if any.Section No. 4
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionna ire Page 3 of 3

ERG w ill describe QA activities conducted for major deliverables, such as summary memoranda, in the Technical Progress Reports (TPR). If data do not meet all the quality specifications but are needed for analysis, ERG will obtain approval to use the data from E PA and will reference the approval in the QA documentation in the TPR

.Section No. 5
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 1 of 1

5. R ECONCILIATION WITH U SER R EQUIREMENTS

Completed detailed questionnaires are the primary data sources that EPA will use to answer the key questions posed in Section 2.2.1 of this QAPP

. T he questionnaire responses will provide detailed descriptions of CBM extraction operations, industry finances, project economics, and produced water management practices

. EPA will use the questionnaire data, in combination with secondary data, to support i ts overall CBM Economic Analysis Task

.

ERG is thoroughly documenting for the public record all primary and existing source data used in any reports or memoranda developed

. Subsequent analyses of questionnaire

responses will clearly describe for the record any limitations of the quality of the questionnaire data or assumptions required for its use as described in Section 6 .0

. Project deliverables will also clearly identify the sources of secondary data collected on this project. The deliverables will clearl y describe the limitations of the data; for example, when information from a source may be dated, ERG will note in the presentation of conclusions that the source may have data quality limitations. However, unless otherwise noted, all data used will meet t he currency, accuracy, reliability, and scope requirements specified in Section 6. 2

.Section No. 6
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 1 of 16

6. A DDENDUM C OVERING THE D ATA A NALYSIS AND U SE OF E XISTING D ATA IN T HE C OALBED M ETHANE E CONOMIC A NALYSIS

6.1 Introduction

This section covers six new tasks that will be condu cted as part of the CBM rulemaking:

1. Task 8.a: SISNOSE assessment and recommendation 2. Task 8.b: Develop information for SBREFA evaluations 3. Task 8.c: Develop impact methodologies
4. Task 8.d: Prepare calculations to analyze preliminary engineering inputs
5. Task 8 .e: Evaluate profile data
6. Task 8.f: Recommend approaches for missing data

This addendum does not modify the quality assurance procedures discussed in the first five sections of this document but incorporates additional discussion of the new tasks. Please refer back to Section 1 .0 of this QAPP for a description of the CBM project.

In response to guidelines issued by the Office of Management and Budget ( 9 ) under Section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001 (Pub lic Law 106

- 554; H.R. 5658), EPA developed Information Quality Guidelines ( 8 ) for ensuring and maximizing the quality of information disseminated by EPA. EPA’s Information Quality Guidelines defines quality as:

**•** Objectivity

- information is accurate, reliab le, and unbiased and its presentation is clear, complete, and unbiased;

**•** Utility

- information is useful for the intended users; and

**•** Integrity

- information is not compromised through corruption or falsification either by accident or by unauthorized access or revision ( 10 )

In executing projects involving the collection and use of existing data, ERG will support EPA in meeting these guidelines by:

**•** Understanding how EPA intends to use the collected data and collecting data appropriate for that use;

**•** Working w ith EPA to develop specifications for the required accuracy, reliability, and objectivity of the data, evaluating collected data against those specifications, and including the results of these evaluations in intermediate and final project deliverables; an d

**•** Presenting the sources of data, assumptions made, and calculations used in all project deliverables. These presentations will be sufficiently detailed and transparent to ensure the reproducibility of the work by qualified third parties.

This section de scribes how ERG will ensure the quality of our collection and use of economic data by using the four

- step plan

- do

- check

- report cycle repeatedly until the project is completed, as summarized below. ERG will collect quality information as we conduct the proj ectSection No. 6
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 2 of 16

and report that information with the project deliverable(s). Table 6

- 1 presents the steps of this four

- step process.

Table 6

- 1. The Four

- Step Plan

- Do

- Check

- Report Cycle

PLAN **•** Identify key questions/objectives

**•** Identify the type and quality of informatio n required to answer the questions/meet the objectives
DO **•** Collect the required data
CHECK **•** Evaluate collected data against the quality specifications
REPORT **•** Report results of the data quality evaluation

PLAN **•** How to use the collected data

**•** Analysis and/ or summary of collected data

**•** Required quality of the analysis results
DO **•** Conduct the planned analysis
CHECK **•** Evaluate analysis results against the quality specifications
REPORT **•** Report results of the quality evaluation of the analysis

The remainder of this QAPP section is organized into the sections below. Please refer to Section 2 .0 for the overall project objectives and management structure.

**•** Section 6. 2 : Project Description and Objectives

**•** Section 6. 3 : Data Needs, Sources, and Quality Criteria

**•** Section 6. 4 : Data Acquisition, Review, Quality Verification, and Reporting

**•** Section 6. 5 : Use of Existing Data

**•** Section 6. 6 : Project Reports (Deliverables)

6.2 Project Description and Objectives

ERG is supporting EPA in assessing the impacts of the costs of candidate t reatment technology options for CBM produced water. Please refer back to Section 1 .0 and Section 2.2 of this QAPP for a general description of the CBM project. The purpose of this section is to describe how ERG will use the detailed questionnaire data , id entified and reviewed according to Sections 2 .0 through 5 .0 of this QAPP , and additional existing data which is described in Sections 6. 3 , 6. 4 , and 6. 5 of this section. ERG will use these data to develop estimates of impacts on CBM projects, firms and othe r affected entities using EPA

- supplied estimates of costs for CBM produced water treatment technology options. In addition, this addendum describes how ERG will identify and extract data subsets from the regulatory database for economic and statistical an alysis at the request of the EPA WAM.

ERG will determine which projects are considered not financially viable (for example, have negative earnings) in the baseline analysis using data in Part B of the detailed questionnaire submitted by respondents. Such d ata include 2008 volumes of gas produced by project, average project wellhead price received for gas in 2008, operating costs, and other expenditures. ERG will use engineering estimates of technology option costs that will further reduce financial healthSection No. 6
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 3 of 16

t o identify how many projects incrementally become non

- viable financially when technology option costs are considered. To the extent feasible, ERG will investigate sources of existing data that might allow more recent financial conditions to be assessed. A dditionally, the detailed questionnaire data will be analyzed to identify if any CBM firms are considered likely to fail in the baseline or incrementally after costs for technology options are used to adjust financial line items as reported. To the extent possible, these line items and project finances will be adjusted for current financial conditions using existing data sources. Revenue data from the detailed questionnaire will be analyzed for small affected firms to determine if technology option costs

co uld pose a significant impact on a significant number of small entities (SISNOSE). Section
6. 5 .1 discusses the quality criteria for identifying methodologies for making these determinations. EPA will evaluate the results of these impact analyses to deter mine which (if any) of the technology options represent Best Available Technology Economically Achievable (BAT).

6.3 Data Needs, Sources, and Quality Criteria

ERG is undertaking an analysis of the impacts of possible alternative produced water management metho ds on CBM operators and their projects using the following primary data, which will be the source of much of the financial data that will be used to characterize firms and projects:

**•** Screener Survey database analyses of numbers and business size of operator s, wells, projects, gas production, and method of produced water management. This database was prepared under the QAPP for the Coalbed Methane Detailed Study Questionnaire , dated January 27, 2010 and approved by EPA March 17, 2010, written by PG Environmen tal, Inc. [ 11 ])

.

**•** The Detailed Questionnaire, Part B, database analyses of firm finances, project finances, and other data that will be used to calculate impacts on firms and projects. This database was prepared under Revision 0 of this QAPP, approved on November 4, 2010, as represented in Sections 1 .0 through 5 .0 of this current document.

**•** Any additional data voluntarily provided by CBM operators that reflect changes from 2008, typical gas and water decline rates by basin, or other information, as request ed by EPA. These data will be handled similarly to existing sources data using the procedures outlined in 6.5 and 6.6 of this QAPP, including tracking, review, and data entry into applicable input databases, proofing, and additional reviews.

ERG will also be adding new existing data sources under the current Work Assignment 9

- 37, including:

**•** Data collected from Securities and Exchange Commission (SEC), state oil and gas commission websites, and CBM operator websites to augment and update profileSection No. 6
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 4 of 16

and other d ata

— SEC filings and CBM operator websites are expected to provide limited updated information on firm finances and size, while the state oil and gas commission websites will provide information on current levels of gas and water production, production dec line rates

, and any changes in ownership or operator status among the surveyed CBM operators.

**•** Additional CBM production cost data prepared by the Department of Energy (DOE) to augment cost data in basins where missing data might be an issue.

**•** DOE data on current state average wellhead prices for gas to provide information for calculating revised project revenues using more recent production data by operator obtained from state oil and gas commission websites.

**•** Compilation of SEC data from publicly owned do mestic oil and gas firms prepared by Oil and Gas Journal to assist in updating the profile beyond the 2008 date of the survey data and to compare publicly held CBM operators to the wider group of oil and gas firms in the industry.

**•** Production cost indices from DOE, which will add to the discussion of natural gas production cost trends. These indices are available from DOE as a graph, which will be downloaded intact with explanatory text for the graph summarized.

**•** DOE’s Annual Report, to provide projected we llhead prices in the U.S. and other background information on the future of natural gas generally and CBM specifically over the next 30 years.

**•** Other information gathered to provide data to assist in developing replacements for missing data, as key missing data points are identified.

**•** Any other information identified as useful during the course of the data gathering and analysis or as requested by EPA.

This section describes how ERG will acquire, review, and document the existing data. Table 6

- 2 lists the criteria that ERG will use to evaluate if existing data source and/or questionnaire data are acceptable for use in analyses supporting the CBM rulemaking. ERG will provide information on the quality of data sources that may be used in analyses, and the E PA WAM will make the final determination of whether the source is suitable for use. If ERG identifies limitations in the quality of the data from data sources that may affect decisions regarding the overall direction of further data collection efforts or t he quality of deliverables, ERG will communicate these issues to the EPA WAM. After a data source is determined to be acceptable for use, ERG may use the data to complete the rulemaking analyses described in Section 6. 5 . Table 6

- 3 discusses the criteria t hat ERG will use to evaluate data for use in replacing missing or erroneous data. Finally, Table 6

- 4 presents the criteria that ERG will use for accepting economic impact estimates.Section No. 6
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 5 of 16

Table 6

- 2. Industry Profile and Updated Production and Financial Data Acce ptance Criteria

Acceptance Criterion: Literature Review/Profile/Financial Data Information Description/Definition Specification Currency (up to date) Does the information reflect current conditions? **•** Sources were published or presented in 2001 or after.

**•** B ecause CBM is a relatively new industry, ERG may use information from sources prior to 2001 if it represents practices currently in place. ERG will document all data collected (see Section 6.3). The EPA WAM will provide further direction on how to incorpor ate or exclude the data in analyses. Accuracy/Reliability Is the information true? ERG assumes that the underlying data and information contained in the following sources are accurate:

**•** Government publications (state and federal reports);

**•** Government

- mainta ined websites and databases;

**•** Peer

- reviewed journal articles;

**•** Presentations at professional technical conferences; and

**•** Publications from recognized industry experts or organizations. The EPA WAM will determine recognized experts. ERG assumes that data from these sources are of sufficient quality to evaluate for use in CBM rulemaking activities. ERG will, however, note if these sources themselves make any disclaimers.

**•** The data are peer reviewed and well respected in the industry, such as the Oil and Gas Jour nal.

Company websites for employment information and parent firm

- subsidiary relationships are considered acceptable sources for privately held firms.

For other sources, ERG will determine, case

- by

- case if the source contains information that appears to b e
accurate.

Regardless of the source, ERG will provide EPA with the following information and EPA will provide further direction on how to incorporate or exclude the data in the analysis:

**•** Source citation information;

**•** Analysis methodology and assumptions (when applicable); and

**•** Data limitations or analysis uncertainty.
Scope Is the information reported from CBM operations? **•** Information pertains to CBM operations and not strictly to conventional oil and gas or shale gas.

**•** Source represents a geographic area i n which CBM operations are active or emerging.Section No. 6
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 6 of 16

Acceptance Criterion: Literature Review/Profile/Financial Data Information Description/Definition Specification

Reasonableness Is the information consistent with EPA’s understanding of CBM operations? **•** ERG will compare information collected from existing data sources to primary sources (e.g., questionnaire data) to con firm the information is reasonable based on documented CBM practices. This comparison will be presented in the QA sections of major deliverables.

Table 6

- 3 . Criteria for Acceptance of Replacement Data for Missing or Erroneous Data

Acceptance Criterion:
M issing or Erroneous Data Description/Definition Specification Currency (up to date) Does the information reflect current conditions? **•** The source for replacement data will primarily be the 2008 Detailed Questionnaire. Additional data will be considered, but 2008 or newer data will be selected preferentially.

Accuracy/Reliability Is the information true? **•** Erroneous data will be identified in the Economics Database by identifying outliers compared to other responses and using the knowledge and judgment of

t he CBM industry or by running calculation checks to identify summing errors in financial data.

**•** First choice for replacement data will be drawn from other responses to the Detailed Questionnaire or from corrected summations of the financial data.

**•** Other choi ces for replacement data might include data associated with standard industry practices (such as typical severance tax rates by state).

Scope Is the information reported from CBM operations? **•** Information pertains to the basin or state in which the operat or is producing.

**•** If firm

- related, information pertains to similar size firms operating in the basin or state in which the firm is producing or is the same information found publicly (e.g., SEC filings for that firm)

.

Reasonableness Is the information con sistent with EPA’s understanding of CBM operations? **•** Erroneous data will be identified based on understanding of CBM operations as well as comparisons to other responses in the Economics Database.

**•** Replacement data for both erroneous and missing data will r eflect what is known about typical operational conditions among CBM operations by basinSection No. 6
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 7 of 16

Table 6

- 4 Impact Estimates Quality Specification
Acceptance Criterion: Economic Impact Estimates Specification Meet requirements for EPA decision making **•** Include ba seline and incremental counts of firm failures and project closures, along with other impacts that EPA might request.

**•** Provide appropriate annualizations of compliance costs by technology option, aggregated by basin and nationally

Standard year(s) used fo r impact analyses matches year or years used for engineering cost analyses **•** All revenue and cost data from the detailed survey are to be updated using appropriate measures of inflation (Producer Price Indices or Consumer Price Indices, depending on the lin e item) to 2008 and to 2010.

Complete **•** All projects and firms currently discharging wastewater and are subject to new proposed requirements will be analyzed.

**•** Non

- respondent firms and projects will be analyzed either based on statistical weighting or match ing to similar firms/projects.

**•** New projects will be developed based on information provided for newer projects in the affected basins.

Accurate **•** Calculations reflect the criteria outlined in the approved economic impact methodologies and replacement data reflect the data as proposed in approved memos suggesting data replacements.

Reasonableness **•** Firm failures, significant firm impact, or project closures identified in analyses are consistent with review of the changes calculated in the key financial data ( e.g., compliance costs exceed available cash flow so firm failure is likely regardless of changes in other financial data).

ERG will prepare a methodology memorandum that discusses potential impact analysis approaches (Task 8.c) and a series of additio nal memoranda that will or might be needed to address Task 8.a (SISNOSE assessment and recommendations), Task 8.b (develop information for SBREFA evaluations), Task 8.e (evaluate profile data), and Task 8.f (recommend approaches for missing data). The EPA WAM will approve these memoranda and select the methodologies to be used for the firm and project economic analyses prior to ERG’s development of economic impact results. These memoranda are discussed in more detail in Section 6.6.

6.4 Data Acquisition and R eview

ERG staff will use the following procedures when collecting published existing data:Section No. 6
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 8 of 16

**•** ERG economists will collect full content and bibliographic information on all newly collected publications, including “date accessed” for information collected from the Internet. ERG researchers will save web pages as PDF files, so that the data source can be added to the electronic project file.

**•** ERG economists will provide bibliographic information for use in the master project file, housed in Chantilly, VA, in a fo rm that will allow the procedures described in the project file management plan to be followed (Appendix A of the Quality Assurance Project Plan for the Coalbed Methane Existing Data Collection [2] ).

**•** ERG economists will submit the publications and meta dat a to the project file index keeper as described in Appendix A of the Quality Assurance Project Plan for the Coalbed Methane Existing Data Collection (2)

. ERG staff will use the following procedures to document data collected via telephone and email contac t.

**•** ERG researchers will document all telephone contacts, questions asked, and responses using a Telephone Contact Report, such as the one depicted in Figure 3

- 1 in Section 3.5.3 of this QAPP.

**•** ERG researchers will save email contacts as PDF files, so that they can be added to the electronic project file.

**•** ERG researchers will ensure that telephone contact reports and email records are added to the CBM Project File.

As discussed in Section 3.5.3 of this QAPP, ERG will store economics questionnaire data colle cted for the CBM project in the economics portion of the CBM regulatory database in Lexington, MA (and in the overall CBM regulatory database in Chantilly, VA, as well). Additionally, ERG will store, separately from the economics questionnaire database, e xisting data collected for the CBM project, and additional input databases developed from a combination of the questionnaire data and existing data sources, as applicable.

ERG will evaluate existing data sources listed in the tracking database. ERG will a lso discuss the procedures for pulling a data subset for input databases, which will include any portion of data from the regulatory database that the EPA WAM requests, for use in project analyses.

6.4.1 Existing Data Tracking Database (Tracking Database) and De velopment of Detailed Data Tables

ERG will store citation and review information for existing data sources in a tracking database which will be used in conjunction with the project file index

. The tracking database summarizes information beyond the bibliog raphic information required for the record index. For each source, the tracking database will identify topics covered, citation information, and anySection No. 6
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 9 of 16

additional sources mentioned in the source . Most of the fields in the tracking database are simple checkbox or yes/no fields that indicate the type of information included in the source. The tracking database does not store actual data, but rather identifies whether or not the source discusses particular topics. ERG will use the fields in the tracking database to identify potential sources that may have useful data to contribute to the rulemaking analysis. The tracking database will indicate the location of any data copied into spreadsheets or other data formats. Existing data sources may not include operator an d project level data, but they will contain information on specific basins, such as production declines or current wellhead prices, that can be used to supplement the primary data collected in the detailed questionnaire. ERG will evaluate the existing sour ces using the criteria specified in Section 6. 3 and provide information on the quality of sources to the EPA WAM. The EPA WAM will determine whether the source is suitable for use in the rulemaking analysis.

ERG will identify data sources that may be used during the rulemaking analysis. These sources may have information relating to current financial conditions, operators who have ceased producing, operators who have sold projects to others, production declines in basins, basin

- wide production information, and other information useful for characterizing the industry. ERG will enter the data into detailed data tables in a format appropriate for use in analyses, such as Microsoft Excel or Access. To review the accuracy of the compilation of the detailed data tables, the ERG Economics Task Lead will assign a project team member to compare at least 20 percent of the entries in the detailed data tables to the original data sources. If the project team member finds errors, an ERG project team member will document the errors and conduct additional checks of the data (the extent will depend on the severity of the original errors and the outcomes of the additional checks).

6.4.2 Development of Data Subsets

ERG will extract data subsets from the regulatory database at the r equest of the EPA WAM. The EPA WAM will request specific data for economic analyses and ERG will create database queries to pull the specific information requested.

ERG will verify that the data housed in the regulatory database has been reviewed accordin g to the procedures outlined in Section 3 .0 and Section 6. 3 of this QAPP. ERG will provide a description and purpose of the data subset as well as documentation of the data subset. ERG will document the data subset by adding a description to all field name s within the data subset and by providing a database dictionary to accompany the data subset. ERG will also document the query development and QC procedures used to create the data subset. All of this documentation will be included in a separate table with in the regulatory database, which will include the following fields:

**•** Query name (for data subset pull);

**•** Name of ERG staff who created query;

**•** Date query was created;

**•** Name of ERG staff who completed QC check;

**•** Date QC check was completed; andSection No. 6
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 10 of 16

**•** Additional note s.

Because the primary data are CBI, data subsets are CBI materials and ERG will manage their creation and distribution as specified in ERG’s 2005 Security Plan for Handling CWA Confidential Business Information , Lexington, MA Office (12)

.

6.5 Use of Existing Data and Quality Verification

ERG will use collected data to estimate impacts on CBM operators resulting from incurring costs for changes to their manage ment , treat ment , and disposal systems using the candidate technology options selected by EPA. ERG will also estimate impacts specific to small firms in the industry. ERG will document the use of existing data for project analyses in memoranda that include:

**•** Purpose of analysis;

**•** Description of analysis methodology;

**•** Data sources used;

**•** Criteria used to select the data sources;

**•** Exceptions to the criteria and rationale for exceptions; and

**•** Documentation of assumptions and equations.

ERG will check the accuracy of the analyses using the procedures outlined in Section 6. 4 of this addendum, which discusses the veri fication and validation methods.

The following sections describe use of data and quality verification according to the four

stage plan

- do

- check

- report cycle.

6.5.1 Develop Methodologies and Evaluate Profile Information (Tasks 8.a, 8.c, 8.d, and 8.f)

ERG will im plement the four

- stage plan

- do

- check

- report cycle to develop the economic impact estimates. The first stage in the four

- part cycle to develop the estimates is to “plan.” ERG will develop methodologies for computing impacts on small businesses, all firms, projects (existing and new), employment, communities, and the production of natural gas.

ERG will work with the EPA Economics Task Leader in developing methodologies for using the data contained in the economic databases that reflect sound analytical appr oaches for assessing impacts. The methodology will identify each of the fields that will be used as inputs to economic analyses. For the SISNOSE evaluation (Task 8.a), the criterion will be to use the EPA

accepted screening approach for evaluating small bu siness impacts, which looks at compliance costs as a percentage of revenues. The two major criteria for the selection of additional recommended methodologies will be:Section No. 6
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 11 of 16

**•** What methodologies are commonly used by the oil and gas industry themselves or the finan cial investment industries for evaluating the financial viability of oil and gas projects and oil and gas firms ; and

**•** What methodologies have been previously used in other ELG analyses related to the oil and gas industry (e.g., those used for the synthetic drilling fluids ELG [13])?

Methodologies previously used to analyze the economic impact of ELGs established for other subsectors of the oil and gas industry have withstood the notice and comment rulemaking

process. However, the two criteria above can be met with the same methodology , although there may be some minor modifications needed to address the few differences between conventional oil and gas projects and CBM projects.

After the EPA Economics Task Leader approves the methodologies, ERG will develop a memorandum (as part of Task 8.c) that describes, in detail, the assumptions, data inputs, equations, algorithms, outputs, outliers and testing procedures that will be used to estimate economic impacts. ERG will compute these impacts following the proce dures approved by EPA.

Another part of the “plan” step is to evaluate the data to be used to create the economic profile (Task 8.b). ERG will work with the EPA Task leader in identifying the key pieces of information needed to create a comprehensive profil e of the industry, including a means of updating the profile to account for industry changes in the intervening years between survey data and a draft profile. (Note that many more analyses can be prepared for a CBI profile than can be performed for a non

- C BI profile. )

The criteria for selecting key pieces of profile data include the types of data that are generally included in all economic profiles used for ELGs to characterize the size, structure, and financial health of the industry and the types of data that are pertinent to characterizing the distinct features of the affected industry. These distinctive features were addressed in the Detailed Study Report and are expected to be expanded using information now available from the Detailed Questionnaire. Add itionally, certain data that are used to develop broader assumptions in the impact analyses might be highlighted in the profile. Also, b ecause the industry has changed significantly since 2008, which is the date of the data in the Detailed Questionnaire an d Screener, ERG will be investigating sources of information to update the profile

. These existing data will be collected using the criteria shown in Table 6

- 2

.
ERG will provide a detailed outline of a profile chapter to the EPA Economics Task Leader. Af ter approval of the contents of the profile chapter, ERG will develop the profile using the data compiled from the detailed questionnaire and existing data. Examples of the type s of data likely to be used in the profile include:

**•** Total and average amounts of CBM gas and water produced by basin, by discharging and non

- discharging projects (in 2008 and updated);Section No. 6
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 12 of 16

**•** Total numbers of non

- discharging and discharging firms in each basin (in 2008 and updated);

**•** Total numbers of non

- discharging and discharging project s in each basin (in 2008 and updated);

**•** Counts of small and large firms by non

- discharging and discharging status (in 2008 and updated);

**•** Average operating costs for CBM wells by basin and by discharging and non

discharging projects;

**•** Changes in the firms ope rating CBM projects since the survey was undertaken;

**•** Average revenues of discharging firms vs. non

- discharging firms, by basin and by size of firm (small vs. large);

**•** Counts of firms by corporate structure types.

**•** Average reported interest rates and hurdle rates.

**•** Average taxable earnings by discharging vs. non

- discharging firms and by small vs. large firms, by basin.

**•** Any changes to financial information since 2008 among publicly held firms (dischargers vs. non

- dischargers , small vs. large, to the extent publ icly available data are available).

**•** Average construction costs of new projects (those begun in 2004 or later).

**•** Other information as requested by the EPA Economics Task Lead or the EPA WAM.

All profile information will be evaluated on the basis of criteria outlined in Table 6

- 2

.
An additional part of the “plan” stage is the evaluation of missing and erroneous data and making recommendations for data replacement. ERG will prepare memos documenting these evaluations and recommendations as a part of the plan ning process.

6.5.2 Calculate Impacts

The second stage in the four

- part cycle to develop the economic impact estimates is to “do.” ERG is supporting EPA’s decision on the economic achievability of technology options. ERG will evaluate the ability of firms and projects to absorb the incremental costs of the technology options and determine, using decision criteria specified in the methodologies approved by the EPA Economics Task Leader whether the firms and projects can afford the incremental costs. In conducti ng this task, ERG will use data from the economics database created from the Detailed Questionnaire as discussed in previous sections of this QAPP, as well

as existing data sources such as state oil and gas commission websites, SEC filings, corporate websi tes, and other sources identified through literature and internet searches and evaluated as discussed in Section 6. 2

.Section No. 6
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 13 of 16

The calculation of economic impact will follow the equations and algorithms outlined in the methodology memo (Task 8.c) and discussed abov e in Section 6. 5 .1. In calculating economic impacts, ERG will:

**•** Following EPA’s approval of recommendations for replacing data (Task 8.f), create the replacement data and input the data into the data subsets created as discussed in Section
6. 4 .2 ;

**•** Develop the projection data needed to provide a rough estimate of project financial conditions out ten years, as will be described in the methodology memo;

**•** Incorporate the projection data and other input data into the spreadsheet calculation tools, subject to th e criteria in Section 6. 3 and the steps discussed in Section 6. 5 .3;

**•** Calculate economic impacts for options selected by EPA; and

**•** Create tables for EPA review, also subject to the steps discussed in Section 6. 5 .3.

At least three calculation tools will be created: a SBREFA tool, a firm impact tool, and a CBM project impact tool. The calculation tools (created in Excel or Access) will use the input data created in the subset databases to determine numbers of firms considered likely to fail incrementally, nu mbers of small businesses likely to incur costs that exceed 1 percent and 3 percent of revenues, and numbers of projects likely to shut in during the first year. The calculation tool for project impacts can also be used to determine reductions in overall project life and production, and identify impacts on federal and state tax collections and impacts on mineral rights owners. The calculation tool for firm failures and that for project closures can also be used to identify the potential impact on employme nt. The economic impact analysis document will record the following elements:

**•** Electronic tool description;

**•** Flow charts (if necessary);

**•** Data sources;

**•** Documentation of assumptions;

**•** Discussion of any boundary or limiting conditions;

**•** Documentation of any defau lt parameter values and their sources; and

**•** Documentation of all equations (and sources, if applicable) used to generate output.

6.5.3 Check Results of Impact Estimates

The third stage in the four

- part cycle to develop the economic impact estimates is to “check. ” ERG will perform a QC check of the Excel or Access tools created for estimating the firm and project economic impacts. The QC check will verify the appropriateness of the approach and verify that original data are correctly input. The QC check also will verify that theSection No. 6
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 14 of 16

tools implement the documented approaches (including those for data replacement). A CBM project team member not involved in the calculations will review any constants used for appropriateness, will check that units are converted properly, a nd will document and check equations used against the documentation and references. To the extent possible, the team member will independently verify 20 percent of the calculations using hand calculations or separate spreadsheets for three representative CBM projects. If any checks fail, then the spreadsheet will be revised and retested. The ERG economics task leader and an ERG senior reviewer will review the results of the calculations to evaluate whether the results make sense (e.g., are on the proper s cale, follow expected trends, or are comparable to data reported by industry) and provide EPA with information appropriate for use in rulemaking decisions.

ERG will document the electronic tool development in database tables or spreadsheets that include:

**•** A list of any data changes including a description, the purpose, and the process used to change the data;

**•** The name of the ERG staff member who performed the QC and the date completed; and

**•** A list of any errors found during QC and the subsequent resolution(s)

.
When changes are made to the electronic tool, ERG will:

**•** Assign a unique name and number to the revised version of the tool (in either Excel or Access);

**•** Maintain a historical file of the changes; and

**•** In a QA spreadsheet, log the changes made to the dif ferent versions and explain what was revised, the date the revision was made, and by whom.

ERG will determine whether the economic impact estimates meet the specifications presented in Table 6

- 4 by comparing the estimates to primary data from the questio nnaires and existing data. ERG will verify and validate the estimated costs and loads by comparing them to actual costs and loads for similar projects. ERG will present any uncertainties about the estimates to the EPA WAM, who will make the final determina tion about whether they are appropriate for use in the rulemaking analysis.

Table 6

- 5 provides a summary of the procedures ERG will use to determine that data used and results generated on this project meet quality specifications.

Table 6

- 5

. Summary of Pro cedures and Results Quality Verification

Action Task QA Actions

PLAN Develop methodology Before delivery to EPA, an ERG senior economist will provide conceptual review of methodology and assumptions. The EPA WAM will approve the final methodology. Devel op calculation approach Evaluate replacement data for missing or erroneous dataSection No. 6
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 15 of 16

Action Task QA Actions

DO Insert approved replacement data A CBM team member who did not do the calculation will check equations against the documentation and references. To the extent possible, the team member will independently verify 20 percent of the calculations using hand calculations or separate spreadsheets for three representative CBM projects for the project impact tool. Additionally all calculations will be checked for 20 percent of t he discharging operators for the SBREFA and the firm impact tools. If the checks fail (i.e., any errors are found), then the spreadsheet will be revised and retested. The ERG Economics Task Leader and an ERG senior reviewer will review estimates to evaluat e whether they are appropriate for the CBM industry. Calculate baseline project and firm financial conditions (closures and failures) Annualize Technology Option

costs Aggregate project costs by option to the operator level Insert appropriate a nnualized, O&M and/or capital costs, depending on the tool and methodology

Calculate numbers of incremental significant impacts on small businesses, project closures, firm failures, and other impacts as specified by EPA.

CHECK Methodology and assumpt ions The ERG Economics task leader and a CBM team member who did not do the initial calculations will review the creation and use of electronic tools. An ERG senior reviewer will review calculation results to evaluate whether they are appropriate for the C BM industry. Spreadsheet and/or database uses the methodology and assumptions Option cost data are properly loaded Data replacements are properly specified and loaded
Results make sense (on both project and overall level)

Results are similar to existing projects and other sources

REPORT Develop project reports (see Section 6. 6 ) The ERG Economics Task Leader and an ERG senior reviewer will review all project reports and evaluate whether they are appropriate for the CBM industry. Additionally, the ERG WAM will also review all project reports. ERG will include the results of the economic impact analyses, the QA procedures, and the results of the quality review in the project reports.

6.6 Project Reports (Deliverables)

The fourth stage in the four

- part cycle that ERG will use to develop the costs and loads calculations is to “report.” Table 6

- 6 lists the key products ERG will produce and how data quality will be reported in each.

ERG will include an evaluation of data quality in all project delive rables. ERG will also identify the sources of data, assumptions made, and calculations used in their development in allSection No. 6
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 16 of 16

project deliverables. These identifications will be sufficiently detailed and transparent to ensure the reproducibility of the work by q ualified third parties. All project deliverables will meet EPA’s standards of transparency, objectivity, integrity, and utility as specified in EPA’s Information Quality Guidelines ( 8 ).

Table 6

- 6 . Key Products
Key products Content Regulatory database **•** Co mplete the portion of the regulatory database that comprises the economics questionnaire database and the tracking database.
Data subsets **•** Relevant subsets of the regulatory database for use by other parties in analyses outside of this work assignment.
F irm, project, and other methodologies **•** Explains estimating baseline and compliance financial conditions at firms and projects and identifying firm failures, project closures

, and other impact

measures. Firm and project impact analysis detailed assumptions **•** Assumptions, equations, inputs, algorithms, outputs, testing procedures, etc. used to estimate impacts on firms, projects

, and other entities. Impact estimates **•** Estimates of impacts of technology options on firms, projects and other entities.Section No. 7
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 1 of 2

7. R EFEREN CES

1. ERG (July 13, 2011). Quality Assurance Project Plan for the Review and Data Entry of the Engineering Portions of the Coalbed Methane Questionnaire. Contract No. 68

- C

- 02

- 095.

2. ERG (August 25, 2011). Quality Assurance Project Plan for the Coalbed Methane Existing Data Collection. Contract No. 68

- C

- 02

- 095.

3. ERG (November 30, 2011). Data Analysis Addendum to the Quality Assurance Project Plan for the Coalbed Methane Existing Data Collection. Contract No. 68

- C

- 02

- 095.

4. Battelle (November 2, 2007)

. Quality As surance Project Plan for WA 2

- 11: Statistical Support for Coalbed Methane Detailed Study; Contract No. EP

- C

- 05

- 030

.

5. U.S. Environmental Protection Agency (March 2001). EPA Requirements for Quality Assurance Project Plans, EPA QA/R

- 5

. EPA, Office of Enviro nmental Information, Washington, DC, EPA/240/B

- 01/003.

6. Battelle (October 19, 2009)

. EPA Contract No. EP

- 05

- 030, Work Assignment 4

- 11: Coalbed Methane (CBM) Study: Final Detailed Questionnaire Sample Design.

7. ERG ( 2009 )

. Selected Projects for the San Juan Judgment Sample. Memorandum from Anne Jones, ERG, to James Covington, III, EPA. October 21.

8. U. S. EPA ( 2002 )

. Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity, of Information Disseminated by the Environmental Protec tion Agency. EPA/260R

- 02

- 008. October.

9. OMB ( 2002 ) . U.S. Office of Management and Budget. Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies (67 FR 8452).

10. EPA ( 2003 ) U.S. Environmental Protection Agency. A Summary of General Assessment Factors for Evaluating the Quality of Scientific and Technical Information , EPA 100/B

03/001. June, 2003.

11. PG Environmental, Inc. (January 27, 2010). QAPP for the Coalbed Methane Detailed
Stu dy Questionnaire. dated 2010 and approved by EPA March 17, 2010, written by PG Environmental, Inc.

12. ERG (2005). Security Plan for Handling CWA Confidential Business Information, Lexington, MA Office. October.Section No. 7
Rev ision No

. 1
Date: March 15 , 2012 QAPP

— CBM Detailed Questionnaire Page 2 of 2

13. EPA (2000). Economic Analysis of Final Effluent Limitations Guidelines and Standards for Synthetic

- Based Drilling Fluids and other Non

- Aqueous Drilling Fluids in the Oil and Gas Extraction Point Source CategoryQAPP

— CBM De tailed Questionnaire

Appendix A

INSTRUCTIONS FOR SURVEY REVIEW & CODINGQAPP

— CBM De tailed Questionnaire

A

- 1 

### INSTRUCTIONS FOR SURVEY TRAC KING, REVIEW & CODING

Prepared for:

James Covington U.S. Environmental Protection Agency Engineering and Analysis Division Office of Water 1200 Pennsylvania Avenue, NW Washington, D.C. 20460

Prepar ed by:

Eastern Research Group, Inc. 110 Hartwell Avenue Lexington, MA 02421

Updated March 10, 2010QAPP

— CBM De tailed Questionnaire

A

- 2 

### INSTRUCTIONS FOR SURVEY TRACKING, REVIEW & CODING

Introduction

ERG

- Lexington is responsible for tracking, reviewing, coding, and data

- entering Part B of the Coalbed Methane Survey (Financial Information)

. Appendix A provides the Operator ID, name of responsible person and contact information for each respondent from which we expect a

survey

.
We will be receiving copies of the complete survey by overn ight courier (we will only be responsible for review and coding of Part B, however)

. All surveys, until otherwise identified as not containing CBI information, will be handled as CBI materials and must be logged into the CBI filing system either by Anne Jo nes, Sara Giberson, or their designate. Once the package is opened, the appropriate tracking sheet column (see Figure 1 at the end of these instructions, which shows a portion of the tracking spreadsheet) must be completed for each Survey ID that is

not co nsidered out of scope (in addition to the CBI logging paperwork). The full tracking sheet is stored electronically in I://econ/coalbed 2007.aj/survey/detailed questionnaire/survey tracking & review/tracking spreadsheet.xls. Once the CBI log

- in and the init ial tracking sheet log

- in is completed, the survey questionnaire will be placed in a prepared wallet packet, labeled with the Survey ID and Respondent Name. A review packet specific to the survey respondent with a review cover page and completeness checkli st for each project we are expecting to be covered in the survey will also be inserted into the wallet packet (see Appendix B)

. The survey will then be ready for an Initial Review.

The tracking data sheet will be kept on the desktop of the middle computer in the CBI room. On each Friday morning, the latest version of the tracking data sheet will be reviewed, copied and uploaded to the network by Sara Giberson, database manager

. The database manager will then summarize the status of the surveys in an email t o the ERG WAM (Anne Jones). For example, a status report might indicate that 35 percent of surveys have been received; of these, all have been logged in, 75 percent have been reviewed and coded, 60 percent have been reviewed, coded, and placed in the data entry file, and 10 percent have data entry completed.



### Review
There are two parts to the review process

. The first part ensures that Section B is present, that financial data has been provided (either in the questionnaire itself or as an attachment) and tha t no projects are missing. It also distinguishes surveys with additional work needed to transcribe data provided by the respondents (i.e., financial reports and/or attachments with information relevant to financial projections

. The reviewer fills out the I nitial Review Sheet (see Appendix C, Form 1) at this stage

. The reviewer will also check off any issues, as noted, on Form 3 (Appendix C). This form is simply a summary of issues and notes across all surveys with initial issues (or special data attachments ). Only those surveys with the issues or data indicated in Form 3 need to be inserted on Form 3. It will serve as a means of summarizing how the Initial Review is proceeding and will ensure that data that will not be inserted into the questionnaire will be identified as needing to proceed to the contractor in charge of gas and water projections.QAPP

— CBM De tailed Questionnaire

A

- 3 The second stage of the review process investigates whether the survey is reasonably complete and readable

. A survey may have a few missing pieces and still be cons idered reasonably complete and ready for data entry; however, after data entry, callbacks might be

necessary

. Some surveys might have gross completeness problems beyond those identified in the Initial Review or readability issues (see Form 2, Appendix C)

. These surveys will require callbacks prior to any attempts at data entry. A checklist on Form 2 will assist in an assessment of whether the survey is complete enough to go through data entry prior to callbacks

. If any of the listed questions on Form 2 are missing answers, it will probably be best to call the recipient back prior to data entry; if there are any doubts, the data manager will make a decision

. Surveys with key pieces of missing information might be held until review and coding of relatively

com plete surveys are finished. All others will go to coding. Following coding, a second review will be performed to ensure the completeness review and coding are correct and readable. At that point, surveys will be ready for data entry

. The following sections present the details of these steps. Section 2.1 discusses the initial review and financial data transcription, while Section 2.2 covers the completeness reviews.

All reviewers and their roles will be assigned in advance on a sheet posted in the CBI

room

. These assignments will be reviewed periodically by the ERG Task Manager, and, if necessary, reassignments will be made based on availability and the inclusion of additional staff. Pen colors for each staff member will also be posted on the assignment sheet

. 

### Initial Review and Financial Data Transcription
The following presents the steps to be taken following the wallet packet setup steps outlined above

. The labeled wallet should contain all survey materials, including possible attachments and cover letter, and the reviewer packet (Appendix B)

.

1. Select a survey wallet and checks to see if the respondent notes that they are out of scope (they did not produce any CBM in 2008)

. To do this, check Table A

- 2 in Part A.

a. Proceed to Step 2 if survey is in scope (Tabl e A

- 2 has wells listed for at least one project). b. If Part A is missing or Table A

- 2 has no entries, check the cover letter or the comments to Part A to see if respondent has noted no CBM was produced. 1

c. If not in scope, the Initial Review Sheet (see Form 1) is checked “Yes” for “Out of Scope.” d. Note on Form 3 that the survey is out of scope and place it in the out of scope file located in Drawer 2 of File Cabinet E.

2. Check to see that Part B is present.

a. If present, proceed to Step 3. b. If not, check “Yes” for “Missing All of Part B” on Form 1.

c. Note on Form 3 that the survey is missing Part B.

1 We may have several of those, because several respondents were not sent the Screener Survey and could not opt
out at that time.QAPP

— CBM De tailed Questionnaire

A

- 4 d. File the survey in the “Initial Problems” file, in the fourth drawer of File Cabinet D. Surveys in this file drawer require further completion of the initial review once
P art B is received.

3. Check Question B2

- 10

. a. If “Yes” is checked, mark “No” for missing financials (the respondent was not required to submit any firm financial information) and proceed to Step 4. b. If “No” is checked, check Questions B2

- 17 through B2

- 23. 1. If a t least one year’s information appears in the answers to most of these questions, mark “No” for missing financials and proceed to Step 4. 2. If no information is contained in these questions but financial reports are attached, mark “No” for “Financial Data fo r B2 Missing” and mark “Yes” for “B2 Financial Data Needs Transcribing” and proceed to Step 4. 2

3. If none of these for any years are completed AND no financial reports are attached, check “Yes” for “Financial Data for B2 Missing” on Form 1. 4. Make a note on Form 3 for that survey ID that financial reports are missing. Proceed to Step 4. 5. 4. Check Table A

- 2 against the projects listed in the review packet. There should be one Table A

- 2 for each project as well as one page labeled Part B3 in the completeness revi ew packet in Appendix B for each project we were expecting the respondents to include in the survey).

a. If project names and count of projects match between Table A

- 2 and the review packet, proceed to Step 5. b. If project names differ between the review sheet and Table A

- 2, or there are more or fewer projects than expected, check the comments to Part A, the cover letter, or comments to Part B for any explanation

. 1. If the explanation is sufficient, mark “No” for Projects missing: 1. Adjust the review forms to matc h (discard omitted project review sheets, add new project review sheets, and/or change project names on the review sheets). 2. Contact the ERG Task Manager (Anne Jones) if there is any concern about the sufficiency of an explanation. 3. Contact Anne Jones to o btain a project number for any new projects. 4. Proceed to Step 5. 2. If there is insufficient explanation, mark “Yes” for “Projects Missing” on Form 1 and note this in Form 3 for that survey ID. Proceed to Step 5.

5. Check the last box of Question B3

- 75 or boxes under Question B3

- 80 and B3

- 81 noting gas or water curves or projections have been attached are X’d

2 We will transcribe data even if they were supposed to have done this themselves (answered No to Question B2

- 15, as long as they sent financial reports).QAPP

— CBM De tailed Questionnaire

A

- 5 a. If attachments referenced are attached (or if any attachments relevant to projections whether or not the boxes indicated are checked), note on For 3 that projection data is attached. Proceed to Step 6. b. If projection information should have been attached (above referenced boxes were checked, but no attachments were found), check “Gas & Water Projections are Missing” on Form 1 and make a note of this on Form 3. Proceed to Step 6.

6. Review Form 1 for all items marked “Yes”.

a. Surveys with boxes marked “Yes” for Issues 3 (missing financial reports) and/or 5 (missing projects) are placed in the “Initial Review Problem” file area. Proceed to Step 7 when directed by t he ERG WAM. b. If Issue 4 is marked “Y” and Issues 3 (missing financial reports) and 5 (missing projects) are both marked “No” (Issue 6 can be “Yes”), the survey is placed in the Financial Data Transcription file area (Drawer 1 of Cabinet C) with a green dot affixed to the wallet. Skip to Step 8.

c. If all issues but Issue 6 are marked “No,” the survey is placed in the Active File (Drawers 3 and 4 in Cabinet E), with a blue dot in place on the wallet noting that initial review is complete. That is, missing projec tion data, while a significant problem, does not interfere with continuing the review, coding and data entry phases. You are done with the Initial Review.

7. Callbacks will be required to resolve Issues 2, 3, 5, and 6.

a. Contact the respondent, asking for the information that is lacking. b. Inform the respondent that additional callbacks might be required at a later stage of review.

c. Complete a telephone log of your conversation. d. When information arrives, data manager will place information in the appropriate wall et and revise Form 1.

e. When all issues on Form 1 have been resolved (with the exception of Issue 6

— missing projections), ensure all Form 1 information is updated and information and proceed with the remainder of the review process: 1. If Part B was missing and has been received, continue with Step 3. 2. If financial reports or projects were missing and have been received, continue with Step 6. 3. If missing projections are received, file them with the survey, update Form 1, and note on Form 3 that projections are av ailable.

8. Transcription:

a. Two economists will work on the transcription of financial data. The first will transcribe the data, the second will review and proofread the transcriptions. b. Financial Data Reviewer 1: 1. Pulls out a survey from the Transcription Fil e 2. Writes in the Survey ID and initials and dates Form 4 (see Appendix C), which is posted in the CBI room, in the “Transcription Begun” Column.QAPP

— CBM De tailed Questionnaire

A

- 6 3. Completes the appropriate section of the Review Cover Sheet for the Survey (see Appendix B) using any pen color but black. 4. Inputs information from the financial reports directly into B2 using the same pen color used to sign the Review Cover Sheet. 5. Completes the transcription.
6. Returns the survey and review packet to the wallet
7. Initials and dates Form 4 in the “Trans cription Completed Column.”

c. Financial Reviewer 2: 1. Checks Form 4 for completed transcriptions 2. Pulls a transcribed survey from the Transcription File. 3. Initials and dates Form 4 in the column “Review of Transcription Begun” for the survey ID pulled. 4. Completes the appropriate section in the Review Cover Sheet in a different pen color than used for the initial transcription. 5. Proofs all transcriptions using the same pen color to mark any changes.
6. Confers with transcriber, data manager, or ERG Task Manager if

nece ssary to resolve discrepancies and to finalize questions for any possible callbacks.
7. Places any callbacks needed, informing respondents that another callback may be needed prior to data entry when further survey reviews are completed. 8. Fills out telephone l ogs to document changes made as a result of conferring with respondent. 9. Inputs all final changes into Section B2. 10. Places the survey and review packet, and any phone logs back into the wallet 11. Affixes a blue dot on the wallet and places the survey in the Ac tive File (Drawers 3 & 4 in Cabinet E). 12. Initials and dates Form 4 (“Transcription Complete”). You are done with Initial Review.

9. Data summary reports (every Friday morning):

a. The database manager reports which survey IDs have been placed in the problem file and why and which are missing projection data, using Form 3. b. The data manager reviews Form 4 for “Transcription Complete” and transfers this information to the survey tracking form.

c. The data manager transfers the copy of the survey tracking spreadsheet f rom the CBI room to the network for review by the ERG WAM.



### Detailed Review
Two persons will undertake each detailed review of surveys that have a blue dot affixed to the wallet. The steps of the detailed review are as follows:

Detailed Reviewer 1 1. Pulls ou t a survey with a blue dot from the Active File.QAPP

— CBM De tailed Questionnaire

A

- 7 2. Initials and dates Form 5 (see Appendix C) in the appropriate column, which will be posted in the CBI room. 3. Opens the reviewer packet (see Appendix B) and fills in the Reviewer Cover Sheet using a pen color different from those used by Financial Reviewer 1 and 2. 4. Turns to the Completeness Review Checklist for B1 and B2 (each survey ID has one such sheet) and the Completeness Review Checklist for B3 (there is one sheet for each project). 5. Using the same pen co lor with which the Cover Sheet was signed, goes through all the questions in Part B of the questionnaire, indicating what, if any missing items are found by checking off and making notations in the Completeness Review Checklist.
6. If, at any point in the Rev iew 1 process, the reviewer needs to place the survey back in the active file, circles the “Review 1 Begun” box for that survey ID on Form 5 and inserts a note into the wallet explaining where in the process the review stopped.
7. Transcribes all issues to b e discussed in a callback on Form 6 (See Appendix C), the Reviewer Flag/Comment Sheet. 8. Reviews Form 2 (Initial Callback Trigger Form) and compares it to missing items in the checklist to see if the survey will need an initial callback. Checks “Yes” if any of the key information is missing in the Form 2 checklist. 9. Alerts the Data Manager to any issues checked on Form 2. 10. Discusses the extent of the key missing data (one missing item might be acceptable; several should be considered a sign that the respondent requires an initial callback). Data manager notes a decision to make an initial callback in the tracking spreadsheet. 11. If an initial callback is required:

a. Returns the survey, review packet, all relevant Forms (including Form 6) to the wallet b. Places a red st ar on the wallet.

c. Places the wallet in the “Detailed Review Problems” file area (Fourth Drawer, Cabinet C). Callbacks are discussed in more detail in Section 4. 12. If an initial callback is not required:

a. Initials and dates “Review 1 Complete” on Form 5. b. Affi xes a silver star to the wallet and returns the survey and all associated forms to the wallet and places the survey in the Active File AND/OR

c. Initials and dates “Coding Begun” on Form 5. d. Begins the coding process (see Section 3, below). 3

e. At the completion of coding, notifies the data manager that the first review and coding is complete so that the information can be entered into the survey tracking spreadsheet. If coding must be interrupted, circles the “Coding Begun” column for that ID on Form 5 and returns the wallet to the active file.

f. Initials and dates “Coding Completed” on Form 5.

g. Affixes a red dot on the wallet to indicate that the first review and coding are complete. h. Returns the wallet to the Active File (Drawers 3 & 4, Cabinet E).

3 Note: if necessary, a person other than Reviewer 1 can code, but no checklist or coding should be re viewed by the same person who completed the checklist or coding. The coder signs into the review cover sheet as well.QAPP

— CBM De tailed Questionnaire

A

- 8 Detailed Review er 2 1. Pulls out a survey with a red dot from the Active File. 2. Initials and dates Form 5 in the appropriate column, which will be posted in the CBI

room. 3. Opens the reviewer packet (see Appendix B) and fills in the Reviewer Cover Sheet using a pen color diffe rent from those used by Financial Reviewer 1 and 2 and Detailed Reviewer 1 (and/or Coder). 4. Turns to the Completeness Review Checklist for B1 and B2 (each survey ID has one such sheet) and the Completeness Review Checklist for B3 (there is one sheet for

ea ch project). 5. Using the same pen color with which the Cover Sheet was signed, goes through all the questions in Part B of the questionnaire, indicating what, if any changes must be made to the Completeness Review Checklist or to the coding instructions.
6. Re fers any legibility issues or discrepancies needing resolution to the Data Manager.
7. Checks the transcription of all issues to be discussed in a callback on Form 6, the Reviewer Flag/Comment Sheet. 8. If, at any point in the Review 2 process, the survey is re turned to the active file prior to completion, circles “Review 2 Begun” on Form 5 and inserts a note into the wallet explaining where in the process the review stopped. 9. At the completion of the second review, returns the survey and all forms to the wallet

, affixes a yellow dot to the wallet and files the wallet in the “Ready for Data Entry” file (Cabinet C).



### Coding
In the coding process, certain items in surveys are marked to ensure data entry operators input the information correctly

. All markings done i n the surveys must be done in colored pen to distinguish ERG markings from respondent markings

. A yellow highlighter pen will be used to highlight any information relating to CBI claims. Several markings or corrections should be done at this time:

**•** Check to see if the CBI box on Page iii is checked, indicating the entire survey is CBI

. If this is the case, on page B

- 1, above the Part B header, write in “Data Entry, check global CBI box.” Highlight this text in yellow highlighter

.

**•** If the CBI box is not chec ked:

1. Check any cover letter to determine if the respondent has claimed the entire survey (or Part B or any particular question in Part B) is CBI

. If the respondent claims the entire survey, or all of Part B, as confidential, write in “Data Entry, check g lobal CBI box” on page B

- 1 as above. Highlight this text in yellow highlighter

.

2. If the cover letter indicates specific questions are confidential, make sure the box next to those questions have been checked “CBI.” If not, check the box, highlight it, and add “marked CBI as per cover letter,”QAPP

— CBM De tailed Questionnaire

A

- 9 in the Reviewer Flag/Comment Sheet, with the relevant question number.

3. Check the comments section to determine if the respondent has declared any questions CBI, even if they are not marked as such

. If questions have been claimed CBI in the comments section, but not marked as CBI in the check boxes, fill in the check boxes for the respondent and highlight them. Note this change in the Reviewer Flag/Comment Sheet.

4. Check the margins of the survey to see if the responden t has handwritten in CBI, confidential, do not disclose, etc. and fill in the check boxes for the respondent and highlight them for any question so designated, if not already checked

. Note this change in the Reviewer Flag/Comment Sheet.

5. Review each CBI b ox in Part B of the survey and if any CBI box for an individual question is checked (whether by the respondent or by ERG), highlight the box.

Following the CBI check, continue with the following general instructions and the question

- by

- question instructio ns listed below. Note that in general,

- 888 is used to designate questions that are blank because they are not applicable to the respondent and is also used to replace “NA” or “not applicable” answers made by respondents. Similarly, the code

- 999 is used f

or blanks where an answer was expected but was not found

. The code

- 999 should not be used to replace any data

. Another code,

- 777, is used to replace “unknown.” It is also used for any data that cannot be deduced from financial forms submitted for transcr iption data (e.g., revenues attributable to oil and gas production)

. Any changes to the answers other than those represented by these codes should be noted in the Reviewer Flag/Comment Sheet, unless otherwise noted

. For example, if a box is checked “no”, w hich would mean the next question should be skipped, but the respondent has provided information in the next question, the reviewer should change the box to “yes” and note in the Reviewer Flag/Comment Sheet that the “no” box seemed to have been checked in error and it was changed to “yes.” Other details of coding are provided below.

**•** Look at the entries for Question B3

- 1. Check the project name and number in Appendix B (the complete list with CBI project names can be found in File E, Drawer 1). If the name checks, replace the name with the number. Write the number at the top of each column in the blank provided. If the respondent has written in the project name there, cross out the name and replace with the number.

**•** If the respondent reports fewer than 3 pro jects on Tables B

- 1, B

- 2, or B

- 3, strike through the columns not used using X’s in each column.

**•** If this respondent is not required to answer anything in Table B

- 2, place a large X through all pages of that table.QAPP

— CBM De tailed Questionnaire

A

- 10 **•** Respondents might use dashes instead of 0 s. Strike dashes and replace with 0s

. This does NOT require a notation in the Reviewer Flag/Comment Sheet.

**•** If respondents have filled in an answer for an “other” question but have not X’d the “other” box, X the box and add this to the Reviewer Flag/Commen t sheet.

**•** Respondents might have used parentheses in place of negative signs in financials

. Strike the parentheses and add a negative sign “

- ”.This does NOT require a notation in the Reviewer Flag/Comment Sheet.

**•** If a respondent has written a comment in th e margin of any question, circle the comment, write COM next to the comment and add the question number next to COM, if question number is not clear.

**•** If a respondent provides a range, write the midpoint of the range as the response

. Circle the range give n by the respondent and write COM under the question number

. Data entry will enter the midpoint. This does not require a notation in the Reviewer Flag/Comment Sheet.

**•** If the respondent writes in “unknown” for any question, circle “unknown” and write COM un der the question number, except as addressed in the specific instructions on questions here. Write in

- 777 as the answer for “unknown,” where “unknown” is not an option for an answer. Also use the code

- 777 to replace “unknown” as an answer to Question B2

- 2 if the respondent has checked “joint venture” in Question B2

- 1 or if they have responded to Question B3

- 2 with “unknown” (which also should be replaced with

- 777).

Specific instructions on questions:

**•** If entries for B1

- 2 are missing, X out entire questi on.

**•** If entry for B1

- 3 is missing, X out entire question.

**•** If B2

- 1 Box 1 is checked, B2

- 2 and B2

- 3 should be blank or NA, and should be marked

- 888.

**•** If Box 2 in B2

- 1 is checked, and only one name appears in B2

- 2,

- 999 should be entered for name 2. Names 2 and 3 are

- 888 for all other respondents.

**•** If Box 2 in B2

- 1 is checked, and only one name appears in B2

- 3,

- 999 should be entered for name 2. However, if “unknown” appears, this entry can remain. Write in unknown for Names 2 and 3 as well. Names 2 and 3 a re

- 888 for all other respondents.QAPP

— CBM De tailed Questionnaire

A

- 11 **•** If B2

- 10 is answered “yes” draw a line after the end of this question and write a note as follows: “Data Entry: you are done with Section B2. Turn to Page B

- 10.” X out Question B2

- 11 and Pages B

- 5 through B

- 9.

**•** If answe rs to questions B2

- 11 through B2

- 13 are missing, note

- 999. If B2

- 12 through B2

- 14 have been marked “disagree,” but no alternative information has been provided in either the comments or in the underscored areas, circle “disagree” box and note COM, then ch eck the “agree.” Box.

**•** If Question B

- 15 has been checked “Yes,” complete the coding of this question, but then skip to Question B2

- 23.

**•** In Question B2

- 19b and b2

- 19c: Respondent marks interest payments and or taxes as “unknown.” Strike out text and mark w ith a

- 888.

**•** Question B3

- 5 through B3

- 8: If respondent has answered “yes” to Question B3

- 4 for a project, enter

- 999 if any of these questions has missing answers; otherwise

use

- 888.

**•** If B3

- 9 is blank and B3

- 10 is blank, mark

- 888 in both questions. If ei ther one is blank and the other is not, mark the blank question with

- 999.

**•** Question B3

- 11. Blanks in vertical should be marked

- 999. Blanks in horizontal should be marked

- 888.

**•** Questions B3

- 12 through B3

- 14. Mark all blanks as

- 888.

**•** If both lines in B3

- 15 are missing, mark both

- 999. If one line or the other is missing, mark the missing line

- 888.

**•** Questions B3

- 18 and B3

- 19, mark

- 888 if B3

- 17 is marked “no”, otherwise mark

- 999.

**•** If B3

- 35 has an entry for either workovers, stimulations or recompletions , mark

- 999 in B3

- 36 for any missing item with an expenditure marked in B3

- 35

. For any item marked “NA” in B3

- 35, mark

- 888 in the matching item in B3

- 36. Otherwise mark any missing items as

- 999 in both questions.

**•** If Question B3

- 37 is answered yes or is missing (

- 999), and Questions B3

- 38 through B3

- 40 are not answered, mark those questions

- 999, otherwise mark

- 888.

**•** If Question B3

- 38 is answered, B3

- 39 and B3

- 40 should have an entry for every line for which B3

- 38 has an answer; if not, missing answers s hould be marked

- 999. Lines not used should be marked

- 888. NOTE: Cross out $ signs and 0s on Question B3

- 39.QAPP

— CBM De tailed Questionnaire

A

- 12 **•** If Question B3

- 41 is answered yes or is missing (

- 999), and Questions B3

- 42 through B3

- 44 are not answered, mark those questions

- 999, otherwise mark

- 888.

**•** If Question B3

- 45 is answered (including with a comment such as “unknown”) and Question B3

- 46 is not answered, mark Question B3

- 46 with

- 888. If Question B3

- 46 is answered with more than 1 lease filled in at least one of the answer options, ma rk Question B3

- 45

- 888

. If Question B3

- 46 shows only one lease, mark B3

- 46

- 888 and transfer the lease type shown in B3

- 46 to B3

- 45. If both Question B3

- 45 and B3

- 46 are missing answers, mark both with

- 999.

**•** If Question B3

- 51 is answered with the 5% check mark, cross out the 5% check mark and write 5 in the “we estimate” line. If both are missing, write 5% in the “we estimate” line. Write “missing” in the margin, circle it and put it in the Reviewer Flag/Comment Sheet.

**•** If Question B3

- 53 is answered yes or is missing (

- 999), and Questions B3

- 54 through B3

- 55 are not answered, mark those questions

- 999, otherwise mark

- 888.

**•** If Question B3

- 56 is answered yes or is missing (

- 999), and Questions B3

- 57 through B3

- 58 are not answered, mark those questions

- 999, otherwise mark

- 888.

**•** If Question B3

- 68 is answered “yes” or is missing and B3

- 69 through B3

- 72 are not answered, mark B3

- 69 through B3

- 72 as

- 999. If B3

- 68 is answered “no,” mark B3

- 69 through B

- 70 as

- 888 and B3

- 71 through B3

- 72 as

- 888 if these latter have not been answered. Replace any “NAs” with

- 888.

**•** If B3

- 68 has an answer and units or estimation method lines are missing requested information, mark

- 999, otherwise, mark units and estimation method

- 888.

**•** If Questions B3

- 78 through B3

- 85 are missing answers, check Table B

- 4 to see if the relevant information has been included there for at least 3 years. If not, check the box “we agree to this approach” and note that a question is missing an answer so the agree box was checked in the Reviewer Flag/Com ment Sheet. If Table B

- 4 contains the information, check the box saying projections have been provided in Table B

- 4 and note this in the Reviewer Flag/Comment Sheet.

**•** If the respondent has provided no information in Table B

- 4, X out all of Table B

4 and no te at the top of the Table “Data Entry

— Skip to page B

- 37” (comments). If some information is provided, all missing items in Table 4 should be marked

- 888. If entire years are missing, X out the columns.

Other issues may arise during the course of the codi ng effort

. Consult with the ERG Task Manager (Anne Jones) for a decision; a decision will be written up and distributed to all coders for future reference

. NOTE: The Reviewer Flag/Comment Sheet should ultimately reflect any coding in which answers have bee n transcribed (other than financial information transcribed in the financial transcription process

.QAPP

— CBM De tailed Questionnaire

A

- 13 

### Initial Problem Callbacks
The Detailed Reviewer 1 assigned the survey is responsible for all initial problem callbacks for that survey. The following steps should be followed for initial callbacks found in the “Initial Problems” file:

1. Pull an assigned survey from the “Initial Problems” file.

2. Follow steps 2 through 7 for Detailed Reviewer 1 (see above). (Note: Missing Part B, goes straight to callback).

3. Ensur es Reviewer Flag/Comment Sheet is complete and prepares questions on the Callback Log Sheet (Form 7, Appendix C)

. If more than one sheet is needed, make sure sheets are stapled together either before or after the callback completion

.

4. Make the callback and note any answers provided in the callback log.

5. When missing information is provided following callback, transcribe information, or include new questionnaire sheets, noting in the callback log that new sheets have been provided and including date (e.g., Pa rt B, relevant project sheets).

6. Enter survey into the review process at differing phases, depending on what information was missing:

a. Missing Part B: Returns to the active file for Initial Review; the Data Manager is informed that it is ready for Initial Re view again.

b. Missing a project’s data: Receives a blue sticker and returns to the Active File for the completion of Detailed Review 1 (any completed portions of the Detailed Review 1 can be skipped).

c. Missing financial reports: receives a blue sticker and is placed in the transcription file area and follows steps as outlined for surveys needing transcription from that point forward.

The following steps should be followed for initial callbacks found in the “Detailed Review Problems” File:

1. Pulls an assigned survey from the Detailed Review Problems file.

2. Reviews the Reviewer Flag/Comment Sheet and transcribe questions to the Callback Log Sheet (Form 7). If more than one sheet is needed, make sure sheets are stapled together either before or after the callback completion

.

3. Makes the callback and documents any answers provided on the callback log sheet.QAPP

— CBM De tailed Questionnaire

A

- 14 4. When all critical data is received (Form 2 data

— either documented in the callback or provided in hardcopy form by the respondent), transcribes it into the questio nnaire.

5. Makes additional callbacks if Form 2 data is still missing, until provided, or until instructed otherwise.

6. Reviews the completeness checklist and updates the list with the new information in place. Affixes a silver star, indicating first review is complete and/or proceeds directly to coding

.

7. Steps in Detailed Review 1 from 12d on can be continued from that point forward.



### Data Entry When the second review is completed, the reviewer has affixed a yellow dot and has placed the file in the data entry f ile drawer (File C, Drawer 3). The data entry personnel will complete all data and follow the instructions provided by the coding. At the completion of each form, the data entry personnel will make a quick QA check of their data entry information. At the c ompletion of data entry, the file is returned to the data entry complete file, File C, Drawer 1), with a gold star attached to show the data entry process is complete.

At the end of the data entry process, a 20 percent check sample (based on numbers of

pr ojects) will be drawn and a complete proofreading of the sample will be done to gauge the accuracy of the data entry

. If the quality check determines that the data entry is accurate, no further proofreading will be done

. If accuracy issues are noted for an y of the data entry personnel, all additional surveys entered by that person will be proofed. Corrections will be made directly to the database and proofed again.



### Post

- Data Entry Callbacks
These callbacks follow the same form as the initial callbacks, exce pt that questions may be added by the ERG Task Manager following analysis of the database to identify out of range responses, totals that do not add, etc. These additional questions need to be added in with any questions derived from the Reviewer Flag/Comm ent sheet if an initial callback was not performed.



### Color codes for file folders

Blue dot

— Initial review complete or initial review data problem resolved, ready for detailed review. Green dot

— Financials require transcription. Red star

— First review started, needs initial callback. Silver star

— First review completed. Red dot

— First review and coding completed. Ready for second review. Yellow dot

— Second review completed. Ready for data entry. Gold star

— data entry completed.QAPP

— CBM De tailed Questionnaire

A

- 15 Figure 1

. Survey Tracking Form

Survey ID Company Survey Receipt (date) Folder Prepared

(date) Financial Info
to be
Transcribed Financial
Info Missing - CALLBACK Completion of B2 (date or

NA) Review & Coding
#1 Complete (date) Review & Coding
#2 Complete (date) Initial Callback Requir-ed (X) Initial CallbackCompleted (date) Data Entry Completed

(date) Database
Data Checks Completed

(date) OID001 OID002 OID003 OID004 OID005 OID006 OID007 OID008 OID009 OID010 OID011 OID012 OID013 OID014 OID015 OID016 OID017 OID018 OID019 OID020 OID021 OID022 OID023 OID024QAPP

— CBM De tailed Questionnaire

A

- 16 APPENDIX A.1

REVIEW PACKET

(LOCATED IN EACH WAL LET)QAPP

— CBM De tailed Questionnaire

A

- 17 

### 2009 Coalbed Methane Questionnaire: Part B: Financial and Economic Information

REVIEWER COVER SHEET

7.1.1.1 Survey ID OID_____

Financial Reviewer 1 Financial Reviewer 2 Detailed Review Reviewer 1 Co der (if different from

Reviewer 1) Detailed Review Reviewer 2

Name

Date Started

Financials Entered/Chec ked (Date) Coding & Review Completed (Date)QAPP

— CBM De tailed Questionnaire

A

- 18 

# PART B1&B2

2009 Coalbed Methane Questionnaire: Part B1 &2: Financial and Ec onomic Information

Overall Survey Completeness Checklist

Survey ID_________

Q# Yes No NA Unread

able Comment Explaining is Provided Checklist Item

Section B1

B1

- 1 Primary Contact Name is present Contact Title is present

Email Address is present Phone number is present Fax number is present Convenient time to call is present

B1

- 2 Primary Contact Name is present Contact Title is present Email Address is present Phone number is present Fax number is present Convenient time to call is present

B1

- 3 B1

- 3 is present (if not present mark “NA”).QAPP

— CBM De tailed Questionnaire

A

- 19 Survey ID_________

Q# Yes No NA Unread

able Comment Explaining is Provided Checklist Item

Section B2

B2

- 1 One and only one box is checked

B2

- 2 Name of owner is present unless box 1 checked in B2

- 1, two or more o wners present if box 2 checked in B2

- 1.

B2

- 3 Name of owner is present unless box 1 checked in B2

- 1, two or more owners present if box 2 checked in B2

- 1 (or “unknown” has been entered).

B2

- 4 One and only one box is checked

B2

- 5 Either a 6

- digit number or an “NA” is entered in the underscored section

B2

- 6 One and only one box is checked

B2

- 7 One and only one box is checked; If “other” is a selection, an explanation has been entered

B2

- 8 One and only one box is checked

B2

- 9 Both underscored lines have a number inserted

B2

- 10 One and only one box is checked
For Respondents not answering “yes” to Question B2

- 10

B2

- 11 One and only one box is checked; If “other” is a selection, an explanation has been entered

B2

- 12 One and only one box is checked; if disagree, comment explaining is providedQAPP

— CBM De tailed Questionnaire

A

- 20 Survey ID_________

Q# Yes No NA Unread

able Comment Explaining is Provided Checklist Item

B2

- 13 One and only one box is checked; if “disagree” is selected, a number is entered on the underscored line or a number or other answer is provided in the

comme nts section One and only one box is checked, if “yes”, numbers are provided on all three underscores provided. For Respondents not answering “yes to Question B2

- 15

B2

- 16 One and only one box is checked; If “other” is a selection, an explanati on has been entered

B2

- 17 to B2

- 19 At least one number is entered each underscored section in each column of each table or an “NA” or “unknown” is written in.

B2

- 20 One and only one box is checked; If “other” is a selection, an explanation has been entered

B2

- 21/ B2

- 22 An entry has been made in each underscored section in each column of each table For All Respondents (other than those answering yes to Question B2

- 10)

B2

- 23/ B2

- 24 A entry has been made on each underscored line

B2

- 2 5 Item is checked if answer to B2

- 15 is yes and financials are attached.QAPP

— CBM De tailed Questionnaire

A

- 21 

# PART B3
2009 Coalbed Methane Questionnaire: PART B3: Project Information

Overall Survey Completeness Checklist

Survey ID_________ Project Name _______________________________

_______ Project ID ________________

Q# Yes No NA Unread

able Comment Explaining is Provided Checklist Item

B3

- 1/ B3

- 2 An entry has been made in the space provided

B3

- 4 One and only one box is checked

B3

- 5 to B3

- 8 An entry has been made in each of the spaces or underscores provided, if relevant (B3

- 4 = yes)

B3

- 9 Should have an entry if B3

- 10 has an entry

B3

- 10 Should have an entry if B3

- 9 has an entry

B3

- 11 At least one entry should be made for either vertical or horizo ntal

B3

- 12 to B3

- 14 An entry has been made in each underscored area

B3

- 15 A minimum of one entry has been provided in the underscored areas

B3

- 16 One and only one box is checked

Table B2

— Only Respondents Marking “No” for Question B3

- 16) for a particular project

B3

- 17 One and only one box is checked

B3

- 18 to B3

- 36 An entry has been made in each of the spaces or underscores provided, if relevant

B3

- 37 One and only one box is checkedQAPP

— CBM De tailed Questionnaire

A

- 22 Survey ID_________ Project Name _______________________________

_______ Project ID ________________

Q# Yes No NA Unread

able Comment Explaining is Provided Checklist Item

B3

- 38 to B3

- 40 An entry has been made in each of the spaces or underscores provided, if relevant

B3

- 41 One and only one box is checked

B3

- 42 to B3

- 44 An entry has been made in each of the spaces or underscores provided, if relevant

Table B3

— All Respondents, All Projects

B3

- 45 to B3

- 50 An entry has been made in each of the spaces or underscores provided, if relevant

B3

- 51 One and only one box is checked

B3

- 52 An entry has been made each of the spaces or underscores provided, if relevant

B3

- 53 One and o nly one box is checked

B3

- 54 to B3

- 55 An entry has been made each of the spaces or underscores provided, if relevant

B3

- 56 One and only one box is checked

B3

- 57 to B3

- 67 An entry has been made each of the spaces or underscores provided, i f relevant

B3

- 68 One and only one box is checked

B3

- 69 Units have been provided, one and only one box is checked for estimation method, if “other” has been selected, an entry has been made in the underscore provided, and a entry has been made i n the space provided in the answer column(s)QAPP

— CBM De tailed Questionnaire

A

- 23 Survey ID_________ Project Name _______________________________

_______ Project ID ________________

Q# Yes No NA Unread

able Comment Explaining is Provided Checklist Item

B3

- 70 to B3

- 74 An entry has been made each of the spaces or underscores provided, if relevant; Units are specified, if relevant Future Costs and Revenues (All Respondents, All Projects)

B3

- 75 One an d only one box has been checked

B3

- 76, B3

- 77 An entry has been made in each of the spaces or underscores provided.

B3

- 78 One and only one box has been checked, underscores associated with a checked box have an entry

B3

- 79 to B3

- 85 One an d only one box has been checked, underscores associated with a checked box have an entry, if applicable

B3

- 86 Optional; if provided make sure it’s readable For respondents who did not check the first box of Question B3

- 75 (remember to consider NAs i f boxes are checked in Questions B3

- 76 through B3

- 85)

B3

- 87 to B3

- 102 An entry has been made each of the spaces or underscores provided, if relevant

B3

- 103 Option; if provided made sure this is readableQAPP

— CBM De tailed Questionnaire

A

- 24 APPENDIX A.2

CBM FORMSQAPP

— CBM De tailed Questionnaire

A

- 25 

### Survey ID____

FORM 1

INITIAL REVIEW

Issue Yes No 1. Out of Scope 2. Missing All of Part B 3. Financial Data for B2 Missing (No Financials) 4. B2 Needs Transcribing 5. Projects Missing 6. Gas & Water Projections Are Missing (Respondent Indicated Attach ment)

Note: Out of Scope marked “Yes” goes in File E, Second Drawer Issues 2, 3, or 5 marked “Yes” goes in File D, Fourth Drawer If Issue 4 is marked yes (but Issues 2, 3, or 5 are not), survey goes in File C, First Drawer. Issue 6 gets logged on For m 3 and placed back in active file with blue sticker

All Others Place Back in Active File with Blue Sticker

Reviewer Initials _________ Date____________________QAPP

— CBM De tailed Questionnaire

A

- 26 

### FORM 2

INITIAL CALLBACK TRIGGER FORM

Issues Yes No

1. B2

- 1, B2

- 4, or B2

- 9 are miss ing

2. B2

- 17e, B2

- 18i, B2

- 19a, B2

- 19d, B2

- 21d, B2

- 22c or B2

- 22d are missing for all years

3. B3

- 1 (if responding for more than 1 project) is missing

4. B3

- 4 = yes and B3

- 5 through B3

- 8 are missing

5. B3

- 11 or B3

- 15 are missing

6. B3

- 16 = no and Table B

- 2 is missing all information

7. B3

- 16 = no and B3

- 20 through B3

- 24 are missing (any one of these)

8. B3

- 16 = no and B3

- 26 through B3

- 29 or B3

- 34 are missing (any one of these)

9. B3

- 48 >0 or not blank and B3

- 47 is missing

10. B3

- 50, B3

- 59 or B3

- 62 through B3

- 66 are missing (any one of these)

11. B3

- 76 or B3

- 79 are missing and respondent has not filled out at least 3 years of Table B

- 4 12. Too many missing or illegible answers

Note: If any issues are marked “Yes” place in Fi le C, Fourth Drawer.QAPP

— CBM De tailed Questionnaire

A

- 27 

### FORM 3

SUMMARY OF INITIAL NOTES AND PROBLEMS FOR PART B (Mark X Only for Survey IDs with Items Missing or Noted)

Survey
ID Out of Scope Missing Part B Missing Financial Reports Missing Project(s) Projection Attachments Missing Proje ctions AttachedQAPP

— CBM De tailed Questionnaire



### Form 4

List of Surveys in Transcription (Place Reviewer Initials and Date in Appropriate Column)

Survey ID Transcrip tion Begun Transcription Completed Review of Transcription Begun Review of Transcription

CompletedQAPP

— CBM De tailed Questionnaire



### Form 5

List of Surveys in Detailed Review & Coding (Place Reviewer Initials and Date in Appropriate Column)

Survey
ID Review 1 Begun Review 1 Completed Coding Begun Coding Completed Review 2 Begun Review 2 CompletedQAPP

— CBM De tailed Questionnaire



### FORM 6

2009 Coalbed Methane Questionnaire: Part B: Financial and Economic Information

REVIEW ER FLAG/COMMENT SHEET

7.1.1.2 Survey ID

______________________

Company Name

______________________

Reviewer 1

______________________

Reviewer 2

______________________

Question No. Description of Problem Date Resolved & Reviewer’s InitialsQAPP

— CBM De tailed Questionnaire



### FORM 7

2009 Coalbed Methane Questionnaire: Part B: Financial and Economic Information

Project Name: CBM Detailed Study

Project No:

Facility ID:

Facility Name:

Contact Name: Contact Phone No: Contact Email A ddress:

Date: Staff Name:

Time: AM PM

This document contains CBI: _____ Yes _____ No Type of Contact:

Email Telephone

General Sub ject(s) Discussed:

CBI = Yes (Y) Question
No. Topics Discussed