**Vermont Department of Environmental Conservation**

**Retail Gasoline Sector Environmental Results Project**

**Quality Assurance Project Plan**

**November 13, 2023**

**Vermont Department of Environmental Conservation**

**103 S. Main Street**

**Waterbury, VT 05671-0404**

**Project Manager: Peter Marshall**

**103 S. Main St./West Office Bldg.**

**Waterbury, VT 05671-0404**

**(802) 241-3868 phone**

**(802) 241-3296 fax**

**peter.marshall@anr.state.vt.us**

# A PROJECT MANAGEMENT

## A1. Approval Sheet

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Peter Marshall, Project Manager Date

Vermont Department of Environmental Conservation

Chief, Hazardous Waste Management & Prevention Section

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Winslow Ladue, Quality Assurance Officer Date

Vermont Department of Environmental Conservation

## A2. Table of Contents

[]{.mark}

[A PROJECT MANAGEMENT 1](#a-project-management)

[A1. Approval Sheet 1](#a1.-approval-sheet)

[A2. Table of Contents 2](#a2.-table-of-contents)

[A3. Distribution List 3](#a3.-distribution-list)

[A4. Project/Task Organization 3](#a4.-projecttask-organization)

[A5. Problem Definition/Background 4](#a5.-problem-definitionbackground)

[Rationale for initiating the project.
4](#rationale-for-initiating-the-project.)

[Objectives of the project. 5](#objectives-of-the-project.)

[Regulatory information, applicable criteria and action limits.
6](#regulatory-information-applicable-criteria-and-action-limits.)

[A6. Project/Task Description 6](#a6.-projecttask-description)

[Project overview. 6](#project-overview.)

[Project summary and work schedule.
7](#project-summary-and-work-schedule.)

[Geographic focus. 9](#geographic-focus.)

[Resource and time constraints. 9](#resource-and-time-constraints.)

[A7. Quality Objectives and Criteria
9](#a7.-quality-objectives-and-criteria)

[Detailed performance measures. 9](#detailed-performance-measures.)

[Quality objectives. 9](#quality-objectives.)

[A8. Special Training/Certification
10](#a8.-special-trainingcertification)

[A9. Documents and Records 11](#a9.-documents-and-records)

[Report format/information. 11](#report-formatinformation.)

[Document/record control. 11](#documentrecord-control.)

[Other records/documents. 11](#other-recordsdocuments.)

[Storage of project information. 11](#storage-of-project-information.)

[Backup of electronic files. 12](#backup-of-electronic-files.)

[B DATA GENERATION AND ACQUISITION
12](#b-data-generation-and-acquisition)

[B1. Sampling Process Design (Experimental Design)
12](#b1.-sampling-process-design-experimental-design)

[B2. Sampling Methods 13](#b2.-sampling-methods)

[B3. Sample Handling and Custody 13](#b3.-sample-handling-and-custody)

[B4. Analytical Methods 13](#b4.-analytical-methods)

[B5. Quality Control 13](#b5.-quality-control)

[Preparation of data collection instruments.
13](#preparation-of-data-collection-instruments.)

[Training. 13](#training.)

[Crosschecking data. 13](#crosschecking-data.)

[Data anomalies. 14](#data-anomalies.)

[Data entry. 14](#data-entry.)

[B6. Instrument/Equipment Testing, Inspection and Maintenance
14](#b6.-instrumentequipment-testing-inspection-and-maintenance)

[B7. Instrument/Equipment Calibration and Frequency
14](#b7.-instrumentequipment-calibration-and-frequency)

[B8. Inspection/Acceptance for Supplies and Consumables
14](#b8.-inspectionacceptance-for-supplies-and-consumables)

[B9. Non-Direct Measurements (I.e., Secondary Data)
14](#b9.-non-direct-measurements-i.e.-secondary-data)

[Key resources/support facilities needed.
15](#key-resourcessupport-facilities-needed.)

[Determining limits to validity and operating conditions.
16](#determining-limits-to-validity-and-operating-conditions.)

[B10. Data Management 16](#b10.-data-management)

[C ASSESSMENT/OVERSIGHT 16](#c-assessmentoversight)

[C1. Assessment and Response Actions
16](#c1.-assessment-and-response-actions)

[C2. Reports to Management 16](#c2.-reports-to-management)

[D DATA REVIEW AND EVALUATION 17](#d-data-review-and-evaluation)

[D1. Data Review, Verification and Validation
17](#d1.-data-review-verification-and-validation)

[D2. Verification and Validation Methods
17](#d2.-verification-and-validation-methods)

[D3. Evaluating Data in Terms of User Needs
18](#d3.-evaluating-data-in-terms-of-user-needs)

## A3. Distribution List

Each person listed on the approval sheet and each person listed under
Project/Task Organization will receive a copy of this Quality Assurance
Project Plan (QAPP). Individuals taking part in the project may request
additional copies of the QAPP from personnel listed under Section A4.

This document has been prepared according to the United States
Environmental Protection Agency publication *EPA Requirements for
Quality Assurance Project Plans* dated March 2001 (QA/R-5).

## A4. Project/Task Organization

Personnel involved in project implementation are listed in Table 1, and
shown as an organization chart in Figure 1.

  ---------------- -------------------------------- ----------------------
  **Table 1:                                        
  Project                                           
  Implementation                                    
  Personnel**                                       

                                                    

  **Individual**   **Role in Project**              **Organizational
                                                    Affiliation**

  Peter Marshall   Project Manager                  VT DEC

                   QA Manager/Officer               VT DEC

  Marc Roy         Supervisor                       VT DEC

  Ted Unkles       UST Program Coordinator          VT DEC

  June Middleton   Outreach/Compliance              VT DEC

  Andrew Shively   Inspections/Verification         VT DEC

  Susan Thayer     Inspections/Verification         VT DEC

  Position to be   Data Management and Project      VT DEC
  hired            Support                          

  ERP Contractor   Support on Project Design and    TBD
                   Implementation (Detailed below)  

  VT DEC IT Group  IT Support/Database Mgt          VT DEC
  ---------------- -------------------------------- ----------------------

The VT DEC Project Manager will be responsible for the following
activities:

-   Conduct outreach with regulated industry and internal/external
    stakeholders

-   Coordinate with Contractor

-   Maintain official, approved QAPP

-   Develop revised QAPP

-   Issue quarterly and annual reports to U.S. EPA

Contractor (to be determined (TBD)) will be responsible for the
following activities:

-   Assist with project design

-   Develop statistical methodology

-   Assist VT DEC Information Technology (IT) staff in the development
    of necessary data systems and hardware infrastructure

-   Assist VT DEC staff in review and presentation of outreach materials

-   Assist VT DEC staff in stakeholder outreach, especially concerning
    the benefits of ERP from experiences of other states

-   Assist VT DEC staff in analysis of project data and presentation of
    results

-   Review data and provide QA/QC

-   

The participating facilities will be responsible for submitting
self-certification materials and, if applicable, returning to
compliance.

**Figure 1: Project Organizational Chart**

See attached project organizational chart.

## A5. Problem Definition/Background 

### Rationale for initiating the project. 

There are approximately 2,600 Underground Storage Tanks (USTs) at
approximately 1,100 facilities in Vermont, and the majority of these
facilities are in the Retail Gasoline Sector (RGS). In the 1960s and
1970s, frequent releases from USTs resulted in extensive soil and
groundwater contamination that required costly remedial actions. In
response to this growing problem, the federal Underground Storage Tank
program was developed (a part of the Resource Conservation and Recovery
Act of 1976 (42 U.S.C. §§ 6991 et. seq.).

The early priorities of this program were to establish: (1) a national
inventory of UST facilities through a registration requirement; and (2)
minimum environmentally protective standards for UST systems and a
milestone by which all subject facilities would need to meet these
standards (December 22, 1998). While the rules and programmatic
requirements did go beyond these two priorities to areas such as spill
and overfill protection, requiring upgrades that would prevent
catastrophic releases (such as through sudden failure of a tank or
piping system due to corrosion) was the primary focus.

Now that the UST system upgrade milestone has passed (and VT facilities
have achieved compliance with this requirement), the focus of the
program has turned to the operation and maintenance (O&M) of UST
systems. Unlike the hardware upgrades of the 1998 deadline, which were a
"one-time" event to attain and sustain compliance, O&M compliance is a
daily, ongoing task. This difference applies to ascertaining compliance
as well -- while a single inspection (or third party evaluation) will
provide enduring documentation of compliance with the upgrade
requirements, compliance with O&M requirements is continually changing
at all 1,100 facilities.

While the goals of the UST Program have remained constant -- to prevent
releases by requiring effective design, operation, and maintenance of
UST systems -- the focus of the program has changed as the sources of
the worst risks have changed. The change in focus of the program has led
to a need for more frequent compliance evaluations for the O&M
requirements, which, when neglected, are now the greatest source of risk
of release.

The Vermont UST Program, with current resources, can inspect
approximately 100 facilities per year. At this rate, it takes over ten
years to inspect each facility once. Significant new resources are not
expected in the future from either EPA or the Vermont Legislature. In
addition, to justify additional resources, a program must be able to
document that the work that it does is effective at solving the problem
it is expected to address. With existing resources and programmatic
operations, the program is: (1) unable to inspect facilities for
compliance with O&M requirements at a level that would adequately
protect the environment; and (2) unable to gather sufficient program
performance data to show either where the problems lie in O&M compliance
or that any of the existing or future approaches to solving these
problems is effective.

### Objectives of the project. 

This project is designed to improve environmental results by the
intended outcomes of: reducing the threat of releases of petroleum
hazardous wastes and materials to groundwater (a significant source of
drinking water in Vermont) and soils through enhanced compliance with
UST, RCRA, UIC, and sector-specific BMPs; reducing air emissions through
enhanced Stage I & II vapor recovery compliance; and reducing hazardous
waste generation through the use of the BMPs.

The ERP model will allow for the measurement of annual compliance
progress for the entire sector, with several environmental laws. This
proposed ERP model consists of an initial round of inspections of a
statistically valid number of facilities, sector-based multimedia
outreach and education, a mandatory multi-media self-certification on a
sector-specific form, a processing fee, random inspections to confirm
self certifications, and data analysis that will result in statistically
valid compliance reports to document project performance. Targeted
inspections will be used to promote submission of self-certifications as
well as assure compliance.

Where facilities indicate they are not in compliance with all applicable
regulations under this project, or where inspections find
non-compliance, a return to compliance plan will be required. All
non-compliance will be documented. When self-certifications indicate
non-compliance, the facility will be required to submit a
return-to-compliance schedule to the VT DEC. Random and targeted
inspections will be used to confirm return to compliance at selected
facilities. These schedules will be tracked in the project database.
When inspections identify facility non-compliance, the VT DEC will
respond by both informing facility personnel on-site and sending a
letter documenting the non-compliance and requiring a return to
compliance by a date certain.

This proposed Environmental Results Project will include the following
components:

-   A workbook that includes Best Management Practices (BMPs) and
    compliance requirements. The workbook is a guide to compliance with
    the self-certification form mentioned below;

-   A BMP checklist for use by the facility in assessing the status of
    compliance, pollution prevention, and other health and safety
    practices;

-   A compliance self-certification form that the facility owners and
    operators are required to complete, sign and return. On the form,
    the facility owners and operators must certify the current
    compliance status of the facility and acknowledge that the facility
    must comply with all applicable environmental laws;

-   A Return to Compliance form that is used to address compliance
    problems identified in the self-certification process that cannot be
    corrected at that time. The form establishes a return to compliance
    schedule and deadline for compliance. This form must be signed and
    returned when a facility cannot certify full compliance with all
    applicable regulatory requirements;

-   Workshops to provide technical and compliance assistance to facility
    owners and operators, and to provide training on the requirements of
    the ERP process;

-   Inspections by the VT DEC to confirm the accuracy of the
    certifications and compliance with the applicable environmental
    regulations; and

-   Ongoing technical and compliance assistance by telephone, on-site
    assistance (as resources allow), and a project-specific web page
    with FAQs and additional resources.

### Regulatory information, applicable criteria and action limits. 

This project focuses on priority environmental issues by targeting an
industry sector that is present statewide with significant potential for
environmental releases and emissions. The project is intended to improve
compliance within the sector with requirements of four
federally-delegated programs: UST, RCRA generator requirements, the
Clean Air Act (CAA) Stage I & II requirements, and the Safe Drinking
Water Act (SDWA) UIC requirements.

This project would establish a sector-specific, cross-media workbook
covering regulatory as well as Best Management Practices (BMPs). The
purpose of the BMPs would be to encourage facilities to go beyond
compliance to reduce waste, pollution, and emissions. This project would
encourage the regulated community to achieve reduced compliance costs by
addressing all compliance and environmental issues at once through
cross-media BMPs (that lead to compliance) rather than narrowly focusing
on whatever problem was identified during the last regulatory
inspection.

This project will build on "lessons learned" from the last decade of
searching for ways to measure prevention efforts by using a mechanism
(ERP) that specifically measures results. The project is likely to
produce quantifiable reductions in hazardous waste generation as well as
quantifiable improvements in compliance with all four prevention
programs. The goal of reducing actual spills and leaks may not be
quantifiable because of the time lag between a spill or leak and its
detection.

## A6. Project/Task Description

### Project overview. 

This project will allow the VT DEC to explore whether an approach
modeled upon the Environmental Results Program (ERP) can help achieve
these goals, while improving regulatory cost-effectiveness. The
Environmental Results Program (ERP) is an innovative approach to solving
high-priority environmental problems in industry sectors largely
comprised of small businesses. The ERP concept combines technical
assistance, self-certification, inspections, and statistically based
performance measurement in order to reduce environmental impacts of
business.

The promise of ERP is that it will cost-effectively reduce environmental
impacts of small businesses that may present a substantial cumulative
environmental risk. Businesses targeted so far by ERP include gas
stations, auto salvage yards, auto body and mechanical repair shops, dry
cleaners, and printers. ERP can help environmental agencies identify
previously unknown facilities, measure performance, increase regulatory
efficiency, and help improve overall environmental performance. ERP is
in part designed to help facilities that want to comply but don't
understand their requirements, and evidence suggests that ERP can
motivate firms to comprehensively review their environmental performance
and take needed action to come into compliance and adopt best practices.

### Project summary and work schedule. 

This project\'s major tasks and timeline are outlined in the table
below.

**Table 2: Schedule of Major Project Tasks**

  ------------------------------------------------------------------------------
  Task/Milestone        Task Description                  Start Date End Date
  --------------------- --------------------------------- ---------- -----------
  Stakeholder Outreach  Outreach to internal and external 10/15/04   09/30/05
                        stakeholders (including targeted             
                        facilities) about the project.               

  Goals identification  Finalize the goals of this        10/15/04   12/30/04
                        project, upon which metrics will             
                        be based                                     

  Develop Logic Model   Develop a logic model with        10/15/04   1/15/05
                        stakeholder involvement. Logic               
                        model can assist in development              
                        of metrics and data needs.                   

  Measures              Development of metrics to be      12/30/04   3/30/05
  identification        tracked by this project.                     

  Prepare legislation   Prepare draft legislation for (1) 10/15/04   12/1/04
                        generic ERP enabling legislation             
                        for agency (not necessary for                
                        this ERP, but helpful), and (2)              
                        fee legislation to cover program             
                        costs after grant period                     

  Revise UST            Draft language for mandatory      10/15/04   12/15/04
  Regulations           self-certification for all UST               
                        facilities (or all UST facilities            
                        in the RGS)                                  

  Formal APA Rule       Average timeframe for selective   1/1/05     7/1/05
  Revision Process      rule revisions is 6 months                   

  Facility              Determine the exact               10/30/04   12/30/04
  identification        characteristics of facilities to             
                        be targeted, and compile a list              
                        of facilities from reliable                  
                        sources.                                     

  Statistical           Development of a statistical      1/1/05     3/1/05
  methodology           methodology to drive performance             
                        measurement and analytical tasks.            

  Data input &          Development and implementation of 11/1/04    6/30/06
  management            an approach to cost-effectively              
                        inputting and managing ERP data,             
                        including primary and secondary              
                        data. Primary data consists of               
                        data from inspection reports and             
                        facility forms (including                    
                        self-certification forms).                   
                        Secondary data sources include               
                        lists of facilities from                     
                        regulatory and private-sector                
                        databases.                                   

  QAPP finalization &   Finalize QAPP based upon results  3/1/05     5/1/05
  approval              of the measures identification,              
                        statistical methodology, and data            
                        management tasks. Primary data               
                        collection will not occur before             
                        relevant parts of the QAPP are               
                        finalized and approved by EPA.               

  Baseline inspections  Inspections at facilities to      5/1/05     7/1/05
                        establish a performance baseline.            
                        Facilities selected at random                
                        from the entire targeted                     
                        population, based upon sample                
                        design from statistical                      
                        methodology.                                 

  Workbook and          Finalization of workbook,         3/1/05     7/1/05
  Certification Form    outreach and assistance                      
  Finalization          materials, web resources, and                
                        certification forms.                         

  Facility              Delivery of compliance/technical  7/1/05     10/31/05
  assistance/Outreach   assistance to facilities, which              
                        is expected to take the form of              
                        workbooks, fact sheets and/or                
                        workshops.                                   

  Self-certification    Implementation of a mandatory     9/1/05     10/31/05
                        facility self-certification                  
                        approach. Self-certification                 
                        refers to the submission of a                
                        legally binding record of a                  
                        facility's compliance and                    
                        beyond-compliance practices.                 

  Self-Certification    Self-Certification and RTC forms  10/31/05   10/31/05
  Deadline              due                                          

  Analysis of           Analysis of Self-Certification    11/1/05    12/15/05
  Self-Certification    data with primary purpose of                 
  Data                  identifying opportunities for                
                        selective follow-up (next step).             

  Selective follow-up   Selective follow-up with          12/15/05   3/15/06
                        self-certifying facilities, based            
                        upon analysis of                             
                        self-certification data. Targeted            
                        follow-up may include phone                  
                        calls, inspections and                       
                        enforcement.                                 

  Post-certification    Inspections at facilities to      4/15/06    5/15/06
  inspections           establish whether sector                     
                        performance measures (and other              
                        measures) have changed since the             
                        baseline. Inspection data also               
                        used to cross-check                          
                        self-certification data at                   
                        inspected facilities. Facilities             
                        selected at random from the                  
                        entire targeted population, based            
                        upon sample design from                      
                        statistical methodology.                     

  Data analysis         Analysis of baseline,             5/15/06    7/15/06
                        self-certification, and                      
                        post-certification data to                   
                        understand change in facility                
                        performance and overall outcomes             
                        of interest. Assessment of                   
                        project efficiency.                          

  Preparation of Final  Draft and finalize final project  8/15/06    10/15/06
  Project Report        report.                                      

  Reporting to EPA      Reporting shall include           12/30/04   9/30/06
                        quarterly, annual and final                  
                        reports.                                     

                                                                     
  ------------------------------------------------------------------------------

### Geographic focus. 

This is a statewide project for the state of Vermont.

### Resource and time constraints. 

## A7. Quality Objectives and Criteria

### Detailed performance measures. 

This project is primarily interested in the following draft performance
measures. Note that one of the tasks of this project involves revisiting
and reaffirming/revising these performance measures.

-   Improved compliance with UST, RCRA, CAA Stage I & II, and SDWA UIC
    Rules

-   Improved facility understanding of UST O&M requirements as measured
    by facility self-certification accuracy

-   Delivery of at least five outreach workshops in different areas of
    state

-   Delivery of at least 1,200 multi-media compliance guides for sector

-   Development of a well-documented methodology for transfer to other
    sectors/states

-   Reduced hazardous waste generation as measured by a reduced need to
    empty sumps and spill buckets, and reduced air emissions by
    increased compliance with vapor recovery requirements.

-   Reduced incidence of petroleum product in sumps, spill buckets,
    interstitial space -- such incidence is the equivalent of a system
    operating without spill and overfill protection (because they are
    already full or have inadequate sensor placement) or without
    secondary containment.

### Quality objectives. 

Quality objectives for these performance measures will be developed as
part of the Measures Identification and Statistical Methodology tasks.
Specific quality objectives for these measures as a group (and, if
necessary, individually) will be provided in the anticipated revision to
the QAPP.

This revision to the QAPP will ensure that the quality objectives for
these performance measures are appropriate for the decisions to be made
based upon those measures. This determination will take into account
best practices for similar projects and resources available for this
project. In part, the Project Manager will rely upon EPA\'s *Generic
Guide to Statistical Aspects of Developing and Environmental Results
Program* (2003) for advice in making decisions related to the optimizing
the following aspects of data quality for this project:

-   Precision

-   Bias

-   Representativeness

-   Completeness

-   Comparability

-   Sensitivity (if applicable)

## A8. Special Training/Certification

The VT DEC and the selected ERP Contractor (TBD) will develop and
deliver mandatory and voluntary training sessions to key parties to
ensure quality data collection, to the extent practicable.

Mandatory intensive in-person training sessions will be delivered to the
following individuals to ensure quality data collection:

-   inspectors who will be collecting baseline and post-certification
    data

-   data-entry personnel who will be processing data from inspections
    and self-certification responses

-   QA/QC personnel (if any additional training is needed to familiarize
    them with the project)

-   Individuals who will be compiling the database containing the
    universe of facilities

Each session will cover proper data collection and QA procedures.
Training will be augmented by debriefing personnel shortly after their
tasks have begun, to correct and clarify appropriate practices.

Voluntary, intensive, in-person training sessions will be offered to the
self-certifying facilities. Facilities will also be provided with clear
written instructions on how to prepare and submit data.

The Project Manager is responsible for ensuring that all personnel
involved with data generation (including state personnel, contractors,
and partners) have the necessary QA training to successfully complete
their tasks and functions. The Project Manager will document attendance
at all training sessions. Attendance records for voluntary trainings may
not include names, given privacy/confidentiality concerns.

The Project Manager is also responsible for ensuring the
self-certification materials sent to facilities clearly document how
facilities should properly prepare and submit their data.

## A9. Documents and Records

[]{.mark}

### Report format/information. 

The format for all data reporting packages will be consistent with the
requirements and procedures used for data validation and data assessment
described in this QAPP.

### Document/record control.

The recording media for the project will be paper, electronic, and
photographic. The project will implement proper document control
procedures for all. For instance, hand-recorded data records will be
taken with indelible ink, and changes to such data records will be made
by drawing a single line through the error with an initial by the
responsible person. The Project Manager will have ultimate
responsibility for any and all changes to records and documents. Similar
controls will be put in place for electronic records and both digital
and film-based photographic records.

The VT DEC Quality Assurance Officer shall retain all updated versions
of the QAPP and be responsible for distribution of the current version
of the QAPP. The VT DEC Quality Assurance Officer and the VT DEC Project
Manager will approve annual updates. The Project Manager shall retain
copies of all management reports, memoranda, and all correspondence
between the VT DEC and all project personnel identified in A4.

### Other records/documents. 

Other records and documents that will produced in conjunction with this
project include:

-   Inspection checklists and reports

-   Self-certification forms

-   Return-to-compliance forms

-   Non-applicability forms

-   Enforcement documentation

-   Facility outreach materials, including workbook, fact sheets,
    brochures, etc.

-   Revised QAPP

-   Readiness reviews (see below)

-   Data handling reports[^1]

-   Quarterly and annual progress reports to EPA

-   Project final report (to include discussion of QA issues
    encountered, and how they were resolved)

### Storage of project information. 

Project information will be stored: (1) in paper files located at the VT
DEC offices in Waterbury, VT; (2) in electronic files in various
locations according to VT DEC standard operating practices; (3) audio
and video tapes located at the VT DEC offices in Waterbury, VT; and (4)
photographs located at the VT DEC offices in Waterbury, VT. All project
records shall be retained for three years or longer as determined by
agency records retention policies and procedures.

### Backup of electronic files. 

### 

This section describes the specific data management process, tracing the
path of the data from their generation to their final use or storage.

Many of the VT DEC data files are located on a central file server
located on the Agency network, which is behind the Agency firewall. The
file server houses production data for many of the Agency's programs,
specifically those programs that send data to US EPA. The file server
also houses program-related files.

All ANR servers are all located in a secure, environmentally controlled
server room, and attached to separate UPS devices and tape backup
systems. Tapes are made, stored, and rotated on a daily, monthly, and
yearly cycle, with tape storage located in a separate State facility
off-site. Each server platform is equipped with RAID controllers as an
additional means of fault tolerance.

Only network and database administrators and their supervisor(s) have
access to the server room through a lock and key mechanism.
Administrator user IDs and passwords to the servers are only know to the
administrators. User IDs and passwords are assigned to all staff who
require access to all applications running on the servers. No separate
userid/passwords are required for the various applications on the
servers since the Agency employs a single network login approach.

###  

# B DATA GENERATION AND ACQUISITION

## B1. Sampling Process Design (Experimental Design)

A key task in this project will be to develop a sound statistical
methodology for collecting and analyzing facility data, in order to draw
inferences related to the selected performance measures. The major
quality objective will be to collect representative data that truly
reflect the conditions of the target population of facilities. Facility
data is of three types: (1) statistical inspection data, which will be
collected by trained VT DEC inspectors from randomly sampled facilities;
(2) judgmental inspection data, based on inspections targeted from
inconsistent responses to self-certification questions; and (3)
self-certification data[^2], which will be collected from facilities
through a mail survey process. Facilities will be required to respond,
so this step is similar to a census. While the precise methods are not
known at this point, they are expected to be built upon the advice given
in EPA's *Generic Guide to Statistical Aspects of Developing and
Environmental Results Program* (2003).

This section of the QAPP will be revised upon completion of the
project-specific statistical methodology.

## B2. Sampling Methods

This section of the QAPP will be revised upon completion of the
project-specific statistical methodology.

## B3. Sample Handling and Custody

Upon completion of paper checklists, inspectors will sign the
checklists. Inspectors will either enter data directly to an electronic
format (such as tablet PC's and PDAs), or from paper checklists into the
electronic database. Facilities will either enter self-certification
data into web-based forms, or mail signed forms to the VT DEC where
data-entry staff will input data into the electronic database.
Electronic signatures would need to be used in connection with facility
self-certifications that are entered into web-based forms by the
facility. This section of the QAPP will be revised upon completion of
electronic data input/management strategies.

Chain of custody is not relevant to this project.

## B4. Analytical Methods

This project will follow well-recognized statistical analytical methods.
This section will be revised upon completion of the detailed statistical
methodology. No physical tests or chemical analyses are anticipated for
this project.

## B5. Quality Control

This project will undertake both proactive and corrective quality
control.

### Preparation of data collection instruments. 

All data collection instruments will be subject to multiple rounds of
review by relevant internal and external stakeholders to help assure the
collection of high-quality and representative data. Data collection
instruments will be prepared in accordance with the guidance on data
collection instruments provided in EPA\'s *Generic Guide to Statistical
Aspects of Developing and Environmental Results Program* (2003).
Specifically, preparation will follow the checklist for data collection
instruments provided in an appendix of that guide.

### Training.

As noted elsewhere in this QAPP, steps will be taken to provide
appropriate QA/QC training to all implementing personnel, particularly
personnel that are collecting and processing data. Facilities receiving
self-certification packets will also receive instructional materials on
how to complete their forms, an opportunity to attend workshops that
will explain how to complete the forms, and a phone number to call to
ask questions.

### Crosschecking data.

Primary data collection forms will be designed in such a way to allow
internal crosschecking of data by comparing answers of different
questions to each other, and such crosschecking will be automatic for
electronically entered data. Further, post-certification inspections
will offer the opportunity to compare inspection results with
self-certification results, if the facilities sampled have submitted
self-certification forms.

### Data anomalies.

Procedures for handling data anomalies (such as outliers and missing
data) will be handled based on guidance prepared in the project-specific
statistical methodology.

### Data entry.

Procedures for entering hand-written data into the database will follow
standard quality control procedures (e.g., 100% verification using
independent double key entry). Detailed quality control procedures for
data entry and acceptance will be prepared during the development and
implementation of a data management strategy. The final QAPP will
reflect the strategy.

## B6. Instrument/Equipment Testing, Inspection and Maintenance

This section is not relevant to this project. The project will not
involve such scientific instruments and equipment.

## B7. Instrument/Equipment Calibration and Frequency

This section is not relevant to this project. The project will not
involve such scientific instruments and equipment.

## B8. Inspection/Acceptance for Supplies and Consumables

This project will use photographic film as the only supply or consumable
requiring inspection and acceptance procedures. All film will be
required to have an expiration date, and film that is beyond that date
will not be used. Any film in damaged or otherwise corrupted packaging
will not be accepted for use.

## B9. Non-Direct Measurements (i.e., Secondary Data) 

This project will rely upon secondary data to identify the facilities in
the target population.

  ---------------- ------------------- ----------------- --------------------
  **Table 3:                                             
  Non-Direct                                             
  Measurements                                           
  (I.e., Secondary                                       
  Data)**                                                

                                                         

  **Data Sources** **Intended Use**    **Rationale for   **Acceptance
                                       Use**             Criteria**

  VT DEC UST       Identifying the     Commonly accepted All records will be
  program database target population,  source of         accepted unless
  of facilities    for the sample      facility list     sample response
                                                         indicates facility
                                                         should not be part
                                                         of target
                                                         population. VT DEC
                                                         will crosscheck any
                                                         facility that
                                                         self-identifies as
                                                         non-applicable to
                                                         this project.

  VT DEC facility  Identifying         Commonly accepted All records will be
  file             background          source of         accepted unless
  information -    conditions at       compliance        placed in file in
  UST, APCD, UIC,  facilities where    information,      error. VT DEC will,
  RCRA             such information    photographs,      on a random basis,
                   could provide       statements by     crosscheck
                   greater precision   facility owners   self-certification
                   to the analysis of  and operators     conditions and
                   background or                         baseline conditions
                   baseline conditions                   with multi-media
                                                         file information.

  Release Data     Identifying         One accepted      Since this is not
  from VT DEC      reduction in        source of release representative data,
  Sites Management prevalence and      data              this will be a
  database         severity of                           project area to
                   releases                              explore. This will
                                                         be a challenge to
                                                         address in the QAPP
                                                         revision process

  Inspection data  A portion of the    More complete     Data will be
  that is not      facility compliance compliance        accepted unless
  observed         data will be that   information.      conditions or
                   which is supplied                     information indicate
                   to inspectors by                      inconsistencies. VT
                   facility personnel,                   DEC inspection
                   but not observed.                     personnel will use
                   An example would be                   available
                   a description of an                   crosschecks from
                   activity that is or                   observed data.
                   is not compliant,                     
                   but the inspector                     
                   does not observe                      
                   the activity                          
                   itself. Provides a                    
                   more complete                         
                   compliance picture.                   

  Return to        Measures of         Commonly accepted Data will be
  compliance data  facility            form of           accepted unless
  from facility    performance         compliance        conditions or
                                       certification     information indicate
                                                         inconsistencies. VT
                                                         DEC personnel will
                                                         use available
                                                         crosschecks from
                                                         data observed on
                                                         follow-up
                                                         inspections.
  ---------------- ------------------- ----------------- --------------------

### Key resources/support facilities needed.

VT DEC will require access to the data sources mentioned above, and this
information will be managed within the database created/utilized for the
overall project. VT DEC does not anticipate any obstacles to this
approach.

### Determining limits to validity and operating conditions. 

Database containing the list of targeted facilities will be designed
such that the original source for all facility data is marked, and
procedures will be in place such that only the Project Manager can
officially remove a facility entry from the target population. In such
cases, facility entry will not be deleted from the database but will be
marked as non-applicable, and corrective data will be provided in fields
parallel to the original data.

[]{.mark}

## B10. Data Management

As part of this project, VT DEC and the ERP Contractor will develop a
data management strategy, and revise the QAPP based upon the strategy.
The Project Manager is responsible for ensuring that that strategy is
developed and that the QAPP is revised to reflect that strategy. Once
revised, this QAPP section on data management will provide information
on the following issues:

-   Data management scheme, from field to final use and storage

-   Standard recordkeeping and tracking practices, and document control
    system (citing relevant agency documentation)

-   Data handling equipment/procedures that will be used to process,
    compile, analyze, and transmit data reliably and accurately

-   Individuals responsible for elements of the data management scheme

-   Process for data archival and retrieval

# C ASSESSMENT/OVERSIGHT

## C1. Assessment and Response Actions

The Quality Assurance Officer will conduct a Readiness Review
immediately prior to the five major data collection tasks: identifying
targeted facilities, baseline inspections, self-certification, targeted
follow-up, and post-certification inspections.. The QA Officer will
report findings to the Project Manager, who will take corrective action
(if any is necessary) before the data collection task begins. Further,
the Project Manager and QA Officer will thoroughly debrief project
implementation staff a short time after beginning their respective
implementation tasks, to identify emerging/unanticipated problems and
take corrective action, if necessary.

## C2. Reports to Management

Three kinds of reports will be prepared: readiness reviews (described
above), regular quarterly and annual progress reports, and project final
report. Progress reports will note the status of project activities and
identify whether any QA problems were encountered (and, if so, how they
were handled). Project final report will analyze and interpret data,
present observations, draw conclusions, identify data gaps, and describe
any limitations in the way the data should be used.

**Project QA Status Reports**

  ----------------- ----------------- ----------------- -----------------
  **Type of         **Frequency**     **Preparer**      **Recipients**
  Report**                                              

  Revised QAPP      Once, before      VT DEC Project    All recipients of
                    primary data      Manager           original QAPP
                    collection begins                   

  Readiness Review  Before each major VT DEC QA Officer VT DEC Project
                    data collection                     Manager, Project
                    task                                Manager
                                                        Supervisor

  Progress Report   Quarterly         VT DEC            U.S. EPA

  Progress Report   Annually          VT DEC            U.S. EPA,
                                                        stakeholders

  Final Project     Once              VT DEC            U.S. EPA,
                                                        stakeholders
  ----------------- ----------------- ----------------- -----------------

# D DATA REVIEW AND EVALUATION

## D1. Data Review, Verification and Validation

The QAPP shall govern the operation of the project at all times. Each
responsible party listed in Section A4 shall adhere to the procedural
requirements of the QAPP and ensure that subordinate personnel do
likewise.

This QAPP shall be reviewed at least annually to ensure that the project
will achieve all intended purposes. All the responsible persons listed
in Section A4 shall participate in the review of the QAPP. The Project
Manager and the Quality Assurance Officer are responsible for
determining that data are of adequate quality to support this project.
The project will be modified as directed by the Project Manager. The
Project Manager shall be responsible for the implementation of changes
to the project and shall document the effective date of all changes
made.

It is expected that from time to time ongoing and perhaps unexpected
changes will need to be made to the project. The Project Manager shall
authorize all changes or deviations in the operation of the project. Any
significant changes will be noted in the next report to EPA, and shall
be considered an amendment to the QAPP. All verification and validation
methods will be noted in the analysis provided in the final project
report.

[]{.mark}

## D2. Verification and Validation Methods

To confirm that QA/QC steps have been handled in accordance with the
QAPP, a readiness review will be conducted before key data
collection/analysis steps, and data handling reports will be prepared
after each step. Standard statistical tests (described below) will be
used to determine the extent to which inferences can be drawn from the
sample data.

## D3. Evaluating Data in Terms of User Needs

This section will be written and finalized after completion of the
project-specific statistical methodology. This section will present the
following information:

-   **Meeting and reporting needs of your project:** a description of
    how the results of the study will be analyzed and evaluated to
    determine whether the needs of the project were met and then
    reported

-   **Mathematical and statistical formulae**: details of formulae that
    will be used to calculate precision, accuracy/bias, completeness,
    comparability and sensitivity (if applicable) of the project data.

-   **Approach to managing unusable data:** Description of what will
    happen if data are unusable, with particular emphasis on the impact
    of such unusability on data representativeness.

[^1]: These records document protocols used in data reduction,
    verification, and validation. Data reduction addresses data
    transformation operations such as converting raw data into
    reportable quantities and units, use of significant figures,
    recording of extreme values, blank corrections, etc. Data
    verification ensures the accuracy of data transcription and
    calculations, if necessary, by checking a set of computer
    calculations manually. Data validation ensures that QC criteria have
    been met.

[^2]: Includes data from self-certification forms, return-to-compliance
    forms, and non-applicability forms.