# Presentation: 448057

## Mike Papp, Joe Elkins, Louise Camalier   EPA - Office of Air Quality, Planning and Standards 

- Precursor Gas Monitoring
- Quality System Development

## Outline

- Quality System Development
- Data Quality Objectives
- Measurement Quality Objectives & Calculations
- Guidance/Training
- Assessments Activities
- Monitoring Org QA responsibilities
- Preliminary Data Collection-Pilot Program
- QAPP Development
- Timetable

## What is a Quality System ?

- A structured and documented management system describing how and by whom an organization assures quality in its work.

**What is a Quality System ?**

- Protect human health and the environment
- Make correct decisions
- Optimize resource use
- Ensure that environmental programs and decisions are supported by data of the type and quality needed for their intended use

- Quality System Goals

**Notes:**
DISTRIBUTE: Handout 3.
EXPLAIN:  This is a diagram of the structure of the EPA Quality System..
EXPLAIN:  The EPA Quality system is based on consensus standards that embody policies for internal EPA organizations, external organizations, and the  American national consensus standard (ANSI).
EXPLAIN:  The EPA Quality policy has 3 functional levels with the basic components of the Quality System being implemented at each level.
EXPLAIN: The policy level establishes the authority and the specifications for the quality system.
EXPLAIN:  The responsibility for planning and implementing the Quality System occurs at the organizational level.  There are 5 required Quality system components here.
 EXPLAIN:  QA System documentation. (The requirement to document the system using a QMP or equivalent document.)
EXPLAIN:  Annual Reporting and Planning of QA activities and budget needs. (The reporting requirement for the QAARWP or related document.) 
Systems Assessments which addresses the requirement of assessing the overall system at least annually. (Assessments that evaluate system effectiveness.)
Training and Communication which ensures that the staff are adequately trained to carry out their duties and that the requirements of the Quality System are adequately communicated.
Supporting Systems Elements which includes those items that can directly impact the quality of environmental data operations, although it may not be immediately obvious.  This includes procurement and computer hardware and software.
EXPLAIN: At the program level, project implementation occurs.  Each component/activity at this level addresses the planning, implementation, or assessment of the environmental data collection activity.

## Where we are, and where are we?

## 1st Stop Systematic PlanningDQOs

**Designed to answer:**
- What do you need?
- Why do you need it?
- How will you use it?
- What is your tolerance for errors?

- All collected data have error.
- Nobody can afford absolute certainty.

**DQO**

**Underlying Principles:**

**Notes:**
EXPLAIN: The DQO process allows the user to answer these specific questions.  It  provides a logical flow for problem solving.

EXPLAIN: In the past, the term DQO has been confused with data quality indicators (precision, accuracy, detection limits).  However, it is important to remember that, while these issues are important to the decision-making process, the purpose of the DQO process is to clearly link sampling and analysis efforts to an action and a decision.

EXPLAIN: The "new" thing about the DQO process is the concept of trying to estimate or control the chance of making a decision error. 

## Slide 6

**Notes:**
Population – Spatial and Temporal uncertainties –   Completeness could effect temporal uncertainty
Some think we should include some spatial uncertainty component

In general, our population parameters are set 
     we don’t question the variability of the site selection and number of sites

Measurement uncertainty is what the QA Staff in MQAG tries to assess and control.

## DQO Promotes Communication

## DQO Game Plan

- Form Workgroup – Done
- 1st Meeting May 5th
- Move through process  (5-9, 2005)
- Report progress
- Parallel Activity
- Gather data on population and measurement uncertainty to support DQOs

## 2nd Stop of Systematic Planning

- The quality control samples and/or performance criteria that provide for an estimate of a data quality indicator for:
- 	- the overall data collection effort
- - a measurement phase
- Performance criteria is established to:
- 	- control data quality
- 	- to meet program DQOs
- 	- can be used to develop validation templates

**2****nd**** Stop of Systematic Planning**
**Measurement Quality Objectives**

**MQOs**

**Notes:**
Population – Spatial and Temporal uncertainties –   Completeness could effect temporal uncertainty
Some think we should include some spatial uncertainty component

In general, our population parameters are set 
     we don’t question the variability of the site selection and number of sites

Measurement uncertainty is what the QA Staff in MQAG tries to assess and control.

## Overall Measurement Uncertainty MQOs

| Data Quality Indicator | QC Type | Comment |
| --- | --- | --- |
| Detectability | MDL standard | 40 CFR Part 136 App B |
| Precision | 1 Point QC Check | 90% Upper Confidence Limit of CV |
| Bias | 1 Point QC Check | 95% Upper Confidence Limit Absolute Value |
| Completeness | Routine Data | Percentage of Collected from Expected  |

## Detectability

- Detection limit is important for this program
- EPA recommends that monitoring orgs. perform MDLs
-    ***MDL:*** *the minimum concentration of a substance that can be measured and reported with 99% confidence that the analyte concentration is greater than zero*
- Using 40 CFR Part 136 App B as a start
- Provides a DL under routine conditions
- Frequency to be determined

## MDL Calculation

- Use a concentration of 2.5 to 5 times the instrument signal/noise
- Run zero gas through analyzer
- Dilute the gas to estimated concentration level and collect readings for a predetermined length of time:
- Suggested: 20-25 1 minute observations, repeated 7 times over the course of 5 -14 days. Average the concentration from these readings.
**Calculate the MDL as:**

- Where               represents the 99th quantile of a Student’s t distribution with (n-1) degrees of freedom and n represents the number of replicate measurements and *s* is the standard deviation.

## Precision and Bias Estimates Start the Same Way

##                   Precision“A measure of mutual agreement among individual measurements of the same property, usually under prescribed similar conditions”

-                   Precision“A measure of mutual agreement among individual measurements of the same property, usually under prescribed similar conditions”

- This represents  a 90% upper confidence limit on the CV estimate

##        Bias“A systematic or persistent distortion of a measurement  process which causes errors in one direction”

-        Bias“A systematic or persistent distortion of a measurement  process which causes errors in one direction”

- Absolute Bias Point Estimate:
- Absolute Bias Upper Bound:

***Where t ******0.95,(n-1)****** is the 95th quantile of a Student’s t distribution with n-1 df and s******d_abs ******is the standard deviation of the absolute value of the relative percent differences***

- This represents  a 95% upper confidence limit on the Bias estimate

## Associating a Sign to the Absolute Bias

- A sign (+/-) is associated with the absolute bias only if the 25th and 75th percentiles of the relative differences have the ***same sign***

| BIAS (%) |  |
| --- | --- |
| CO (Sampler A): |   |
| OLD BIAS | NEW BIAS |
| -0.7 | 14.6 |
| CO (Sampler B): |   |
| OLD BIAS | NEW BIAS |
| -22.9 | -27.7 |

## Initial Estimates from OAQPS

| Make/
Model | API
(CO) | Thermo (CO) | API 
(SO2) | Thermo (SO2) | Thermo (NOy) |
| --- | --- | --- | --- | --- | --- |
| MDL Obs. | 20 ppb
(15-31 ppb 
CL) | 17 ppb
(13-26 ppb 
CL) | 420 ppt
(330 – 670 
ppt CL) | 230 ppt
(180 – 360 
ppt CL) | 46 ppt
 (35-73 ppt CL) |
| Precision Obs. | 18.3% | 24.3% | 1.3% | 1.3% | 2.2% |
|  Bias Obs. | 14.6% | -27.7% | +6.26% | +11.3% | +11% |

-                      (in Lab)

## Not to Worry

- OAQPS will be developing a guidance document of the Statistics
- Including data examples and Excel spreadsheets for use
- Should be available in AQS for reporting/assessment as needed.
- Calculated in the DAS systems?

## Guidance /Training

- QA Guidance- QA Handbook Volume II
- Revised upon completion of  DQOs/MQOs.
- May develop stand alone guidance depending on PMcoarse
- Training
- Will follow lead of monitoring group to include:
- Hands on at Regional forums
- APDLN segments
- Other EOG venues

## EPA Assessment Activities

- National Performance Audit Program –Through The Probe (TTP)
- DQO/MQO will determine need and frequency of TTP
- Plan to develop one audit vehicle to test technical capability.
- Technical Systems Audits
- Will occur at Regional level
- Frequency may/may not change
- National Standards Certification
- Proposal on table to certify precursor gas standards through ORIA
- Annual P and B Assessments
- On AQS

## Monitoring Organization QA Responsibilities

- Pilot Testing
- Help develop data set to get a handle on the data quality indicators.
- QA Project Plan Development
- Once DQOs/MQOs developed
- Revision of QAPP to include precursor gas  data quality requirements.

## Sequence of QA Events

- 05

- DQO Development

- DQO Validation

- 06

- 07

- MQO Development

- Guidance Development

- SLT QAPP Development

- 08

- Fiscal Year