# Presentation: 411147

## An Overview of the  Essential Elements of a PT/EQA Program

- Michael A. Noble MD FRCPC
- Chair, Clinical Microbiology Proficiency Testing program
- University of British Columbia, Vancouver Canada
- and
- Division of Laboratory Systems, CDC

**Notes:**
Slide 1
Many thanks to the organizers for giving the opportuniry to chat with you today.  My presentation today will be an overview on the topic of setting up and operating an EQA program.  As some of you may be aware, my own experience in chairing an EQA or PT program spans a little less than 20 years addressing a variety of topics directly related to clinical and environmental microbiology.
While there will be information, most will not be of full detail.  The speakers also on the schedule will provide much more specific information.

## Laboratories monitor Quality

- To ensure their information is accurate
- To ensure their information is timely
- To ensure their information is appropriate
- To ensure their information is interpretable
- To demonstrate they provide quality patient care

**Notes:**
Slide 2
Allow me to start at the very beginning.  Why odes EQA exist?  What is the point of the exercise?  The answer is because it provides a measurement tool for measuring laboratory quality.  Measuring quality is something that all laboratories do.  Laboratories need to ensure that the information they generate and provide is accurate, timely, clinically appropriate, and useful.  If it is we can say that the laboratory is proficient; if it is not, we say the laboratory has an opportunity for improvement.

## Self

- Monitoring Laboratory Quality

- External
- Inspections

- PT/EQA

**Self**

**Inspection**

**Notes:**
Slide 3
Laboratories have a number of devises that can be used to monitor quality.  Laboratories can look at complaints that regularly get generated by patients or physicians.  They can look to reports created by  accreditation or other official body reports, or they can create their own examination timetimes and review their own procedures without outside prompt. If a program does exist, the laboratory can engage an EQA program.

## What is PT/EQA 

- A quality monitoring scheme
- Delivery of samples that closely simulate clinical materials of known composition
- A technique to challenge a laboratory’s routine methods and procedures.
- To program to assess laboratory performance and the ability to determine the “correct” result.
- To program to assess the overall ability and performance of a group of laboratories.

**Notes:**
Slide 4
Taken at its most basic, PT programs providedsamples of known unknown composition, with the expectation that they will be able to generate a proper laboratory report.

## The Benefits of PT/EQA

**Health Care System**
- A valuable assessment tool
- An indicator of laboratory performance
- A mechanism to provide notice and education

**The Laboratory**
- A valuable tool for: education
- Self assessment
- System confidence
- Public confidence

**The Patient**
- Knows the system is working together to ensure the public receives accurate and useful information

**Notes:**
Slide 5
Why perform such a service?  Monitoring EQA provides benefits to all involved.  Done properly, It truly create a win-win-win result.  

## The Rules for the PT/EQA Game

- 

- How to get the mostbenefit from proficiency testing samples.

**Notes:**
Slide 6.  
The challenge is to us is how to provide the best program respecting the rules for programs.

## Rules for PT/EQA Providers

- United States: Clinical Laboratory Improvement Amendments (1988)
- World Health Organization (WHO)
- 	Requirements and Guidance for External Quality  Assessment Schemes for Health Laboratories 1999
- International Organization for Standardization (ISO)
- 	ISO/IEC Guide 43-1: 1997 (E)Proficiency testing by interlaboratory comparisons
- No specific rules set in Canada

**Notes:**
Slide 7
A number of organizations have laid down specific requirements for operating EQA programs.  The include the WHO, ISO and the United States.  As far as I am aware, only the United States has made following specific requirements mandatory.

## ISO/IEC Guide 43-1: 1997 (E) Proficiency testing by interlaboratory comparisons 

- 	Part 1: Development and operation of proficiency testing schemes

**5.5.3**
- The test items or materials to be distributed in the scheme should ***generally be similar in nature*** to those routinely tested by participating laboratories
**6.2.4**
- Participants should be advised to treat proficiency test items ***as if they were performing routine tests*** (unless there are some special requirements in the design of the proficiency test which may require departure from this principle).

**Notes:**
Slide 8
These guidelines or regulations tend to have a lot in common.  All of them speak to quality management and confidentiality of information.  Another aspect clearly defined is the importance of ensuring that the materials sent to the participating laboratories similar in nature to the materials routinely tested.  Another aspect is the programs are encouraged to urge their participants to use the same procedures as they normally use to process routine samples. 
An example of this not out recently happened with our program.  A few years ago we received a letter  from an irate participant who said he had spend over $5000 working through one of our simulated urine samples.  They had identified all 5 organisms present, had performed susceptibilities, and genetically typed all the gram negative bacilli.  We wrote back to them an told them while were most certainly interested in their results, we had anticipated a result of “mixed flora”, please repeat, and enquired why they would consider doing such excessive work on an obviously contaminated specimen.  They responded with apology that in their normal routine they never would have worked up any specimen to a similar degree.  They have since changed their approach to proficiency samples.

## PT/EQA Business Plan

**Is there a need?**
- 	Are the laboratories ready for PT?
**Is there a demand?**
- 	 Is there already an existing, operating satisfactory PT program?
**Is the demand specific?**
- 	Which analytes need to be challenged?
**Does the demand have limitations?**
- 	Cost, Resources, Time, Number of samples
**Can you meet those demands and limitations****	**mandated authority	resources
**Are there options to developing a new program**
- 	Purchase service		Co-share	No program

**Notes:**
Slide 9
Before embarking on establishing a program, please be aware that EQA takes a lot of time, effort, energy and money.  It is important to not embark on this journey lightly.  All programs must develop a realistic business plan to ensure the new program is filling a need and demand, and that the resources are available.

## Minimum Organizational Requirements

- Staff
- Experience and understanding of the clinical laboratory
- A **commitment** to quality management
- A **commitment** to detail
- A **commitment** to confidentiality and privacy
- Facilities and Equipment
- A safe and secure facilities which may include on-site laboratory
- Safety appropriate to the materials being handled.
- Reliable, versatile, and flexible communication capabilities
- Network
- Viable and stable client group
- Reference groups and individuals
- Licensing and accrediting bodies
- Regulatory authorities
- Funding source.
**Political will**

**Notes:**
Slide 10
As a minimum, EQA programs need personnel, facilities, equipment, and a network of clients and the ability to reach them.

## EQA cycle

**Notes:**
Slide 11
For the rest of this presentation I would like to briefly take you through the EQA cycle.  Just as the laboratory cycle is divided into the pre-analytic, analytic, and post-analytic phases.  A similar structure is seen with the EQA cycle. 

## Challenge Selection Issues

- 1:	Which analytes to challenge
- 2:	Challenges per send-out survey
- 3:	Send-outs surveys per year
- 4:	What range to consider
- 5: 	What complexities to incorporate

**Notes:**
Slide 12
In this cycle we begin with selecting the challenges that the program is going to submit to the participants.  Key questions need to address, what, how often, how many, what range, and what complexity.  The decisions are best made by a collective group of experts.  In my program, a committee of participants in involved in this decision process.  Issues such as regulatory requirements, client requirements, and prevailing standards and program feasibility will be a necessary part of this discussion. 

## Purchase or Produce

**Purchase**
- Easier
- Less expensive
- Import and Transport issues
- May not address regional issues

**Produce**
- Meet specific needs
- Meet regional needs
- Ensure quality
- Need Infrastructure
- Requires larger staff budget
- Need special location and equipment

*Sometimes, the best plan is a** ****blended co-share solution****.*

**Notes:**
Slide 13
The next critical decision is whether it is better to purchase the materials that your program wants to challenge, or make them yourself, or consider a combination of the two.  Clearly there are advantages and disadvantages to all three sides.  Each program ultimately will make the decision on their own.  As a guide, I only encourage you to consider only materials that will address the specific needs for your participants, in your geographic local.

## Sample Demons

- Nonrelevancy

- NonReliability

**Notes:**
Slide 14
Samples that are not relevant to your participant group, or are so artificial that they can not be worked on using routine procedures, or are inconsistent or not reliable are in my opinion a waste of time 

## Quality Control and Validation are integral to program credibility

- Are we sending what we think we are sending?
- Have the samples been altered in production?
- Did we send the same thing to all laboratories?
- Did all laboratories receive the same thing?

**Notes:**
Slide 15
Regardless of source,  all materials must be validated before they are sent out.  You must be confident of what you have sent out, and that you know what result you expect to get upon return.  All samples should be the same both when you sent them and also when they were received.

## Quality Control for  PT/EQA  Samples:Purchase and process

- Process Controls			  /p
- Detection-oriented controls	P/p
- Internal confirmation control	P/p
- External confirmation control	P/p
- Post survey retrieval and re-test	P/p

- Incubator,Refrigerator
- Spectrophotometer

- Gram checks
- Parasites

- Reference Laboratories and Group  Results

- Contamination and Growth Failure ISO MIL-STD 105E

- Trial Runs to Determine Sample Target

**Notes:**
Slide 16
In our program, we tend to make most of our own materials, and for us we do a 5 point quality control procedure.  We monitor process controls, and do practice runs to ensure that we can meet a specified target.  We check our products ourself, and submit them to reference laboratories outside our program to confirm.  Finally when we get significant answers amiss, we get the sample back to determine if the discrepancy in our fault.

## Challenge Validity

- Is the challenge fair?

- Validity

- Open
- Selection

- Material
- Validation

- Internal
- Q.C.

- 80% Reference
- Laboratory Agreement

- 50% Group Agreement

**Notes:**
Slide 17
All challenges must be validate for interpretation and fairness.  If 80% of our reference group can not get the right answer, we become suspect of the sample and the question we asked.  If 50% of the complete participant group can not get the right answer, we would not consider the fair challenge.

## Reference Laboratories

- CMPT Reference Pool3-5% of total group10% of advanced group

- A group of laboratories selected for being the most technically sophisticated and least likely to make errors.  If this group can’t get the correct answer, it is unreasonable to expect expert performance from other laboratories.

**Notes:**
Slide 18 
In our program, the reference group of laboratories is selected from the whole group.  Other programs can use another process.

## Communication pathways

- Are critical

**Notes:**
Slide 19
EQA programs are shippers of materials, and at the same time receivers and suppliers of information.
Shipping of biological materials must be safe, secure, and timely to ensure stability.

## Sample Packaging

- 	In many countries and for international transport, primary packaging requirement  is known as UN Standard Packaging.

**Water-tight**
**container**

**Contained Absorbent**

**High Resistance**
**Container**

**Box with Markings ****and Documentation**

**Notes:**
Slide 20
Packaging of all biological materials must comply with prevailing regulations.  Fortunately for us in Canada, we have one set of requirements for domestic transport, and another for international.  An  EQA program cannot afford to be in conflict with regulation.

## Post Analytic Data Management

- Results Capture
- Results Analysis
- Results Summation
- Results Reports Generation.
- Reports Transport to Laboratories
- Reports Transport to Required Authorities
- Correspondence and Communications
- Education materials

**Notes:**
Slide 21
Once the participants have received and processed their samples, and returned their information to your centre, the critical steps need to be followed without error.  You will hear a presentation later on this topic.

## EQA Reports 

- We demand that laboratories must generate reports that are **TIMELY**, **ACCURATE**, and **UNDERSTANDABLE**.
- The laboratories, authority bodies, and ourselves can and must  demand the same of our own reports!!

## Educational Opportunities 

- PT programs have the unique platform to provide real-time and continuous laboratory education in a variety of formats

- Meetings &
- Conferences

- Newsletters &
- Websites

- Manuscripts

**Critiques ****&**
**Reports**

**Notes:**
22
And finally, but perhaps most importantly EQA programs are about education.  Every program even those heavily endowed with regulatory requirement provide opportunities for participants and program to learn new information.  EQA programs must grasp these opportunities.

## In Summary

**EQA  is a program that provides:****	****	****Quality management**
**	****	****Education**
**	****	****Research and Development**
**	****	****Social Responsibility**
**	****	****	**

**Notes:**
Slide 23
In summary, done well, EQA programs are about Quality management, about education, research and development, and social responsibility...

## In Summary

**Being a EQA provider demands:****	****	****	****	****Knowledge and Expertise**
**	****	****	****Commitment**
**	****	****	****Good Communication Skills**
**	****	****	****Respect for Confidentiality**

**Notes:**
Slide 24
Provided that the providers are knowledgeable, committed, communicate well, and respect confidentiality 

## EQA

- Education

- Information

- Quality Management