# Presentation: 225603

## Workgroup 4: Ensuring the Quality of PT/EQA Programs

- Bill Donohue
- Chris Hamilton
- Claus Heuck
- David Kelly

- Lloyd Penberthy
- Helen Pollock
- Harry Richardson
- Nick Serafy, Jr.

- Wkgp 4 -- Subgp 1

- Feb 26, 2002

## WHAT ARE THE ADVANTAGES AND DISADVANTAGES OF PT/EQA PROGRAMS THAT ARE MANAGED FOR EDUCATIONAL AND/OR REGULATORY PURPOSES?

- Regulatory programs should include an educational component
- Educational Approach
- Advantages: 1) _provides an educational component that may not otherwise be available_; 2) _samples can be more challenging_; 3) particularly advantageous for developing countries and for initial program start-up; 4) allows for less threatening consultation and follow-up to correct problems; 5) more timely response from program; and 6) _more likely to lead to improved quality_
- Disadvantages: 1) may not include poorly performing labs that could really benefit; 2) _labs may continue testing without improving performance_; 3) improvement may occur slowly; and 4) _perception of cost/benefit may result in lower participation_

- Wkgp 4 -- Subgp 1

- Feb 26, 2002

**Notes:**
4-1q1

## WHAT ARE THE ADVANTAGES AND DISADVANTAGES OF PT/EQA PROGRAMS THAT ARE MANAGED FOR EDUCATIONAL AND/OR REGULATORY PURPOSES?

- Regulatory approach
- Advantages: 1) mandated participation provides more data improving relevance of statistics; 2) _increased response rate_; 3) _increased public confidence_; and 4) _authority to enforce minimum consensus standards - laboratories are more responsive to improving the quality of testing following poor PT results_
- Disadvantages: 1) _samples are often  less challenging_; 2) labs may give samples additional attention/testing; 3) may lack meaningful educational component; 4) _evaluation criteria may be inappropriate_; 5) _sanctions in the absence of education are not useful_; 6) slow to respond to rapidly changing technologies; and 7) due process/regulatory process may lead to inappropriate public confidence and confidence in non-regulated analytes

- Wkgp 4 -- Subgp 1

- Feb 26, 2002

## HOW CAN THE HARMONIZATION AND STANDARDIZATION OsF PT/EQA PROGRAMS BE IMPROVED AT BOTH THE INTERNATIONAL AND NATIONAL LEVEL?

- Develop an international consensus framework for essential components of a quality EQA scheme; possibly through ISO Technical Committee 212 – a standard for EQA providers for medical labs (ILAC Guide 13 but more specific for medical labs) and a similar document for developing countries
- Hold regular EQA forums for sharing information and discussing the effectiveness of the consensus standards
- Develop mentoring and collaboration between programs
- When national regulations are developed or revised they should consider appropriate international standards
- Guidelines should include all common processes for a quality EQA program including quality management and technical aspects (resulting in a single/unified accreditation)

- Wkgp 4 -- Subgp 1

- Feb 26, 2002

**Notes:**
4-1q2

## WHAT ARE THE FACTORS THAT ENSURE THE SUSTAINABILITY OF PT/EQA PROGRAMS? 

- The most important factors in sustaining a PT/EQA scheme are:
- the political commitment needs to be evident (e.g., a legally-mandated scheme);
- adequate human, financial, and other resources; and
- confidence in an independent program developed through good communication, high-quality materials, and demonstration of the value of the scheme.

- Wkgp 4 -- Subgp 2

- Feb 26, 2002

**Notes:**
4-2q1

## TO WHAT EXTENT ARE TECHNICAL ISSUES AND FINANCIAL ISSUES IMPORTANT TO PROGRAM SUCCESS? 

- Political/Financial factors include: a) political will and financial commitment; b) the perceived value of the program; c) communication and documentation of the value; d) an element of accountability; e) use of recognized standards; f) absence of costs/fees; and g) use of partnerships
- Technical Factors include: a) availability of quality control materials; b) use of information technology; c) education and sustained training; and d) information management
- An intangible factor preventing sustainability is geographic barriers

- Wkgp 4 -- Subgp 2

- Feb 26, 2002

**Notes:**
4-2q1

## WHAT ORGANIZATIONAL ELEMENTS OF PROGRAMS HAVE BEEN SUCCESSFUL?

- Elements of successful program include:
- dedication, commitment
- continuity
- integration into everyday activities
- standardization, consistency
- documentation, procedure manuals
- supported by professions, government, industry, NGOs
- those that are legally-mandated
- ease of use, access to materials, information
- high-quality approach, confidence in the program

- Wkgp 4 -- Subgp 2

- Feb 26, 2002

**Notes:**
4-3q1

## WHAT ARE THE ADVANTAGES AND DISADVANTAGES OF HAVING PT/EQA PROGRAMS MANAGED BY A LABORATORY THAT IS LINKED TO A CLINICAL SERVICE?

- advantage is a higher level of resources, greater expertise
- disadvantages: 1) potential for conflict of interest; and 2) bias
- a linked program may be positive way to start but not to sustain (depends on money, resources)
- an independent approach is generally better but it requires higher levels of communication, greater resources

- Wkgp 4 -- Subgp 2

- Feb 26, 2002

**Notes:**
4-3q1

## SHOULD PT/EQA PROGRAMS BE RESPONSIBLE FOR DISCLOSING TO PARTICIPATING LABORATORIES INFORMATION ABOUT THE QUALITY OF THE MATERIALS THEY DISTRIBUTE?

- Proficiency testing providers should offer information regarding specimen source, special handling, and how it differs from patient specimens, where appropriate.  This information should be included with the specimen shipment.
- The PT provider should be responsible for performing internal checks prior to release of performance results to labs to validate the quality and integrity of the specimens.

- Wkgp 4 -- Subgp 3

- Feb 26, 2002

**Notes:**
4-3q2

## IF A USER SUSPECTS THAT THEIR POOR PERFORMANCE IS DUE TO POOR QUALITY SAMPLES, HOW CAN THEY DETERMINE IF THEIR ASSUMPTION IS CORRECT?

- Once a lab has ruled out internal factors contributing to poor performance, it should notify the PT provider that it suspects the specimen may have been compromised and ascertain if the provider has data to support the claim.
- It is appropriate for the PT provider to offer information including inter-laboratory comparisons, error trends observed with other laboratories, and trouble-shooting assistance.

- Wkgp 4 -- Subgp 3

- Feb 26, 2002

**Notes:**
4-3q2

## SHOULD PROVIDERS BE RESPONSIBLE FOR DESCRIBING HOW LABS SHOULD INTERPRET THEIR PERFOMANCE IN THE PROGRAM? IF SO, WHAT INFORMATION SHOULD BE AVAILABLE?

- Providers are responsible for giving such information as: 1) lab result; 2) mean; 3) target value; 4) standard deviation; 5) n-value; 6) coefficient of variation percent; 6) acceptable range; 7) composition of peer groups; and 8) a summary of all participants, as appropriate, for qualitative and quantitative analyses
- The responsibility for interpretation of performance results rests with the lab

- Wkgp 4 -- Subgp 3

- Feb 26, 2002

**Notes:**
4-4q1

## SHOULD PT/EQA PROGRAMS BE SELF-REGULATED OR UNDERGO EXTERNAL REVIEW?

- _External review required_: 1) to increase credibility/confidence within the program; and 2) to demonstrate competence to participants
- Recommend a limited number of standards for assessing PT providers; goal to have a single standard

- Wkgp 4 -- Subgp 4

- Feb 26, 2002

**Notes:**
4-4q2

## IF AN EXTERNAL REVIEW WAS CONDUCTED, WHO SHOULD CONDUCT SUCH A REVIEW?

- Establish a funding mechanism to support external review as required
- For-profit program - programs pay
- Developing Countries - Aid from agencies
- Government programs - seek targeted funding
- Goal of having external auditing administered by international lab administrations (e.g., WHO, CAREC, PAHO) in a partnership arrangement

- Wkgp 4 -- Subgp 4

- Feb 26, 2002

**Notes:**
4-5q1

## WHAT ARE THE RELEVANT QUALITY INDICATORS FOR PT/EQA PROGRAMS?

- Customer feedback via annual scientific meetings and appropriate surveys of services provided and required
- Professional overview by relevant professional organisations or steering committees to provide guidance and review areas such as: communication, scientific value, scope of testing and support services, frequency, staffing, turn-around-times, clinical relevance, etc.

- Wkgp 4 -- Subgp 4

- Feb 26, 2002

**Notes:**
4-5q2

## HOW CAN PT/EQA PROGRAMS EVALUATE THE QUALITY, CONSISTENCY, ACCURACY, AND DEGREE OF DIFFICULTY OF THEIR CHALLENGES?

- Use the External Review Standard or Guidelines as a basis of internal audits to document conformance, e.g., ILAC, ISO

- Wkgp 4 -- Subgp 4

- Feb 26, 2002

**Notes:**
4-6q1

## DOES PT/EQA HAVE A ROLE TO PLAY IN THE STANDARDIZATION OF ANLYTICAL SYSTEMS AND TEST METHODS?IF SO, WHAT ROLE SHOULD IT PLAY?

- _Yes_, PT/EQA organizers are uniquely aware of the range of performance by all users of a given analytical or test system
- The role should be to share information on the performance of analytical systems with both client labs and manufacturers

- Wkgp 4 -- Subgp 5

- Feb 26, 2002

**Notes:**
4-6q2

## SHOULD MANUFACTURERS OF INSTRUMENTS AND TEST KITS WORK WITH PT/EQA PROGRAMS TO ASIST IN STANDARDIZATION OF TEST SYSTEMS?

- Manufacturers should be able to join a PT/EQA scheme as a participant
- PT/EQA scheme organizers should provide information to the manufacturers to assist in the investigation and resolution of performance differences and/or bias
- PT/EQA scheme organizers should encourage collaboration between manufacturers and users of test systems to resolve performance issues that may exist specifically in resource-challenged countries

- Wkgp 4 -- Subgp 5

- Feb 26, 2002

**Notes:**
4-6q2

## SHOULD PT/EQA PROVIDERS ALSO INSPECT THE LABORATORIES PARTICIPATING IN THEIR PROGRAM?  WHAT ARE THE ADVANTAGES AND DISADVANTAGES OF SUCH INSPECTIONS?

- Yes, inspection may be an acceptable approach provided it is in the context of support of quality improvement
- Countries should be encouraged to develop a legislative and regulatory framework that governs diagnostic (medical/clinical) lab practice

- Wkgp 4 -- Subgp 5

- Feb 26, 2002

**Notes:**
4-6q2

## HOW COULD INSPECTIONS ASSIST LABORATORIES IN RESOURCE-LIMITED COUNTRIES?

- Inspections could provide opportunity for educational training that will improve the quality of the laboratory (the visits should not have a punitive intent)
- Inspection could:
- assist in understanding laboratories’ problems;
- carry out resource-needs assessment;
- help to develop SOPs;
- provide opportunities for knowledge and skill training;
- provide advice on using PT data; and
- form a basis for PT/EQA providers to develop partnerships with donor organizations to meet resource needs (reagents, equipment etc.)

- Wkgp 4 -- Subgp 5

- Feb 26, 2002

**Notes:**
4-6q2

## FOR WHAT TYPES OF TESTING WOULD IT BE BETTER TO RE-ANALYZE PATIENT SPECIMENS RATHER THAN SENDING OUT SAMPLES TO DETECT POOR PERFORMANCE?

- If no external PT/EQA is available – e.g., new technology (PCR) or where there is no gold standard (cytology)
- If both external and internal re-examination and PT/EQA are available both are beneficial because they fill different functions

- Wkgp 4 -- Subgp 6

- Feb 26, 2002

**Notes:**
4-6q2

## WOULD THIS BE A BETTER INDICATION OF ACTUAL LABORATORY PERFORMANCE?

- Advantages of internal re-analysis are: 1) easy start-up; 2) provides immediate results; and 3) valuable, especially in developing countries

- Wkgp 4 -- Subgp 6

- Feb 26, 2002

**Notes:**
4-6q2

## HOW WOULD THE COST COMPARE WITH TRADITIONAL PT/EQA? 

- Traditional PT/EQA lends itself to automation with information technology resulting in lower costs
- Costs are the same if not more to do re-examination because both external monitoring and internal re-examination incur costs

- Wkgp 4 -- Subgp 6

- Feb 26, 2002

**Notes:**
4-6q2

## WHAT WOULD BE THE LOGISTICAL CONCERNS ABOUT ESTABLISHING A RE-EXAMINATION OF PATIENT SPECIMENS? 

- Logistics of establishing a re-examination program to achieve equal benefit to PT/EQA programs require more samples and more time and thus greater costs.

- Wkgp 4 -- Subgp 6

- Feb 26, 2002

**Notes:**
4-6q2

## Summary

- The major elements to ensure success in managing PT/EQA programs are:
- government commitment and professional dedication;
- consumer satisfaction; and
- demonstration that quality of laboratory measurement improves the efficiency and effectiveness of health care.

- WORKGROUP 4

- Feb 26, 2002