# Presentation: 532615

## Role of PT in Laboratory Quality Management

**Dr. Jean-Claude Libeer**

- Wkgp 1 -- Subgp 1

- Feb 26, 2002

## Advantages – reaches wider audience than regulatory PT and reduces punitive atmosphere, which encourages discussion:
identifies technical needs for new equipment and methods;
assesses staff competency and assists in improving staff confidence; 
improves communication with clinicians;
improves the quality of testing, which ultimately improves patient outcomes 

- identifies technical needs for new equipment and methods;
- assesses staff competency and assists in improving staff confidence;
- improves communication with clinicians;
- improves the quality of testing, which ultimately improves patient outcomes

**FROM A LABORATORY’S PERSPECTIVE, WHAT ARE ****STRENGTHS AND WEAKNESSES**** OF PARTICIPATING IN AN EDUCATIONAL PT/EQA PROGRAM?**

- Wkgp 1 -- Subgp 1

- Feb 26, 2002

**Notes:**
1-1q1

## Disadvantages – increased costs for testing and evaluation
may be difficult to obtain and requests validated high quality samples;
educational objectives are not well understood (communication problems) so may not participate;
since not mandatory, may not have government support or recognition;
educational programs may be co-opted for regulatory purposes;
lab may not follow up if samples are not graded 

- may be difficult to obtain and requests validated high quality samples;
- educational objectives are not well understood (communication problems) so may not participate;
- since not mandatory, may not have government support or recognition;
- educational programs may be co-opted for regulatory purposes;
- lab may not follow up if samples are not graded

**FROM A LABORATORY’S PERSPECTIVE, WHAT ARE STRENGTHS AND WEAKNESSES OF PARTICIPATING IN AN EDUCATIONAL PT/EQA PROGRAM?**

- Wkgp 1 -- Subgp 1

- Feb 26, 2002

**Notes:**
1-1q1

## Essential elements:
identify specific target audience;
provide focused objectives and clear standardized instructions;
provide interlaboratory comparison and timely feedback;
recognize differences between disciplines, tests, methodologies, clinical practice patterns; 
“Problem-based control” (a problem identified:)
assists lab in determining how problem occurred;
offers new and emerging technology and organisms to increase laboratory awareness

- identify specific target audience;
- provide focused objectives and clear standardized instructions;
- provide interlaboratory comparison and timely feedback;
- recognize differences between disciplines, tests, methodologies, clinical practice patterns;
- “Problem-based control” (a problem identified:)
- assists lab in determining how problem occurred;
- offers new and emerging technology and organisms to increase laboratory awareness

**WHAT PT PROGAM ELEMENTS ARE ESSENTIAL FOR PT/EQA TO BE A USEFUL EDUCATIONAL EXPERIENCE?**** **

- Wkgp 1 -- Subgp 1

- Feb 26, 2002

**Notes:**
1-1q1

## WHAT EDUCATIONAL OPPORTUNITIES SHOULD PROGRAMS BE ENCOURAGED OR REQUIRED TO PROVIDE? 

- Data interpretation - include all instrument or all method comparisons, percentile for lab scores, trends, timely turnaround of results (making retesting possible), clear description of target audience, graphical or visual information
- Supplemental information (interpretive guidelines)
- Provide information on commonly misidentified cells or organisms, common errors, feedback on medical relevancy

- Wkgp 1 -- Subgp 1

- Feb 26, 2002

**Notes:**
1-1q2

## WHAT EDUCATIONAL OPPORTUNITIES SHOULD PROGRAMS BE ENCOURAGED OR REQUIRED TO PROVIDE? 

- Communication and training - provide clear description of target audience and educational objectives, program manuals and handbooks, newsletters, summaries from users, face to face customer service, training; information to lab students, interns
- Background information – provide information on how to interpret results, reference to standards and guidelines, structured approach for remedial action

- Wkgp 1 -- Subgp 1

- Feb 26, 2002

**Notes:**
1-1q2

## FROM A LABORATORY’S PERSPECTIVE, WHAT ARE THE STRENGTHS AND WEAKNESSES OF PARTICIPATING IN A MANDATED PT/EQA PROGRAM? 

- Mandated programs encourage more participation and treat all laboratories equally.
- Provision of more data promoting standardization.
- Any type of program is acceptable so long as it has strong links with the scientific/ professional community.

- Wkgp 1 -- Subgp 2

- Feb 26, 2002

**Notes:**
1-2q1

## FROM A LABORATORY’S PERSPECTIVE, WHAT ARE THE STRENGTHS AND WEAKNESSES OF PARTICIPATING IN A VOLUNTARY PROGRAM? 

- Voluntary programs provide an easier/faster starting point for PT/EQA.
- Unrepresentative data of the entire laboratory field.
- Stronger educational component.
- With voluntary programs only conscientious quality-minded laboratories participate.

- Wkgp 1 -- Subgp 2

- Feb 26, 2002

**Notes:**
1-2q2

## WHAT ARE THE DIFFERENCES IN MANDATED PROGRAMS OFFERED BY GOVERNMENT, ACADEMIC, OR COMMERCIAL SPONSORS? 

- Governmental schemes usually linked with laboratory license/accreditation
- Academic schemes mostly focus on clinical outcome and education
- Commercial schemes essentially focus on analytical outcome

- Wkgp 1 -- Subgp 2

- Feb 26, 2002

**Notes:**
1-2q1

## WHAT ARE SOME OF THE BENEFITS A LABORATORY CAN DERIVE FROM PARTICIPATION IN PT/EQA PROGRAMS? 

- PT is an important source of education and training
- may provide the only opportunity for laboratory peer interactions in resource limited countries.
- Identifying problems with the equipment, processing, or other testing component to correct deficiencies
- also provides support and justification for resources (e.g. new equipment purchases)

- Wkgp 1 -- Subgp 3

- Feb 26, 2002

**Notes:**
1-3q1

## WHAT ARE SOME OF THE BENEFITS A LABORATORY CAN DERIVE FROM PARTICIPATION IN PT/EQA PROGRAMS? 

- Validation and verification of testing performance and methods within the laboratory
- Improving the quality of service through:
- confidence building of laboratory staff, clinicians, and patients
- increased communication with clinicians, other laboratories and vendors.

- Wkgp 1 -- Subgp 3

- Feb 26, 2002

**Notes:**
1-3q1

## HOW MIGHT A PT/EQA PROGRAM BE USED TO TRAIN LABORATORY STAFF AND ASSESS THE SKILLS, KNOWLEDGE, AND ABILITIES OF LABORATORY STAFF? 

- Provides validated samples that can be retained in a library and re-used for training, monitoring reproducibility, and improving staff performance.
- Identifies training needs within a laboratory for individual staff and, externally, for groups of laboratories

- Wkgp 1 -- Subgp 3

- Feb 26, 2002

**Notes:**
1-3q1

## HOW MIGHT A PT/EQA PROGRAM BE USED TO TRAIN LABORATORY STAFF AND ASSESS THE SKILLS, KNOWLEDGE, AND ABILITIES OF LABORATORY STAFF? 

- PT/EQA can be responsive (educational materials) to meet needs of laboratories in different countries/situations
- tailor reports and data analyses to address specific problems and training needs.
- Promotes standardization of methods, such as interpretation and reporting of laboratory results

- Wkgp 1 -- Subgp 3

- Feb 26, 2002

**Notes:**
1-3q1

## WHAT ARE THE BENEFITS TO A LABORATORY WHEN A PT/EQA PROGRAM IS USED AS PART OF THE ACCREDITATION OR LICENSING PROCESS? 

- PT/EQA is first step in resource limited countries and may serve to trigger on-site evaluation of lab in addition to identifying priority issues for evaluation.
- Since accreditation provides a measure of process/standards, PT is one measure of outcomes.
- PT may provide a better measure of  performance in large laboratories where accreditation may only provide a cursory process check for each testing service

- Wkgp 1 -- Subgp 3

- Feb 26, 2002

**Notes:**
1-3q2

## WHAT ARE SOME OF THE CHANGES PT/EQA PROGRAMS NEED TO MAKE IN THEIR PROGRAM TO ENHANCE THEIR USE FOR ACCREDITATION OR LICENSURE? 

- The data from PT programs may be used to set performance expectations and focus the on-site evaluation/accreditation process.
- The greatest benefit occurs when qualified staff conduct the accreditation process to address PT performance deficiencies.

- Wkgp 1 -- Subgp 3

- Feb 26, 2002

**Notes:**
1-3q2

## HOW CAN A LABORATORY ENSURE THAT PT/EQA RESULTS ARE USED TO ENHANCE LABORATORY PERFORMANCE? 

- The purpose of  PT needs to be understood
- an external mechanism to monitor, improve test performance
- PT should be considered a positive, not a  negative tool
- Labs need to have specific procedures for handling
- when possible, testing PT samples like patient specimens

- Wkgp 1 -- Subgp 4

- Feb 26, 2002

**Notes:**
1-4q1

## HOW CAN A LABORATORY ENSURE THAT PT/EQA RESULTS ARE USED TO ENHANCE LABORATORY PERFORMANCE?

- Use PT samples to evaluate the lab’s processes, not to evaluate an individual
- If problems occur, these should reflect process problems
- Contact EQA provider and/or manufacturer to ensure problems are not due to sample matrix

- Wkgp 1 -- Subgp 4

- Feb 26, 2002

## HOW CAN A PT/EQA PROVIDER ASSIST A LABORATORY IN DETERMINING THE ROOT CAUSE OF QUESTIONABLE RESULTS?

- Establish a continuous mechanism for providing and receiving timely feedback
- Assess performance for all participants in each testing event; individual participant performance evaluation over time
- Review the lab’s corrective action plan

- Wkgp 1 -- Subgp 4

- Feb 26, 2002

**Notes:**
1-4q1

## HOW CAN A PT/EQA PROVIDER ASSIST A LABORATORY IN DETERMINING THE ROOT CAUSE OF QUESTIONABLE RESULTS?

- Consultation site visits to evaluate processes
- Provide additional samples when needed
- If PT/EQA problem is method dependent, EQA provider should contact the manufacturer.

- Wkgp 1 -- Subgp 4

- Feb 26, 2002

**Notes:**
1-4q1

## CAN AND SHOULD PT/EQA PROGRAMS ADDRESS THE ISSUE OF THE UNCERTAINTY ASSOCIATED WITH MEASURING A LABORATORY’S PERFORMACE ASSESSMENT? 

- The credibility and utility of PT/EQA programs are dependent upon the identification, characterization, and, where possible, resolution of sources that introduce uncertainty.
- Design elements and rigor of challenge must consider the education and training of analysts, particularly in resource-restricted countries.

- Wkgp 1 -- Subgp 5

- Feb 26, 2002

**Notes:**
1-5q1

## CAN AND SHOULD PT/EQA PROGRAMS ADDRESS THE ISSUE OF THE UNCERTAINTY ASSOCIATED WITH MEASURING A LABORATORY’S PERFORMACE ASSESSMENT? 

| Sources of Uncertainty  | Responsible Partner  | Comment  |
| --- | --- | --- |
| Specimen Design/Preparation
-Matrix
-Homogeneity / viability | PT/EQA provider 
Manufacturer | Behavior should be similar to authentic patient samples; commutability  |
| Value Assignment  | PT/EQA provider
Calibration lab | Increasing uncertainty from use of CRM to use of participant mean/median |
| Stability
-Timeframe for analysis
- Storage conditions
-Transportation/handling | PT/EQA
Provider/
Manufacturer |  |

- Wkgp 1 -- Subgp 5

- Feb 26, 2002

**Notes:**
1-5q1

## CAN AND SHOULD PT/EQA PROGRAMS ADDRESS THE ISSUE OF THE UNCERTAINTY ASSOCIATED WITH MEASURING A LABORATORY’S PERFORMACE ASSESSMENT? 

| Sources of Uncertainty  | Responsible Partner  | Comment  |
| --- | --- | --- |
| Specimen Processing Variables  | laboratory  | Process as patient specimens; follow provider instructions  |
| Analysis | Laboratory/
Manufacturer | Measurement uncertainty related to bias and imprecision  |
| Reporting Requirements  | PT Provider and Laboratory  | Unique reporting requirements may not be consistent with routine practices  |

- Wkgp 1 -- Subgp 5

- Feb 26, 2002

**Notes:**
1-5q1

## WHAT WEIGHT SHOULD BE PLACED ON PT/EQA PERFORMACE COMPARED WITH OTHER MEASURES OF PERFORMANCE MEASURES? 

- PT is a significant _objective_ measure (indicator) of laboratory performance.
- Outcomes are used to drive evaluations of laboratory practices and other measures of laboratory performance.
- In some instances, e.g., resource restricted countries, PT/EQA may be the _only_ indicator of laboratory performance.
- If PT primarily evaluates the analytic phase of laboratory testing it must not be used as the sole process for measuring laboratory performance.

- Wkgp 1 -- Subgp 5

- Feb 26, 2002

**Notes:**
1-5q2

## WHAT WEIGHT SHOULD BE PLACED ON PT/EQA PERFORMACE COMPARED WITH OTHER MEASURES OF PERFORMANCE MEASURES? 

- PT outcomes must be evaluated by the lab regardless of provider assessment of laboratory performance, i.e., successful or unsuccessful performance.
- Not all PT programs are equivalent, e.g., performance evaluation criteria.

- Wkgp 1 -- Subgp 5

- Feb 26, 2002

**Notes:**
1-5q2

## HOW COULD PT/EQA PROGRAMS BE USED AS A VEHICLE TO INTRODUCE STANDARDIZED TESTING ALGORITHMS, CRITICAL LIMITS, REPORTING PROTOCOLS, ETC.  THROUGHOUT A COUNTRY OR REGION? 

- EQA is a powerful tool for improving desired testing algorithms, standardized nomenclature and standardized protocols in the following ways:
- Information can be used to define/describe “state of the art” of lab practice and formulate relevant performance criteria.
- PT providers should provide PT samples in a clinical context (case histories) when possible.
- PT providers should require reporting in standardized units and nomenclature

- Wkgp 1 -- Subgp 6

- Feb 26, 2002

**Notes:**
1-6q1

## FROM A LABORATORY’S PERSPECTIVE, WHAT WOULD BE THE ADDED VALUE OF PARTICIPATION IN A PROGRAM THAT HAS MET INTERNATIONALLY ACCEPTED REQUIREMENTS TO BE AN ACCREDITED PT/EQA PROGRAM? 

- It provides a higher degree of confidence that the laboratory is being evaluated fairly by defined procedures
- It is essential for compulsory PT programs to be accredited by a third party.
- It provides the potential for arbitration by an independent body.

- Wkgp 1 -- Subgp 6

- Feb 26, 2002

**Notes:**
1-6q2

## Summary

- Role of EQA/PT in laboratory management will depend on:
- Type of EQA organization
- Goals of the schemes
- Resources
- Level of quality desired

- WORKGROUP 1

- Feb 26, 2002

## SUBGROUP LEADERS AND RECORDERS

- Nancy Bestic
- Humberto Tibúrcio
- Sharon Ehrmeyer
- Beverly Metchock
- Richard Jenny
- Tadashi Kawai

- Nancy Anderson
- Ruth Tembwe
- John Ridderhof
- Rhonda Whalen
- Gene Baillie
- Judy Delaney

- Workgroup 1

- Feb 26, 2002

## Slide 29

- Workgroup

- Feb 26, 2002