# Presentation: 305865

## Analytic Process Control

**Quality Control  for**** Quantitative Tests**

## Quantitative QC Materials

_**Calibrator**_**:** a solution which has a known amount of analyte weighed in or has a value determined by repetitive testing using a reference or definitive test method
_**Control**_**: **material or preparation used to monitor the stability of the test system within predetermined limits

## Calibration

- The process of testing and adjustment of an instrument, kit, or test system, to provide a known relationship between the measurement response and the value of the substance being measured.
_**Calibration verification**_: run calibrators like patient specimens to confirm calibration throughout the reportable range

## Types of Control Materials

- Assayed
- mean calculated by the manufacturer
- must verify in the laboratory
- Unassayed
- less expensive
- must perform data analysis
- “Homemade” or “In-house”
- pooled sera collected in the laboratory
- characterized
- preserved in small quantities for daily use

## Sources for Control Materials

- Commercial product
- Diagnostic samples (qualitative QC)
- Obtained from:
- Another laboratory
- EQA provider

## Analysis of Control Materials

- Need data set of at least 20 points, obtained over a specified time period
- Calculate mean, standard deviation, coefficient of variation; determine target ranges
- Develop control charts (e.g., Levey-Jennings charts), plot results

## Establishing Control Ranges

- Select appropriate controls
- Assay them repeatedly over time (e.g., 30 days)
- at least 20 data points
- Make sure any procedural variation is represented:
- different operators
- different times of day
- Determine the degree of variability in the data to establish acceptable range

## Measurement of Variability

- A certain amount of variability will naturally occur when a control is tested repeatedly.
- Variability is affected by operator technique, environmental conditions, and the performance characteristics of the assay method.
- The goal is to differentiate between variability due to chance from that due to error.

**Measures of Central Tendency**

- Data are frequently distributed about a central value or a central location
- There are several terms to describe that central location, or the ‘central tendency’ of a set of data

## Measures of Central Tendency

- Median = the value at the center (midpoint) of the observations
- Mode = the value which occurs with the greatest frequency
- Mean = the calculated average of the values

## Calculation of Mean

**Calculation of Mean**

**X = Mean**
**X****1 ****= First result**
**X****2 ****= Second result**
**X****n**** = Last result in series**
**n – Total number of results**

## Calculation of Mean: Outliers

- 192 mg/dL
- 194 mg/dL
- 196 mg/dL
- 196 mg/dL
**160 mg/dL**
- 196 mg/dL

- 200 mg/dL
- 200 mg/dL
- 202 mg/dL
**255 mg/dL**
- 204 mg/dL
- 208 mg/dL
- 212 mg/dL

## Calculation of Mean

- 192 mg/dL
- 194 mg/dL
- 196 mg/dL
- 196 mg/dL
- 196 mg/dL
- 200 mg/dL
- 200 mg/dL
- 202 mg/dL
- 204 mg/dL
- 208 mg/dL
- 212 mg/dL
- Sum = 2,200 mg/dL

- Mean = the calculated average of the values
- The sum of the values (X1 + X2 + X3 ... X11) divided by the number (n) of observations
- The mean of these 11 observations is (2200  11) = 200 mg/dL

## Normal Distribution

- All values are symmetrically distributed around the mean
- Characteristic “bell-shaped” curve
- Assumed for all quality control statistics

## Normal Distribution

**Frequency**

**	****4.7’****	****4.8’****	****4.9’****	****Mean****	****5.1’****	****5.2’****	****5.3’**

## Normal Distribution

- Mean

## Accuracy and Precision

- The degree of fluctuation in the measurements is indicative of the “precision” of the assay.
- The closeness of measurements  to the true value is indicative of the “accuracy” of the assay.
- Quality Control is used to monitor both the precision and the accuracy of the assay in order to provide reliable results.

## Precision and Accuracy

- Imprecise and inaccurate

**Precision and Accuracy**

**Systematic Error**

**Random Error**

## Precise and Accurate

## Measures of Dispersion or Variability

- There are several terms that describe the dispersion or variability of the data around the mean:
- Range
- Variance
- Standard Deviation
- Coefficient of Variation

## Range

- Range refers to the difference or spread between the highest and lowest observations.
- It is the simplest measure of dispersion.
- It makes no assumption about the shape of the distribution or the central tendency of the data.

## Calculation of Variance (S2)

## Calculation of Variance

- Variance is a measure of variability about the mean.
- It is calculated as the average squared deviation from the mean.
- the sum of the deviations from the mean, squared, divided by the number of observations (corrected for degrees of freedom)

## Degrees of Freedom

- Represents the number of independent data points that are contained in a data set.
- The mean is calculated first, so the variance calculation has lost one degree of freedom (n-1)

## Calculation of Standard Deviation 

## Calculation of Standard Deviation

- The standard deviation (SD) is the square root of the variance
- it is the square root of the average squared deviation from the mean
- SD is commonly used (rather than the variance) since it has the same units as the mean and the original observations
- SD is the principle calculation used in the laboratory to measure dispersion of a group of values around a mean

## Standard Deviation and Probability

- For a set of data with a normal distribution, a value will fall within a range of:
- +/- 1 SD 68.2% of the time
- +/- 2 SD 95.5% of the time
- +/- 3 SD 99.7% of the time

## Calculation of Coefficient of Variation

- The coefficient of variation (CV) is the standard deviation (SD) expressed as a percentage of the mean
- Ideally should be less than 5%

## Standard Deviation and Probability

- In general, laboratories use the +/- 2 SD criteria for the limits of  the acceptable range for a test
- When the QC measurement falls within that range, there is 95.5% confidence  that the measurement is correct
- Only  4.5% of the time will a value fall outside of that range due to chance; more likely it will be due to error

## Excercise

**Standard deviation worksheet**

## Monitoring QC Data

## Monitoring QC Data

- Use Levey-Jennings chart
- Plot control values each run, make decision regarding acceptability of run
- Monitor over time to evaluate the precision and accuracy of repeated measurements
- Review charts at defined intervals, take necessary action, and document

## Levey-Jennings Chart

- A graphical method for displaying control results and evaluating whether a procedure is in-control or out-of-control
- Control values are plotted versus time
- Lines are drawn from point to point to accent any trends, shifts, or random excursions

## Levey-Jennings Chart

## Chemistry Case Study

- The supervisor of the chemistry laboratory noticed that the quality control results for the month of February for all 3 levels of ALT control were running below the mean.
- The technologist operating the instrument for this particular assay admitted that it had been difficult to get the control values to fall within the assay’s control limits.
- Several of the analytic runs were out of control.
- Instrument calibration and calibrator values looked fine

## Chemistry Case Study (Cont.)

- Technologist tried to resolve the problem by:
- Repeating runs using fresh aliquots of control material
- Repeating runs using an aliquot from new vial of the same lot of control
- Only those analytic runs that had control values within the acceptable range were accepted.

## Levey-Jennings Charts by month

- Let’s discuss these further on the next slide.

## Levey-Jennings charts by month

- What do these charts show?

**Notes:**
Graphic techniques used for quality control purposes are control charts. Control charts are very important tools in identifying shifts, trends, and outliers
Control charts provide a graphic display of the control results in order to aid in evaluating whether the analytic procedure is in control or out of control.
Control results are plotted on the charts vs time or sequential run number. Lines are generally drawn from point to point to accent any trends, systematic shifts, and random excursions.

Measurement of all 3 controls had shifted below the mean.  (Click for lines to display on chart, click again to hide) 
The control chart revealed a shift of the control values below the mean and values >2 SD from the mean.
Inspection of the control charts revealed a shift in the control values across the run and within the run below the mean for all 3 levels of the control material. (Mouse click to highlight)

## What the charts show

- Measurement of all 3 controls had shifted below the mean.
- Inspection of the control charts revealed a shift in the control values across the run and within the run below the mean for all 3 levels of the control material.

## Corrective Action

- When quality control results are out of range
- identify the type of error causing the quality control failure
- determine the best possible approach to resolve the deficiency.

**Notes:**
Systematic

## What kind of error do we have in this case?

- Random error?
- Systematic error?

## Random Error

- Random error is defined as imprecision of the test system causing a scatter or spread of control values around the mean.
- The exact magnitude of random error cannot be predicted. It is estimated by repetitive testing or precision studies.

## Common causes of random error

- air bubbles in the reagent
- reagent lines, sampling, or reagent syringes
- improperly mixed/dissolved reagent
- pipette tips not fitting properly
- a clogged pipetter (clot)
- imprecise pipetter
- the power supply
- power fluctuations.

## Systematic Error

- Systematic error is defined as a systematic change in the test system that displaces the mean of the distribution from its original value.
- Systematic error of an analytic system is predictable and causes shifts or trends on control charts.

## Common causes of systematic error 

- a change in the reagent or calibrator lot number
- wrong calibrator values
- improperly prepared reagents
- deterioration of reagents or calibrators
- inappropriate storage of reagents and calibrators
- change in sample or reagent volumes due to pipetter misadjustments or misalignments
- change in temperature of incubators and reaction blocks
- deterioration of a photometric light source
- change in procedure from 1 operator to another.

## Questions to be considered:

- This shift in control values in one direction indicates an increase in what kind of error?
- Should the supervisor recalculate the control limits using the current mean?

**Notes:**
Systematic error

No – he should identify the problem.

## Questions to be considered:

- Should the technologist continue to reject and repeat out-of-control runs but report patient results from assays that meet the laboratory’s quality control acceptability criteria?
- How should the technologist try to locate the cause of the problem and take appropriate action to resolve the problem?

## Case Resolution

- The systematic shift was attributed to analysis of a new batch (shipment) of the current control lot.
- It was discovered that the technologist started having problems with the control values when the new batch of control material was used.
- New control material of the same lot was obtained from the manufacturer and tested. The new controls gave the expected values with results near to the estimated mean.

## When new batch of control material arrives:

- The acceptable quality control limits for each level of control should be determined by repetitive testing in the laboratory.
- In this case, each level of the new (phase-in) quality control material should be assayed in parallel with current (in-use) quality control material at least 20 times for a minimum 2-week period.
- The mean, standard deviation (SD), and coefficient of variation (%CV) should be calculated to determine the expected level of variation.
- The quality control acceptable limits should be established and new control charts created.

## For each test, the following should be established:

- Statistical control rules (e.g., Westgard) should be carefully selected to maximize error detection and minimize false rejection.
- The best set of control rules will vary from method to method and will be based on the quality (allowable error) required by the test.

## Periodic Review

- Periodic review of control charts to check assay performance is essential and detects problems such as shifts, trends, random errors, imprecision, and outliers.
- Periodic review can include such elements as:
- daily review of control values and control charts by the technologist before accepting assays
- weekly or monthly review of quality control data and charts by the supervisor or director of the laboratory
- periodic audits

## Conclusion

- The technologist in this case study was facing an out-of-control problem.
- Simply repeating out-of-control runs is not a good practice and should be discouraged.
- Rerunning control material or even repeating control material and patient samples does seem like a good temporary solution, but it merely delays troubleshooting and resolving the problem.
- In the end, it is worthwhile to spend the time necessary to find the cause of the problem and follow the appropriate corrective action.