Cholesterol Measurement: Test Accuracy and Factors That Influence Cholesterol Levels
Published: 1994-12-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Pursuant to a congressional request, GAO reviewed the National Heart, Lung, and Blood Institute's National Cholesterol Education Program (NCEP), focusing on the: (1) different techniques for measuring cholesterol in laboratory settings; (2) accuracy of these measurement techniques; (3) factors that influence individual cholesterol levels; and (4) potential effect of cholesterol measurement variability.
section_title: What GAO Found
paragraphs: GAO found that: (1) the natural daily variation in cholesterol levels and instrument measurement errors make it impossible to pinpoint individual cholesterol levels; (2) over 160 different devices with different technologies and chemical formulations are available to perform cholesterol tests; (3) standard cholesterol tests measure two cholesterol components, total cholesterol, and a related blood fat; (4) research, clinical, and hospital laboratories tend to produce reasonably accurate and precise cholesterol measurements, but little is known about cholesterol measurements in other settings such as physicians' offices and public health screenings; (5) two federal agencies have developed reference methods and quality control testing materials for manufacturers and laboratories to use in assessing their equipment's performance; (6) although cholesterol measurement methods have improved in recent years, there is a large variance in the accuracy and precision of tests performed across a broad range of devices and analytical settings; (7) there has been no overall evaluation of the different instruments and technologies laboratories use to conduct cholesterol tests; (8) biological and behavioral factors, many of which are uncontrollable, cause individual cholesterol levels to vary and may account for up to 65 percent of total variation in individual cholesterol levels; (9) the methods for collecting and handling blood specimens affect cholesterol measurements; and (10) uncertain cholesterol measurements can affect individual diagnoses and treatment decisions.

section_title: Letter
paragraphs: In response to your request, we are submitting this report, which examines how cholesterol and its subfractions are measured in different laboratory settings and discusses what is known about the accuracy of these measurement techniques. The report also discusses various analytical and biological factors that can affect an individual’s cholesterol levels. Furthermore, the report assesses the potential effect of cholesterol measurement variability and the classification and treatment of patients.
We are sending copies to interested congressional committees and government agencies, and we will make copies available to others upon request. If you have any questions or would like additional information, please call me at (202) 512-2900 or Kwai-Cheung Chan, Director of Program Evaluation in Physical Systems Areas, at (202) 512-3092. Major contributors to this report are listed in appendix II.
subsections: section_title: Introduction
paragraphs: The National Institutes of Health (NIH) emphasizes lowering cholesterol as an important aspect of preventing coronary heart disease. In 1985, NIH’s National Heart, Lung, and Blood Institute (NHLBI) initiated the National Cholesterol Education Program (NCEP), which has undertaken a major effort to encourage individuals to measure, track, and reduce their cholesterol levels (notably total and low-density lipoprotein (LDL) cholestrol) with the objective of reducing mortality and morbidity from coronary heart disease. The focus on cholesterol reduction has come at a time when increased emphasis has also been given to modifying other risk factors associated with heart disease such as cigarette smoking and hypertension.
One aspect of the efforts to broaden awareness of cholesterol as a risk factor has been to encourage individuals to “know your cholesterol number.” This advice has been heeded by the public. According to data compiled by the Centers for Disease Control and Prevention (CDC) from 47 states and the District of Columbia, the percentage of adults who reported having had their total cholesterol checked in the past 5 years ranged from 56 percent in New Mexico to 71 percent in Connecticut (median across the states sampled: 64 percent). The percentage of persons who had been told their cholesterol is high by a health professional ranged from 14 percent in New Mexico to 21 percent in Michigan (median across the states sampled: 17 percent).
For a widespread cholesterol-lowering campaign to be credible, however, test results must be accurate across the diverse devices and settings in which cholesterol is measured. This is because the guidelines for treating elevated cholesterol are predicated on test results that place an individual into different risk categories. In this report, we discuss what is known about the accuracy of cholesterol testing, including how it is measured, factors that hinder accurate measurements, and efforts to improve the accuracy of cholesterol tests.
subsections: section_title: Background
paragraphs: Coronary heart disease is one of the leading causes of death for both men and women in the United States, accounting for 478,530 deaths in 1991, according to the American Heart Association (AHA). Of these deaths, 52 percent were men and 48 percent were women. Approximately 6.3 million people alive today in the United States have a history of heart attack, chest pain, or both; of this group, 44 percent are 60 years of age and older, 25 percent are 40 to 59 years old, and 31 percent are younger than 40.
Further, 1.5 million Americans are expected to suffer a heart attack in 1994. The death rate from heart attack in the United States, however, declined 32 percent between 1981 and 1991. Reasons cited as contributing to this decline include improved medical care of patients and preventive measures in the population.
AHA estimates that total costs associated with coronary heart disease are $56.3 billion per year. Of this figure, $37.2 billion is spent on hospital and nursing home services, $8.7 billion on physicians and nurses services, and $2.4 billion on drugs. Lost output associated with heart disease is valued at $8 billion.
Because of the large sums being spent on treatment, to say nothing of the attendant psychological and social costs, prevention has been emphasized. NHLBI has established several education programs, such as NCEP, to inform the public about different risk factors associated with coronary heart disease and to provide guidelines for reducing risks that are modifiable. Other programs include the National High Blood Pressure Education Program, which began in 1972; the Smoking Education Program (1985); and the Obesity Education Initiative (1991).
section_title: NCEP Guidelines
paragraphs: A consensus development conference of scientific experts brought together by NIH in 1984 concluded that the risk of coronary heart disease is positively related to increased levels of serum cholesterol and that lowering elevated cholesterol levels can reduce coronary heart disease risk for individuals. The conference experts based their conclusions on the accumulated evidence from a large body of epidemiological, animal, metabolic, and clinical studies. Of major importance was the results of the Lipid Research Clinics Coronary Primary Prevention Trial, a large randomized study completed in 1984 that provided evidence that treatment to lower high cholesterol levels in patients can reduce the risk of coronary heart disease. The conference experts further recommended plans for establishing the National Cholesterol Education Program, which began in 1985.
NCEP has convened several expert panels and issued a series of guidelines, reports, and educational materials on the management and control of cholesterol for health care professionals and the general public. The program emphasizes two parallel approaches: (1) a clinical approach that attempts to identify and treat individuals who are at high risk and (2) a broader population approach that aims to reduce cholesterol levels for the entire population.
Clinical guidelines for reducing elevated cholesterol levels in adults over 20 years of age were first issued by the Adult Treatment Panel in 1987 and were subsequently updated in a second expert panel report in 1993. These guidelines cover the classification of cholesterol, patient evaluation, and dietary and drug treatments. In 1990, NCEP outlined population strategies to lower total and LDL cholesterol by encouraging all Americans to be aware that elevated cholesterol is a potential risk factor for coronary heart disease, have their cholesterol measured at regular intervals, and modify their diet. NCEP published another report in 1991 that addressed cholesterol issues in children and adolescents. It emphasized strategies for encouraging the nation’s youths to reduce their intake of saturated fat and cholesterol as well as identifying and treating those whose high serum cholesterol levels put them at increased risk for heart disease as adults. The recommendations made in NCEP reports are disseminated and implemented through 40 agencies, such as AHA, that conduct health education and information activities.
subsections: section_title: Adult Treatment Guidelines
paragraphs: The current NCEP adult treatment guidelines emphasize classification and treatment decisions based on a person’s risk status, which is defined not only by serum cholesterol levels (including total cholesterol and its low-density lipoprotein (LDL) and high-density lipoprotein (HDL) components) but also by what other coronary risk factors are present. Those with symptoms of coronary heart disease or with at least two other coronary heart disease risk factors are considered candidates for more intensive treatment.
Hypertension (>140/90 mm Hg, or on antihypertensive medication)
Current cigarette smoking
Diabetes
Family history of myocardial infarction or sudden death before age 55 in father or male sibling, before age 65 in mother or female sibling
Age: male >45 years of age or female >55 years of age or postmenopausal and not on estrogen replacement therapy
Low HDL cholesterol (<35 mg/dL)
A negative risk factor is
HDL cholesterol >60 mg/dL.
The guidelines recommend that all adults have their total cholesterol measured at least once every 5 years and that HDL cholesterol be measured at the same time. As shown in figure 1.1, adults without evidence of existing coronary heart disease are classified initially into three levels based on total cholesterol levels—desirable (below (<) 200 mg/dL), borderline high (200-239 mg/dL), and high (equal to or above (>) 240 mg/dL). An HDL cholesterol level of less than 35 mg/dL is considered low and a contributing risk factor for coronary heart disease. The cutpoints for total cholesterol are based largely on epidemiological data that have shown that the risk of heart disease increases as cholesterol levels rise. For example, in 361,000 men screened for the Multiple Risk Factor Intervention Trial, those at or above the 90th percentile of total cholesterol, about 263 mg/dL, had a four times greater risk of death from coronary heart disease than those in the bottom 20 percent (< 182 mg/dL).
As indicated in figure 1.1, individuals are recommended for a followup lipoprotein analysis depending on an assessment of their total cholesterol and HDL cholesterol levels in conjunction with the presence or absence of other coronary heart disease risk factors. Thus, those who would be candidates for a subsequent lipoprotein analysis include individuals with (1) high total cholesterol (‡ 240 mg/dL), (2) borderline-high cholesterol (200-239 mg/dL) and low HDL cholesterol (<35 mg/dL), or (3) borderline-high cholesterol (200-239 mg/dL), higher HDL cholesterol (>35 mg/dL), and two or more risk factors.
Lipoprotein analysis includes measurement of fasting levels of total cholesterol, HDL cholesterol, and triglycerides and the calculation of LDL cholesterol, which is derived by a mathematical formula. The subsequent classification of adults based on LDL cholesterol levels is shown in figure 1.2. NCEP also classifies LDL cholesterol into three levels—desirable (<130 mg/dL), borderline-high risk (130-159 mg/dL), and high risk (>160 mg/dL).
Decisions for beginning diet or drug treatment are then based on these levels in combination with other risk factors (see table 1.1). Thus, candidates for diet therapy without known symptoms of coronary heart disease include those with high LDL cholesterol (>160 mg/dL) or those with borderline-high LDL cholesterol (130-159 mg/dL) plus two or more risk factors.
NCEP recommends diet therapy as a first line of treatment for most patients except those at particularly high risk who may warrant drug intervention immediately, such as individuals with existing coronary heart disease. NCEP’s recommended step I and step II diets are designed to reduce consumption of saturated fat and cholesterol and to promote weight loss in overweight patients. If diet therapy is ineffective at lowering LDL cholesterol levels, then drug treatment is advised. NCEP has developed a series of guidelines for administering different types of drugs that are available to lower cholesterol. It should be noted that initiating drug treatment commits patients to long-term therapy, which may be for the rest of their lives.
Some perspective on what these treatment categories and recommendations mean for Americans can be seen in recently collected, nationally representative data from the first phase of the National Health and Nutrition Examination Survey (NHANES III). These data indicate that the average total serum cholesterol level is 205 mg/dL for men 20 years old and older and 207 mg/dL for women 20 years old and older. As shown in figure 1.3, women tend to have lower total cholesterol levels compared to men up until the ages of 45 to 54, at which time it increases to levels above those of men. This difference may be attributed, in part, to menopause, which influences women’s lipid and hormonal levels. Whether this increases women’s coronary heart disease risk is not clear, according to some research. Overall, women appear to have higher HDL cholesterol levels then men do, which may also account for part of this difference.
While trend data indicate that cholesterol levels have declined since the early 1960’s, 52 million U.S. adults, or 29 percent, have an LDL cholesterol level that is classified as borderline-high or high according to the NCEP guidelines and that, when combined with other risk factors, makes them candidates for dietary therapy. Of the 52 million adults mentioned above, about 12.7 million have cholesterol levels sufficiently elevated that they might be candidates for drug therapy (about one third of this group would be patients with coronary heart disease).
section_title: Cholesterol Measurement Recommendations
paragraphs: NCEP’s Laboratory Standardization Panel (LSP) has issued two reports on cholesterol measurement. The first report, issued in 1988, focused attention on the importance of accurate measurements. In the report’s introduction, the panel stated: “the current state of reliability of blood cholesterol measurements made in the United States suggests that considerable inaccuracy in cholesterol testing exists.” That report, along with press articles critical of cholesterol testing in 1987, drew attention to the need for more consistent and replicable results.
In addition to outlining the state of the art in cholesterol testing, the NCEP/LSP reports describe factors that can affect test accuracy and reliability: analytical problems (laboratory analyzer inaccuracy and imprecision) and preanalytical factors (biological variation, disease, conditions under which a sample is taken). The second report published in 1990 also contains a number of recommendations to improve laboratory testing systems. These include using only analytical systems whose standardization process is linked to the National Reference System for Cholesterol (NRS/CHOL, discussed in chapter 3), participating in external surveillance programs (proficiency testing), comparing results with other laboratories, and using quality controls to monitor analytical performance.
Recognizing the problem of measurement variability in cholesterol testing, the Adult Treatment and Laboratory Standardization Panels recommended that total and LDL cholesterol be measured on two separate occasions and averaged together. If the cholesterol results differ by 30 mg/dL or more, then a third test should be conducted and the three tests averaged together to assess an individual’s cholesterol level.
NCEP/LSP established the goal that a single serum total cholesterol measurement should be accurate within +8.9 percent. This goal of +8.9 percent was effective in 1992, replacing the interim goal of +14.2 percent that had been established in 1988. NCEP has not previously issued goals for HDL and LDL cholesterol measurement; however, an expert panel convened by NCEP has recently developed such goals and they are expected to be published shortly.
The Clinical Laboratory Improvement Amendments of 1988 (Public Law 100-578) also mandated that the Secretary of Health and Human Services (HHS) establish performance standards such as quality control, quality assurance, and personnel regulations. HCFA testing requirements for total cholesterol, authored by CDC, stipulate a +10 percent criterion for acceptable performance for proficiency testing purposes. HCFA testing requirements for HDL cholesterol for acceptable performance on proficiency testing specimens is +30 percent of the established target value.
Although the NCEP guidelines advocate multiple measurements, there has been concern by some researchers that, in practice, physicians may not take measurement variability into account when making treatment decisions about cholesterol. Given that the NCEP classification levels for cholesterol are relatively narrow and that the average cholesterol levels for the U.S. population are in the borderline-high category at about 205 mg/dL, there is potential that patients can be misclassified. That is, measurement errors can lead to individuals with “true” levels below the high cutpoint of 240 mg/dL for total cholesterol or 160 mg/dL for LDL cholesterol being put on treatment (termed a false positive) or conversely those with “true” levels above the cutpoints not being treated (a false negative).
section_title: Objectives, Scope, and Methodology
paragraphs: In discussions with the requester, we agreed to focus our review of cholesterol measurement on the following evaluation questions:1. How is cholesterol measured? (See chapter 2.) 2. What is known about the accuracy and precision of cholesterol measurement techniques? (See chapter 3.) 3. What factors influence cholesterol levels? (See chapter 4.) 4. What is the potential effect of uncertain measurement? (See chapter 5.)
To answer these questions, we identified and reviewed relevant scientific literature published mainly since 1988 and synthesized data across studies to address these questions. We selected this period because it covered the time since the first NCEP cholesterol measurement goals were issued, permitting a benchmark by which later testing could be judged. We conducted our bibliographic search using on-line data bases of medical literature. Other sources included articles recommended by experts in the field and the bibliographies of articles published in medical and related research journals. We identified and reviewed approximately 125 books and articles relevant to cholesterol measurement in this manner.
We supplemented our review of the medical literature with interviews with a range of individuals who have expertise in the field. These included government agency officials involved with cholesterol measurement and testing issues at CDC, the Food and Drug Administration (FDA), HCFA, NIH, and the National Institute of Standards and Technology (NIST). We also interviewed manufacturers of analyzers in private industry, university researchers, and representatives of organizations that conduct proficiency testing for laboratories. In order to have a better understanding of the testing process, we visited a major hospital laboratory facility to discuss quality control issues and challenges facing practitioners. We also visited a major manufacturer of analyzers to learn more about the production process (quality control procedures, analyzer calibration, potential sources of inaccuracy) as well as industry concerns about the accuracy and precision of cholesterol testing. We did not, however, independently evaluate laboratory performance in any of the different settings where cholesterol tests are conducted across the country.
section_title: Cholesterol Measurement: Methods, Settings, and Analyzers
paragraphs: In this chapter, we answer the first evaluation question: How is cholesterol measured? The discussion begins with an overview of cholesterol’s role in the body and analyzes how total cholesterol, HDL, LDL, apolipoproteins, and triglycerides are measured, focusing on laboratory techniques. We also describe the range of settings where cholesterol testing is done and review the type of analyzers for sale in the U.S. market.
Cholesterol measurement focuses mainly on determining levels of total, HDL, and LDL cholesterol. Triglyceride levels are also included in lipid profiles. Cholesterol is commonly tested in a variety of settings ranging from large health fairs to more specialized clinical laboratories. No national data are available on the number of laboratories that conduct cholesterol tests, the number of cholesterol testing devices in use in laboratories, or the number of such tests that are done each year. The universe of U.S. laboratories that conduct different types of medical tests is large, however, with some 154,403 having registered with HCFA by October 1993. While HCFA data indicate that physicians’ offices predominate in the testing arena, the distribution of cholesterol tests is not ascertainable from these data. Test results may be less accurate from such settings because of the type of devices used and less staff expertise in conducting tests.
In addition to the broad range of settings where measurements are conducted, a large number of analyzers on the market measure cholesterol (45 manufacturers make 166 test systems that measure total cholesterol). Because some of these analyzers are used with different chemical formulations to conduct cholesterol tests, standardizing measurements is a complex task (a topic taken up in chapter 3). A related measurement issue is the use of enzymatic materials in cholesterol analyzers. While enzymatic materials have permitted improvements in ease of use, they are difficult to characterize chemically because they may deteriorate or vary with time, introducing potential measurement inaccuracy.
subsections: section_title: Cholesterol’s Role in the Body
paragraphs: While considerable attention has been given to the negative consequences of elevated total and LDL cholesterol levels, cholesterol is essential to body processes, affecting the production of steroid hormones and bile acids as well as being a structural component of cellular membranes.Cholesterol is a fat-like substance (lipid) manufactured by the body and is also ingested directly through foods such as eggs, which contain cholesterol. In addition, certain saturated fats raise the blood cholesterol level more than any other nutrient component in the diet. If you eat a “standard” American diet, two thirds of the cholesterol in your body is manufactured by your cells—the remainder is derived from your diet. Thus, an elevated cholesterol level may be the result of a diet heavy in saturated fat and cholesterol; it is also possible that the liver is manufacturing high levels of cholesterol and triglyceride or that cholesterol is being removed too slowly from the body.
Cholesterol is transported in blood plasma through lipoproteins. The three major classes of lipoproteins include LDL (containing 60 to 70 percent of the total serum cholesterol), HDL (containing 20 to 30 percent of the total serum cholesterol), and VLDL (very low density lipoproteins, which are precursors of LDL and contain 10 to 15 percent of the total serum cholesterol). Triglycerides are also an important lipid in the blood and are usually measured in conjunction with cholesterol values. More recently, increased scientific attention has been given to the apolipoprotein “families,” the subcomponents that make up these types of cholesterol, because they may be better predictors of certain risks associated with coronary heart disease such as degenerative changes in arterial walls. At present, however, research on this topic continues to be developed and tests for measuring apolipoproteins cannot be done in most laboratories.
section_title: Total Cholesterol Measurement
paragraphs: Of the different cholesterol types, total cholesterol is the best understood and documented, in large part because of work done at NIST and CDC to standardize measurement techniques (see chapter 3). In general laboratory practice, total cholesterol measurement is commonly accomplished by several different enzymatic methods using a variety of reagent materials.The various procedures used make standardization of technique across different reagents and instrument configurations difficult.
section_title: HDL Cholesterol Measurement
paragraphs: HDL cholesterol, sometimes referred to as the “good” cholesterol, has become recognized as an important coronary heart disease risk factor. HDL is the smallest in size of the lipoproteins and its major subcomponents are apo AI, or apolipoproteins AI, and apo AII. Because a validated reference method has not been developed for HDL measurement, a patient specimen comparison with CDC’s procedure is considered the best means to assess accuracy. HDL cholesterol is difficult to measure accurately, however, and current criteria under the Clinical Laboratory Improvement Amendments of 1988 for acceptable laboratory performance are that a sample must be +30 percent of a test target value, a relatively broad range even with lower HDL values. CDC officials we interviewed pointed out that considerable scientific work remains before HDL measurement accuracy is as well understood as total cholesterol currently is. This would include developing accurate reference materials that could be used to evaluate how well analyzers are measuring HDL cholesterol.
section_title: LDL Cholesterol Measurement
paragraphs: Low-density lipoprotein cholesterol, sometimes referred to as the “bad” cholesterol, is considered to be the principal fraction that causes plaque to build up on arterial walls. No error standards for LDL cholesterol measurement have been established under the Clinical Laboratory Improvement Amendments of 1988 or NCEP, although NCEP expects to issue such standards shortly. Direct measurement of LDL cholesterol can be accomplished through ultracentrifugation methods; however, such methods are expensive and time consuming to conduct and therefore not generally available in most cholesterol test settings.
In practice, LDL cholesterol is calculated from other laboratory measurements using the Friedewald formula: LDL = total cholesterol − HDL − (triglycerides/5). Among the several limitations to the Friedewald formula is that a patient should be fasting when the specimen is taken. The formula cannot be used for individuals with extremely high triglyceride levels (400 mg/dL and above) and several rare lipid conditions. The most crucial constraint related to the Friedewald formula is that because it relies heavily on the accuracy and precision of total cholesterol, HDL, and total triglycerides, potential measurement error is compounded.
section_title: Triglycerides
paragraphs: Triglyceride levels are usually measured along with lipoprotein levels because they are considered an important health indicator for certain diseases, including coronary heart disease in some patients. Triglycerides are also important to measure because they are used to calculate LDL cholesterol with the Friedewald equation. Enzymatic methods are used to analyze triglyceride levels, although the calibration of such methods is not linked to a validated definitive or reference method. As with HDL and LDL cholesterol, the CDC method (in this case, a chemical chromatropic acid method) is considered the best means for comparing accuracy. Current criteria under the 1988 amendments for acceptable laboratory performance are that a sample must be +25 percent of a proficiency test target value.
section_title: Apolipoprotein Measurement
paragraphs: As analytical capabilities have increased, attention has also turned to the apolipoproteins, which make up HDL and LDL cholesterol. This interest is linked to finding other relevant markers for coronary heart disease risk. For example, recent research has focused on apolipoprotein B-100 (apo B), which is an integral component of four major lipoproteins—LDL, VLDL, intermediate density lipoprotein, and lipoprotein(a)—and apo AI, the major protein component of HDL. At present, several assay methods are available to measure different apolipoprotein components; however, these methods have not yet been standardized. Another practical difficulty in using these apolipoproteins is that a comprehensive, statistically sound study has not yet been undertaken that can be used as a comparative reference base.
section_title: The Use of Ratios to Determine Coronary Heart Disease Risk
paragraphs: One issue addressed in the medical literature concerns combining cholesterol levels to determine a ratio that is used to evaluate a patient’s risk of developing coronary heart disease—for example, a total cholesterol or LDL to HDL ratio. In some instances, individuals are advised to achieve a specific ratio as an indicator of an acceptable cholesterol level. Such ratios have been useful estimators of coronary heart disease risk in some population studies; however, NCEP emphasizes that HDL and LDL cholesterol levels are independent risk factors with different determinants and should not be combined for clinical decisionmaking.
section_title: Test Settings
paragraphs: Widespread awareness of elevated total cholesterol levels as a potential coronary heart disease risk factor has led to patient testing in a variety of settings. These range from traditional clinical settings (hospitals, physician office laboratories) to mass screenings (such as health fairs). No national data are available on the number of laboratories that conduct cholesterol tests, the number of cholesterol testing devices in use in laboratories, or the number of such tests that are done each year.
The Clinical Laboratory Improvement Amendments of 1988 changed federal regulation of laboratories and expanded federal oversight to virtually all testing laboratories in the nation. The amendments required all laboratories to register with HCFA and established testing and quality control standards, including provisions for conducting inspections to ensure that laboratories are maintaining proper controls and records. In implementing the act, the Secretary of HHS established three categories of laboratory tests: (1) simple tests, (2) tests of moderate complexity, and (3) tests of high complexity. Waivers are given to laboratories that conduct only simple tests such as dipstick or tablet reagent urinalysis. Cholesterol tests are in the moderate complexity group, meaning that laboratories that perform such tests should comply with regulations under the amendments for personnel standards, quality control, and proficiency testing (these tests evaluate accuracy and precision using quality control materials).
As of October 1993, 154,403 laboratory facilities in the United States had registered with HCFA. HCFA officials estimate that there may be as many as 50,000 additional laboratories that should have registered with HCFA but have not, making it impossible to determine the universe of such facilities. Table 2.1 categorizes laboratories that had registered with HCFA. Of the registered laboratories, the majority, 90,673 (58.7 percent), are located in physicians’ offices.
Oversight of the laboratories listed in table 2.1 varies, depending on the level of tests performed and several factors. A large number, 67,000, conduct only tests that are not medically complex and are therefore exempt from regulation; 6,500 are accredited by a state agency; 24,000 are accredited by nongovernment proficiency testing groups; 16,000 conduct microscopic tests under HCFA oversight. HCFA coordinates biannual, on-site inspections by state agencies and HCFA regional office laboratory consultants for the remaining 41,000 laboratories.
HCFA expects to have 180 state agency surveyors nationwide who will work under 10 different HCFA regional offices. On-site inspections will consist of examining a sample of laboratory tests based on volume, specialties, clients, and the number of shifts over which equipment is used. In their inspections, surveyors will look at the following five areas: patient test management and organization, results of proficiency tests, personnel qualifications, quality assurance procedures, and use of daily quality controls.
HCFA staff began laboratory inspections under the 1988 amendments in September 1992 and they hope to have the first cycle of visits and certifications completed by March 1995. The initial emphasis of inspections has been to educate and inform laboratory personnel about pertinent regulations. HCFA staff responsible for overseeing laboratory inspections stated that of the 6,200 survey visits that had been made by August 1993, 500 laboratories were found to have major deficiencies (the nature of these problems was not specified).
HCFA survey and certification officials and NCEP have expressed concern that cholesterol testing in physicians’ offices or screening settings may differ from that done in clinical and research settings. For instance, clinical laboratories or hospitals may be more likely to have well-established quality control programs and large analyzers while physicians’ offices or health fairs may be limited to less reliable desk-top analyzers and less expertise in conducting tests and maintaining analyzers (see the discussion of analyzer types in chapter 3). HCFA staff stated that physicians’ offices often send specimens for HDL and LDL tests to larger laboratories, which have the capability to do these tests.
While enforcement under the 1988 amendments is relatively new, each of the groups most affected—HCFA, laboratory personnel, and proficiency testing service providers—views it differently. HCFA officials noted from their experience overseeing laboratories that the traditionally unregulated segment of the medical testing market, physicians’ office laboratories, sees the regulations as a burden and an added cost. In contrast, laboratories that have maintained a high-quality testing program believe the regulations represent minimum standards for running a quality testing program. Proficiency testing service providers have had to confront problems with the quality control materials they use to assess and transfer accuracy among laboratories, attempting to balance the limits of these materials with how they are used to judge laboratory performance. All agree, however, that meeting the standards adds to the cost of testing.
There are also several types of nonmedical settings in which testing is routinely undertaken: health fairs, shopping malls, and the workplace. In some cases, only a small amount of blood taken from the finger (capillary source) is used to conduct such a cholesterol analysis. These testing environments are subject to a variety of potential problems, however: poorly trained personnel taking samples, inappropriate patient preparation, incorrect specimen collection, or improperly calibrated analyzers. There is also concern that in nonmedical settings individuals may be given test results without proper interpretation. An additional concern is that those who need to be referred for further medical consultation or a more detailed cholesterol profile may not receive that advice.
section_title: Cholesterol Analyzers
paragraphs: FDA reviews and clears diagnostic devices, including those that measure cholesterol. Following section 510(k) of the Food, Drug, and Cosmetic Act of 1976, device manufacturers must notify FDA that they intend to market a device. FDA then determines whether the device is accurate, safe, effective, and substantially equivalent to a legally marketed “predicate” device—that is, one that was on the market when this law was passed. If the agency determines that a device does not meet 510(k) guidelines and deems it not substantially equivalent, then it must be reviewed as a new product. According to agency officials, FDA’s review of cholesterol measurement devices takes into consideration information provided by manufacturers on intended use, test type and methodologies, performance characteristics (derived from actual assays), analytical performance for 40 normal and 40 abnormal specimens across the range of cholesterol levels, and label wording (intended use statement and conditions).
FDA officials indicated that the agency requests that cholesterol device manufacturers compare their analyzers to the accuracy and precision methods of the National Reference Method Laboratory Network for total cholesterol measurement (see chapter 3). However, FDA does not formally require that analyzers be “traceable” to this method because there are devices on the market that have not established “traceability” to the reference method (traceability refers to the ability of a device to closely duplicate the accuracy attained by the reference method).
CDC has compiled a list of total and HDL cholesterol analyzers currently in use. As of April 1994, there were 166 test systems (made by 45 different manufacturers) available to measure total cholesterol. For HDL cholesterol, 143 test systems, made by 41 manufacturers, have been identified. (Some manufacturers have as many as 11 “systems” that use the same technology.)
FDA-cleared cholesterol analyzers encompass three types of devices: large stationary analyzers used in clinical laboratories, desk-top analyzers, and home test kit analyzers. Desk-top analyzers can be used in a variety of settings (medical and nonmedical) to provide relatively quick cholesterol test results, whereas large analyzers are capable of performing multiple tests on many analyses for hundreds of specimens a day. The latter are usually found in large independent laboratories, hospital laboratories, and the offices of major testing organizations that serve the medical community.
The third type is a home test kit, designed for sale directly to consumers. Currently, one device, the AccuMeter (manufactured by ChemTrak) is being marketed in the United States. The approval of this device has been somewhat controversial in the clinical chemistry field because of concerns about the reliability of its measurements. Apart from possible technical problems is the related issue of whether a person may incorrectly interpret his or her cholesterol level after using the device or initiate a self-treatment program without proper medical feedback and monitoring.
Cholesterol analyzers currently on the market primarily use enzymatic methods, high-technology equipment, and computerized data processing systems. Enzymatic methods offer advantages over older chemicals because they are safer and can be used in an automated laboratory environment, both distinct improvements. Nonetheless, enzymatic methods are also considered to be difficult to characterize chemically, thus adding to the uncertainty of tests done with them. FDA draft guidelines for approving cholesterol testing devices note that because enzymatic materials may deteriorate or vary, analyses done with them may be imprecise. A related concern noted by HCFA officials is that each analyzer and reagent combination has its own “method” for measuring cholesterol, making it difficult to assess accuracy using standardized testing materials.
An additional perspective on these devices was provided by a hospital laboratory administrator who observed that the devices used in his laboratory are self-contained “black boxes” that rely heavily on computer technology that must be regularly calibrated as part of a routine quality control process. Unlike the older instruments these have replaced, he noted, these newer devices are easier to use than the older systems. However, their complexity also means that it is hard to determine whether something may be wrong inside the device.
section_title: The Accuracy of Cholesterol Measurements
paragraphs: In this chapter, we answer the second evaluation question: What is known about the accuracy and precision of cholesterol measurement techniques? The discussion first focuses on national accuracy goals and efforts to standardize cholesterol measures. This is followed by an analysis of recently published literature that compares test results from different settings.
Standards for cholesterol testing have evolved from the late 1980’s, when NCEP first established the goal that total cholesterol measures should be accurate within +14.2 percent. By 1992, NCEP lowered its total cholesterol measurement goal to +8.9 percent. HCFA established a similar total cholesterol goal (+10 percent) as well as the goal that HDL cholesterol tests should be within +30 percent of its correct value, when judged by quality control testing. To date, an LDL cholesterol measurement goal has not been established, although one is expected soon.
Evaluating the extent to which laboratories across the country are providing medical personnel and patients with accurate total cholesterol test results is difficult. While an accepted national reference system exists, and network laboratories can provide traceability to an accuracy standard, participation by laboratories has been limited particularly to clinical and research settings. Additional information is collected through proficiency testing surveys that indicate that laboratory precision has improved over time but, again, the number of participating laboratories is small. The lack of information on accuracy in actual laboratory settings makes it impossible to know whether the goals established for total and HDL cholesterol measurement are being met and how well LDL cholesterol is being measured. Because these test results are key to making treatment decisions in NCEP guidelines, such data are arguably important.
Two collaborative research efforts, one by the College of American Pathologists (CAP) and CDC and the other by Veterans Affairs (VA) and CDC, highlight weaknesses of the current system of monitoring cholesterol laboratory tests. The reliance on processed quality control materials for evaluating analyzer accuracy was found to be problematic because of what are termed matrix effects. Processed materials tend to act differently from fresh serum samples on many instrument reagent systems and produce different test results. The studies found that total cholesterol tests done on fresh serum samples in a select group of clinical settings met NCEP accuracy standards whereas with processed control materials, there was greater inaccuracy. Finding ways to address matrix problems is important because processed control materials are key to assessing accuracy across laboratories and serve as the basis for enforcing the Clinical Laboratory Improvement Amendments of 1988.
With regard to desk-top analyzers, there are sufficient concerns about the reported accuracy and precision of total, HDL, and LDL results provided by several devices, even when tested under optimal operating conditions, to warrant further scrutiny of their performance. Consumers should be aware of the potential uncertainty associated with test results produced by these devices, particularly in screening settings. Several studies we reviewed found misclassification rates ranging from 17 to nearly 50 percent.
One new development in the cholesterol testing arena are home test devices, which measure total cholesterol. While these may prove to be useful, questions about their precision and accuracy should not be overlooked—particularly in light of their direct availability to consumers. Broader concerns about how individuals may interpret results and what they might do with that information in terms of failing to seek out appropriate medical consultation and possible treatment are too important to be ignored.
subsections: section_title: Accuracy, Precision, and NCEP Goals
paragraphs: NCEP’s 1990 report, Recommendations for Improving Cholesterol Measurement, established performance goals for assessing the accuracy of individual laboratory testing programs. The report recommended that by 1992 the total error associated with a single serum total cholesterol measurement should be within +8.9 percent (0.05, 2-tailed test). Total error is defined in terms of two main measurement components: bias and precision. Bias is the extent to which a series of test results deviate from the “true” value, within acceptable limits (<3 percent according to NCEP), whereas precision refers to the consistency and reliability of repeated results within acceptable limits (<3 percent according to NCEP). A cholesterol analyzer, for example, could be very precise yet inaccurate because of the poor calibration of an analyzer or the deterioration of the reagents being used. The difference between bias and precision can be illustrated with the following brief example. Suppose that a total cholesterol specimen whose “true” value is 200 mg/dL were tested 10 times on the same analyzer. If the analyzer gave a reading of 220 mg/dL each time it tested the specimen, the analysis would be biased—that is, it would be 10 percent over the “true” value. However, the analysis would be precise in that it consistently gave the same result when testing the specimen—the precision error would be zero.
While test results need to be unbiased and precise, there is the question of how accurate a test need be at particular cholesterol levels. It has been suggested that greater variability may be acceptable at levels well above or below the NCEP cutpoints—for example, at total cholesterol readings of 160 mg/dL or as high as 350 mg/dL. Arguably, accuracy becomes more important near the 240 mg/dL cutpoint than at 350 mg/dL, where there is less doubt about a patient’s risk category.
section_title: The National Reference System for Cholesterol
paragraphs: The National Reference System for Cholesterol (NRS/CHOL) grew out of work undertaken by the National Committee for Clinical Laboratory Standards in 1977 to establish an accuracy base for cholesterol testing.Rather than requiring that all laboratories use the same analyzers and methods to achieve standardization, emphasis is given to having test results traceable to an accepted accuracy standard. NRS/CHOL consists of a hierarchy of approved methods and materials used to assess cholesterol measurement accuracy. These include basic measurement units and definitive methods (NIST), primary reference materials (NIST), reference methods (CDC), secondary reference materials (NIST and CDC), field methods, and patients’ results. These are integrated into an accuracy base that can be transferred through a national laboratory network to device manufacturers and the broad range of laboratories where cholesterol is measured.
One component of NRS/CHOL involves expertise at NIST, where the definitive method for measuring total cholesterol was developed. The definitive method assigns the “true” value to a specimen through a process in which all potential sources of inaccuracy and interference are evaluated. The definitive method uses an isotope dilution mass spectrometric technique. Because it requires special equipment and costly materials and is time-consuming, the definitive method is not considered transferable to clinical laboratories. This method is also used for the highly specialized purpose of developing and testing standard reference materials that are used by manufacturers and in other settings such as research lipid laboratories.
CDC oversees another piece of NRS/CHOL: it uses what is termed the modified Abell, Levy, Brodie, and Kendall (abbreviated Abell-Kendall) reference method for total cholesterol measurement. When the reference and definitive methods have both been used to test the same samples, the reference method’s results have been shown to be about 1.5-percent higher than those of the definitive method.
CDC disseminates the reference method through the National Reference Method Laboratory Network for Cholesterol Standardization. This includes nine laboratories located throughout the United States and four overseas. Because the reference method is expensive and labor intensive, it is not considered practical for use in most clinical laboratories. Consequently, it is used primarily by research laboratories and manufacturers, two settings in which closer traceability to the definitive method is essential.
The network provides a support system that would permit a laboratory or manufacturer to gauge its total cholesterol test accuracy and standardize its measurements. This can be done by splitting samples with a network laboratory and comparing results. Participation in the network is relatively low when one takes into consideration the number of laboratories in the United States. In 1991, for example, 170 laboratories applied for a certificate of traceability and 58 percent passed (if a laboratory fails, it can reapply for certification). In 1992, 167 laboratories applied for a certificate of traceability and 79 percent passed.
Although participation is low, CDC officials estimate that 95 percent of the types of instrument systems most common in U.S. laboratories have been certified through the reference network as meeting NCEP standards mentioned earlier in this chapter (a list of these analyzers is published in Clinical Chemistry News). CDC representatives caution that the reference laboratories test only an analytical system for potential to meet these standards. Day-to-day consistency in a laboratory requires rigorous quality controls that help ensure that an analyzer will perform as it is capable of performing. In other words, if an analyzer is not maintained properly, it will not provide results that are constantly accurate.
Another way in which NRS/CHOL attempts to transfer accuracy to laboratories is through “quality control” substances called standard reference materials, which are the link between the definitive and reference methods and manufacturers of analyzers and reagent systems. These include CDC and NIST-produced and certified materials (which are made in stabilized, frozen, or lyophilized, or freeze-dried, forms) that are assigned target values for cholesterol using the reference or definitive methods.
section_title: Proficiency Testing Services
paragraphs: Proficiency testing (outside surveillance) services have an increasingly important role in efforts to achieve accuracy and standardization of clinical laboratory tests because they provide the basis for interlaboratory comparison of test results and accuracy across analyzers. Proficiency testing programs send quality control materials that participating laboratories analyze and the results are compared with a target value determined by CDC’s reference method. These test results are divided into peer groupings (by instrument type), permitting laboratory staff to judge how their results compare with laboratories using the same method as well as the CDC reference method result. CAP and the American Association of Bioanalysts are two major groups involved in this work. CAP’s Comprehensive Chemistry Survey has 12,000 subscribers that use its service to evaluate several different clinical chemistry tests. This service is not generally used by smaller laboratories. The American Association of Bioanalysts does similar types of proficiency testing.
National trends through 1990 in interlaboratory comparability (that is, the degree to which established test values vary from one laboratory to the next) for total cholesterol are as follows: 1949, 23.7 percent; 1969, 18.5 percent; 1980, 11.1 percent; 1983, 6.4 percent; 1986, 6.2 percent; 1990, 5.5 to 7.2 percent. These data indicate that interlaboratory precision in those clinical laboratories participating in the CAP survey improved considerably from variability of about 24 percent in 1949, to 1983, when it appears to have leveled off at the 6-percent range. These differences between laboratories suggest that method and laboratory-specific biases contributed to overall inconsistency in cholesterol analyses. Another indicator of precision is consistency within individual laboratories. CAP data indicate intralaboratory precision for cholesterol measurements (where participating laboratories analyze the same quality control materials repeatedly over an extended period) improved from 4.1 percent in 1975 to 3.5 percent in 1985.
section_title: Matrix Effects
paragraphs: Efforts to achieve standardized, accurate cholesterol measurements through NRS/CHOL and proficiency testing programs have encountered serious problems with the use of quality control (reference) materials. These are termed “matrix” effects and arise when “cholesterol recovered from the control material matrix may not compare with that typically recovered from fresh patient specimens.” This is because the matrix surrounding the cholesterol quality control material interferes with the analysis, causing erroneous results (matrix effects do not arise when analyzing fresh blood samples). This is a function of instrument design, reagent composition, method employed, and the material formulation. Because these quality control materials are key to transferring accuracy and quality control in NRS/CHOL and assessing precision in proficiency testing programs, matrix effects present considerable problems. While most attention has focused on matrix effects in quality control materials used to standardize total cholesterol measures, there is also concern that HDL cholesterol control materials may be subject to these effects.
Recent interest in the problems presented by matrix effects is linked to the Clinical Laboratory Improvement Amendments of 1988, which required that proficiency testing be used to evaluate the quality of laboratory results. Matrix problems can make it impossible to assign a target value to quality control material that will apply to all routine testing methods. Industry and academic research efforts are underway to address the measurement problems associated with matrix effects but practical solutions are not yet available. Research has focused on establishing correction factors to account for the matrix error component (derived from comparisons of test results using fresh samples and quality control materials) as well as on developing new analytical systems and quality control materials that can accurately measure both fresh patient and quality control materials.
subsections: section_title: CAP and CDC Collaborative Study
paragraphs: A recently published CAP and CDC collaborative study examined matrix effects on cholesterol tests. A total of 997 laboratories that participate in the CAP survey were selected (selection method was not specified) to analyze both a freshly frozen serum pool and a lyophilized (freeze-dried) CAP chemistry quality control sample simultaneously, permitting comparisons and bias to be calculated. Laboratories that had submitted incomplete data or had results considered to be outliers (defined in this study as a pooled within-run coefficient of variation across three samples that exceeded 10 percent or a within-run bias of any sample of 25 percent or more relative to the reference method value) were excluded from the analysis. Laboratories that participated in the study were drawn from CAP survey participants, which are mainly hospital laboratories. They are, thus, not representative of small independent laboratories such as those found in physicians’ offices, or even hospitals. While the ability to generalize from this study is limited, the authors make several points that have important consequences for cholesterol measurement.
The CAP and CDC study classified the cholesterol analysis methods into 37 instrumentation and reagent groups. This figure indicates the range of instruments and reagent combinations that regulators must work with in attempting to achieve standardization. Across this group of instruments, they found that “26 (70%) of  37 methods evaluated had statistically significant calibration bias compared with the reference method. The calibration bias of 13 methods (41%) exceeded the NCEP 3% limit for bias.” When the investigators adjusted the results to compensate for matrix effects, “92% to 93% of adjusted results met the NCEP 8.9% total error goal relative to the reference method due to superior interlaboratory precision of some of the biased methods.” For the fresh-frozen serum sample that was analyzed, test results (N = 900) had a mean bias of 0.1 percent that was nearly identical to the reference method and a coefficient of variation of 4.6 percent, the latter figure slightly exceeding the 1992 NCEP/LSP goal.
Thus, 70 percent of enzymatic methods used to measure cholesterol in the CAP and CDC study were subject to matrix effects when testing quality control material. The implication of this for NRS/CHOL is that the use of fresh human samples, such as by splitting samples with a member of the National Reference Method Laboratory Network and comparing results, may be a better means to transfer accuracy than the use of processed quality control materials. Given the number of laboratories in the nation and the limited number of National Reference Method Laboratories, this would be a difficult if not impossible task. Table 3.1 lists the 37 instrument and reagent systems and calibration bias relative to the reference method.
section_title: Veterans Affairs Laboratory Study
paragraphs: A study similar to the CAP and CDC investigation was undertaken in 112 VA laboratories in conjunction with CDC. Because VA has the nation’s largest hospital system, it provides insight into large-scale efforts to standardize cholesterol measurements. Briefly, the VA research group asked participating laboratories to conduct analyses of fresh serum samples and 1990 CAP quality control materials, permitting comparisons of how well instruments analyzed both types of specimens. This study team found “significant matrix-effect biases with the CAP Survey materials in six of the eight major peer  groups, despite the fact that accuracy of cholesterol measurements was maintained with fresh serum samples.”The authors concluded that “CAP PT  materials used currently do not behave in a manner identical to fresh human serum when measuring cholesterol on many, but not all, analytic systems.” Table 3.2 presents the study findings.
The VA study authors noted that the biases that arise from matrix effects will cause incorrect conclusions about the accuracy of laboratory procedures done on fresh patient specimens. Further, matrix effects will “severely hamper interlaboratory accuracy transfer, standardization efforts, and monitoring performance of a laboratory’s testing accuracy . . . .”
section_title: Accuracy and Precision of Desk-Top Instruments
paragraphs: Cholesterol is often measured with small, portable and semiportable devices called desk-top analyzers, in either a physician’s office or a nontraditional setting such as a health fair. Desk-top systems generally use the same kinds of enzymatic methods employed in laboratory settings.
NCEP guidelines do not differentiate between desk-top analyzers and those used in laboratories; all such devices are held to the same overall accuracy standards. A recent study that summarized desk-top analyzers concluded: “In general, desk-top analyzers give fairly accurate measurements on average, but tend to be somewhat more variable than laboratory-based methods in individual samples.” The same article links this difference in part to the use of fingerstick blood samples with these analyzers, the results of which are likely to differ from venous samples. Other factors that contribute to their measurement variability include lack of operator training and use of such devices in field settings where frequent transportation and changes in temperature and humidity can affect test results.
We identified 13 recent studies that evaluated desk-top analyzer performance. We discuss several of these studies in this section, focusing on those that permit comparison of data across devices. The first study evaluated five analyzers under tightly controlled conditions: Analyst (DuPont), Ektachem DT-60 (Eastman Kodak), Reflotron (Boehringer-Mannheim Diagnostics), Seralyzer (Ames Division, Miles Laboratories), and Vision (Abbott Laboratories). In terms of accuracy of total cholesterol measurements, the Ektachem DT-60 and the Vision had biases of less than 2.0 percent, within the current NCEP bias goal of <3 percent. The three other instruments had biases ranging from 5.2 percent to 10.4 percent, thus exceeding the NCEP goal (see table 3.3).
Only three of the five analyzers tested could conduct HDL and LDL cholesterol analyses. Across the three HDL cholesterol levels tested, Kodak Ektachem DT-60 had results that were approximately 6.0-percent higher than the true value while the Analyst and Vision analyses of the low HDL cholesterol measure were –12.7 percent below and 29.7 percent above the true value, respectively. LDL cholesterol measures, derived with the Friedewald equation, conducted on the Kodak Ektachem DT-60 and Vision had an error of less than 3.0 percent while the Analyst exceeded 17 percent at both LDL cholesterol levels tested. The consequence of such error is that the correct total, HDL, and LDL cholesterol value is systematically over- or underestimated.
Data on the precision of these analyzers are presented in table 3.4. Note that the coefficient of variation of the Reflotron and Seralyzer for total cholesterol is 10 percent or higher, exceeding the current NCEP precision goal of <3 percent.
Another perspective on the data in the preceding tables is how the results could influence the risk category into which a patient is classified (desirable, borderline-high risk, or high risk for coronary heart disease). Two instruments, the Kodak Ektachem DT-60 and Abbott Vision, correctly classified 95 percent and 94 percent of the total cholesterol specimens, respectively. The Analyst, Reflotron, and Seralyzer correctly classified 74 percent, 83 percent, and 75 percent of patient total cholesterol specimens, respectively.
A second study we reviewed that was published in 1993 also evaluated five desk-top devices used to measure cholesterol in screening environments, assessing bias, precision, and patient misclassification error for capillary and venous whole blood and venous plasma. The devices were the Ektachem DT-60 (Kodak), Liposcan (Home Diagnostics), QuickRead (Photest), Reflotron (Boehringer-Mannheim), and Vision (Abbott). The authors concluded that none of these devices met the NCEP performance recommendations regarding bias and precision. Of interest were findings regarding average percentage bias, which differed for capillary and venous whole blood (see table 3.5) and misclassification rates (see table 3.6), which ranged from false negative rates as high as 37 to 48 percent for the Liposcan to false positives up to 38 and 34 percent for the QuickRead. Misclassification into false positive categories was 18 percent for the Vision, 14 percent for the Reflotron, and 7 percent for the Ektachem DT-60.
section_title: Home Test Devices
paragraphs: Home test kits to measure total cholesterol have also been cleared by FDA and have recently begun to be marketed directly to consumers (they have been available to physicians since 1991). Total cholesterol results obtained with the AccuMeter, currently the only such device being marketed in the United States, for 100 patients were compared with a CDC-standardized laboratory at the Medical College of Virginia. While AccuMeter’s results met NCEP guidelines for measurement bias (<3 percent) for both capillary and venous blood when using a mean bias measure, these researchers found that mean absolute percentage bias was 5.7 percent and 5.2 percent, respectively. In addition, 18 to 20 percent of samples fell outside the +8.9 percent of the reference result, the level the NCEP established for acceptable total error for single cholesterol measurements. Figures for precision, from 40 total cholesterol assays done in duplicate from three pools of human serum with mean concentrations of 182 mg/dL, 223 mg/dL, and 266 mg/dL, exceeded NCEP/LSP guidelines (<3 percent precision error); the coefficients of variation were 4.5 percent, 5.4 percent, and 5.8 percent, respectively. The authors noted that approximately 5 percent of the devices did not function properly and could not provide a cholesterol reading.
We met with FDA officials to discuss their decision to permit marketing of the AccuMeter under 510(k) regulations. They explained that it met the criteria of “substantial equivalence” to an analyzer currently being marketed, therefore complying with existing regulations, although the device does not meet NCEP standards for precision and accuracy (as judged by “traceability” to the Abell-Kendall reference method).
section_title: Factors That Influence the Variability of Cholesterol Levels
paragraphs: Even if a single cholesterol measurement were analytically accurate and precise, it would not reflect how a person’s cholesterol can vary from day to day. Total, HDL, and LDL cholesterol levels vary over time and are influenced by what are termed preanalytic or biological factors that include behavioral (exercise, diet, alcohol consumption), clinical (disease, pregnancy), and sample collection conditions. In this chapter, we answer our third evaluation question: What factors influence cholesterol levels?
Scientific literature indicates that some variation and fluctuation of an individual’s total, HDL, and LDL cholesterol is normal and to be expected. For instance, in some individuals, week-to-week fluctuations can be dramatic while in others virtually no change may occur over the same time period. Overall, biological variation of total cholesterol is reported to average 6.1 percent; HDL cholesterol variation averages 7.4 percent; LDL biological variability, 9.5 percent; triglycerides, 22.6 percent. These findings suggest that variation in cholesterol levels is normal and, for some individuals, can be quite pronounced. The implication for testing, particularly for patients near a cutpoint (such as 240 mg/dL) is that repeated measurements may be necessary. In light of measurement uncertainty for HDL and LDL, multiple measures of these subfractions may be warranted, particularly before making a diagnosis.
Other factors—diet, exercise, alcohol intake—appear to have differing effects on individuals’ cholesterol levels. The amount of the effect varies depending on the amount and duration of intake and physiological factors. In some, it may not have a large effect on total and LDL cholesterol levels. This may be partially related to the estimates that one third of an individual’s cholesterol level is linked to diet while the body produces the remaining two thirds. The evidence regarding regular exercise points to the benefits associated with such activity, as measured by changes in lipid levels. While alcohol intake can have a positive effect on cholesterol levels, consumption must be balanced with the potential risks associated with it. The potential effect of diet on cholesterol levels was noted in the 1990 NCEP report on cholesterol measurement, which recommended that patients maintain their usual diet and that their weight be stable for at least 2 weeks before their cholesterol level is measured.
Clinical factors such as disease, pregnancy, and some medications (diuretics, beta blockers, oral contraceptives) can also alter cholesterol levels. How a blood specimen is taken can also have a crucial role in cholesterol analysis. Some research has found that fingerstick (capillary) samples differed markedly from venous samples when analyzed by the same device while other researchers have called for more standardized specimen collection techniques.
subsections: section_title: Biological Variation
paragraphs: Cholesterol levels within a person vary over time, depending on a number of factors. As discussed in chapter 1, for example, as people age, their total cholesterol level tends to increase. However, cholesterol levels can also vary considerably between measurements because of what is termed intra-individual biological variability—that is, normal fluctuations in cholesterol levels are estimated to account for about 65 percent of the total intra-individual variation for both total and HDL cholesterol and about 95 percent of the variation for triglycerides. Studies have linked other types of biological variation to diet, alcohol intake, smoking, and physical activity. The body of literature on this subject is large: a 1992 article reviewed more than 300 publications, most of which had been published within the previous 5 years.
A recent statistical synthesis of findings from 30 studies published between 1970 and 1992 provides considerable information on intra-individual biological variation—that is, the normal fluctuation in cholesterol levels referred to above. According to this review, total cholesterol is the most stable lipid, with the day-to-day biological variation averaging 6.1 percent; variations in HDL cholesterol concentrations, 7.4 percent; LDL biological variability, 9.5 percent; triglyceride, 22.6 percent.
The number of subjects in the selected studies of total cholesterol variability ranged from small (less than 20) to quite large (14,600). Not surprisingly, the number of specimens and the sampling intervals varied as well. Two large studies analyzed two specimens, taken 1 month apart, while another study with a smaller number of subjects analyzed specimens taken twice a week for 10 weeks. Results for HDL variability were based on 16 studies, triglyceride variability 19 studies, and LDL variation 10 studies.
Two recent articles have also reported similar findings. One study compared total cholesterol and HDL measurements taken from 40 male subjects 1 week apart. The authors found a relatively wide range of variability in some patients: one patient’s total cholesterol declined dramatically from one week to the next, dropping from nearly 300 mg/dL to just over 220 mg/dL, while several others’ total cholesterol level scarcely moved between the two tests (the coefficient of variation for single measurements was 6.8 percent for total cholesterol and 10.5 percent for HDL, slightly higher than the figures from the statistical synthesis reported earlier).
Another study of cholesterol variability tracked 20 subjects 22 to 63 years old measuring their total, LDL, and HDL cholesterol weekly for 4 weeks. The authors found variations of more than +20 percent in the serum levels of total cholesterol, LDL, and HDL in 75 percent, 95 percent, and 65 percent of the subjects, respectively. More important, 40 percent moved in or out of one of the risk categories, and 10 percent moved two categories—from “desirable” to “high risk.”
Other research has found that LDL and total cholesterol levels within individuals vary by season, both averaging 2.5-percent higher in the winter than the summer. The HDL cholesterol level, however, has not been found to vary seasonally. Women are affected by another aspect of biological variability; total cholesterol concentrations may average 20 percent lower during the luteal phase (the period immediately after ovulation) of the menstrual cycle.
section_title: Behavioral Factors
paragraphs: Cholesterol levels vary because of behavioral factors, and some of this variability can influence short-term measurements. For example, strenuous exercise 24 hours prior to having a blood specimen taken can elevate an individual’s HDL cholesterol level. Likewise, moderate alcohol consumption can increase HDL and decrease LDL cholesterol levels. Behavior over longer periods of time can also affect cholesterol levels—diet, alcohol consumption, exercise. The relevance to the measurement theme of this report is that there is more to variation in cholesterol levels than inaccurate laboratory tests.
subsections: section_title: Diet and Cholesterol
paragraphs: Consumption of certain saturated fatty acids and, to a lesser extent, cholesterol is linked to higher serum LDL cholesterol values. In terms of diet, an increase in cholesterol intake of about 100 mg (per 4,200 joules) raises plasma cholesterol by about 10 mg/dL. Progressively higher cholesterol intakes exceeding 500 mg appear to have smaller incremental effects on cholesterol levels. The same study points out that dietary cholesterol is incompletely and variably absorbed by individuals, ranging from 18 to 75 percent. Further, people with the highest LDL cholesterol levels appear to have the highest percentage of absorption of dietary cholesterol. “blood cholesterol responses of individuals differ substantially in response to changes in dietary lipids . . . . For the same increase in dietary cholesterol or saturated fat, the cholesterol levels of most persons will increase, but some will remain essentially unchanged and a few will increase dramatically.”
As noted in chapter 2, only one third of an individual’s cholesterol is derived from diet and the remaining two thirds are manufactured by the liver.
In terms of the contribution that diet can make to cholesterol reduction, the 1993 NCEP guidelines state that men who follow the step I diet could expect their total cholesterol level to be reduced 5 to 7 percent while those who follow the more restrictive step II could expect an 8-to-14 percent reduction. These estimates are based on models derived from metabolic ward studies (done on institutionalized patients), which closely monitored and controlled individuals’ adherence to their diet. Some researchers have noted that such reductions can be difficult to achieve in a “free living” population.
section_title: Alcohol Intake
paragraphs: Published epidemiological studies have demonstrated a relationship between alcohol intake and changes in cholesterol profiles. The amount of change attributed to alcohol depends on the amount consumed, individual susceptibility, genetic variables, and diet. Moderate alcohol intake (defined as several drinks a day) appears to increase HDL cholesterol and may be associated with reduced risk of coronary heart disease. Greater alcohol consumption is also associated with a lowering of LDL cholesterol and an increase in triglycerides. In one study, it is estimated that 4 to 6 percent of the variance of HDL cholesterol levels in the population may be linked to alcohol consumption.
section_title: Exercise
paragraphs: Exercise has been shown to influence cholesterol levels and has received increased attention as having a preventive effect on coronary heart disease. Researchers have found that exercise that is strenuous and promotes endurance causes LDL, triglycerides, and apo B to decrease while raising HDL and apo AI levels. Other evidence regarding exercise points to the benefits of brisk walking. One study found that previously sedentary women who walked an average of 155 minutes per week decreased their total cholesterol level by 6.5 percent compared with a decrease of 2.2 percent in control subjects, and the HDL level of walkers increased 27 percent, compared with a 2-percent increase in controls. A recent article suggests that the effect of these changes depends on the volume, intensity, and type of exercise undertaken, a slight variation on earlier work.
Apart from longer-term effects, acute exercise also causes a significant rise in HDL levels such that it is recommended that patients avoid any strenuous exercise 24 hours prior to having a blood specimen taken.
section_title: Obesity
paragraphs: Obese individuals have been found to have higher total and LDL cholesterol and triglyceride levels and lower HDL cholesterol when compared to nonobese members of control groups. When an obese individual loses weight, a decline in triglyceride level occurs (about 40 percent); total and LDL cholesterol levels are found to decline about 10 percent while the HDL level increases about 10 percent. The implication for cholesterol measurements, particularly for obese individuals who repeatedly gain and lose weight, is that such fluctuations can be the source of significant variation in lipoprotein levels. In fact, NCEP/LSP recommended that an individual’s weight be stable and that he or she maintain his or her usual diet for at least 2 weeks prior to having cholesterol measured.
section_title: Clinical Factors
paragraphs: A person’s cholesterol profile can be affected by acute, infectious, and metabolic diseases, and some types of medications have been linked with elevated levels in some patient groups.
subsections: section_title: Disease
paragraphs: Several conditions are associated with increased cholesterol levels. Diabetes mellitus and hypothyroidism are cited as the most common of these, with total cholesterol and LDL cholesterol levels being elevated in 30 percent of the patients with the latter condition. Patients with diabetes mellitus sometimes have elevated triglycerides, and higher levels of insulin are positively associated with unfavorable levels of total and LDL cholesterol, triglycerides, apo B, and blood pressure, and negatively with HDL cholesterol components.
Acute myocardial infarction is associated with decreases in levels of total cholesterol, LDL, apo AI, and apo B. Indeed, lipid levels after a heart attack are affected to such a degree that it is recommended that blood specimens be obtained within 24 hours of the event; if they cannot be taken within 24 hours, then they should not be taken for 3 months because the test will not accurately reflect the patient’s usual lipid level. Other diseases such as Tay-Sachs, rheumatoid arthritis, and infections can also alter lipid profiles. In addition, familial hypercholesterolemia and other related disorders are associated with increased blood cholesterol levels.
section_title: Drug-Induced Variations and Pregnancy
paragraphs: Medication can also alter lipid levels. Diuretics, some beta blockers, and sex steroids have been cited as changing lipid levels. Oral contraceptives high in progestin can increase serum total and LDL cholesterol and decrease HDL cholesterol levels, while contraceptives with high estrogen content can cause opposite changes. Similar changes have been found in postmenopausal women taking estrogen supplements.
Pregnancy is associated with changes in lipid profiles in the second and third trimesters, when total and LDL cholesterol, triglyceride, apo AI, apo AII, and apo B are significantly increased. Because of these changes, lipid levels are affected to the degree that testing is not recommended until 3 months postpartum or 3 months following cessation of lactation.
section_title: Laboratory Factors
paragraphs: How a blood specimen is collected and handled may affect lipid levels. For example, blood cholesterol samples are often drawn when the patient is in a fasting state, particularly when a lipid profile is to be taken. This is because eating a typical fat-containing meal causes a patient’s lipid profile to change, an effect that lasts about 9 hours. Typically, triglyceride levels increase as does very-low-density lipoprotein (VLDL), while LDL cholesterol falls significantly. “the most reliable screening measurements were obtained when the analyses were performed in venous plasma samples by a qualified clinical laboratory. . . . The most-variable measurements were obtained with the capillary samples, and these measurements seemed to be most prone to misclassification overall.”
A 1993 article briefly discusses the difference between venous and capillary samples, pointing out “contradictory results” (that is, some studies reporting either higher or lower capillary results than venous results, depending on the various procedures and devices tested) and a lack of consensus in the literature about such differences. The study’s authors conclude that “capillary collection technique is critical and must be standardized to obtain reliable cholesterol results.”
How the specimen is taken and prepared for analysis also can affect lipid level measurements. Here factors such as the knowledge and experience of the laboratory technician are important. For example, the length of time a person is sitting or standing prior to having the specimen taken has been demonstrated to influence cholesterol test results. Patients should remain seated for at least 15 minutes before a venous sample is taken, and if a tourniquet is used, it should be applied for less than 1 minute before the specimen used for a lipid analysis is taken. Proper storage of samples is also important to avoid changes in the composition of samples and to ensure accurate measurement results. Use of a standard collection policy by trained laboratory technicians can help minimize variability associated with these factors.
section_title: The Potential Effect of Measurement Uncertainty and Agency Comments and Our Response
paragraphs: In this chapter, we discuss the study’s fourth evaluation question: What is the potential effect of uncertain measurement? This is followed by our conclusions and discussion of agency comments.
subsections: section_title: Addressing Measurement Error
paragraphs: Progress has been made in improving analytical accuracy in cholesterol measurement, with the development of better methods and materials in recent years. Yet, despite the attention cholesterol has received, it continues to be difficult to measure with accuracy and consistency across the broad range of devices and settings in which it is analyzed. While several studies have found that accuracy with patient samples was good, problems with matrix effects from using processed quality control materials have occurred, thus making it difficult to adequately assess accuracy among laboratories. In addition, the lack of information on accuracy in many laboratory settings where patients are likely to be tested, such as commercial laboratories, physicians’ offices, and mass screening locations, makes it impossible to know whether the accuracy goals established for total and HDL cholesterol are uniformly being met.
Even if one could be certain that a laboratory could provide reasonably accurate and precise test results, biological and behavioral factors such as diet, excercise, or illness cause an individual’s cholesterol level to vary. It has been estimated that such factors may account for up to 65 percent of the total variation in an individual’s reported cholesterol measurement. Studies have documented that some individuals’ cholesterol level can vary dramatically from week to week while others’ remains relatively constant. Although some biological variation can be controlled for, by having patients maintain their weight and diet for a modest period prior to measurement, many factors cannot be controlled.
Total error from both analytical and biological variability can be considerable, as shown in tables 5.1 and 5.2, where calculations are made for hypothetical total and HDL cholesterol test results at different specified levels. For the purposes of this analysis, which is intended to illustrate the potential range of variability around an actual or known cholesterol level, we used the current goals for total analytical error (+8.9 percent for total cholesterol according to NCEP and +30 percent for HDL according to HCFA) and what is currently known about biological variability from a synthesis of studies (6.1 percent for total cholesterol and 7.4 percent for HDL cholesterol). Both analytical and biological variability can of course be lower or higher than these figures, depending on a combination of factors.
The results in tables 5.1 and 5.2 show that a single cholesterol measurement may be highly misleading with respect to an individual’s actual cholesterol value. A total cholesterol value that is known to be 240 mg/dL, for example, may vary as much as 16 percent or range from 201 to 279 mg/dL, when using these error rate assumptions. Similar estimates for HDL cholesterol measurements are presented in table 5.2.
The implication of these estimates is that cholesterol levels should be thought of in terms of ranges rather than absolute fixed numbers. Compensating for variation by using the average of at least two cholesterol measurements is in line with the current NCEP guidelines and recent literature on the subject. The most recent NCEP Adult Treatment Panel recommends that a second test be done when an initial measurement has found that total cholesterol exceeds 200 mg/dL and HDL is under 35 mg/dL. In terms of HDL and LDL cholesterol, which have been documented to have analytical and biological variation somewhat higher than total cholesterol, more variability can be expected. CDC officials we interviewed emphasized that considerable scientific work remains before HDL measurement is as well understood as total cholesterol. Authors of a recent study in Clinical Chemistry therefore recommend that as many as four HDL and LDL cholesterol tests be done before making treatment decisions.
S. J. Smith et al., “Biological Variability in Concentrations of Serum Lipids: Sources of Variation among Results from Published Studies and Composite Predicted Values,” Clinical Chemistry, 39:6 (1993), 1021. equal to 0.19; with four specimens, the relative range should be less than or equal to 0.21.
section_title: Implications of Measurement Variability
paragraphs: Having accurate and precise cholesterol measurements is important, given the central role that cholesterol measurement has in classifying, evaluating, and treating patients deemed at risk of coronary heart disease. As noted in chapter 1, the average total cholesterol level for U.S. adults 20 years old and older is about 205 mg/dL, which puts them within the NCEP-defined borderline-high risk category. Moreover, 29 percent of U.S. adults, 52 million people, have a cholesterol level that is classified as too high, making them candidates for dietary therapy. Of this group, an estimated 12.7 million adults, one third of whom have established coronary heart disease, might be considered candidates for drug therapy to lower their cholesterol level. Once drug therapy is initiated, it may need to be maintained for life.
Although the NCEP guidelines recognize the problem of measurement variability and the guidelines stress the need for multiple measurements, important consequences can be associated with measurement error. The potential exists, for example, that physicians may not account for measurement problems and may base decisions about patients on incorrect test results. In a worst-case scenario, two types of diagnostic errors could occur: false-positive or false-negative screens. A false-positive screen could result in treating individuals who in fact have a desirable total, HDL, and LDL cholesterol level. A false-negative result would incorrectly reassure an individual that his or her cholesterol level is low. The risk of misclassification would be greatest for those whose measured cholesterol levels are closest to one of the cutpoints. There is less ambiguity when values are well above or below a cutpoint. The likelihood of such errors occurring, however, is greater if physicians rely on only a single cholesterol measurement in making treatment decisions.
Continuing efforts are needed to improve the accuracy and precision of lipid measurements so that medical decisions to initiate and continue treatment to lower elevated cholesterol levels can be both effective and efficient. To minimize misclassification problems, it is also important to ensure that physicians who evaluate and treat patients with elevated cholesterol levels are knowledgeable about measurement variability and the need to conduct multiple tests.
section_title: Agency Comments and Our Response
paragraphs: Officials from HHS reviewed a draft of this report and provided written comments, reproduced in appendix I. In addition, HHS provided draft technical comments that we have incorporated in the text where appropriate.
Overall, HHS officials believed that cholesterol measurement has improved substantially in recent years and that accuracy in laboratories across the country is better than what is presented in our report. Regarding general comments on the need for better standardization materials (lyophilized serum pools without matrix effects), we agree that this is a major challenge that must be addressed if measurement is to be improved. This point was made in NCEP’s 1990 report on cholesterol measurement, indicating that this is not a new problem but rather one that was noted previously.
HHS did not concur with a recommendation we included in the draft report concerning an assessment of whether problems of patient “misclassification” result from measurement variability. It indicated that information on misclassification already exists and that additional work would only provide further definition of the issue rather than solving known problems such as the effect of matrix effects on measurement accuracy. We recognize that some information on this issue does exist and also understand that further efforts are currently under way, particularly by NIH and CDC, to assess how the NCEP guidelines are being implemented in practice and to evaluate overall laboratory performance. We have deleted our draft recommendation from the final report because these ongoing agency efforts should respond to our concerns about misclassification. We encourage HHS to continue this work and provide the results to the Congress and the general public.
The agency also suggested that the discussion of diet and clinical trials that we included in the draft was too brief. We have deleted this discussion from the final report and will address it in more depth in a later report we are preparing on the clinical trial base of information that supports the NCEP guidelines.