PROCEEDINGS OF THE 

 NISTIR 6742
i 

NISTIR 6742 

Proceedings of the Workshop on Measurement 

Traceability for Clinical Laboratory Testing and 

In Vitro Diagnostic Test Systems 

Ellyn S. Beary, Editor 

May 2001 

U. S. DEPARTMENT OF COMMERCE 

Donald L. Evans, Secretary 

TECHNOLOGY ADMINISTRATION 

Karen H. Brown, Acting Under Secretary for Technology 

NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY 

Karen H. Brown, Acting Director
ii 

Compiled by Gretchen Kennedy 

For information contact NIST/CSTL 

Telephone:301-975-8300 

Facsimile:301-975-3845 

E-Mail: cstl@nist.gov 

Web Sites:www.nist.gov 

www.cstl.nist.gov 

Any mention of commercial products is for information only. It does not imply recommendation or 

endorsement by the National Institute of Standards and Technology, nor does it imply that the 

materials or equipment mentioned are necessarily the best available for this purpose. 

Version of February 2001 
iii 

TABLE OF CONTENTS 

(Note: The Agenda Items are linked, and Bookmarks are available to 

facilitate navigation through the document) 

I. INTRODUCTION AND WELCOME........................................................... 1 - 7 

II. AGENDA..................................................................................................... 8 - 9 

III. MANUSCRIPTS AND VIEW GRAPHS ................................................. 11 - 266 

• Kim Carneiro, Ph.D. ....................................................................................11 

• Emil Voelkert, Ph.D. ....................................................................................31 

• Hratch G. Semerjian, Ph.D. ........................................................................59 

• Lothar Siekmann, Ph.D. ..............................................................................85 

• Heinz Schimmel, Ph.D................................................................................115 

• Neil Greenberg, Ph.D.................................................................................133 

• John Eckfeldt, M.D., Ph.D..........................................................................161 

• Willie E. May, Ph.D. ..................................................................................189 

• George G. Klee, M.D., Ph.D. .....................................................................223 

• Joan Walsh Cassedy, M.B.A.......................................................................247 

IV. POSTER SESSIONS............................................................................ 267 - 288 

V. BREAKOUT SESSION REPORTS AND DISCUSSION ...................... 289 - 339 

• Neil Greenberg, Ph.D.................................................................................291 

• William F. Koch, Ph.D...............................................................................298 

• Gary L. Myers, Ph.D..................................................................................302 

• Don Powers, Ph.D......................................................................................309 

• Greg Miller, Ph.D. .....................................................................................315 

VI. SUMMARIES AND ACTION PLAN.................................................... 341 - 346 

• William F. Koch, Ph.D...............................................................................341 

VII. REGISTRATION LIST......................................................................... 347 - 360 
iv 
1 

WELCOME 

Dear Colleague, 

A workshop on Measurement and Traceability for Clinical Laboratory Testing and In 

Vitro Diagnostic Test Systems will be held at the National Institute of Standards and 

Technology (NIST) on November 2 and 3, 2000. It is becoming increasingly evident that 

a formal national traceability structure will be essential for U.S. clinical device 

manufacturers to compete in the global marketplace. One of the driving forces for this 

workshop is the proposed ISO standard for Calibration traceability of In Vitro Diagnostic 

Medical Devices, which is being drafted in response to the European Union Directive. 

This Directive requires that all IVD assays be traceable to the highest available reference 

material or method. NIST is committed to provide the nation’s measurement 

infrastructure, and indeed, has many primary standard reference materials that provide 

Traceability in the clinical arena. But more is required to properly position the U.S. 

clinical community to compete in this changing environment. 

This workshop will provide a forum for all stakeholders from manufacturers of clinical 

devices and secondary reference material producers, to government agencies and 

professional associations. The goal is to assess the measurement needs of the clinical 

community, and collaboratively produce a plan that addresses international standards 

issues and assures U.S. manufacturers of an accessible national traceability system. 

This letter serves as a call to action. Your participation is important to assess the current 

and future needs of the integrated clinical community and to ensure a vital U.S. role in 

the world market. 

Sincerely, 

Deputy Director 

Chemical Science and Technology Laboratory, NIST 

Immediate Past Present, NCCLS 
2 

OVERVIEW 

The National Institute of Standards and Technology (NIST), in cooperation with the 

NCCLS (National Committee for Clinical Laboratory Standards) and with the 

sponsorship of the Centers for Disease Control and Prevention (CDC), the Advanced 

Medical Technology Association (AdvaMed), the College of American Pathologists 

(CAP), and the American Association for Clinical Chemistry (AACC), is planning a 

workshop on measurement traceability for clinical laboratory testing and in vitro 

diagnostic test systems for the IVD industry, professional organizations, government, and 

national and international standards developing organizations. The workshop will be held 

on the NIST-site in Gaithersburg, Maryland, November 2-3, 2000. The purpose of the 

workshop is to actively involve all interested parties in the solution to establishing the 

means and processes for assuring measurement traceability and reference systems for the 

healthcare community. 

Topics to be Discussed: 

The workshop will include scientific experts who participated in development of the 

standards for measurement traceability in ISO Technical Committee 212 (ISO/TC 212). 

Speakers will discuss the technical requirements for establishing measurement 

traceability, and present practical approaches that manufacturers may follow to satisfy the 

requirements of these ISO standards. Additional presentations will focus on the state of 

the art in the calibration of IVD test systems, review global resources already available to 

manufacturers, and identify areas for which improvements are needed. Breakout sessions 

will be scheduled for participants to discuss reference measurement methods, reference 

materials, and laboratories that perform reference measurement procedures 

Recommendations will be developed to address needs in the area of clinical measurement 

traceability and how progress can be accelerated through cooperation among the 

stakeholders. The emphasis will be on solutions that are in compliance with international 

standards and industry’s needs. 

Stakeholders: 

 IVD manufacturer’s representatives involved in product development, quality 

assurance and regulatory affairs 

 Representatives of regulatory agencies and notified bodies 

 Providers of proficiency testing programs, laboratory accreditation, and external 

quality assessment materials 

 Providers of reference methods and materials for clinical laboratory analysis, 

supporting measurement traceability of commercial IVD systems 

 Laboratory professionals involved in standardization of laboratory methods 
3 

Background: 

Traceability to internationally recognized and accepted standards is an important 

component in assuring the accuracy and comparability of clinical laboratory 

measurements. In addition, the global marketplace is presenting new demands for 

measurement traceability. NIST has a long history of providing such measurement 

standards in several fields including the clinical laboratory community and is continuing 

its efforts to develop new reference methods and materials for important health-status 

markers to meet on-going and future needs for traceability. NCCLS is a globally 

recognized, voluntary consensus standards-developing organization that enhances the 

value of medical testing within the healthcare community through the development and 

dissemination of standards, guidelines, and best practices. NCCLS has the Secretariat 

responsibility for the ISO Technical Committee 212 (ISO/TC 212) on Clinical 

Laboratory Testing and In Vitro Diagnostic Test Systems (TC 212), and is the home of 

the National Reference System for the Clinical Laboratory (NRSCL), a collection of 

broadly understood reference systems intended to improve the comparability of test 

results, consistent with the needs of medical practice. Recently, a new opportunity has 

emerged that applies new pressure on the quest for traceability and the demand for 

reference systems. Prompted by the European Union’s In Vitro Diagnostics Directive 

(IVDD), the European Committee for Standardization’s Technical Committee 140 

(CEN/TC 140), in vitro diagnostic systems, began drafting a standard on metrological 

traceability. By working closely together, CEN/TC 140 and ISO/TC 212 will develop 

identical European and ISO standards on this topic. Full implementation of the IVD 

Directive is expected by December 2003 and will require that calibration of all IVD 

assays be traceable to the highest available reference material or method.
4 

SPONSORS 

The National Institute of Standards and 

Technology (NIST), located in Gaithersburg, 

Maryland, is an agency of the U.S. Department 

of Commerce’s Technology Administration. 

NIST’s mission is to promote U.S. economic growth by working with industry to develop 

and apply technology, measurements, and standards. NIST’s Chemical Science and 

Technology Laboratory (CSTL) works to provide the chemical measurement 

infrastructure to enhance U.S. industry’s productivity and competitiveness, assure equity 

in trade, and improve public health, safety, and environmental decision-making. This is 

accomplished through the development and critical evaluation of analytical methods, and 

provision of certified reference materials and reference data that support measurement 

accuracy and traceability for diverse communities such as those performing health-related 

measurements. 

http://www.nist.gov/ 

The College of American Pathologists (CAP) is headquartered in 

Northfield, Illinois with its Division of Government and 

Professional Affairs located in Washington, DC. The College is a 

medical society serving more than 15,000 physician members and 

the laboratory community throughout the world. It is the world's 

largest association composed exclusively of pathologists and is 

widely considered the leader in providing laboratory quality 

improvement programs. The mission of the College, the principal 

organization of board-certified pathologists, is to represent the interests of patients, the 

public, and pathologists by fostering excellence in the practice of pathology and 

laboratory medicine worldwide. 

http://www.cap.org/ 

The American Association for Clinical Chemistry (AACC), 

headquartered in Washington DC, is an international 

scientific/medical society of clinical laboratory professionals, 

physicians, research scientists and other individuals involved with 

clinical chemistry and other clinical laboratory science-related 

disciplines. Founded in 1948, the society currently has 11,000 members in 93 countries. 

The AACC was formed to further the public interest by encouraging the study, advancing 

the science, and improving the practice of clinical chemistry. To this end, they establish 

standards for education, training and certification of clinical chemists and sponsor 

activities that promote scientific knowledge in health-related fields. 

http://www.aacc.org/ 
5 

The Centers for Disease Control and Prevention (CDC), located 

in Atlanta, Georgia, is an agency of the U.S. Department of 

Health and Human Services. In fulfillment of its mission “to 

promote health and quality of life by preventing and controlling 

disease, injury, and disability”, the CDC monitors public health, maintains national health 

statistics, supports immunization services, guards against international disease 

transmission, fosters health environments, and supports research into disease and injury 

prevention including occupational injuries. 

http://www.cdc.gov/ 

NCCLS, headquartered in Wayne, PA, is a globally recognized, 

voluntary consensus standards-developing organization that 

enhances the value of medical testing within the healthcare 

community through the development and dissemination of 

standards, guidelines, and best practices. NCCLS is 

fundamentally and strategically committed to facilitating development of medical-testing 

standards worldwide and to global harmonization through the NCCLS and ISO 

standardization processes. NCCLS has the Secretariat responsibility for the ISO 

Technical Committee 212 (ISO/TC 212) on Clinical Laboratory Testing and In Vitro 

Diagnostic Test Systems (TC 212), and is the home of the National Reference System for 

the Clinical Laboratory (NRSCL), a collection of broadly understood reference systems 

intended to improve the comparability of test results, consistent with the needs of medical 

practice. 

http://www.nccls.org/ 

AdvaMed (formerly HIMA) is headquartered in Washington, DC and 

is the largest medical technology trade association in the world, 

supported by more than 800 medical device, diagnostic products and 

health information systems manufacturers of all sizes. AdvaMed is 

the only medical device association that operates globally to promote 

a legal, regulatory and economic climate that advances health care by 

assuring worldwide patient access to the benefits of medical technology. Together, 

AdvaMed companies form a strong and effective partnership to support policies and 

practices that provide timely delivery of new medical technologies throughout the global 

health-care market. 

http://www.himanet.com/ 

Industry Sponsors

The following corporation sponsors provided funding support. 

BD (Becton, Dickinson and Co.) Ortho-Clinical Diagnostics, Inc. 

Beckman Coulter, Inc. Roche Diagnostics 

Diagnostics Products Corporation 
6 

Workshop on Measurement Traceability 

For Clinical Laboratory Testing 

And In-Vitro Diagnostic Test Systems 

November 2-3, 2000 

Steering Committee 

Basil Doumas, Co-Chair 

Medical College of Wisconsin 

Neil Greenberg, Co-Chair 

Ortho-Clinical Diagnostics 

Carolyn Jones, Co-Chair 

Advanced Medical Technology Association (formerly HIMA) 

William Koch, Co-Chair 

National Institute of Standards and Technology 

Ellyn Beary, National Institute of Standards and Technology 

W. Gregory Cooper, Bio-Rad Laboratories 

John Eckfeldt, University of Minnesota 

Chandra Jain, Beckman Coulter, Inc. 

David Kelly, NCCLS 

Arthur Kessner, Beckman Coulter, Inc. 

George Klee, Mayo Clinic 

Willie May, National Institute of Standards and Technology 

Peter Maxim, Food and Drug Administration 

Kenneth McNeil, Dade Behring, Inc. 

W. Gregory Miller, Virginia Commonwealth University 

Gary Myers , Centers for Disease Control 

Debbie Nadel, American Association for Clinical Chemistry 

Donald Powers, Powers Consulting Services 

Terry Pry , Abbott Laboratories 

Jean Rhame, American Association for Clinical Chemistry 

John Thompson, Dade Behring, Inc. 

Michael Welch, National Institute of Standards and Technology 

Beth Ann Wise, NCCLS 

The steering committee was supported by many others who contributed to the success of 

the workshop, and we are most grateful for all their efforts.
7 

NIST Organizing Committee

Ellyn Beary Chemical Science and Technology Laboratory, NIST 

Jeanice Brown Thomas Analytical Chemistry Division, NIST 

Jennifer Colbert Standard Reference Materials Program, NIST 

Michael Epstein Analytical Chemistry Division, NIST 

Amy Grafmuller Analytical Chemistry Division, NIST 

Kathleen Kilmer Conference Program, NIST 

William Koch Chemical Science and Technology Laboratory, NIST 

Willie May Analytical Chemistry Division, NIST 

Donna Sirk Analytical Chemistry Division, NIST 

Kimberly Snouffer Conference Program, NIST 

Teresa Vicente Conference Program, NIST 

Michael Welch Analytical Chemistry Division, NIST 
8 

AGENDA

Traceability Workshop: November 2 – 3, 2000 

Invited Talks, November 2, 

Chair: Basil Doumas, Ph.D., Medical College of Wisconsin, USA 

Introduction Welcome from NIST Deputy Director Karen Brown, Ph.D. 

9:00 – 9:15 NIST, USA 

Invited Talk 1 EC IVD Directive – Expectations and Kim Carneiro, Ph.D. 

9:15 – 9:45 Implications Danish Institute of 

 Fundamental Metrology 

 Denmark 

Invited Talk 2 Standardization Activities to Support the Emil Voelkert, Ph.D. 

9:45 – 10:15 Implementation of the IVD-Directive: Roche Diagnostics 

 Harmonized Standards and the European Germany 

 Free Market CEN/TC 140, Chair 

 

Invited Talk 3 Status of MRAs and International Hratch G. Semerjian, Ph.D. 

10:15 – 10:45 Standardization NIST, USA 

Invited Talk 4 Traceability of Calibration for IVD Lothar Siekmann, Ph.D. 

11:15 – 11:45 Industry: Process for Credentialing University of Bonn, Germany 

 Materials, Reference Methods, and 

 Reference Laboratories in Europe 

Invited Talk 5 European Reference Materials Activities Heinz Schimmel, Ph.D. 

11:45 – 12:15 in the Clinical Field Institute for Reference 

 Materials & Measurements 

 (IRMM), Belgium 

 

Invited Talk 6 IVD Directive Requirement for Calibration Neil Greenberg, Ph.D., 

12:15 – 12:45 Traceability from the U.S. Industry Ortho Diagnostics, USA 

 Perspective 

Invited Talk 7 History of Reference Systems for John Eckfeldt, M.D., Ph.D., 

 2:00 – 2:30 Clinical Measurements CAP 

 Fairview-University Medical 

 Center, USA 

Invited Talk 8 Development of Reference Methods Willie E. May, Ph.D. 

2:30 – 3:00 and Reference Materials in the U.S. NIST, USA 

Invited Talk 9 Importance of Commutable Human George G. Klee, M.D, Ph.D. 

3:30 – 4:00 Specimens and Test Value Mayo Clinic, USA 

 Distributions for Assay Calibration 

Invited Talk 10 Reference Systems Models from Other Joan Walsh Cassedy, MBA 

 4:00 – 4:30 Industries ACIL, USA 

 

 

Poster Session all day, November 2 

 Chair: Michael J. Welch, Ph.D., NIST, USA
9 

Breakout Sessions, November 3, 2000 

Breakout Session ♦ Convener Topic of Breakout 

Session 1 

Neil Greenberg, Ph.D., D.A.B.C.C. 

Ortho Clinical Diagnostics 

Title: Priorities for National and International Investments 

in New or Improved Reference Systems in Support of 

Clinical Laboratory Measurement 

Objective: To identify and rank the factors that are 

necessary for determining priorities for development of 

new reference systems. 

 ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦

Session 2 

Don Powers, Ph.D. 

Private Consultant 

Title: Reference Materials and Reference Measurement 

Procedures to Support Traceability Requirements of the 

IVD Directive. 

Objective: To identify practical ways to close important 

gaps in available reference measurement procedures and 

materials, and to explore ways to maximize cooperation 

among existing organizations developing and providing 

reference systems. 

 ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦

Session 3 

Greg Miller, Ph.D. 

Medical College of Virginia Hospitals at 

Virginia Commonwealth University

Title: Impact of Method-Material Matrix Interactions on 

Calibration Traceability Protocols for Successful 

Harmonization of Patient Results. 

Objective: To develop a statement of limitations in the 

commutability of reference materials and the appropriate 

use of such reference materials for traceability and 

harmonization of results. 

 ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦

Session 4 

William F. Koch, Ph.D., F.A.C.B. 

NIST/NCCLS 

Title: Development of International Consensus on the 

Credentialing Process for Reference Systems 

Objective: To develop an international consensus on the 

credentialing process for reference systems. 

 ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦

Session 5 

Gary L. Myers, Ph.D., F.A.C.B. 

Centers for Disease Control and Prevention 

Title: Creating and Sustaining Reference Method 

Laboratory Networks 

Objective: To develop criteria necessary for the 

implementation and maintenance of reference 

laboratory networks 

Afternoon Session: Reports from Breakout Session 

 Chair: Willie E. May, Ph.D., NIST, USA 
10 
11 

IVD Directive − Expectations and Implications 

Kim Carneiro 

Danish Institute of Fundamental Metrology 

1. THE IVD DIRECTIVE 

 1.1 What is a European Directive? 

Within the framework of common European legislation, the “new approach” Directives play an important role. A directive is a legislative document issued by the European Council of Ministers and the European Parliament, to be implemented by the member states. Implementation is means that the member states must issue appropriate national legislation to make the directive function in the particular member state. There is a 

time limit for the implementation, which for the IVD directive is December 2003. 

The purpose of a directive is to ensure that “devices”, such as products and materials that are sold in Europe are functional and safe. However, the directive itself provides 

no details. Instead it is states the essential requirements for devices in question for the 

well being of European citizens. It is then up to the manufacturer or importer to demonstrate that the products meet these requirements. 

Associated with any directive are one or more voluntary EN/ISO standards that 

detail the essential requirements and tell the producer how to test the device in order to 

demonstrate conformity with the directive. These standards prescribe a way to qualify 

compliance with the directive; but if the producer wishes to demonstrate compliance in 

another way, he is free to do so as long as complete compliance is demonstrated. One 

reason why the standards are voluntary is to avoid impediments of technological progress, for instance in testing methods. For the same reason details of test methods are also 

seldom given. At any point in time, a manufacturer may use test methods other than those 

mentioned in the standard to demonstrate conformity with the directive. 

If a product is subject to more than one directive, careful considerations must be 

given to resolve any discrepancies in compliance requirements. Sometimes compliance 

must be with both directives; sometimes one has priority over the other. 

Although the tests required may be done by any competent laboratory (and competence is increasingly demonstrated through accreditation), the conformity assessment 

itself and the associated type approval are done by a notified body. Hence, each member 

state will inform the European Commission about which body is authorized to issue the 

pattern approval. 

Once the product has been type approved, each product is marked with a CE-mark 

by the producer/importer. The CE-mark serves as a manufacturer’s declaration that “this 

product complies with all relevant EU-directives.” Manufacturers that perform CEmarking must register, and are subjected to inspection. 
12 

Although the above describes a typical case, there are variations on the theme. 

Certain products must be initially verified for proper functioning, while others must be 

re-verified. In the case of potential severe danger in the case of malfunctioning of the 

product, each product must be inspected by an independent competent body. In short, the 

complexity of the device and the potential hazard due to improper functioning are reflected in the tests. 

 1.2 An Example: The Toy directive 

Before going to the IVD directive, let us take a look at another example, namely 

the toy directive. This states requirements for toys that are essential for their harmlessness 

in the hands of children. It distinguishes between children below and above 3 years, because the behavior in playing with toys changes at that age. 

The voluntary standard EN 71 specifies how to test toys. For instance it gives the 

dimensions of a standard throat of a 3-year-old child that the toys (or parts thereof) may 

not easily pass through. If the toy does not pass this test, the CE-marking must be accompanied by a clear statement such as “this toy is not intended for the use of children below 

3 years of age.”

In Denmark the notified body for the toy directive is the Agency for Consumer 

Protection. Any producer or importer may CE-mark (i.e. declaring compliance with the 

directive). 

If the toy in question must be connected to an electrical supply, the low voltage 

directive comes into play. However, in the case of a children’s lamp (for instance with 

the shape of a doll) the lamp must be considered safe as a toy and not as an electrical 

appliance. This means that the lamp must be equipped with a 12 V transformer and 

cannot be connected directly to the 220 volt supply. 

 1.3 The IVD Directivei

This directive, issued on 27 October 1998 for implementation in the member 

states by December 2003, applies to in vitro diagnostics medical devices and their accessories. The details of its scope and definitions are stated in Article 1. 

Article 3 relates to the essential requirements which are detailed ANNEX 1. It consists of two parts A and B: 

A: GENERAL REQUIREMENTS (1/2 page) 

Clause 1 of part A states the following: 

“The devices must be designed and manufactured in such a way 

that, when used under the conditions and to the purposes intended, they 

will not compromise, directly or indirectly, the clinical conditions or the 

safety of the patients, the safety or health of users or, where applicable, 

other persons, or the safety of property. Any risk which may be associated 

with their use must be acceptable and be compatible with a high level of 

protection of health and safety.” 

Clause 3 makes reference to metrology in the following way: 
13 

“The devices...must achieve performances in terms of ...accuracy, 

repeatability, reproducibility...as stated by the manufacturer. 

The traceability of values assigned to calibrtors and/or control 

materials must be assured through available reference measurements 

and/or available reference materials of higher order.”

Hence, clinical measurements must possess the metrology quality of traceability. 

This has so far only been sporadically the case. 

B: SPECIFIC REQUIREMENTS (4 pages) detailed under the following 

headings: 

 Chemical and physical properties 

 Infection and microbial contamination 

 Manufacturing and environmental properties 

 Devices which are instruments or apparatus with measuring 

function 

Here is stated that accuracy limits must be stated by the manufacturer, and that units must 

be legal units of the EU. 

 Protection against radiation 

 Requirements for medical devices connected to or equipped with 

an energy source 

 Requirements for devices for self-testing 

 Information supplied by the manufacturer

The rest of the directive details the implementation of the directive including the 

setting up of a Committee on Medical Devices (Article 7). This committee may examine 

any question connected with the implementation of the directive. 

 1.4 The “Voluntary” Standardii

The standard associated with the IVD directive will be discussed in a subsequent 

lecture. 

2. EXPECTATIONS 

Within the general spirit of European Directives, the expectations of the IVD directive may be discussed under the following three headlines: 

 2.1 Better European Healthcare by Protecting the Patient 

By stating the general and specific requirement for clinical instrumentation, one 

expects (of course) that the patient, the end-user of the clinical investigation, will get a 

better treatment and be protected from the hazards of the clinical devices. Each of the 8 

points in the specific requirements has to be read in that light. 

Of particular interest is the new focus on measurement traceability as a means of 

improving laboratory chemistry. 
14 

 2.2 Less Costs Due to Less Double Analysis 

No less important is the fact that one expects significant reductions in the costs of 

healthcare, costs that are in Europe predominantly included in the public budget. Although the directive at least initially imposes costs on the manufacturer of medical devices, these costs have to be weighed against the significant reductions in erroneous and 

superfluous analytical activities. Let us look at two examples: 

Saving Money From Repetitive Measurements

According the “German Health Report 1998” (available on the internet at 

www.gbe-bund.de) the costs of repeat measurements amount to 1.5 billion US $ per year 

in Germany. This is a number that speaks for itself. 

Evolution of Danish Public Chemical Laboratories

An example that illustrates that the raising of quality standards may lead to more 

efficient laboratories in the public sector, let us consider the evolution of public chemical 

laboratories in Denmark during the last ten years. Although not clinical laboratories, their 

fate may well be indicative of what will happen to the clinical laboratories in the next few 

years. 

Up to 1990, some 70 of the 228 local administrations in Denmark operated general-purpose laboratories for the surveillance of the environment and foodstuff (MLKlaboratories). In order to offer its services to other clients, one laboratory applied for accreditation; but just before being granted the accreditation from Department of Commerce, the Department of the Interior ruled that this would be illegal, with respect to the 

operating laws of the laboratories. 

During 1992-93, the Department of the Interior came to appreciate the value of 

accreditation as a means of controlling the quality of the laboratories, and cost reductions 

were expected. Suddenly, many laboratories saw accreditation as a necessary evil for survival, and this opened a “race for accreditation”, so that some 30 laboratories applied for 

accreditation. 

In 2000, there are 11 accredited public laboratories, which are all highly specialized. 

Hence, by setting strict quality standards for the operation of its laboratories, the 

Department of the Interior caused considerable structural changes, increased laboratory 

competence, and significantly reduced the public costs. 

2.3 Openness and Fairness 

Although the system of new approach directives has sometimes been accused of 

being complicated and unfair to producers outside the European Union, the idea is to create an open and fair market, subjected to regulations that are essential. The system has the 

following virtues. 

• The directives state the essential requirements in an unambiguous 

way, without prescribing specific tests that could constitute technical 

barriers to trade. 
15 

• The voluntary standards give practical solutions to the requirements. 

Other solutions are not excluded. 

3. IMPLICATIONS 

A European directive puts demands on the manufacturer, who is held responsible 

for the compliance of his products. This of course is also the case for the IVD directive. 

Manufacturers must ensure that their instruments can be calibrated against reference 

standards and procedures (implicit from the traceability requirement), that repeatability 

and reproducibility are quantified and documented, and that accuracy is substantiated by 

uncertainty calculations. These points conveniently may be categorized as establishing a 

metrology infrastructure appropriate for in vitro diagnostics. 

The IVD directive will probably also lead to a global coordination, as well as to a 

common approach to quality assurance. These three topics are discussed below. 

 3.1 The Establishment of a Metrology Infrastructure for Laboratory Medicine 

Traceability in Chemistry

National 

Laboratory A

National 

Laboratory B

Accredited Laboratory Accredited Laboratory

Analytical Laboratory Analytical Laboratory

Producer’s test Purchaser’s test

Traceability

Tested product

Equivalence

FIGURE 1. 

The establishment of a metrology infrastructure for the IVD directive is conveniently discussed in using the general scheme for chemical traceability above. This scheme 

is itself an adaptation of the scheme that has been developed for most physical quantities 

under the auspices of the meter convention during the last 125 years. A producer (lower 

left) of a clinical test-device performs measurements that are traceable to some analytical 

laboratory, which uses stable control materials and validated procedures in its analysis. 

The measurements of that analytical laboratory is itself traceable to the certified reference 

materials and methods of an accredited laboratory, which in turn uses reference materials 
16 

that have been certified at a national laboratory (NMI). Seen from the upper corner of the 

scheme, the national laboratories have disseminated traceable measurements to the producer. Hence, the producer is traceable to the international system of units, the SI, or 

some other reference system. 

The NMI has another important task, namely to ensure equivalence with other 

NMIs that do similar dissemination in their countries. When the scheme works properly, 

all users are sure to arrive at the same measurement results, so that repetitive measurements can be avoided, and one has arrived at the situation where a product is “once tested 

– globally accepted”. In IVD terms this phrase would be “once analyzed, accepted by all 

doctors”. 

However, that “everything works properly” requires quality assurance and includes comparisons between laboratories. 

The ideal situation of traceability to the SI is only the rare example in the IVD 

analytical reality. However, it is useful to discuss the opposite situation, namely no traceability at all in terms of the scheme below: 

Absence of Traceability: Repeatability Only

Producer’s test Purchaser’s test Tested product

FIGURE 2. 

The second figure illustrates that in the absence of traceability, one can only study 

repeatability, i.e. the stability of the measurements without changes to the circumstances 

under which the measurements are performed. Since the repeatability of a chemical 

measurement typically contributes with only a small fraction to the total uncertainty of 

the measurement, the lack of traceability implies lack of compliance with the IVD directive. 
17 

What the IVD directive requires is traceability to some reference (material and 

procedure) of higher order, possibly the SI. This is demonstrated in the figure below: 

This figure shows the requirement of the IVD directive, namely that traceability 

to some stated reference is achieved and documented, which ensures that both reproducibility and accuracy can be stated by the manufacturer. In some cases, the stated reference 

is the SI. 

Traceability to the SI (LEFT) or other (RIGHT)

National 

Laboratory A

Accredited Laboratory

Analytical Laboratory

test test

Traceability

Traceability

Accredited Laboratory

Analytical Laboratory

FIGURE 3. 

3.2 Global Coordination 

There are three reasons why the IVD directive will lead to global coordination: 

• The IVD industry is a global one. Hence, whichever region sets essential requirements the industry will ensure proliferation to other regions. 

• The task of setting up the metrology infrastructure exceeds the capabilities of any region. Therefore, regions must unite forces to deliver 

the necessary knowledge for its practical implementation. 

• Finally, patients travel globally, acquire global diseases and will 

eventually request global healthcare. 
18 

 3.3 A Common Approach to Quality Assurance 

Although a common approach to traceability has been achieved globally amongst 

NMIs, the way in which traceability is disseminated within each country has not been 

fully agreed upon; and different approaches to quality assurance are still prevailing. As 

quality in measurement is of paramount importance in the IVD field, a global approach to 

quality should emerge, so that the above global coordination can become effective. 

4. CONCLUSIONS 

The European IVD directive is designed for the well being of Europeans. Since it 

has global implications it will be followed up by global research. 

In order to make it work, a coordinated approach has to be adopted for the basic 

concepts of quality in measurements. An infrastructure must be developed for the IVD 

sector. 

5. ACKNOWLEDGMENTS 

I would like to thank my colleagues in Metrology in the Healthcare initiativeiii for 

introducing me to the field of IVD and to share with me their knowledge of clinical 

chemistry. I would also like to thank my colleagues in the various groups within the 

framework programs for research and technological development of the European Unioniv v

 who have taught me that metrology in chemistry has facets other than metrology in 

physics. 

References 

i Directive 98/79EC of the European Parliament and of the Council of 27 October 

1998 on in vitro diagnostic medical devices. 

ii prEN ISO 17511 (CEN/TC 140) In vitro diagnostic medical devices– Measurement 

of quantities in samples of biological origin- Metrological traceability of values assigned to calibrators and control materials. 

iii Consultative meeting on measurements in healthcare (minutes June 23, 2000). 

iv Metrology in Chemistry. Current Activities and Future Requirements in 

Europe (EUR 19074 EN, ISBN 92-828-7465-6). 

v Metrology in Chemistry. Position paper of the High Level Expert Group on Measurement and Testing under the 5th Frame Work Program. 
19 

KIM CARNEIRO 

Danish Institute of Fundamental Metrology 

Kim Carneiro, M.Sc., Ph.D., is the current director of the Danish Institute of Fundamental Metrology (DFM), a Member of the Board of the SYMBION Foundation, and Chairman of the Board of Image Metrology ApS. Dr. Carneiro received his Ph.D. in physics 

from the University of Copenhagen in 1974, followed by a postdoctoral research position 

at Brookhaven National Laboratory that lasted until 1976. He then took a professorship 

at the University of Copenhagen from 1976 to 1986 with a sabbatical at IBM Research in 

San Jose, CA from 1982-1983. In 1986, Dr. Carneiro accepted his current position at 

DFM where he worked on the construction of quality assurance at the DFM, helped build 

the metrological organization in Denmark, served as Scientific Advisor to IPQ-Portugal 

and served as Strategic Advisor to INMETRO Brazil. Dr. Carneiro is a member of the 

Danish Physical Society, Solid Section, holding the position of Chairman from 1976-

1979, the European Physical Society, the Danish Academy for Natural Sciences, and the 

Academy of Technical Sciences, ATV. He is currently a member of numerous standardisation committees, evaluation committees, and technical working groups, including the 

EUROMET Committee, serving as chairman from 1994-1998, the UN-ECE working 

party on technical harmonisation and standardisation policies as Co-rapporteur for Metrology, the Management group of the Centre for Fundamental Metrology Manager, the 

Steering Group for European Metrology in Chemistry of the European Commission, and 

the Supporting group for 5th Framework programme “Competitive and Sustainable 

Growth” of the Danish Research Ministry. 
20 
21 

IVD Directive – Expectations and Implications 

Visuals Presented By 

Kim Carneiro 

Danish Institute of Fundamental Metrology 
22 

EC IVD Directive

Expectation and Implications

Kim Carneiro

Danish Institute of Fundamental Metrology

Matematiktorvet 307, 

DK 2800 Kgs. Lyngby, Denmark

www.dfm.dtu.dk, kc@dfm.dtu.dk

What is a European Directive?

(Layman’s view)

• A legislative document issued by the European 

council of ministers and the European Parliament

to be implemented by the member states

• It is stating the essential requirements for “devices”

for the wellbeing of European citizens

Compliance is stated in voluntary standards (EN/ISO)

Compliance with several directives: preference or both? 

• Conformity assessment is done by notified bodies

Testing is done by competent (accredited) laboratories

• CE-marking is done by the producer/importer 
23 

Example: The toy directive

• The toy directive states requirements for toys that 

are essential for their safety

• The standard EN 71 tells you how to test toys

Distinguish between children below and above 3 years

• In Denmark the notified body is the Agency for 

Consumer protection

• Any producer or importer may CE-mark (i.e. 

declaring compliance with the directive). 

Registration required

• Preference over low-voltage directive:

Children’s lamps must be supplied by 12 V

The IVD directive

• Applies to in vitro diagnostics medical devices and 

their accessories

• Essential requirements:

A: GENERAL REQUIREMENTS (1/2 page)

They must achieve performances in terms of .. accuracy, 

repeatability, reproducability...stated by the manufacturer

The traceability of values assigned to calibrators and/or 

control materials must be assured through available

refererence measurements and/or available reference 

materials of higher order.

B: SPECIFIC REQUIREMENTS (4 pages)
24 

The IVD directive:

Expectations

• Better European healthcare by protecting the 

patient

• Less costs due to less double analysis

• Openness and Fairness

Less costs (1)

• German Health report 1998 (www.gbe-bund.de):

Costs of repeat measurements amounts to 1,5 billion 

US$ per year in Germany.

• Raising quality standards leads to more efficient 

laboratories
25 

Less costs (2)

Evolution of Danish Public Laboratories

• Up to 1990

Some 70 of the 228 Local administrations in Denmark 

operate general purpose laboratories for the surveillance 

of environmental and foodstuff (“MLK-laboratories”)

One applies for accreditation; but that is illegal

• During 1992-93

About 30 laboratories applies for accreditation

• In 2000

There are 11 accredited public laboratories (specialised)

The IVD directive:

Implications

• The establishment of an IVD metrology 

infrastructure

• Global coordination

• A common approach to quality assurance
26 

What is Metrology?

Repeatability

Traceability to SI or other

Uncertainties (Accuracy)

Comparisons (Reproducibility)

Traceability in length

National Laboratory A National Laboratory B

Accredited Laboratory Accredited Laboratory

Calibration Laboratory Calibration Laboratory

Producer’s test Purchaser’s test

Traceability

Tested product

Equivalence
27 

Traceability in Chemistry

National 

Laboratory A

National 

Laboratory B

Accredited Laboratory Accredited Laboratory

Analytical Laboratory Analytical Laboratory

Producer’s test Purchaser’s test

Traceability

Tested product

Equivalence

Absence of Traceability: Repeatability Only

Producer’s test Purchaser’s test Tested product
28 

Traceability to the SI (LEFT) or other (RIGHT)

National 

Laboratory A

Accredited Laboratory Accredited Laboratory

Analytical Laboratory Analytical Laboratory

test test

Traceability

Traceability

The “Measurement in Healthcare”

Initiative (1)

• The need for an IVD metrology infrastructure has 

been stated as an “Expression of Interest” to the 5th 

Framework Programme for Research and 

Technological Development

• Two dedicated calls for proposals have been issued

A: Preparation of reference materials and methods

B: Dissimination of knowledge: Networking
29 

The “Measurement in Healthcare”

Initiative (2) • Partners:

IFCC, NMIs (including NIST), Academia, Industry

• A: Preparation of reference materials and methods

Distinguish between 

SI traceable (PTB coordinator)

Non-SI traceable (IFCC coordinator)

• B: Dissimination of knowledge: Networking

The infrastructure must

Build on existing competent organisations

Single out a few reference laboratories with obligation to 

maintain knowlegde and reference systems

Conclusion

• The European IVD directive

• Is designed for the well being of Europeans

• Has global implications

• Will be followed up by global research
30 

References (1)

• DIRECTIVE 98/79/EC OF THE EUROPEAN 

PARLIAMENT AND OF THE COUNCIL of 27 

October 1998

on in vitro diagnostic medical devices

• prEN ISO 17511 (CEN/TC 140)

In vitro diagnostic medical devices- Measurement of 

quantities in samples of biological origin- Metrological

traceability of values assigned to calibrators and control 

materials

References (2)

• Metrology in Chemistry

Current Activities and Future Requirements in Europe

(EUR 19074 EN, ISBN 92-828-7465-6)

• Metrology in Chemistry

Position paper of the High Level Expert Group on 

Measurement & Testing under the 5th Frame Work

Programme

• Consultative meeting on measurements in 

healthcare (minutes June 23, 2000)
31

Standardization Activities to Support the 

Implementation of the IVD Directive

Emil Voelkert 

Roche Diagnostics GmbH 

Germany 

HARMONIZED STANDARDS AND THE FREE EUROPEAN MARKET

Europe has seen rather important changes in the past twenty years. The countries 

in Western Europe are growing together economically and, to some extent, also politically. One of the major ongoing trends is the disappearance of trade barriers between the 

countries. 

This implies not only the removal of trade barriers, like excise taxes or other import restrictions between the European countries, but goes much further on to harmonization and mutual recognition of regulations, standards and technical specifications. 

Presently, all products that are legally manufactured and marketed in one European country can move freely throughout the community. There are possible exceptions, 

but they are acceptable only if a legitimate purpose is pursued, e.g. environmental protection, health issues. 

The harmonization process was initially rather slow, because elaborate technical 

specifications had to be generated and unanimously agreed upon by the European Council and the European Parliament. 

Since 1993, a different philosophy, the “New Approach” was implemented. It involves a regulatory approach, defining only “Essential Requirements” in rather general 

terms that the products must meet. The Essential Requirements specify the elements necessary to protect public interests. They are mandatory and only products complying with 

the Essential Requirements may be placed on the market and put into service. 

The actual technical specifications that the products must meet to fulfil the Essential Requirements, are defined in Harmonized Standards. As before, the application of 

standards is voluntarily and manufacturers can always choose to demonstrate fulfilling 

the Essential Requirements by other means. However, adherence to the Harmonized 

Standards leads to presumption of conformity with the Essential Requirements. This approach provides a high level of protection. Safeguard clauses allow contesting the conformity of a product or are against failure or shortcomings of Harmonized Standards. 

This development occurred and is occurring in an environment, where quality assurance concepts and accreditation systems are continuously developed and applied. At 

the same time mutual recognition of testing and certification is supported by the European member states in order to minimize differences in the quality infrastructures that 

exist between the states and the industrial sector, involving metrology systems, testing 

laboratories, etc. These activities extend beyond the European Union in order to develop 

this approach to a global scale. 
32

Although the Harmonized Standards do not constitute a specific category amongst 

the European standards, they have nevertheless a special meaning in the context of the 

European directives. They are very relevant to the presumption of conformity regarding 

the fulfillment of the Essential Requirements of products. 

The European Commission mandates the standards after consultation with the 

member states, and the mandate is transmitted to one of the European standard organizations (CEN, CENELEC, ETSI). The European standard organization accepts the mandate 

and develops a work program. The standards will then be prepared according to General 

Guidelines that are agreed upon by the European Commission and the European standard 

organizations. The procedure involves the participation of all interested parties, i.e. users, 

consumers, manufacturers and public authorities. 

After the standards have been developed and agreed upon, they are adopted by all 

European standard organizations. They are being transposed into national standards in all 

member states and any contradicting national standards have to be withdrawn. 

The European standard organization transmits reference to the corresponding directive of the European Commission that will publish this in the Official Journal of the 

European Commission. The approach does not foresee a process under which public authorities verify or approve the contents of the standards. 

Common Technical Specifications (CTS) fulfill a similar function as Harmonized 

Standards, and adherence to the CTS also leads to the presumption of compliance with 

the Essential Requirements to which the specifications refer. They are developed by experts from all interested constituencies for selected products, which are mainly used for 

the evaluation of safety of blood products. Presently only CTS are drafted for only Annex 

II, list A products. Also in the case of the CTS, the application is voluntarily. In principle 

the manufacturer can show by other means that this product maintains at least the same 

level of performance and safety as demanded by the CTS. In practice, the CTS will 

probably always be applied. In addition, references of the CTS to the Essential Requirements are published in the Official Journal of the European Communities. 

Increasingly, standardization work is shifted from a national to a European or international global level. Between the two standardization organizations, CEN and ISO, 

there exists an agreement to avoid duplication of work by these two organizations, and 

sets out procedures for cooperation and mutual recognition of developed standards. The 

two Technical Committees (TC) charged with the generation of standards for laboratory 

medicine and in vitro diagnostics are ISO/TC 212 – "Clinical laboratory testing and in 

vitro diagnostics test systems" on the international level and CEN/TC 140 – "In vitro diagnostics medical devices" on the European side. These two technical committees cooperate closely and in many instances experts participate in the working groups of both 

TCs. 

There are, however, some differences in focus of the two corresponding committees, ISO TC 212 and CEN/TC 140. They are as follows:. CEN/TC 140 is occupied with 

the development of “Harmonized Standards” mandated by the Commission. In view of 

the fact that the IVD Directive regulates IVD products, the emphasis is on products and 

processes relevant to the manufacturers. ISO/TC 212 has a wider scope that includes 

laboratory medicine, as well as and user aspects. However, there are several agenda items 
33

that are jointly developed under the Vienna Agreement, that are either ISO lead or CEN 

lead. 

The working groups and the national standardization organizations in the CEN 

area have to develop the standards with a view of the technical coherence at national and 

European level. This is of importance, because of the implied compulsory national implementation and the obligation to withdraw any conflicting national standard. Another 

goal is to coordinate with other international activities, especially ISO. 

CEN and ISO Technical Committees involve national standardization organizations that assume the role of the secretariat. These are: NCCLS in the case ISO/TC 212; 

and, DIN in the case of CEN/TC 140. 

National “mirror committees” play an important role at the different stages of development of the standards. They provide the necessary input from experts of the interested parties on a national level and provide the platform for commenting and voting on 

the standards. 

In the IVD Directive (98/79/EC) there are three major references to standards and 

CTS: 

In Article 5 - Reference to standards, the publication of references in the Official Journal of 

the European Communities is described. Member states shall presume compliance with the Essential Requirements when manufacturers adhere to harmonized standards and CTS. 

In Article 6 - Committee on Standards and Technical Regulation describes the role of this 

committee assisting the European Commission. 

In Article 8 - Safeguard clause. The right of member states is described to take appropriate 

interim measures when – amongst other things – standards are applied incorrectly or show shortcomings themselves.

With this philosophy and in this environment, the European standard organizations are charged with the development of standards for this purpose and in this context. 

The general principles that are applied to these tasks involve the participation of 

all interested parties in a consensus process to develop documents that fit the purpose laid 

down in the principles of the new approach for the directives. Formal adoption is done by 

a vote of all CEN national members. The votes are weighted in order to account for differences in the population of the European countries. 

There are several mechanisms that can be used to generate standards. In most 

cases, a Technical Committee is charged to generate a suitable document. Delegates of 

national delegations are invited for working groups in order to represent their expertise 

and their national points of view. During this period they have to consider relevant work 

and national regulations falling within the scope of their tasks. Another route is to transfer the execution of the standard work to ISO, as formalized by the Vienna Agreement. 

This will lead to a global standard, which will become a “harmonized” standard in the 

European countries. In some areas another approach might be taken, the “questionnaire 

procedure”, whereby an appropriate existing reference document (often as ISO document) is adopted. 
34

The development of standards goes usually through four stages. At first, a draft is 

generated by experts from all interested constituencies, manufacturers, health authorities 

and users, based on the scope and justification of the mandate. The document drawn up 

by these experts goes on to the enquiry stage where comments are invited via the national 

standards institutes. After considering and accepting or rejecting the comments the national standard institutes will vote on the acceptance or rejection of the proposed standard 

using their weighted votes. When the majority of votes are in favor, the standard will be 

accepted and published. 

There are also interfaces to other committees dealing with topics related to medical devices – IVDs are medical devices - on more general terms. This is necessary to 

achieve a consolidated perspective in the whole field. 

The subsequent tables give an overview of the present status of published standards and projects of both CEN/TC 140 and ISO TC 212. 

Work in CEN/TC 140 is spread over several working groups focussing on 

different aspects. Some of the projects are developed in parallel, either under CEN or 

under ISO lead. A few of the projects listed are revisions of earlier versions. 

TABLE 1 

CEN/TC 140 In vitro diagnostic medical devices 

WG Topic Title Document 

WG 1 Labelling and performance evaluation 

 Information supplied by the manufacturer with in vitro diagnostic reagents for 

professional use 

EN 375 rev 

 Information supplied by the manufacturer with in vitro diagnostic reagents for 

self-testing 

EN 376 rev 

 Instructions for use for in vitro diagnostic instruments for professional use EN 591 rev 

 Instructions for use for 

 in vitro diagnostic instruments for self-testing 

EN 592 rev 

 Performance evaluation of in vitro diagnostic medical devices prEN 13612 

z Requirements for marking of in vitro diagnostic instruments EN 1658 

CEN/TC 257 Graphical Symbols for use in the labelling of medical devices EN 980 

WG 2 GMP for IVDs 

 Stability testing of in vitro diagnostic medical devices PrEN 13640 

 Sampling procedures used for acceptance testing of in vitro medical devices - 

Statistical aspects 

PrEN xxx 

(in print) 

 Elimination or reduction of risk of infection related to in vitro diagnostic 

medical devices 

prEN 13641 

z In vitro diagnostic systems - Guidance on the application of EN 29001 and EN 

46001 and of EN 29002 and EN 46002 for in vitro diagnostic medical devices 

EN 928 

WG 3 Quality management in the medical laboratory 

 Quality management in the medical laboratory ISO/DIS 

15189:1999 
35

TABLE 1 (cont’d) 

WG 

4 

Reference Systems 

z In vitro diagnostic medical devices - Measurement of quantities in samples of biological 

origin - Presentation of reference measurement procedures 

EN 12286 

 In vitro diagnostic medical devices - Measurement of quantities in samples of biological 

origin - Description of reference materials 

EN 12287 

 In vitro diagnostic medical devices - Measurement of quantities in samples of biological 

origin -Metrological traceability of values assigned to calibrators and control materials 

prEN ISO 

17511 

(in print) 

 In vitro diagnostic medical devices - Measurement of quantities in samples of biological 

origin -Metrological traceability of values for catalytic concentration of enzymes assigned to 

calibrators and control materials 

prEN ISO 

18153 

(in print) 

 Requirements for reference measurement laboratories in laboratory medicine prEN ISO 

15195 

(in print) 

WG 

5 

Specimen containers 

 In vitro diagnostic systems - Transport packages for medical and biological specimens - 

Requirements, tests 

EN 829 

 Single-use receptacles for human venous blood specimen collection prEN ISO 6710 

(in preparation) 

 In vitro diagnostic medical devices - Single-use receptacles for the collection of specimens, 

other than blood, from humans 

prEN xxx 

(in preparation) 

WG 

6 

Staining in Biology 

 In vitro diagnostic medical devices - Information supplied by the manufacturer with in vitro 

diagnostic reagents for staining in biology 

EN 12376 

WG 

7 

Culture media 

z In vitro diagnostic medical devices - Culture media for microbiology - Performance criteria 

for culture media 

EN 12322 

 In vitro diagnostic systems - Culture media for microbiology - Terms and definitions EN 1659 

WG 

8 

IVDs for self-testing 

 Determination of performance criteria for in vitro blood glucose monitoring systems for 

management of human diabetes mellitus 

ISO/DIS 

15197:2000 

prEN 15197 

 General requirements for in vitro diagnostic medical devices for self-testing prEN 13532 

WG 

9 

Use of external quality assessment schemes 

 Use of external quality assessment schemes in the assessment of the performance of in vitro 

diagnostic procedures 

prEN xxx 

(in print) 

 parallel 

ISO/TC 212 

There are also several standards that have been harmonized, i.e. references to the 

Essential Requirements have been published in the Official Journal of the European 

Commission; they are marked by "z". 
36

In ISO/TC 212 there are three Working Groups dealing with the areas “Quality 

Management in the Clinical Laboratory”, “Reference Systems” (parallel work with 

CEN/TC 140) and “IVD Products". 

TABLE 2 

ISO/TC 

212 

Clinical laboratory testing and in vitro diagnostic test systems 

WG Topic Title Document 

WG 1 Quality management in the Clinical Laboratory 

 Quality management in the medical laboratory ISO/DIS 

15189 ** 

 Safety management for medical laboratories ISO/CD 

15190 * 

WG 2 Reference Systems 

 Measurement of quantities in samples of biological origin - Contents and description 

of reference measurement procedures utilizing nominal and ordinal scales 

ISO/DIS 

15193 

 Measurement of quantities in samples of biological origin - Contents and description 

of reference materials 

ISO/DIS 

15194 

 Clinical Laboratory Medicine - Requirements for laboratories performing reference 

procedures 

ISO/CD 

15195 * 

 Metrological traceability of values assigned to calibrators and control materials ISO/CD 17 

511 * 

 Traceability of assigned values for catalytic concentration of enzymes in calibrators 

and control materials 

ISO/CD 

18153 * 

WG 3 IVD Products 

 Determination of analytical performance goals for laboratory procedures based on 

medical needs 

ISO/CD 

15196 

 Determination of performance criteria for in vitro blood glucose monitoring systems 

for management of human diabetes mellitus 

ISO/DIS 

15197 

 Clinical Laboratory Medicine 

Recommendation for validation of user quality control 

ISO/CD 

15198 * 

 Performance of measurement systems for self-testing of oral anticoagulant therapy ISO/AWI 

17593 

 

 Information supplied by the manufacturer with IVD reagents for staining in biology ISO/DIS 

19001 

Status: * DIS pending parallel CEN 

August 

2000 

 ** FDIS pending 
37

Most of the standards from both committees are either in the stage of voting or 

enquiry. 

By their mandate, the Harmonized Standards are developed primarily with the 

view of the IVD-Directive as well as the fulfillment of the Essential Requirements in 

mind. However, in most cases attempts were also made to include perspectives from 

areas outside Europe. This is most obvious in the case of standards that are developed in 

parallel both on ISO and CEN levels. It is definitely in the interest of the manufacturers, 

and to a very large extent also of the users, to have a framework of identical standards on 

a global scale. But there are situations where, due to differences in the national 

regulations, some problems exist that have to be considered. Many of these 

complications arise from discrepancies in nomenclature and definitions of terms in their 

legal context. There are several cases where the meanings of terms differ, due to the 

regulatory environment, mainly between USA and Europe. In the long run, 

harmonization of definition is very desirable also in this area. 

There are a few areas that merit special consideration. Due to the linguistic 

heterogeneity of the European countries the question of languages has a much higher 

importance than in the USA, and poses some problems for manufacturers in the first 

place. Therefore, special approaches have been developed in the context of labeling. 

Another basic principle that underlies the European approach is the requirement for 

metrological “traceability”, which is in fact the general theme of this workshop. 

There have been some delays in the final preparation of the standards, which put 

some time pressure on the work of the standardization: the CE mark can already be 

applied as of 7 June 2000. So far, only four standards have been formally harmonized for 

IVD products. Most of the mandated standards are, however, in a late stage of 

development, either at the voting stage or in enquiry. On the assumption that the standard 

will be accepted, perhaps with some minor modification, they will provide the required 

framework even at this stage. 

On the verge of transposing the IVD Directive into national law, the requirement 

for labeling in the national language came up in several countries of the EU, often 

because of political motives. While this demand is without any doubt justified for 

products for self-testing and lay use, the situation is different for products used by 

professionals in central laboratories. In those cases, medical technologists are usually 

highly trained and have, in many countries, a command of at least one major European 

language besides their native language. There are now 11 different major languages in the 

European Economic Area and assuming that the European Union expands eastwards, 

there might be another dozen of languages to consider. Thinking in extremes, this would 

then either require separate products for almost two dozens of countries, or products that 

have labels and contain instructions for use in all the required national languages. 

Compromise solutions will have to be chosen. 

In addition, the meaning of “labeling” is in fact not identical in the US and 

Europe, and moreover it is open to (national) interpretation. This includes labels on bottles and packs, it may include instructions for use, handbooks, software and even promotional material. Considering the fact that only 4-5 million people speak some of the European languages, this demand for national languages constitutes an additional burden for 
38

manufacturers and may in fact inadvertently create a trade barrier. It becomes rather obvious that, for example, the translation of software for an analyzer system is not economically feasible when only a few units are placed onto the national market. Therefore, 

compromises have to be reached for products that are intended for use by professionals. 

Approaches in this direction are foreseen in some of the standards. They include the use 

of symbols, which will be very helpful for labels on bottles and packs, as well as the possibility of providing detailed instructions for use by alternate means other than the insert 

in the pack, e.g. system manuals, CD ROMs, Internet, Fax polling etc... 

The main topic of this workshop is, of course, traceability. In the description of 

the Essential Requirements for IVDs, traceability of values assigned to calibrators and/or 

control material must be assured to available reference measurement procedures and/or 

available reference materials of higher order. This has to be implemented by the 

manufacturers and the information pertaining to the traceability of the calibration has to 

be supplied to the user in the instructions for use. This requirement by the IVD Directive 

demands the application of reference measurement procedures and materials, and supports further development. For this purpose, five standards have been elaborated: 

ISO/DIS 15193 "Contents and description of reference measurement procedures utilizing 

nominal and ordinal scales", ISO/DIS 15194 "Contents and description of reference materials", ISO/CD 15195 "Requirements for laboratories performing reference procedures", 

ISO/CD 17511 “ Metrological traceability of values assigned to calibrators and control 

materials” and ISO/CD 18153 “Metrological traceability of assigned values for catalytic 

concentration of enzymes in calibrators and control materials”. The last two of these are 

of particular relevance to manufacturers 

The approach for metrological traceability described in the Standard ISO/CD 

17511 describes a spectrum of different situations, ranging from cases where the complete traceability chain can be established, to situations in which the chain ends at a 

manufacturer's master calibrator. The essential idea is shown here in a somewhat simplified scheme: 

secondary calibrator

prim ary calibrator

working calibrator

product calibrator

prim ary reference

measurem ent procedure

secondary reference

measurem ent procedure

m an ufa c turer’s se le c te d

measurem ent procedure

m a nu fa ctu re r’s s ta nd ing

measurem ent procedure

user’s routine

measurem ent procedure

routine sam ple

Result

definition of SI-unit
39

To establish and describe the traceability chain, first of all, it is necessary to define the 

intended use of the product (with respect to the medical decision it will support, the system to which it will be applied, blood, plasma, urine etc. and the kind of quantity that is 

measured). The primary or secondary calibrator (e.g. NIST material, WHO) or the reference measurement procedure of highest order must be decided upon and a transfer protocol has to be established. 

Both reference measurement procedures and materials should be internationally 

agreed upon and accepted, preferably with the support of a recognized scientific 

organization. The reference laboratories should be accredited or at least a quality 

management system should be established. 

There are some hurdles to take with regard to the actual application of this 

approach to a wide range of analytes. In cases where the entity is a clearly defined 

molecule, traceability to the highest possible metrological level is possible and can be 

successfully applied. It has led in the past to an improvement of quality and 

comparability of measurements of patient results. This applies at present to some 30 

analytes of the approximately 400 to 600 types of quantities routinely determined in the 

medical laboratory. 

The traceability for the majority of these parameters will have to end at a lower 

hierarchical level. However, it is also the aim to develop materials and procedures for 

these analytes thus leading to a higher metrological level. On the way toward that goal, 

problems of different nature have to be solved. Very often analytes are mixtures of 

chemical entities or are heterogeneous by nature. Protein binding plays a role and 

immunological methods inherently show variability, because of the possibility (or 

probability) of being directed towards different epitopes. Last but not least, matrix effects 

may play a role and must be taken into account. 

The standard, due to the metrological approach, also requires the determination of 

the uncertainty along the traceability chain for calibrators and controls. Uncertainty is, of 

course, increasing as one moves down the traceability chain. The determination of 

uncertainty is, however, not trivial, particularly in situations where several calibrators are 

used to establish a calibration curve. 

Information on the uncertainty of calibrator set values must be made available to 

users on request. 

Reference materials and reference measurement procedures will provide “Golden 

Standards” for the calibration of methods. How to proceed further and how to address 

some of the issues in this field, is the topic of this workshop. It is encouraging to see that 

these efforts are taken up on a global scale with initiatives arising both in the USA and 

Europe. Improvement of quality of laboratory data is to be expected from these 

approaches, and will provide further steps in comparability of patient results, hopefully 

on a global scale. However, it should be kept in mind that measurements of clinically 

useful parameters are not an aim per se, but rather the fulfillment of medical needs. 

To summarize now the importance and the impact on standards, in particular, with 

perspective to the European situation: 
40

The harmonized standards are still voluntary in their application but they are 

(almost) legally binding. The same is true for the Common Technical Specifications. 

Both play a very important role at the presumption of conformity with the Essential 

Requirements for IVDs. They are designed to guarantee a high level of quality of 

performance and safety. To achieve this, the documents describe the necessary 

framework of conditions that have to be met. Their global application is in the interest of 

manufacturers and users, as several of the major standards for this purpose have been 

developed in parallel, and with close cooperation between ISO/TC 212 and CEN/TC 140. 

Additional information on the Internet: 

HTTP://www.Newapproach.org/NewApproach/Concepts.htm

http://www.iso.ch/wtotbt/wtotbt.htm

http://www.cenorm.be/standardization/standardization.htm

http://www.wssn.net/WSSN/
41

EMIL VOELKERT 

Roche Diagnostics GmbH 

Emil Voelkert, Ph.D., is the Senior Vice President of Integrated Health Care Diagnostics 

at Roche Diagnostics GmbH Mannheim. Dr. Voelkert received his Ph.D. from the 

University of Saskatchewan, Canada and in 1970 joined Boeringer Mannheim in 

Mannheim, Germany, serving as Head of the Applications Laboratory Diagnostics. In 

1979, he became the Head of Evaluation Clinical Chemistry and then Head of Product 

Management German Sales Organization Diagnostics in 1985. He was Head of 

Marketing German Sales Organization, Laboratory Systems from 1991 to 1996. He then 

took the position of General Manager Boeringer Mannheim Switzerland from 1996 until 

1998, when he assumed his current position at Roche. Dr. Voelkert is a member of a 

number of standards organizations, including Chairman of the CEN/TC1 140, the EDMA 

Chairman of WP Standardisation, KAM, a member of the NCCLS Board of Directors, 

and both a board member and chairman of WP science and technology at VDGH. 
42
43

Standardization Activities to Support the Implementation of the 

IVD Directive 

Visuals Presented By 

Emil Voelkert 

Roche Diagnostics GmbH 

Germany
44

Standardization activities to support 

the implementation of the IVD-Directive

Harmonized standards and the European free market

NIST Workshop

Gaithersburg, Nov. 2-3, 2000

E. Voelkert, Roche Diagnostics GmbH

NIST Workshop EV

Underlying idea:

Free trade in a single market

Any product lawfully produced and 

marketed in one Member State must be 

admitted to the market of any other 

member state

this implies the prevention of barriers, 

mutual recognition and technical 

harmonization
45

NIST Workshop EV

New Approach: Basic philosophy

l Legislative harmonization limited to Essential 

Requirement, which products must meet

l Technical specifications laid down in 

harmonized standards or in Common Technical 

Specifications

l Application of standards is voluntary, however

l adherence to these standards leads to the 

presumption of conformity with the 

corresponding essential requirements

l Safeguard clause to protect against failure or 

shortcomings of standards

NIST Workshop EV

Harmonized Standards

l mandated by the European Commission

l prepared according General Guidelines agreed between 

the European Commission and the European Standards 

Organizations

t participation of all interested parties (manufacturers, 

users, consumer associations trade unions)

t participation of public authorities only in certain areas 

(health, safety, etc ...)

t uniform application throughout the community

l adopted by all European standards organizations

l contradicting national standards have to be withdrawn

l references are published in the Official Journal of the 

European Communities
46

NIST Workshop EV

Common Technical Specifications

l For selected devices, mainly used for the evaluation of 

safety of blood supply and organ donations (Annex II 

products)

l developed by experts, replacing national specifications

l similar role as harmonized standards - manufacturers 

are required to comply with CTS -

exceptions are possible, however, alternate solutions have to 

maintain a level at least equivalent thereto

l references published in the Official Journal of the 

European Communities

adherence to these specifications leads to the 

presumption of compliance with the essential 

requirements to which the specifications refer to

NIST Workshop EV

Aspects of standardization work

l Increasingly Standardization is shifted from a national to a 

European and to a global level

l Duplicate work by CEN and ISO is avoided ("Vienna 

Agreement" foresees "fast track procedures" for mutual 

recognition)

l Underlying philosophy:

CEN: Standards mandated by European Commission related 

to IVD-Directive - product focus

ISO: Global acceptance - laboratory focus

l In CEN and ISO national standardization institutes take the 

role of the secretariat (DIN, NCCLS)

l national "mirror committees" are established to find a 

consensus and provide input via the national standardization 

institutes
47

NIST Workshop EV

Standardization Organizations 

Focus on In vitro diagnostics

CEN Comité Européen de Normalisation

European Federation of national standards bodies

CEN/TC 140 In vitro diagnostic medical devices

secretariate: DIN

ISO International Organisation for Standardization

World-wide federation of national standards bodies

ISO/TC 212 Clinical laboratory testing and in vitro 

diagnostic test systems

secretariate: NCCLS

NIST Workshop EV

Basis in the IVD-Directive

l Article 5 - Reference to Standards

Presumed compliance with the Essential 

Requirements when adhering to standards and CTS

l Article 6 - Committee on Standards and 

Technical Regulations

assisting the Commission in matters related to 

standards

l Article 8 - Safeguard clause

Right of the Member States to take appropriate 

interim measures ... when standards are applied 

incorrectly or there are shortcomings of the 

standards themselves
48

NIST Workshop EV

CEN Principles

l Integration of all interested parties

l Consensus - voluntary agreement by interested 

parties

l formal adoption by weighted majority vote of all 

CEN National Members

l technical coherence at national and European 

level

t compulsory national implementation

t withdrawal of conflicting standards

l Co-ordination with other international activities, 

especially ISO

NIST Workshop EV

CEN Routes

l ISO

transfer of execution of standard work to ISO 

(and vice versa), formalized by "Vienna 

Agreement"

l Questionnaire procedure

use of an appropriate "reference" document, 

often an ISO document

l Technical Committee

delegates from national delegations to 

represent expertise and national points of view. 

Relevant work falling within its scope has to be 

considered

This path can also be used to modify ISO work
49

NIST Workshop EV

The Stages at the Development

l Draft

Experts from manufacturers, health 

authorities and users develop a draft, 

based on scope purpose and justification 

of the standard

l Enquiry

Comments are invited via the national 

standards institutes

l Voting

National standards institutes vote on 

acceptance or rejection of proposed 

standard (weighted votes)

l Publication

NIST Workshop EV

Actually, the situation is more complex...

There are several other TCs at CEN and ISO that also 

have an influence:

t ISO/TC 176 Quality management and assurance

t ISO/TC 210 Quality management and general 

aspects for medical devices

t CEN/TC 204 Sterilisation of medical devices

t CEN/TC 257 Symbols and information provided 

with MD; Nomenclature for 

regulatory data exchange

t ISO/TC 210/ Application of risk management to 

IEC/SC 62A medical devices
50

NIST Workshop EV

CEN/TC 140

WG Topic Title Document

WG 1

Information supplied by the manufacturer with in vitro diagnostic reagents 

for professional use EN 375 rev

Information supplied by the manufacturer with in vitro diagnostic reagents 

for self-testing EN 376 rev

Instructions for use for in vitro diagnostic instruments for professional use EN 591 rev

Instructions for use for in vitro diagnostic instruments for self-testing EN 592 rev

Performance evaluation of in vitro diagnostic medical devices prEN 13612

Requirements for marking of in vitro diagnositc instruments EN 1658

Graphical Symbols for use in the labelling of medical devices EN 980

WG 2

Stability testing of in vitro diagnostic medical devices PrEN 13640

Sampling procedures used for acceptance testing of in vitro medical 

devices - Statistical aspects

PrEN xxx 

(in print)

Elimination or reduction of risk of infection related to in vitro diagnostic 

medical devices prEN 13641

In vitro diagnostic systems - Guidance on the application of EN 29001 and 

EN 46001 and of EN 29002 and EN 46002 for in vitro diagnostic medical 

devices EN 928

GMP for IVDs

CEN/TC 257

In vitro diagnostic medical devices

Labelling and performance evaluation

CEN/TC 140 Projects (1)

Harmonized Standard

NIST Workshop EV

CEN/TC 140 Projects (2)

WG 3

Quality management in the medical laboratory

ISO/DIS 

15189:1999

WG 4

In vitro diagnostic medical devices - Measurement of quantities in samples 

of biological origin - Presentation of reference measurement procedures EN 12286

In vitro diagnostic medical devices - Measurement of quantities in samples 

of biological origin - Description of reference materials EN 12287

In vitro diagnostic medical devices - Measurement of quantities in samples 

of biological origin -Metrological traceability of values assigned to 

calibrators and control materials 

prEN ISO 17511

(in print)

In vitro diagnostic medical devices - Measurement of quantities in samples 

of biological origin -Metrological traceability of values for catalytic 

concentration of enzymes assigned to calibrators and control materials

prEN ISO 18153

(in print)

Requirements for reference measurement laboratories in laboratory 

medicine

prEN ISO 15195

(in print)

WG 5

In vitro diagnostic systems - Transport packages for medical and biological 

specimens - Requirements, tests EN 829

Single-use receptacles for human venous blood specimen collection

prEN ISO 6710

(in preparation)

In vitro diagnostic medical devices - Single-use receptacles for the 

collection of specimens, other than blood, from humans

prEN xxx 

(in preparation)

Specimen containers

Reference Systems

Quality management in the medical laboratory
51

NIST Workshop EV

WG 5

In vitro diagnostic systems - Transport packages for medical and biological 

specimens - Requirements, tests EN 829

 ISO/TC 76 Single-use receptacles for human venous blood specimen collection

prEN ISO 6710

(in preparation)

In vitro diagnostic medical devices - Single-use receptacles for the 

collection of specimens, other than blood, from humans

prEN xxx 

(in preparation)

WG 6

In vitro diagnostic medical devices - Information supplied by the 

manufacturer with in vitro diagnostic reagents for staining in biology EN 12376

WG 7

In vitro diagnostic medical devices - Culture media for microbiology - 

Performance criteria for culture media EN 12322

In vitro diagnostic systems - Culture media for microbiology - Terms and 

definitions EN 1659

WG 8

Determination of performance criteria for in vitro blood glucose monitoring 

systems for management of human diabetes mellitus

ISO/DIS 15197:2000

prEN 15197

General requirements for in vitro diagnostic medical devices for self-testing prEN 13532

WG 9

Use of external quality assessment schemes in the assessment of the 

performance of in vitro diagnostic procedures

prEN xxx

(in print)

parallel ISO/TC 212

Use of external quality assessment schemes

IVDs for self-testing

Culture media

Staining in Biology

Specimen containers

CEN/TC 140 Projects (3)

NIST Workshop EV

ISO/TC 212 Projects (1)

WG 1

Quality management in the medical laboratory ISO/DIS 15189 **

Safety management for medical laboratories ISO/CD 15190 *

WG 2

Measurement of quantities in samples of 

biological origin - Contents and description of 

reference measurement procedures utilizing 

nominal and ordinal scales ISO/DIS 15193

Measurement of quantities in samples of 

biological origin - Contents and description of 

reference materials ISO/DIS 15194

Clinical Laboratory Medicine - Requirements for 

laboratories performing reference procedures ISO/CD 15195 *

Metrological traceability of values assigned to 

calibrators and control materials ISO/CD 17 511 *

Traceability of assigned values for catalytic 

concentration of enzymes in calibrators and 

control materials ISO/CD 18153 *

* DIS pending

** FDIS pending

Quality management in the Clinical Laboratory

Reference Systems
52

NIST Workshop EV

ISO/TC 212 Projects (2)

WG 3

Determination of analytical performance goals 

for laboratory procedures based on medical 

needs ISO/CD 15196

Determination of performance criteria for in 

vitro blood glucose monitoring systems for 

management of human diabetes mellitus ISO/DIS 15197

Clinical Laboratory Medicine

Recommendation for validation of user quality 

control ISO/CD 15198 *

Performance of measurement systems for selftesting of oral anticoagulant therapy ISO/AWI 17593

Information supplied by the manufacturer with 

IVD reagents for staining in biology ISO/DIS 19001

* DIS pending

IVD Products

NIST Workshop EV

Special aspects

l Globalisation and global recognition 

l Harmonization of Nomenclature and Definitions

l Labeling

t Languages 

t Use of symbols

l Traceability

t Reference methods / -material

t Uncertainty

l Timeline (IVD-Directive is effective as of 7.6.2000)
53

NIST Workshop EV

Labeling

l What is labeling?

t Labels on bottles and pack

t instructions for use

t software

l The European languages

t there are 11 different (major) languages in the EU

t there are at least another 11 in the Eastern parts

t at the transposition of the IVD-Directive more countries 

demand their national language

l Choices

t one version for all countries

t 11 country specific versions of products

NIST Workshop EV

Labeling

• Use of Symbols

e.g. Use by

Store between 

(Temperature)

• Use of alternate means of providing detailed 

information (instructions for use)

• System manual

• CD-ROM

• FAX on demand

• Internet
54

NIST Workshop EV

Traceability - Basis

l The IVD-Directive:

l The Standards ISO/CD 17511 and ISO/CD 18153

Metrological traceability of values assigned to calibrators and 

control materials

Metrological traceability of assigned values for catalytic 

concentration of enzymes in calibrators and control materials

Essential Requirements (Annex 1)

General requirements ...

The traceability of values assigned to calibrators and/or control materials must

be assured through available reference measurement procedures and/or available

reference materials of a higher order.

8. Information supplied by the manufacturer ...

(k) information appropriate to users on:

- internal quality control including specific validation procedures,

- the traceability of the calibration of the device;

NIST Workshop EV

The standard prEN ISO 17511 describes 

several situations

l Quantities traceable to SI-units

l International conventional reference 

measurement procedure and international 

calibration material available

l International conventional reference 

measurement procedure available

l International conventional calibration material 

available

l Neither international conventional reference 

measurement procedure and international 

calibration material available
55

NIST Workshop EV

Traceability - the approach

secondary calibrator

primary calibrator

working calibrator

product calibrator

primary reference 

measurement procedure

secondary reference 

measurement procedure

manufacturer’s selected 

measurement procedure

manufacturer’s standing 

measurement procedure

user’s routine 

measurement procedure

routine sample

Result

definition of SI-unit

NIST Workshop EV

Traceability chain

l Define intended use of IVDMD

t medical decision

t system (serum, plasma, urine, ...)

t kind of quantity

l Describe traceability chain to methods of higher 

metrological order (including uncertainty)

l Characterization of calibrators used (e.g. NIST, 

WHO ...)

l Responsibility of manufacturer starts with 

product calibrator and ends at highest reference 

used by manufacturer

l Establish transfer protocol
56

NIST Workshop EV

Traceability - considerations

l Reference method internationally agreed

l Reference material suitable for intended 

purpose

t analyte clearly defined

t matrix

l International recognition of methods and 

materials, supported by scientific organizations

l Reference laboratories accredited

NIST Workshop EV

Traceability - problems

l Analytes - of the approx. 400 to 600 analytes routinely 

investigated in the clinical laboratory there are only some 

30, which are clearly described chemical entities

t Analytes are often mixtures

t Heterogeneity of many analytes (e.g. ferritin)

t Analytes bound to proteins and in free state

t Immunological procedures show inherent 

variability (different epitopes)

t Matrix influences

l Uncertainty - has to be indicated (on request)

t a priori calculation of uncertainty is often 

rather complex

t useful for routine purposes ?
57

NIST Workshop EV

Conclusions

l Harmonized standards and Common Technical 

Specifications are (almost) legally binding 

l they are an important element at the 

presumption of conformity

l they guarantee a high level of performance and 

safety

l they tend to be "horizontal"

l their global application is in the interest of 

manufacturers and users

l there is a close co-operation between 

ISO/TC 212 and CEN/TC 140
58
59

Mutual Recognition of Measurements: How Do They Impact 

 International Trade? 

Hratch G. Semerjian and Ellyn S. Beary 

National Institute of Standards and Technology 

Introduction 

Advances in technology, globalization of trade, and accelerated growth of international trade are the primary factors driving economic growth. These changes, in turn, 

have created an increasing need for strengthening the national metrology infrastructure 

and ensuring mutual recognition of measurements performed in each country. International trade is growing at a rate of 15% per year, and tariff-based barriers are being replaced with technical barriers to trade. To support this rate of growth and to overcome the 

technical barriers to trade, measurements performed by industry to characterize their 

products and for conformity assessment must be globally recognized. Each nation must 

therefore ensure the quality and accuracy of their measurement system to promote mutual 

recognition in the global marketplace. 

Technology is one of the main driving forces for economic growth; according to a 

recent study, in the United States alone, more than 50% of the economic growth is attributed to technological advances [1]. Advanced technologies such as microelectronics, biotechnology, and nanotechnology, require measurements of higher spatial resolution, sensitivity, and selectivity. In addition, the deregulation of many mature industries, such as 

natural gas and electricity, requires more frequent and accurate measurements to help ensure equity in trade. 

The federal role in metrology is becoming ever more 

important as the health of domestic industries becomes increasingly dependent on global 

trade. In the U.S., the federal 

government has the constitutional responsibility to provide 

the weights and measures for 

the nation, and this responsibility is vested in NIST (Figure 

1). NIST maintains both basic 

and derived SI units (International System of Units) and 

also provided standards and 

calibrations that are traceable 

to NIST. NIST invests apFigure 1: NIST Measurements and Standards 

Infrastructure 
60

proximately $500 million per year (0.7% of the federal R&D funds) in metrology programs, and supports a $10 billion private sector investment in measurements and standards. More than half of the $7.6 trillion per year U.S. GDP in sales is supported by this 

measurement infrastructure. 

The measurement and standards infrastructure developed by NIST is designed to 

address needs of rapidly growing, high technology industries, as well as mature industries. NIST provides measurements and standards for these industries not only to provide 

better tools to characterize products, but also tools to improve processes, enhance process 

yields, and promote competitiveness in the global market. This paper provides an overview of the international agreements that support global trade, and describes the role 

NIST plays in facilitating mutual recognition of measurements and standards. 

Measurements and Standards for International Trade 

Traceability and Comparability: The metrology infrastructure to ensure global recognition of measurements and standards must have two major elements: a) vertical traceability system to ensure the quality and accuracy of measurements; and b) comparability 

of the standards realized and maintained by the National Metrology Institutes (NMIs) of 

different countries. Accreditation of measurement capabilities and conformity assessment 

for certification of products require demonstration of measurement traceability to national 

standards and comparability among the national standards of the trading partners (Figure 

2). The VIM (International Vocabulary of Basic and General Terms in Metrology) definition of traceability is: 

“... the property of the result of a measurement or the value of a standard whereby 

it can be related to stated reference, usually national or international standards, 

through an unbroken chain of comparisons all having stated uncertainties.”

Indeed, traceability carries an 

additional requirement in that it 

can only exist when scientifically rigorous evidence is collected on a continuing basis 

showing that the measurement 

is producing documented results for which the total measurement uncertainty is quantified. A national traceability 

system links field measurements to the SI by providing 

tools such as reference materials, reference data, and calibrations. 

Figure 2: NMIs are responsible for national traceability

and international comparability. The function of comparability is becoming more critical in global trade. 
61

NMIs have a unique and pivotal role to play in ensuring the comparability of 

physical and chemical measurements. NMIs must be active in advancing the state of 

measurement science to support the needs of their own domestic industry. NIST, the NMI 

of the United States, was founded on March 3, 1901 as the National Bureau of Standards, 

and was the U.S. federal government’s first physical science research laboratory. NIST 

provides traceability to the SI and derived units, ensuring that products and goods sold 

meet the claimed specifications. 

In addition to providing vertical traceability, the NMI also serves to link the national infrastructure to the international measurement system. NMIs have been collaborating and carrying out international comparisons of their national measurement standards 

for more than one hundred years. However, the ad hoc recognition that resulted from 

these long-standing interactions among the NMIs is no longer considered sufficient to 

promote international trade and to ensure equity in trade. The NMI function of establishing comparability is more critical in today’s global market, and must be established on a 

more formal and systematic basis. The International Committee of Weights and Measures 

(CIPM) has played a critical role in the negotiation and implementation of a Mutual Recognition Arrangement (MRA), as a mechanism to demonstrate comparability of measurements and standards between NMIs. Details of this new MRA will be described later 

in this paper. 

All NMIs interact on a global basis with other NMIs and strive to have their 

measurement and standards capabilities recognized by their counterparts. These efforts 

can facilitate the harmonization of systems of measurement and standards and eventually 

lead to mutual recognition among trading partners. Today’s more formalized system reflects a major change in philosophy, where measurements and data from each NMI are 

openly shared and accessible to all. In formal comparisons, degrees or levels of comparability may be demonstrated among participants, and is defined in terms of a “degree of 

equivalence” that is needed for “agreement for the purpose at hand” to be decided by the 

end user. This formal international measurement infrastructure spear-headed by the metrological community is in harmony with the EU/US Mutual Recognition Agreement designed to facilitate international trade. 

The EU/US Mutual Recognition Agreement: This international trade agreement 

was negotiated at the highest level of governments in recognition of EU/US 

economic interdependence. This agreement, signed in October 1998, provides 

for mutual recognition of specified conformity assessment activities undertaken in the exporting country to the regulatory requirements of the importing 

country. It covers testing activities, product certification, and quality system registration, 

where relevant [2]. Since NMIs provide the traceability to the SI required for these services, mutual recognition of the capabilities of the NMIs is a prerequisite for the mutual 

recognition of metrology services in general. This treaty was designed to address regulatory issues in the following areas: Avionics, Environmental Protection, Pharmaceutical 

and Medical Devices, Electromagnetic Compatibility and Interference, and Occupational 

Health and Safety. It is estimated that this treaty will eliminate duplicative product testing 

on approximately $60 billion worth of traded goods. [3] 
62

EU IVD Directive: The European Community expressed the need for legislative 

directives that would create a single European market as far back as the 1980s. In fact, it 

was the medical products industry throughout Europe that identified the need for a set of 

directives to regulate its products. This industry group sought the 

assistance of the European Commission to develop directives with 

the goal of setting specific rules to regulate the distribution of 

medical devices and diagnostic products throughout the European 

market, and requiring only a single approval for each product [4]. 

It is not surprising that the EU/US Mutual Recognition Agreement of 1998 targeted 

“Pharmaceutical and Medical Devices” as a focus area. While this agreement engaged 

the NMIs, the EU Directive 98/79/EC on In Vitro Diagnostic (IVD) Medical Devices that 

was issued around the same time, 

was the impetus for the industry to 

take notice. This IVD Directive 

was issued in October 1998 and 

required that all IVD devices sold 

in Europe after December 2003 

bear the CE mark. This mark 

would indicate traceability of its 

measurement systems to “standards of a higher order” [5]. The implementation timetable 

provides the following options: CE marking of IVD Devices could begin as early as June 

2000, and devices already on the market prior to December 7, 2003 could be sold without 

CE marking until December 7, 2005. Health status markers that are impacted by the IVD 

Directive are classified into two categories: “List A” includes approximately 100 welldefined chemical species, potentially traceable to SI units; and “List B” consists of more 

than 500 less well-defined chemical species, potentially not traceable to SI units. Since it 

is generally agreed that the phrase “standards of a higher order” refers to nationally and 

internationally recognized Certified Reference Materials (CRMs), it is clear that international collaboration and cooperation is required to meet the industry need for new standards. NIST’s CRMs sold under the trademark Standard Reference Material (SRM®) are 

widely recognized and used worldwide. 

NIST’s Standard Reference Materials Program: The 

SRM Program at NIST is of long standing, and has produced chemical and physical standards for almost 100 

years. These standards continue to promote the development of new technologies, and are as relevant at the 

turn of this century as they were at the turn of the last. 

NIST standards have ushered in many eras of technological advances in the U.S. from the industrial age to the new age of DNA based technologies. Approximately 1300 SRMs are currently available for use in: (1) industrial materials production and analysis; (2) environmental analysis; (3) healthcare measurements; 

and (4) basic measurements in science and metrology. SRMs are crucial reference points 

in the establishment of a comprehensive measurement system for the entire nation [6]. 

This system has met the needs of U.S. industry and commerce for a century and continThe CE Marking of in vitro diagnostic devices 

applies to reagents and reagent products, calibrator materials, or instruments including specimen receptacles intended by the manufacturer 

for the in vitro examination of human tissue, 

blood, or fluid samples for the purpose of proving information about a patient’s state of health. 
63

ues to evolve to satisfy more demanding measurement requirements. SRMs are used 

both in the U.S. and abroad to provide traceability, i.e., an unbroken chain that connects 

field measurements to SI units. The process of traceability is ongoing, not stagnant, and 

NIST continually strives to provide its customers with the robust, flexible, and globally 

recognized measurement and standards infrastructure that is necessary to compete in the 

21st century market place. Healthcare related standards, of particular interest to the IVD 

manufacturing community, are discussed in the following section. 

Measurements and Standards for Healthcare 

Chemical metrology is at the heart of accurate medical diagnosis and the development of measures to improve our health and ensure long life. In the U.S., about $1.5 

trillion are spent each year on health care, which corresponds to about 14 % of our GDP 

[7]. About 13% of these expenditures are for measurements. It is estimated that over one 

third of these measurements are performed for non-diagnostic purposes, such as quality 

assurance and quality control (QA/QC) measurements and retests at a cost of about $50 

to $60 billion annually. Clearly, improvements in the reliability of chemical measurements in this area would have a significant economic impact for the U.S. Indeed, improvements in the accuracy of just one analyte (cholesterol) is estimated to have resulted 

in cost savings of about $100 million per year. 

NIST works closely with the American Association for Clinical Chemistry (AACC), the 

Centers for Disease Control and Prevention (CDC), the College of American Pathologists 

(CAP), the National Conference of Clinical Laboratory Standards (NCCLS), the Food 

and Drug Administration (FDA), and other organizations interested in health-related 

standards to help prioritize our standards activities and the development of SRMs. 

For more than 20 years NIST has 

developed, maintained, and refined 

“Definitive Methods” for 12 health 

status markers to support the national reference system for clinical 

measurements (Figure 3). “Definitive Methods” are robust analytical 

methods for which all sources of 

uncertainty are known and quantified. Such methods have been instrumental in improving the accuracy and reliability of clinical laboratory measurements. For example, 

in the area of cholesterol measurements, it has been estimated that 

the measurement uncertainty was reduced from ± 18% relative in 1969 to between ± 5.5 

to 7.5% relative by 1994 [8]. The improvement in accuracy has been attributed to the 

release a pure cholesterol SRM in 1969, and the serum cholesterol SRMs released in 

1981 and 1988. In addition to using NIST definitive methods for the certification of 

Figure 3: NIST Standards are Currently Available for 12 Health Status Markers 

• Calcium Cancer, Blood Clotting

• Chloride Kidney Function

• Cholesterol Heart Disease

• Creatinine Kidney Function

• Glucose Diabetes

• Lithium Antipsychotic Treatment

• Magnesium Heart Disease 

• Potassium Electrolyte Balance 

• Sodium Electrolyte Balance 

• Triglycerides Heart Disease

• Urea Kidney Function

• Uric Acid Gout 
64

SRMs, they are also used to value-assign high-priority serum pools that serve as the anchor point for CDC developed reference methods. These standards are also used by the 

College of American Pathologist (CAP) for proficiency testing of more than 20,000 U.S. 

clinical laboratories. Improved accuracy of measurements, facilitated by this program, 

has led to better diagnosis, treatment, and reduced healthcare costs. Maintaining these 

anchor points for the clinical measurements reference system also facilitates the development and use of new technologies that are better, faster, and less expensive. 

A new generation of health status 

markers, emerging now, shows great 

promise from the clinical diagnostic 

perspective, but offers new and more 

difficult challenges for standardization. Many of the new markers are 

proteins, peptides, or other large 

biomolecules, usually present at very 

low concentrations. Because of the 

large market for tests for these new 

markers, many different approaches 

have been developed commercially, 

resulting in vast disagreements 

among results obtained with different 

manufacturer test kits. NIST has focused on basic research to establish 

reference systems for several new 

biomarkers shown in Figure 4. 

NIST views its SRMs as a cornerstone to traceability. NIST also works to ensure recognition of its standards worldwide, and to promote fair trade practices. One mechanism in 

which NIST ensures mutual recognition of its traceability structure worldwide is through 

proactive participation in the CIPM) Mutual Recognition MRA. This MRA provides the 

mechanism for implementation of the EU/US Mutual Recognition Agreement. 

CIPM Mutual Recognition Arrangement 

The CIPM MRA was signed in October 

1999 by the directors of the NMIs of 

thirty-eight member states of the Metre 

Convention, and representatives of two 

international organizations. Since then, 

additional members have joined the MRA. 

The MRA provides the framework for 

“mutual recognition of national measurement standards, and of calibration and 

measurement certificates issued by National Metrology Institutes” [9]. 

Figure 4: Standards under Development for 

New Health Care Markers 

• Troponin Heart Attack Marker

• Homocysteine Risk of Heart Disease

• Glycated Diabetes Status

 Hemoglobin 

• Cortisol Endocrine Function

• Thyroxine Thyroid Function

• Cadmium Heavy Metal Toxicity

• Folic Acid Neural Tube Defects

• Mercury Heavy Metal Toxicity

• Speciated Iron Hemochromatosis, 

Anemia 

• Human Serum Renal Failure 

Albumin 

• Prostate Specific Prostate Cancer

 Antigen 

• P53 Dna Breast Cancer

• Thyroid Stimulating Thyroid Function

Hormone 

Convention du Metré 

As far back as 1875 when the Convention

du Metré was signed in Paris, the necessity for international collaboration and

agreement on metrological issues was

recognized. The United States, while not

a world power at the time, was one of the

seventeen original signatory nations. The

Convention remains the basis of all international agreement on units of measurement. Currently, there are 49 Member

States, including all the major industrialized countries.
65

The CIPM MRA has as its objectives: a) to establish the degree of equivalence of 

measurement standards maintained by NMIs; b) to provide for the mutual recognition of 

calibration and measurement certificates issued by NMIs; and thereby c) to provide governments and other parties with a secure technical foundation for wider agreements related to international trade, commerce, and regulatory affairs. Implementation of the 

MRA involves international key and supplementary comparisons and quality systems and 

demonstration of competence by NMIs. The CIPM Consultative Committees are responsible for identifying key comparisons that are needed to demonstrate comparability for a 

broad range of measurements. Consultative Committees are organized to focus on the 

following metrology areas: amount of substance (mole) (CCQM), electricity and magnetism (CCEM), ionizing radiation (CCRI), length (CCL), mass and related quantities 

(CCM), photometry and radiometry (CCPR), thermometry (CCT), and acoustics, ultrasound, and vibration (CCAUV). Results of these comparisons and demonstrations are 

maintained in a database on the World Wide Web. 

The BIPM Key Comparisons Database was originally developed by NIST for the 

BIPM and is now operated by the BIPM (http://kcdb.bipm.org/BIPM-KCDB/). This database is defined in the text of the CIPM MRA as “the database maintained by the BIPM 

which contains Appendices A, B, C, and D of the Mutual Recognition Arrangement” and 

are as follows: 

Appendix A is the list of the NMIs that are signatories to the arrangement, 

Appendix B contains the results of key and supplementary comparisons, 

Appendix C contains calibration and measurement capabilities (CMCs), 

Appendix D is the list of key and supplementary comparisons. 

The principal components of the CIPM MRA 

are contained in Appendices B and C. Appendix B 

lists the metrology areas, 

protocol specifications, participating laboratories, participating countries, measuring conditions, results, 

and associated uncertainties 

for Key and Supplementary 

Comparisons. CIPM Key 

Comparisons (KCs) are carried out by the CIPM Consultative Committees or the 

BIPM; these are carried out 

at the highest metrological 

level. Participation in the 

CIPM KCs is open to all 

signatories of the CIPM 

EUROMET

Key

Comparisons

SIM

Key

Comparisons

BIPM

and

Consultative Committee (CC)

Key Comparisons

APMP

Key

Comparisons

other regional

Key

Comparisons

other regional

Key

Comparisons

NIST

NRC

CIPM

BIPM 

Figure 5: The figure shows the interconnectivity among 

the RMOs, the BIPM, and the CIPM Consultative Committees. The RMOs serve to link their respective regions to 

the international measurement infrastructure. 
66

MRA. Key Comparisons are also carried out by regional metrology organizations 

(RMOs) such as SIM (Systema Interamericano de Metrologia), EUROMET (European 

Collaboration on Measurement Standards), and APMP (Asia-Pacific Metrology Programme). The RMO KCs must be linked to corresponding CIPM Key Comparisons by 

means of joint participants (Figure 5). RMOs may also carry out Supplementary Comparisons that address regional needs, not covered by Key Comparisons, to provide confidence in calibration and measurement certificates. 

The CIPM CCQM (Consultative Committee on Amount of Substance) is the committee responsible for the realization of the SI unit “mole”, and demonstrating the comparability of all chemical composition related measurements. Currently, the CCQM Key 

Comparisons are focused on measurements related to health care, food and nutrition (including drinking water), environment, advanced materials, commodities, forensics, and 

general analytical applications (e.g., calibration solutions, high purity materials, pH, conductivity, etc). 

NIST is the pilot (lead) laboratory for several of the CIPM Key Comparisons, including the CCQM-K6, cholesterol in human serum. This comparison has been completed 

and its report is in a final draft stage. Planned Key Comparisons that address clinical 

needs are glucose in serum and creatinine in serum. One of the Key Comparisons that 

has been completed is CCQM-K2 comparison, lead and cadmium in water. The data have 

been reviewed and approved for posting in the database; therefore they can be discussed 

here as an example. Along with the analytical results, a statement of the degree of equivalence is included, 

which is defined as 

the magnitude (Di) 

and uncertainty in 

the deviation (Ui) 

from the key comparison reference 

value. An example 

of a “Matrix of 

Equivalence” from 

the BIPM Key 

Comparisons Database, along with the 

graphic representation of the same 

data, are shown in 

Figures 6 and 7, respectively. A given 

authority can use 

the results in the 

BIPM Key Comparisons Database 

to determine if the 

Figure 6: A Matrix of Equivalence is shown for CIPM Key 

Comparison (CCQM – K2) in the results section of the BIPM 

Key Comparisons Database. Each cell in the matrix shows the

degree of equivalence between two NMIs.
67

degree of equivalence 

demonstrated between 

two NMIs (Dij and Uij, 

corresponding to each 

cell in the matrix) is sufficient for the purpose at 

hand, such as regulatory 

compliance or international trade. 

Appendix C of the BIPM 

Database provides detailed information on the 

existing metrological capabilities of each NMI. 

These calibration and 

measurement capabilities 

statements (CMCs) are 

developed and submitted 

by the participating NMI 

and provide the following 

information: the participating organization and country, metrology or calibration area, applicable range of measurements, uncertainties obtained by the institution, and the means of traceability to the SI. 

Currently, Appendix C is populated only in the areas of electricity and magnetism, and 

length. A draft template suggested for the amount of substance capability declaration has 

been prepared and includes the following categories: matrix, measurand, dissemination 

range of measurement capability, associated uncertainties, range in certified values of 

reference materials, and expanded uncertainties of certified values, source of traceabililty, 

measurement technique, and link to Appendix B if applicable. 

A Joint Committee of Regional Metrology Organizations and the BIPM, known 

as the JCRB, provides oversight for results included in Appendix B, and is responsible 

for determining the degree(s) of equivalence of results from individual NMIs as discussed 

above. The statement of the degree of equivalence is a very important component of the 

CIPM MRA, as it demonstrates comparability at the NMI level. 

The CIPM Key Comparisons that are carried out by its members are at the highest 

metrological level with traceability to the SI units. In the Americas, NIST, the National 

Research Council (NRC) of Canada, and the Centro Nacional de Metrologia (CENAM) 

of Mexico are members of the CIPM Consultative Committee on Amount of Substance, 

and as such take part in the CIPM Key Comparisons. However, in order to expand trade 

throughout each region, comparability of measurements is extended to non-members 

through Regional Metrology Organization (RMO) activities. 

Figure 7: The graphic depiction of the “degree of equivalence” for CCQM – K2 indicating how each NMI’s value compares with a reference value, and more importantly, how they

compare to each other. 
68

Systema Interamericano de Metrologia (SIM) is the regional metrology organization (RMO) of the Americas and thus includes the United States. 

The metrology organizations that make up SIM are known as: NORAMET 

(North America), CAMET (Central America), CARIMET (Carribean), 

ANDIMET (Andes Region), and SURAMET (Southern America). Regional cooperation leads to a wider harmonization of measurements and standards, facilitates free flow of trade, and is a necessary first step to globalization. NIST, CENAM, and 

NRC Canada link SIM countries to the BIPM, and as such, provide greater opportunities 

for international trade and commerce. 

Figure 8 shows the traceability and comparability link between two specific 

RMOs, SIM and EUROMET, through the NMIs that are members of the CIPM Consultative Committees. In addition to the KCs, RMOs can carry out Supplementary Comparisons (SCs) among its members to provide linkage to the BIPM/CIPM membership. Secondary Reference 

Laboratories may 

participate in SCs 

organized by their 

own RMO, SCs of 

other RMOs, as 

well as in Laboratory Accreditation 

(LAC) and Proficiency Testing (PT) 

Programs. The fundamental concept 

here is to establish 

confidence in the 

capabilities and the 

quality of measurements provided by a 

particular laboratory. Therefore, 

Secondary Reference Laboratories and Field Laboratories can demonstrate comparability 

via LAC and PT programs, as well as via traceability to the NMIs participating in CIPM 

Key Comparisons. 

NIST’s Chemical Science and Technology Laboratory staff provide leadership for SIM 

by chairing the Chemical Metrology Working Group of SIM, and by serving as the U.S. 

representative to the JCRB, in order to assure the effective, fair, and metrologically sound 

implementation of the MRA throughout the world. 

Figure 8: Comparability at the highest level is established 

among NMIs. However, for specific purposes and when mutually accepted among all parties, comparability between laboratories lower in the traceability structure may also be established 

through accreditation processes and proficiency testing. 
69

Summary 

The EU/US Mutual Recognition 

Agreement is a treaty that was negotiated to 

promote equity in trade by removing technical 

barriers. This treaty provides for mutual recognition of standards and conformance infrastructure. While the “Pharmaceutical and 

Medical Devices” area explicitly addresses 

health (and safety) issues, all of the areas 

specified in the treaty deal with health and 

safety regulations. This crosscutting focus indicates the value placed on health-related issues at the highest level of governments, and 

an understanding that economic, environmental and human health issues are intertwined. 

The EU IVD Directive, to become fully effective in December 2003, reminds us 

of the importance of traceability for international trade, as well as for improving accuracy 

of measurements. It is estimated that approximately 60 % of the IVD devices sold in 

Europe are imported from the U.S. U.S. manufacturers recognize the urgent need to be 

active participants in building the measurement and standards infrastructure that ensures 

continued and open commerce for their products. 

The CIPM MRA recently developed by the metrological community is an overarching framework that facilitates the implementation of the EU/US Mutual Recognition 

Agreement and provides comparability of measurements and standards worldwide. An 

important part of the CIPM MRA is the BIPM Key Comparisons Database that demonstrates equivalence of measurements among signatory NMIs. While the Key Comparisons are conducted at the NMI level, comparability is expanded on a regional basis 

through each RMO. The comparability of measurement capabilities demonstrated 

through the Key Comparisons is designed to facilitate trade by eliminating the need for 

retesting a product at every port of entry. The linkage through the national traceability 

structure helps to ensure that the measurements performed by industry to characterize 

their products are recognized in the global marketplace. It is clear that such a system is 

required to sustain the growth of global trade, and to remove technical barriers to trade. 

Over the past few years, the international metrological community has been building the 

framework for fair and efficient trading practices by demonstrating comparability of selected measurements through the implementation of the CIPM MRA. The IVD manufacturers and the medical community along with the NMIs must now work in partnership to 

identify the highest priority measurement standards for the timely implementation of the 

regulations imposed by the IVD Directive thus meeting its requirement of traceability to 

“standards of a higher order.” 

“Global economic integration, in 

itself is neither a panacea nor a 

curse. It is, however, a powerful 

phenomenon that incorporates 

many aspects of technological, 

economic, political and cultural 

change. We cannot, nor should we, 

wish to stop it. We can and must, 

however, shape it so that it advances our prosperity and values.”

Alan P. Larson, 

U. S. Under Secretary of State for 

Economic, Business and Agricultural Affairs, February, 2000 
70

References 

1. M. Boskin, L. Lau, “Generalized Solow-Neutral Technical Progress and Postwar 

Economic Growth,” NBER Working Paper, W8023, December 2000. 

2. “Agreement on Mutual Recognition between the European Community and the United 

States of America,” December 1, 1998. (Text available on the internet at 

http;//www.useu.be). 

3. “The US/EU Partnership for the 21st Century,” remarks given by U.S. Under Secretary of State Eizenstat at the Nixon Center, October 16, 1998. (Text available on 

the internet at http://www.useu.be/DOCS/Index.html). 

4. M. Suppo, “EU Starts Five-year Countdown for the IVD Directive,” IVD Technogy, 

January, 1999 (Text available on the internet at: 

 http://www.devicelink.com/ivdt/archive/99/01/006.html) 

5. "Council Directive 98/79/EC of the European Parliament and of the Council of 27 

October 1998 on In Vitro Diagnostic Medical Devices," Official Journal of the 

European Communities L331, December 1998. (Text available on the Internet at 

http://www.edma-ivd.be). 

6. Trahey, N. M., editor, NIST Standard Reference Materials Catalog 1998 99 edition, 

NIST Spec. Publ. 260, January 1998 (Text available on the internet at 

http//www.nist.gov). 

7. U.S. Industry and Trade Outlook ’99, U.S. Department of Commerce/International 

Trade Administration, McGraw-Hill Companies, 1999. 

8. “Cholesterol Measurement – Test Accuracy and Factors that Influence Cholesterol 

Levels,” General Accounting Office Report GAO/PEMD-95-8, December 1994. 

9. “Mutual Recognition of National Measurement Standards and of Calibration and 

Measurement Certificates Issued by National Metrology Institutes,” Edited by the 

BIPM, Paris, France, September 1999. 
71

HRATCH G. SEMERJIAN 

National Institute of Standards and Technology 

Chemical Science and Technology Laboratory 

Dr. Semerjian, Ph.D., has been the Director of the Chemical 

Science and Technology Laboratory (CSTL), one of seven 

Laboratories in NIST, since 1992. He is responsible for 

chemical engineering, biotechnology, analytical chemistry, 

process technology, and physical chemistry research activities at NIST. The mission of CSTL is to promote the U.S. 

economy by providing technical leadership for the Nation’s 

measurement and standards infrastructure, and assuring the 

availability of essential data and measurement capabilities. 

CSTL provides these infrastructural capabilities for U.S. industry, government agencies, and the scientific community, 

to enhance U.S. industry’s productivity and competitiveness, 

assure equity in trade, and improve public health, safety and 

environmental quality. CSTL has a total staff of about 450, 

and an annual budget of about $60 M. Dr. Semerjian received his B.Sc. with honors in 

Mechanical Engineering from Robert College (Turkey) in 1966, and his M.Sc. (1968) and 

Ph.D. (1972) in Engineering from Brown University. He served as a Lecturer and PostDoctoral Research Fellow in the Chemistry Department at the University of Toronto. He 

then joined the research staff of Pratt & Whitney Aircraft Division of United Technologies Corp. in East Hartford, CT. In 1977, Dr. Semerjian joined the National Bureau of 

Standards (now the National Institute of Standards and Technology). His research has 

focused on combustion processes, pollutant formation, and the application of optical 

techniques to chemically reacting systems. Dr. Semerjian has authored over 100 publications. He is an active member of AIChE, ACS, ASME, AIAA, AAAS, the Combustion 

Institute, and the Council for Chemical Research. He has also served on advisory and 

review boards for the Department of Energy, National Science Foundation, Environmental Protection Agency, National Oceanic and Atmospheric Administration and the 

Department of Defense. Awards he has received include AM&F Corp. Fellowship at 

Robert College, the Fullbright Fellowship, C.B. Keen Fellowship at Brown, the U.S. Department of Commerce Meritorious Federal Service (Silver Medal) Award in 1984, and 

the U.S. Department of Commerce Distinguished Achievement in Federal Service (Gold 

Medal) Award in 1995; he was also selected as the DoC "Federal Engineer of the Year" 

in 1991. In 1996, he was elected a Fellow of the American Society of Mechanical Engineers. In 1997, he received the Brown Engineering Alumni Medal. Dr. Semerjian served 

as Acting Deputy Director, NIST, January through October, 1997. In 2000, Dr. Semerjian was elected a member of the National Academy of Engineering. 
72
73

Mutual Recognition of Measurements: How Do They Impact 

International Trade? 

Visuals Presented By 

Hratch G. Semerjian 

National Institute of Standards and Technology 
74

Mutual Recognition of Measurement Capabilities Mutual Recognition of Measurement Capabilities

and International Standardization and International Standardization

Hratch G. Hratch G. Semerjian, Director , Director

IVD Traceability Traceability Workshop Workshop

Gaithersburg, MD Gaithersburg, MD

November 2, 2000 November 2, 2000

Chemical Science and Chemical Science and 

Technology Laboratory Technology Laboratory

National Institute of Standards and Technology National Institute of Standards and Technology

Forces for Forces for 

Measurement System Change Measurement System Change

•• World-wide economic growth wide economic growth 

rests heavily on the rests heavily on the 

advancement of technology advancement of technology

• Advancing technology requires Advancing technology requires 

new measurement capability

• Measurement standards and Measurement standards and 

certificates should be certificates should be 

recognized worldwide recognized worldwide

• Measurements must be traceable Measurements must be traceable 

to the international system of to the international system of 

units (SI) units (SI)
75

Traceability vs Traceability vs Comparability Comparability

NMI: National Measurement Institute

Seller Buyer

Specifications Requirements

Measurements Measurements

NMI NMIa b

comparability

traceability

trade

Approx. $500 M/yr NIST 

investment (0.7% of 

federal R&D)

Undergirds ~$10 B/yr of 

private sector investment 

in measurements and 

standards

Impacts U.S. economy -

More than half of $7.6 

T/yr U.S. GDP in sales 

supported by 

measurement

LEVERAGE

ADVANCING TECHNOLOGY...

• Is vital for commerce and international trade

• Accounts for ~50% of U.S. economic growth

• Drives demand for new measurements and standards

• Requires that NIST maintain state-of-the art scientific facilities

Impact of NIST Measurement and Standards Programs Impact of NIST Measurement and Standards Programs
76

International Comparability Methods International Comparability Methods 

A Major Change A Major Change

• Expanding national and regional metrology systems 

(beyond the 48 Convention du Mètre signatories)

• Improving turn-around time in comparability exercises 

to match product life-cycles

• International agreement that comparison data shall 

serve as the basis for mutual recognition of national 

measurement and calibration certificates

• Realizing increased leverage for the key comparisons

Agreement on Mutual Recognition Agreement on Mutual Recognition 

between the US and the EU between the US and the EU

1. Avionics

2. Environmental Protection

3. Pharmaceuticals and Medical Devices

4. Electromagnetic Compatibility and Interference

5. Occupational Health and Safety

Purpose:

“... to facilitate bilateral trade between the United States and the European 

Community, ... and mutual recognition of conformity assessment activities 

as an important means of enhancing market access ...”

Regulatory Areas included in agreement:
77

Measurements and Standards for Health Care Measurements and Standards for Health Care

+HDOWK&DUHDQG

3KDUPDFHXWLFDOV

Human 

Mitochondrial 

DNA

Fluorescence Standards 

under development

Fat-Soluble Vitamins, 

Carotenoids, & Cholesterol in 

Human Serum

Structural Biology &

Genomics

Biomimetic Membrane 

Technology

High Priority Clinical Diagnostic Markers

Troponin-I Heart attack

Glycated Hemoglobin Diabetes

p53 DNA Breast cancer

Bilirubin Liver function

TSH Thyroid function

Speciated Iron Anemia

PSA Prostate Cancer

Health care costs amount to ~ 14% of the GDP, an estimated $1.5 Health care costs amount to ~ 14% of the GDP, an estimated $1.5 trillion trillion

DNA Damage 

and Repair

Lab-on-a-Chip

Health Care Markers: EU IVD Directive Health Care Markers: EU IVD Directive

Magnitude and Scope:

• U.S. Spends ~ $1.5 trillion on Health Care (14% of GDP)

– ~13% of this amount is associated with measurements ($140B)

– Non-diagnostic measurements cost ~$39B

– Improvement in accuracy of one marker alone (cholesterol) is estimated to 

have saved $100M per year

New Driver: EU IVD Directive to go into effect 2003

• Worldwide in vitro diagnostic device market is ~$20B;

• >60% of European market is supplied by U.S. and Japan

Stated Purpose of Directive

• Eliminate trade barriers within Europe by ensuring access to the entire EU 

market with one single product approval (CE Mark)

Essential Requirements

• IVD Calibrators and/or control materials must be traceable to “standards of a 

higher order”

– nationally/internationally recognized certified reference materials

Implementation

• First IVD product with CE Mark may be placed from June 2000 onwards

• All new IVD products must have mark by December 2003

• Existing IVD products may be sold without the CE mark until December 2005
78

Measurement Traceability Measurement Traceability

VIM Definition:

The property of the result of 

a measurement or the value 

of a standard whereby it can 

be related to stated 

references, usually national 

or international standards, 

through an unbroken chain 

of comparisons all having 

stated uncertainties.

Basic Units

Definitive Methods

Primary

Reference

Materials

Reference Methods

Secondary

Reference

Methods

Field Methods

• Reference method 

development

• Production of highaccuracy, secondary 

reference materials

• Critical quality assurance 

applications

• Field method 

development

• Preparation of working 

reference materials

• Routine quality 

assurance

Field Applications

Additional Requirement:

Traceability only exists when 

scientifically rigorous evidence 

is collected on a continuing 

basis showing that the 

measurement is producing 

documented results for which 

the total measurement 

uncertainty is quantified. S.D. Rasberry, ASQC Quality Congress Transactions (1983)

A National Traceability System A National Traceability System

• Tools for the dissemination of 

measurement standards

– reference materials

– reference data

– calibrations

– special arrangements for 

greater leverage

SI

Units

RMOs

NMIs

Private and State

Lab Measurements

Field Measurements
79

Environment Health Care

Industrial Process Control &

Materials Performance

Standard Standard

Reference Reference

Materials Materials 

Global Science

Nutrition

CIPM Mutual Recognition Arrangement CIPM Mutual Recognition Arrangement

Objectives: Objectives:

• Establish the degree of equivalence of national 

measurement standards maintained by NMIs

• Provide for the mutual recognition of calibration 

and measurement certificates issued by NMIs

• Provide a secure technical foundation for wider 

agreements related to international trade, 

commerce and regulatory affairs
80

CIPM Mutual Recognition Arrangement CIPM Mutual Recognition Arrangement

Process: Process:

• International comparisons of measurements, to 

be known as key comparisons

• Supplementary international comparisons of 

measurements

• Quality systems and demonstration of 

competence by NMIs

Outcome: Outcome:

• Statements of the measurement capabilities of 

each NMI in a database publicly available on the 

Web

CIPM Key and Supplementary CIPM Key and Supplementary 

Comparisons Comparisons

EUROMET

Key

Comparisons

SIM

Key

Comparisons

BIPM

and

Consultative Committee (CC)

Key Comparisons

APMP

Key

Comparisons

other regional

Key

Comparisons

other regional

Key

Comparisons

NIST

NRC

CIPM

BIPM
81

Metrology Areas Metrology Areas

• Amount of substance (mole)

• Electricity and Magnetism

• Ionizing Radiation

• Length

• Mass

• Photometry and Radiometry

• Thermometry

• Acoustics, Ultrasound, and Vibration

Principal MRA Components Principal MRA Components

• Comparisons

– Metrology areas

– Protocol specifications

– Participants

– Measuring conditions

– Results and 

uncertainties

• Capabilities

– Measurement or 

calibration area

– Applicable range

– Uncertainties attainable

– Means of traceability to 

the SI

Appendix B Appendix B Appendix C Appendix C

Recorded in International Comparisons Database Recorded in International Comparisons Database 

developed by NIST and maintained by BIPM developed by NIST and maintained by BIPM
82

CIPM Key Comparison CIPM Key Comparison - Example Example

CIPM MRA Appendix B CIPM MRA Appendix B - Example Example
83

CIPM MRA Appendix C CIPM MRA Appendix C - Example Example

Template for Amount of Template for Amount of 

Substance Measurement Substance Measurement 

Capability Declarations Capability Declarations 

with selected NIST examples. with selected NIST examples. 

Linking Vertical Systems of Traceability Linking Vertical Systems of Traceability

Assuring Global Comparability of Measurements Assuring Global Comparability of Measurements traceability

comparability

SI SI

RMO (SIM) RMO (EUROMET)

NMIs NMIs

Secondary

Ref. Labs

Secondary

Ref. Labs

Field Labs Field Labs

CIPM KC’s

RMO KC’s, SC’s

RMO SC’s

LAC PT

LAC PT
84

Summary Summary

• Health Care has great impact on the economy and everyday life

• US-EU MRA provides framework for promoting international trade

• CIPM MRA facilitates the mutual recognition of measurements 

performed in other countries

• EU IVD Directive identifies an area where current standards are 

inadequate

A cooperative international effort is 

required to meet the challenges of 

providing necessary measurement 

standards
85

Traceability of Calibration for IVD Industry: 

Process for Credentialing Reference Materials, 

Reference Methods and Reference Laboratories in Europe 

Lothar Siekmann 

University of Bonn • Institut für Klinische Biochemie 

The concept of measurement traceability has been established in general chemical 

metrology and is now also being introduced to the field of clinical chemical analyses. 

Traceability provides probably the most important strategy to achieve standardisation in 

laboratory medicine aimed at comparable measurement results regardless of the method, 

the measurement procedure (test kit) and of the laboratory where analyses are carried out. 

According to the ‘Vocabulary in Metrology (VIM)’ and the ‘Guide to the Expression of Uncertainty in Metrology (GUM)’ measurement traceability is defined as: 

property of the result of a measurement or the value of a standard whereby it can 

be related to stated references, usually national or international standards, through 

an unbroken chain of comparisons all having stated uncertainties 

Traceability of a value attributed to a routine sample, a calibrator or a control material is established by a series of comparative measurements using measurement procedures and reference materials in a chain of increasing hierarchical order as shown in 

Fig.1. Since each link in the traceability chain contributes to the uncertainty of the result 

it is advisable to omit as many steps as possible. In terms of metrology it would be ideal 

to omit all in between steps of the traceability chain and to measure the routine sample 

directly by use of a primary reference procedure; this of course is not feasible. 

The complete traceability chain as presented here is valid only for those measurable quantities that can have a value expressed in SI units. When primary or secondary 

calibrators are not available, the traceability chain for many measurands in laboratory 

medicine ends at a lower level, e.g. at the manufacturer’s selected measurement procedure. In a case where a manufacturer detects a new diagnostic marker and defines the 

measurable quantity by establishing a measurement procedure for this marker, the 

maunufacturer’s measurement procedure will form the top of the traceability chain. Nevertheless even in this simple situation the principles of the traceabilty concept are applicable. 

An inevitable precondition for establishing traceable results to calibrators and 

control materials is the specificity of the measurement procedures applied. Results of 

measurement cannot be traceable when the procedure applied partially detects components that are not consistent with the definition of the measurand. 

Traceability is not really a new fundamental concept in our science. Many years 

before traceability has been mentioned in general chemical metrology, reference meas-
86

urement procedures and reference materials have been established in clinical chemistry. 

The early developments in this field in the U.S., particular by the NIST colleagues 

Cohen, Ellerbe, Hertz, Mandel, Schaffer, Sniegoski, Welch, White and others [1-12] as 

well as by individual scientists such as Cali [13], Tietz [14-16], Eckfeld [17], and Doumas [18-19] as well as the relevant NCCLS standards [20-23] probably had an important 

influence on the development of the concept of traceablility in general chemical metrology. Some basic experimental work for the development of reference measurement procedures and reference materials had already been undertaken in Europe, e.g. by Björkhem 

[24-26], Büttner [27-29], Külpmann [30- 31] Schumann [32-33], Siekmann [34-43], 

Stöckl [44-46], Thienpont [47-53], and others. 

FIGURE 1. Calibration Hierarchy and Traceability 

SI-Unit

(definition) BIPM

Primary calibrator

Secondary calibrator

Mf. ́s working

(master) calibrator

Mf. ́s product 

calibrator

Secondary reference

measurement procedure

NMI

ACL

NMI

ACL

Mf. ́s reference

measurement procedure

Mf. ́s standing

measurement procedure

End-user ́s routine

measurement procedure

Routine sample

RESULT

NMI

ACL

ACL

MCL

ML

Mf.->

End-user

Mf. and/or

End-user

MCL

Traceability

BIPM: International Institute of Weights and Measures

NMI : National Metrological Institute

ACL : Accredited Calibration Laboratory

MCL : Manufacturer ́s Calibration Laboratory

ML : Manufacturer ́s Laboratory 

Mf : Manufacturer

End-user

End-user

Primary reference measurement procedure

ML

μc(y)

In 1970, a long time before the concept of traceability became popular, the technique of isotope dilution mass spectrometry had been developed for the first time in a 

clinical chemical reference laboratory and was applied as a reference procedure for the 

measurement of estrogens in human body fluids [35]. Ever since that time, this technique 
87

provides one of the most powerful tools for establishing reference method values for 

many substrates and metabolites in calibrators, controls and reference materials. 

In view of the globalisation of economics, science and, hopefully, also social developments, the process of credentialing, the traceability concept and its implementation 

and acceptance, is not only a regional (European) or national task, but it is a global task. 

Traceability is a task that concerns all members of our scientific community involved in 

the field of clinical chemical analyses. 

Such as: 

• Legislative bodies, when issuing regulations or directives concerning measurements in laboratory medicine, e.g. FDA, CEC. 

• The national metrology institutes, e.g. NIST, which are responsible 

for dissemination of the SI units (Système International d’Unités) 

and for establishing correctness of measurement according to national legislation. 

• International organisations issuing reference materials, e.g. NIST, 

IRMM, WHO. 

• International and national standardisation organisations, e.g. ISO, 

CEN, NCCLS. 

• International and national scientific societies, e.g. IFCC. 

• External quality assessment organisations, e..g. CAP, Reference 

Institut of Bioanalysis (DGKC). 

• Reference laboratories. 

• Diagnostic kit manufacturers. 

• Clinical chemical laboratories when applying commercial or 

home-made diagnostic tests, and 

• Physicians determining diagnosis and therapy on the basis of 

laboratory results. 

Credentialing traceability in clinical chemistry implies the demonstration of applicability and usefulness of the concept as a basis of credibility. In practice, reference 

systems are to be established consisting of reference measurement procedures, reference 

materials and reference laboratories, which should preferably be accredited and organised 

within a network. 

The introduction of such reference systems has been proposed in clinical chemistry for about 30 years. However, neither the international scientific community nor any 

national or international body have addressed the question of which agency or authority 

should be responsible for formal authorization of these reference systems; including reference materials, reference procedures and reference laboratories. 
88

For reference materials it may not be too difficult to solve the problem. Materials, 

which fulfil the requirement for higher metrological order standards, are now provided by 

the NIST and the IRMM. The WHO, although usually of lower metrological order, issues additional useful materials. Authorization of these materials is a question of mutual 

acceptance, which, in view of the “Implementing Arrangement for Cooperation in the 

Fields of Metrology and Measurement Standards” signed by the directors of the NIST 

and the EU`s Research Directorate, has almost been achieved. 

For the authorization of reference procedures and reference laboratories the situation is somewhat more complicated and the question of how the concept of traceablity 

should be implemented arises. There is no simple answer and probably there is no general 

rule that can be applied to all situations. The new International Standards dealing with 

`traceability` (prEN ISO 17511) and the `requirements for reference laboratories` 

(ISO/DIS 15195) developed by CEN TC140 and ISO TC212, can at least give some 

guidance for the credentialing process for reference laboratories and reference procedures. In fact, the strategy for establishing reference systems depends on the nature of the 

analyte. 

For low molecular substances - electrolytes, organic substrates and metabolites 

like cholesterol, creatinine or steroid hormones - as well as for many drugs, the only 

meaningful way is to aspire to results that are traceable to the SI units. For these measurands, where traceability to SI units is achievable, national metrology institutes as legal 

custodians of units and measurements hold the highest authority (Fig 2). The metrology 

institutes are connected globally with each other by so-called CCQM (Consultative 

Committee on Amount of Substance) key comparisons, where they demonstrate, by the 

means of ring trials, their ability to perform measurements of the highest available metrological level, usually by the use of so-called primary methods. Such key comparisons 

were carried out for cholesterol, creatinine and glucose. A group of European metrology 

institutes has now started a global initiative to address the problem of traceability in clinical chemical measurements in collaboration 

with the NIST and the metrology institutes in 

Australia and Japan. 

In addition to the national metrology institutes 

there exist a number of highly specialised reference laboratories most of which are situated 

in university hospitals and at manufacturer’s 

sites. Usually these laboratories have developed their own reference procedures. Some of 

them have long-standing experience and perform measurements at a high metrological 

level. 

According to the ISO standard on reference laboratories in laboratory medicine their 

competence may be approved by the national 

metrology institutes, e.g. by accreditation 

according to ISO 17025. The German 

metrology institute (PTB) has so far accredited 

two reference laboratories; the first one is the 

National Metrology Institutes

Accredited Reference Laboratories

(universities, hospitals, manufacturers)

Manufacturer‘s Standardisation

Laboratories

Routine Laboratories

FIGURE 2. Hierarchy of Laboratories
89

reference laboratories; the first one is the author’s laboratory which mainly serves the 

proficiency testing organisation ‘Reference Institute of Bioanalysis’ of the German 

Society of Clinical Chemistry (DGKC) and also establishes target values for calibrators 

and control materials of commercial diagnostic kits; the second laboratory is that of an 

industrial diagnostic kit manufacturer; further accreditations will follow in due course. 

The competence of reference laboratories concerning the environmental, personnel and management performance may be approved by accreditation; this still leaves a 

question that sometimes appears in letters to the editors of scientific journals, e.g. Clinical 

Chemistry. The question is: “When is a ‘proposed’ reference method a reference 

method?”. The answer here is that the competence of a reference laboratory should not 

only be evaluated according to its management quality laid down in a quality manual and 

monitored by regular inspections of the laboratory, but also on the basis of the documented reference procedures and – most importantly – on the results of parallel comparative measurements. In this way the accrediting body also approves the measurement procedures and their performance. Consequently, accreditation is not valid for all measurements of the laboratory but only for particular measurands for which agreement of results 

with those of the laboratory of the accrediting body and thereby traceability to the SI has 

been demonstrated in comparative measurements. 

Therefore, accreditation of a reference laboratory for a particular measurable 

quantity also contains an approval of the measurement procedure that includes the measurement principle, e.g. IDMS as a ‘primary method’ for the measurement of cholesterol, 

the complete standard operating procedure and the uncertainty of results. This may serve 

as an answer to the question “When is a ‘proposed’ reference method a reference 

method?”

In Europe the concept 

of traceability was promoted 

during the last 15 years by the 

organisers of external quality 

assessment schemes (EQAS). 

In the German proficiency 

testing system in particular the 

use of reference measurement 

procedures for several measurands is prescribed by legislation since 1988. As a result 

the “Reference Institute for 

Bioanalysis“ has established 

reference measurement procedures for electrolytes, metabolites and substrates, hormones 

and drugs (Fig. 3). Reference 

methods for 13 of these 20 

analytes have been developed 

using the analytical principle 

of the so-called ‘primary 

DGKC Reference Procedures

• Electrolytes:

Calcium

Chloride 

Lithium

Magnesium

Potassium

Sodium

• Metabolites and Substrates:

Cholesterol

Creatinine

Glucose

Total Glycerol

Uric Acid 

Urea

Bilirubine

Lactate

• Enzmes;

AST GGT

ALT AMYLASE

CK

• Hormones:

Aldosterone

Cortisol

Estradiol-17ß

Estriol

Progesterone

17-Hydroxy-progesterone 

Testosterone

Thyroxine

• Drugs:

Theophylline

Digoxin 

Digitoxin 

• Total Protein

FIGURE 3. Reference Procedures Established by the 

Reference Institute of Bioanalysis (DGKC)
90

method’ isotope dilution mass spectrometry in the reference laboratories of the DGKC. 

These include creatinine, urea, cholesterol, total glycerol, uric acid and glucose, as well 

as the steroid hormones and thyroxine. The reference methods are now applied regularly 

for the setting up of target values in the control samples of internal and external quality 

assessment as well as for certifying matrix reference materials of the IRMM. 

We currently share the system with partners in Portugal, the Czech Republic and 

occasionally in Denmark by exchanging samples for external quality assessment with 

reference procedure target values. 

How the introduction of the concept of traceability improved the performance of 

diagnostic tests since 1988 may be of some interest. A look at the list of routine method 

target values for creatinine, uric acid, total cholesterol and total glycerol in the control 

material of one manufacturer issued before 1988, clearly shows that a large scatter of up 

to 30 % existed, depending on which method or test kit was used (Fig 4). This situation 

was particularly untenable considering the fact that only one value for creatinine concentration in serum can be the "true" one. Obviously any progress towards improving the 

comparability of analytical results from different laboratories is hindered as long as 

methods with a known or even unknown bias are accepted. 

FIGURE 4. Procedure Dependent Target Values in a Commercial Control Serum

This unsatisfactory situation also became visible in external quality assessment. 

For example, two different samples were distributed in a routine ring trial of the DGKC 

for cholesterol to about 1300 laboratories and the results were then displayed in a 

YOUDEN diagram (Fig. 5). Each dot in this diagram represents the two results from one 

laboratory, whereby the result for sample A can be read from the abscissa and that for the 

sample B from the ordinate. A laboratory with its dot just in the middle of the screen is in 

full agreement with the target value, which here is the reference method value certified 

by isotope dilution mass spectrometry. 

CREATININE μmol/l CHOLESTEROL mmol/l 

------------------------------------------------------------- -----------------------------------------------

Enzymatic/ PAP 151 CHOD - Iodide 4.02

Enzymatic UV system 161 CHOD - PAP 4.30

Jaffe without deproteinisation (Merck) 168 CHOD - Katalase 4.61

Jaffe after deproteinisation (Boehr.) 177 Peridochrom 4.69

Jaffe without deproteinisation (Boehr.) 189 Liebermann-Burchard 5.49

URIC ACID μmol/l TRIGLYCERIDES mmol/l

------------------------------------------------------------- -----------------------------------------------

Fully enzymatic (Boehr./Merck) 457 Fully enzymatic (Behr.) 1.15

UV - system (Boehr.) 476 Fully enzymatic (Merck) 1.34

UV - system (Merck) 539 Fully enzymatic (Roche) 1.30

Phosphotungstic acid (Goed.) 583 Enzymatic (Boehr.) 1.36
91

Participants ́ results 

from this survey for cholesterol in 1987 clearly show 

that three different groups of 

data have been reported according to three different 

methods of cholesterol determination. The participants 

with relatively high cholesterol results had used the 

Liebermann-Burchard procedure, which was still in 

use in 1987. The group with 

low cholesterol values had 

applied the cholesterol oxidase iodide method, and the 

data of laboratories using the 

CHOD/PAP method are 

situated in the middle of the 

screen. Until 1988 participants results were evaluated 

by comparison with the 

means of their peer group 

according to the different 

methodological principles 

used. Differences of up to 

50% between the peer group 

target values could be observed for cholesterol measurements. In view of the fact 

that there can be only one true cholesterol concentration value in a serum, this situation 

was clearly untenable. 

After introducing the reference procedure values for cholesterol, based on IDMS 

measurements, the different peer group target values have now been replaced by 

reference method values, which in our case are represented as the exact middle of the 

screen. The corresponding limits of acceptance are shown as the solid square. As a 

consequence, methods with inherent systematic errors like the Liebermann-Burchard 

method and the CHOD-iodide method, disappeared from the market. Today only 

methods which are within the limits of acceptance with the reference method values 

established by isotope dilution mass spectrometry exist. 

In fact, 12 years ago there was an unacceptably wide scatter of method-dependent 

target values for many clinical chemical parameters. In order to improve accuracy in 

clinical chemistry it was absolutely essential to replace these method-dependent target 

values with reference method values. 

FIGURE 5. YOUDEN Diagram of a Collaborative Survey for

Cholesterol (1987) 

The three broken-line squares show the methoddependent evaluation limits for the Liebermann-Burchard

method (upper-right), for the CHOD-PAP methods (middle) and the CHOD-Iodide method (lower-left). The solidline square in the middle shows the acceptance limits on

the basis of the IDMS reference method value as target. 
92

The measurement of hormone concentrations in human body fluids has proved to 

be a valuable diagnostic tool in the field of clinical endocrinology. Thyroxine and the 

various steroids are the most commonly determined hormones and are usually measured 

by radio immunoassay (RIA) or by enzyme immuno assay (EIA) with a fairly high 

degree of sensitivity. However, a manufacturer’s list of aldosterone-, cortisol-, 

progesterone- and oestradiol-17ß target concentrations in a commercial serum pool 

indicates that given the same sample and using immunoassay, assigned values varied 

considerably from one test kit to another (Fig 6). For cortisol and aldosterone the range of 

results was between 100% and 200%; for progesterone and oestradiol-17ß determinations 

the results differed by a factor of 7. This was probably due to the different qualities of the 

antibodies and reagents used in the various commercial test kits. What could a consensus 

value mean in such a context? A target value based on a consensus mean or median was 

of little use in judging test kits that gave such variable results. 

Aldosterone Cortisol Progesterone Estradiol

pmol/l nmo/l nmo/l pmol/l

_____________________________________________________________________

ABBOTT 121.9

AMERSHAM 113.1

BAXTER DADE DIR 104.8 2.16 396.4

BAXTER DADE AG ER 244.1

BAXTER DADE AD EXT 196.0

BECTON DICINSON 88.0

BIOCLONE 1.91

BIOMERIEUX 2.54 539.6

BIOTEX PREMIX 99.4 70.6 3.72 759.9

CAMBRID ́GE MEDICAL 120.8 0.86

CIBA CORNING 110.3

CLINICAL ASSAYS 99.3

CYBERFLUOR FIAGEN 88.2

DIANOSTIC PRODUCTS 207.7 113.1 3.12 119.3

DUPONT RIANEN 135.1

EURODIAGNOSTICS 115.8

FARMOS DIAGN. 99.3 4.67 394.9

IMMUNCHEM COV. COAT 110.3 5.41 348.7

LEECO 113.1 2.99 144.2

MALLINCKRODT 88.3

NML RIA 96.6

NMS PHARMACEUTICALS 3.18 205.5

PANTEX IMMUNO DIRECT 143.1

PANTEX IMMUNO 118.6 4.13 190.8

PANTEX IMMUNOCOAT 132.4 7.00 154.9

PHARMACIA DELFIA 99.9 790.0

RSL 169.2 4.77 117.4

SCLAVO LISO PHASE 277.4 126.9 3.82

SERONO 112.0

SIBAR ELISA 121.3 1.27

SORIN 165.9 68.9 2.86 139.5

SYVA EMIT 137.9

TECHLAND RIA 4.77

VITEK SYSTEMS 110.0

FIGURE. 6 Target Concentrations for Steroids Hormones in a Commercial Control Serum 

Using method-dependent assigned values for external quality control means having many different target values for the same analyte in the same control serum is a very 

impractical and, from a theoretical point of view, very unsatisfactory procedure since 
93

they generate many different results for a substance of known molecular weight and with 

a defined number of molecules. 

It, therefore, seemed imperative to establish a methodology that would provide 

the basis for the development of reference methods. As a result, the target values for the 

collaborative surveys of the DGKC for steroid hormones have been determined by 

reference methods since 1977 and more recently also for thyroxine. 

Only recently we had to reply to a complaint of a manufacturer who suspected 

that the bad performance of his customers in our proficiency system surveys for 

progesterone was due to commutability problems of the quality control materials used in 

our ring trials. The unsatisfactory performance of the test became visible as an increasing 

bias of the test kit results at lower progesterone lower concentrations (Fig 7, sample B). 

FIGURE 7. YOUDEN Diagram (left) and Test-Kit Specific Evaluation (right) of a Collaborative 

 Survey for Progesterone 

In order to validate the commutability of our control materials it was necessary to 

perform split sample measurements with patient samples using the test kit in parallel to 

the IDMS reference procedure for progesterone. At the first glance a good correlation 

between the test kit and the IDMS reference procedure could be observed (Fig. 8a). 

However the difference plot of the same data revealed for both the patient sera 

and the ring trial results a considerable bias in relation to the reference procedure at low 
94

progesterone concentrations (Fig 8b). The reason for the bad performance of the test was 

obviously a lack of specificity rather than a lack of commutability of the control materials. At even lower progesterone concentrations the bias increased up to 1000 %. It should 

be noted that the kit manufacturer, unfortunately, did not issue any lower limit of determination for his measurement procedure. 

PROGESTERONE

MEDIANS IN

RING TRIALS

y = 2,98 + 0,87 x

n = 40, r = 0,681

Median (x) = 14,2

Median (y) = 15,75

Median (y-x)% = 10,3

PROGESTERONE

PATIENT SAMPLES

y = 1,84 + 0,96 x

n = 42, r = 0.99

Median (x) = 19,2

Median (y) = 17,9

Median (y-x)% = 17,4

0

10

20

30

40

50

60

70

0 10 20 30 40 50 60 70

IDMS / Progesteron [nmol/l] 

D PC-R IA / Progesteron [nmol/l] 

0

10

20

30

40

50

60

70

0 10 20 30 40 50 60 70

IDMS / Progesteron [nmol/l] 

DPC-RIA / Progesteron [nmol/l] 

FIGURE 8a. Method Comparison for Progesterone 

Test-Kit D – IDMS - Regression Analysis

PROGESTERONE

MEDIANS IN

RING TRIALS

 Test-Kit D

PROGESTERONE

PATIENT SAMPLES

 Test-Kit D

-100

0

100

200

300

400

500

600

700

800

900

1000

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30

Progesteron [nmol/l]

Deviation (%)

-100

0

100

200

300

400

500

600

700

800

900

1000

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30

Progesteron [nmol/l]

Deviation (%)

FIGURE 8b. Method Comparison for 

Progesterone 

Test-Kit D – IDMS - Plot of Difference

During the early years of external quality control, the accuracy of unconjugated estriol in serum proved to be astonishingly high. This changed dramatically towards the end of 1981 (Fig. 9). 

Especially when the control 

samples contained conjugated 

estriol, the ranges of participants’ results were significantly higher than the mass 

spectrometric target values. 

 Estriol

-50

-30

-10

10

30

50

70

Year

Deviation [%]

 1980 1981 1982 1983 1984 1985 1986 1987 

FIGURE 9. 

Results of Collaborative Surveys for Estriol 

Columns Indicate the Range Between the 

16th and 84th Percentile of Distribution of 

Each Survey
95

As it turned out, just at that time a kit manufacturer, who dominated the 

estriol-determination market in Germany, started using a new antibody. This obviously gave rise to cross-reactions with the conjugated steroid. Meanwhile a small 

group of participants who used their own laboratory methods to determine estriol continued to produce results that accorded with mass spectrometric values. It was possible to convince the manufacturer that this state of affairs required correction and, 

mainly due to this, results have improved greatly since 1985. However, it must be assumed that in the time span from 1981 to 1984 test kits did not only fail to measure 

estriol in control samples but also in patient samples. Estriol determinations are 

mainly used to monitor fetal well-being in the last months of pregnancy. Since not 

only estriol but also estriol conjugates are elevated in this period, we suspect that nonconjugated estriol was probably overestimated due to the test kit ́s lack of specificity 

not only in control samples, but also in patient samples. 

For non-SI traceable quantities the strategy for introducing traceability has to 

be different. This concerns a large number of analytes for which no defined molecular 

structure can be assigned, such as for many enzymes, proteo-hormones, tumour 

markers, cardiac markers. Before it is possible to establish reference systems 

(reference procedures, materials and laboratories) the measurand under consideration 

must first be defined. Whenever possible, a global consensus on the definition of the 

measurand should be achieved. Consequently, definition of the measurand and 

establishment of reference systems is the objective of several working groups and 

committees of the Scientific Division of IFCC. 

In many instances a selected and agreed reference measurement procedure 

forms the basis of the definition of the measurand and thereby represents the top of 

the hierarchical traceability chain. This is particularly true for establishing reference 

systems for the catalytic concentrations of enzyme activities. In 1999 members of the 

IFCC working group and some enzyme reference laboratories decided to establish 

new 37° measurement procedures as ‘IFCC reference methods’ on the basis of the 

existing 30° IFCC methods and to certify enzyme reference materials for ALT, GGT, 

CK and LD in collaboration with the IRMM. The project was conducted in three 

steps: 

1. Discussion and decision on primary procedures for the measurement of 

catalytic activities as ‘IFCC reference methods’. Some experimental work was 

necessary for this in order to achieve optimised conditions for the measurement 

protocol. The members of the working group agreed upon standard operating 

procedures (SOP). The standard operation procedures include control and 

reporting of traceability of all individual steps of the procedures, e.g. for mass, 

volume, temperature, photometric wavelength and absorbance measurements. 

2. The performance of laboratories applying the standard operating procedure 

was demonstrated in feasibility studies by analysing several commercial control 

materials. Depending on the enzyme, 10 – 12 laboratories from hospitals and 

diagnostic kit manufacturers were involved in the project. 
96

3. In the certification campaign the participating laboratories were asked to carry 

out the measurements on at least three different occasions. A material from the 

feasibility study was included in the analytical series for internal quality assessment. 

As shown in Fig. 10a and 10b the certification campaign for four different 

BCR enzyme reference materials (ALAT, GGT, CK and LD) demonstrates that: 

1. The 95 % confidence interval of the laboratory results is less than 2.5 % for all 

enzymes under investigation, which shows the excellent metrological performance of the participating laboratories from the Far East (Japan) to the Far West 

(California). 

2. The standard operating procedures, which were developed in the course of the 

study, can be used as a reference points for the definition of the measurands as 

the top of the traceability chain. The procedures will now be published as IFCC 

reference methods. 

95 96 97 98 99 100 101 102 103 104 105

C F A S / 3 7 ° C 

1 9 9 8 , n = 1 3

A S A H I / 3 7 ° C 

1 9 9 8 , n = 1 3

B e c k m a n 1 / 3 7 ° C 

1 9 9 8 , n = 1 2

B e c k m a n 2 / 3 7 ° C 

1 9 9 8 , n = 1 2

C R M 4 2 6 / 3 0 ° C

1 9 9 2 , n = 9

C R M 4 2 6 / 3 7 ° C

1 9 9 2 , n = 8

C R M 4 2 6 / 3 7 ° C

1 9 9 9 , n = 1 2

 R e la tiv e R e c o v e ry a n d 9 5 % -C o n fid e n c e In te rv a l (m e a n = 1 0 0 % ) [% ]

A L A T - a c c e p te d r e s u lts

95 96 97 98 99 100 101 102 103 104 105

C F A S / 3 7 ° C 

1 9 9 8 , n = 1 2

A S A H I / 3 7 ° C 

1 9 9 8 , n = 1 3

B e c k m a n 1 / 3 7 ° C 

1 9 9 8 , n = 1 2

B e c k m a n 2 / 3 7 ° C 

1 9 9 8 , n = 1 1

C R M 3 1 9 / 3 0 ° C 

1 9 8 6 , n = 9

C R M 3 1 9 / 3 7 ° C 

1 9 9 3 , n = 6

C R M 3 1 9 / 3 7 ° C 

1 9 9 9 , n = 1 2

 R e la tiv e R e c o v e r y a n d 9 5 % -C o n fid e n c e In te rv a l (m e a n = 1 0 0 % ) [% ]

G G T - a c c e p te d r e s u lts

FIGURE 10a. Results of the Certification of BCR Enzyme Reference Materials for ALAT and GGT. 

The bar graphs show the 95 %-confidence intervals obtained in the certification experiment in 1999 

(top) in comparison to those of former certification campaigns using different methods (temperatures). 

The lower bar graphs show the 95 %-confidence intervals obtained from a feasibility study in 1998 

before the certification campaign using different commercial calibrator preparations
97

95 96 97 98 99 100 101 102 103 104 105

C F A S / 3 7 ° C 

1 9 9 8 , n = 1 2

A S A H I / 3 7 ° C 

1 9 9 8 , n = 1 2

B e c k m a n 1 / 3 7 ° C 

1 9 9 8 , n = 1 0

B e c k m a n 2 / 3 7 ° C 

1 9 9 8 , n = 1 0

C R M 2 9 9 / 3 0 ° C , 

n=9

C R M 6 0 8 / 3 0 ° C

1 9 9 5 , n = 5

C R M 6 0 8 / 3 7 ° C

1 9 9 9 , n = 1 2

 R e la tiv e R e c o v e ry a n d 9 5 % -C o n fid e n c e In te rv a l (m e a n = 1 0 0 % ) [% ]

C K

94 95 96 97 98 99 100 101 102 103 104 105 106

C F A S / 3 7 ° C 

1 9 9 8 , n = 1 2

A S A H I / 3 7 ° C 

1 9 9 8 , n = 1 2

B e c k m a n 1 / 3 7 ° C 

1 9 9 8 , n = 1 0

B e c k m a n 2 / 3 7 ° C 

1 9 9 8 , n = 1 0

C R M 4 0 4 / 3 0 ° C 

1 9 9 4 , n = 8

C R M 4 0 4 / 3 7 ° C 

1 9 9 4 , n = 8

C R M 4 0 4 / 3 7 ° C 

1999

 R e la tiv e R e c o v e ry a n d 9 5 % -C o n fid e n c e In te rv a l (m e a n = 1 0 0 % ) [% ]

L D - a c c e p te d r e s u lts

FIGURE 10b. Results of the Certification of BCR Enzyme Reference Materials for CK and LD. 

The bar graphs show the 95 %-confidence intervals obtained in the certification experiment in 

1999 (top) in comparison to those of former certification campaigns using different methods 

(temperatures). The lower bar graphs show the 95 %-confidence intervals obtained from a 

feasibility study in 1998 before the certification campaign using different commercial calibrator 

preparations. 

So far reference systems for the measurement of catalytic activity 

concentrations for four different enzymes have been successfully established and can 

now be used for assigning traceable values to calibrators and control materials. 

The IFCC enzyme project, which has been conducted together with the 

IRMM, could be regarded as a model for the development of reference systems in 

other fields of interest. 
98

In summary it can be stated that for SI- traceable measurands the credentialing 

process has to some extent been successful, although the full implementation of the 

traceability concept on a global basis still requires considerable efforts. 

For non-SI-traceable quantities the predominant objective must be an agreement on the definition of these quantities before reference systems comprising reference procedures, materials and laboratories can be established. 

References 

1. Cali JP, Mandel J, Moore L, Young DS. A referee method for the determination of 

calcium in serum. National Bureau of Standards Special Publication 260-36, 1972. 

2. Velapoldi RA, Paule RC, Schaffer R, Mandel J, Murphy TJ, Gramlich JW. A reference method for the determination of chloride in serum. National Bureau of Standards 

Special Publication 260-67, 1979. 

3. Velapoldi RA, Paule RC, Schaffer R, Mandel J, Machlan LA, Gramlich JW. A reference method for the determination of potassium in serum. National Bureau of Standards Special Publication 260-63, 1979. 

4. Velapoldi RA, Paule RC, Schaffer R, Mandel J, Moody RJ. A reference method for 

the determination of sodium in serum. National Bureau of Standards Special Publication 260-60, 1978. 

5. Velapoldi RA, Paule RC, Schaffer R, Mandel J, Machlan LA, Garner EL, et al.. A 

reference method for the determination of lithium in serum. National Bureau of Standards Special Publication 260-67, 1979 

6. Cohen A., Hertz HS, Mandel J, Paule RC, Schaffer R, Sniegoski LT, et al. Total serum cholesterol by isotope dilution/mass spectrometry: a candidate definitive method. 

Clin. Chem. 1980; 26: 854-860. 

7. Welch MJ, Cohen A, Hertz HS, Ng KJ, Schaffer R, Van der Lijn P, et al. Determination of serum creatinine by isotope dilution mass spectrometry as candidate definitive 

method. Anala. Chem. 1986; 58: 1681-1685. 

8. Schaffer R, Mandel J, Sun J, Cohen A, Hertz HS, Neese JW. Standard reference materials: evaluation by an ID/MS method of the AACC reference method for serum glucose. National Bureau of Standards Special Publication 260-80, 1982. 

9. White V E, Welch J, Sun T, Sniegosky LT, Schaffer R, Hertz HS, et al. The accurate 

determination of serum glucose by isotope dilution mass spectrometry – two methods. 

Biomed. Mass Spectrom. 1982; 9:395-405. 

10. Ellerbe P, Sniegosky LT, Welch MJ. Isotope dilution mass spectrometry as a candidate definitive method for determining total glycerides and triglycerides in serum. Clin. 

Chem. 1995; 41:397-404. 
99

11. Ellerbe P, Cohen A, Welch M, White V E. Determination of serum uric acid by isotope dilution mass spectrometry as a new candidate definitive method. Anal. Chem. 

1990; 62:2173-2177. 

12. Welch MJ, Cohen A, Hertz HS, Ruegg FC, Schaffer R, Sniegisky LT, et al. Determination of serum urea by isotope dilution mass spectrometry as a candidate definitive 

method. Anal. Chem. 1984; 56:713-719. 

13. Cali JP, Rationale for reference methods in clinical chemistry. Pure Appl. Chem. 

1976; 45:63-68. 

14. Tietz NW. A model for a comprehensive measurement system in clinical chemistry. 

Clin. Chem. 1979; 25:833-839. 

15. Tietz NW, Rodgerson DO, Laessig RH. Are clinical laboratory proficiency tests as 

good as they can be? Clin. Chem. 1992; 38: 473-475. 

16. Tietz NW, Accuracy in clinical chemistry – does anybody care? Clin. Chem. 1994; 

40: 859-861. 

17. Eckfeldt JH, Lewis LA, Belcher JD, Singh J, Frantz Jr ID. Determination of serum 

cholesterol by isotope dilution mass spectrometry with a bench top capillary gas chromatograph/mass spectrometer: comparison with the national reference system’s definitive and reference methods. Clin. Chem. 1991; 37: 1161-1165. 

18. Doumas BT, Kwok-Cheung PP, Perry BW, Jendrzejczak B, McComb RB, Schaffer 

R, et al. Candidate reference method for determination of total bilirubin in serum: development and validation. Clin. Chem. 1985; 31: 1779-1789. 

19. Doumas BT, Bayse DD, Carter RJ, Peters jr. T, Schaffer R. A candidate reference 

method for the determination of total protein in serum. I Development and validation. 

Clin Chem. 1981; 27: 1651-1654. 

20. National Committee for Clinical Laboratory Standards. Development of definitive 

methods for the national reference system for the clinical laboratory; approved guideline. Villanova (PA): NCCLS; 1991 NCCLS Publication NRSCL1-A. 

21. National Committee for Clinical Laboratory Standards. Development of reference 

methods for the national reference system for the clinical laboratory; approved guideline. Villanova (PA): NCCLS; 1991 NCCLS Publication NRSCL2-A. 

22. National Committee for Clinical Laboratory Standards. Development of certified 

reference materials for the national reference system for the clinical laboratory; approved guideline. Villanova (PA): NCCLS; 1991 NCCLS Publication NRSCL3-A. 

23. National Committee for Clinical Laboratory Standards. Source book of reference 

methods, reference materials, and related information for the clinical laboratory; proposed guideline. Villanova (PA): NCCLS; 1994 NCCLS Publication NRSCL 12-P. 

24. Björkhem I, Blomstrand R, Lantto O, Svensson L, Öhman G. Towards absolute 

methods in clinical chemistry: application of mass fragmentography to high-accuracy 

analyses. Clin. Chem. 1976; 22: 1789-1801. 

25. Björkhem I, Blomstrand R, Eriksson S. Use of isotope dilution-mass spectrometry 

for accuracy control of different routine methods used in clinical chemistry. Scand. J. 

Clin. Lab. Invest. 1980; 40: 529-534. 
100

26. Björkhem I, Bergman A, Falk O, Kallner A, Lantto O, Svensson L, et al. Accuracy 

of some routine methods used in clinical chemistry as judged by isotope dilution-mass 

spectrometry. Clin. Chem. 1981; 27: 733-735. 

27. Büttner J. Reference methods as a basis for accurate measuring systems. Eur. J. Clin. 

Chem. Clin. Biochem 1991; 29: 223-235. 

28. Büttner J. Reference materials and reference methods in laboratory medicine: a challenge to international cooperation. Eur. J. Clin. Chem. Clin. Biochem. 1994 38: 1401. 

29. Büttner J. The need for accuracy in laboratory medicine. Eur. J. Clin. Chem. Clin. 

Biochem 1995; 33: 981-988. 

30. Külpmann WR. Grundprinzioien von Referenzmethoden am Beispiel der 

Elektrolytbestimmungen. Fresenius Z. Anal. Chem. 1986; 324: 205-207. 

31. Külpmann WR, Lagemann J, Sander R, Maibaum P. A comparison of reference 

method values for sodium, potassium and chloride with method dependent assigned 

values. J. Clin. Chem. Clin. Biochem. 1985; 23: 865-874. 

32. Schumann G, Büttner J. A candidate reference method for creatinine with HPLC. 

Fresenius Z. Anal. Chem. 1986; 321: 279-280. 

33. Schumann G, Büttner J. Determination of glucose in reference materials by isotope 

dilution – mass spectrometry. Fresenius Z. Anal. Chem. 1988; 330: 424-425. 

34. Siekmann L. Reference methods. IN: Lawson AM, editor. Mass Spectrometry. Berlin, New York: Walter de Gruyter, 1989: 647-681. 

35. Siekmann L, Hoppen HO, Breuer H. Zur gas-chromatographischmassenspektrometrischen Bestimmung von Steroidhormonen in Körperflüssigkeiten 

unter Verwendung eines Multiple Ion Detectors (Fragmentographie). Fresenius Z. 

Anal. Chem. 1970; 252: 294-298. 

36. Siekmann L, Spiegelhalder B, Breuer H. Bestimmung von Aldosteron im Plasma 

des Menschen unter Verwendung der Kombination Gaschromatographie 

Massenspektrometrie. Fresenius Z. Anal. Chem. 1972; 261: 377. 

37. Siekmann L. Determination of steroid hormones by the use of isotope-dilution mass 

spectrometry: a definitive method in clinical chemistry. J. Steroid. Biochem. 1979; 11: 

117-123. 

38. Siekmann L, Breuer H. Determination of cortisol in human plasma by isotope dilution - mass spectrometry. J. Clin. Chem. Clin. Biochem. 1982; 35: 532-536. 

39. Siekmann L. Determination of estradiol-17ß in human plasma by isotope dilutionmass spectrometry. J. Clin. Chem. Clin. Biochem. 1984; 22: 551-557. 

40. Siekmann L. Determination of uric acid in serum by isotope dilution-mass spectrometry. J. Clin. Chem. Clin. Biochem. 1985; 23: 129-135. 

41. Siekmann L. Determination of creatinine in human serum by isotope dilution-mass 

spectrometry. J. Clin. Chem. Clin. Biochem. 1985; 23: 137-144. 

42. Siekmann L. Reference methods for total cholesterol and total glycerol. Eur. J. Clin. 

Chem. Clin. Biochem. 1991; 29: 277-279. 

43. Siekmann L. Measurement of thyroxine in human serum by isotope dilution-mass 

spectrometry. Biomed. Mass Spectrom. 1987; 14: 683-688. 
101

44. Stöckl D, Reinauer H. Development of criteria for the evaluation of reference 

method values. Scand. J. Lab. Invest. 1993; 53 Suppl. 212: 16-18. 

45. Stöckl D, Reinauer H. Candidate reference methods for the determination of target 

values for cholesterol, creatinine, uric aid, and glucose in external quality assessment 

and internal accuracy control. I. Method setup. Clin. Chem. 1993; 39: 993-1000. 

46. Stöckl D, Franzini C, Kratochvila J, Middle J, Ricos C, Siekmann L, Thienpont LM. 

Analytical specifications of reference methods; compilation and critical discussion. 

Eur. J. Clin. Chem. Clin. Biochem. 1996; 34: 319-337. 

47. Thienpont LM, Stöckl D. Reference methods and reference method values. Klin. 

Biochem. Metab. 1994; 2: 4-9. 

48. Thienpont LM, Franzini C, Kratochvila J. Middle J, Ricos C, Siekmann L, Stöckl D. 

Analytical quality speciications for reference methods and operating specifications for 

networks of reference laboratories. Recommendations of the European EQAOrganizers Working Group B. Eur. J. Clin. Chem. Clin. Biochem. 1995; 33: 949-957. 

49. Thienpont LM, Van Nuwenborg JE, Stöckl D. Ion chromatography as a potential 

reference methodology for the determination of total calcium and magnesium in human 

serum. Anal. Chem. 1994; 66: 2404-2408. 

50. Thienpont LM, De Leenheer AP, Stöckl D. Reinauer H. Candidate reference methods for the determination of target values for cholesterol, creatinine, uric acid and glucose in external quality assessment and internal accuracy control. II Method transfer. 

Clin. Chem. 1993; 39: 1001-1006. 

51. Thienpont LM, Van Landuyt K, Stöckl D, De Leenheer AP. Candidate reference 

method for determining serum creatinine based on isocratic high performance liquid 

chromatography: validation with isotope dilution – gas chromatography/mass spectrometry and application for accuracy assessment of routine test-kits. Clin. Chem. 1995; 

41: 995-1003. 

52. Thienpont LM, Verhaeghe PG, Van Brussel KA, De Leenheer AP. Estradiol-17ß

quantified in serum by isotope dilution-gas chromatography-mass spectrometry: reversed phase C18 high performance liquid chromatography compared with immunoaffinity chromatography for sample pretreatment. Clin. Chem. 1988; 34: 2066-2069. 

53. Thienpont LM, De Brabandere VI, Stöckl D, De Leenheer AP. Development of a 

new method for the determination of thyroxine in serum based on isotope dilution-gas 

chromatography mass spectrometry. Biol. Mass Spectrom. 1994; 23: 475-482 
102

LOTHAR SIEKMANN 

University of Bonn • Institut für Klinische Biochemie 

Lothar Siekmann, Ph.D., is a Professor of Clinical Chemistry at the University of 

Bonn, Germany. He is the Director of the Department for Clinical Chemical Analysis 

and Technology as well as the Director of the Reference Institute of Bioanalysis of the 

German Society of Clinical Chemistry. His current research includes development of 

isotope dilution mass spectrometry and the development of reference measurement 

systems in internal and external quality assessment. Dr. Siekmann has received the 

Gabor Szasz Prize of the German Society for Clinical Chemistry for “Development 

and Application of Reference Methods in Clinical Chemistry”. He is a member 

numerous of standards organizations and committees, including being the Chairman 

of the Commission of Standardization of the German Society of Clinical Chemistry, 

Convener of an S, M&T Project (BCR) of the European Union for the Certification of 

Reference Materials for Creatinine in Serum, Coordinator of an IFCC/IRMM Project 

for Establishing Reference Systems for Measurement of Catalytic Concentrations of 

Enzyme, an Expert in Working Groups on Steroid Hormones of the European Union, 

and a member of working groups of CEN TC 140 and ISO TC 212. 
103

Traceability of Calibration for IVD Industry: Process for 

Credentialing Reference Materials, Reference Methods and 

Reference Laboratories in Europe 

Visuals Presented By 

Lothar Siekmann 

University of Bonn • Institut fur Klinische Biochemie
104

Traceability in Laboratory Medicine: 

Process for Credentialing Reference Materials, 

Reference Methods and Reference Laboratories 

_____________________________________________

Lothar Siekmann Lothar Siekmann

Institut f Institut für Klinische Biochemie, Universit r Klinische Biochemie, Universität Bonn

Reference Institute of Bioanalysis, DGKC Reference Institute of Bioanalysis, DGKC

 

Measurement Traceability 

_______________________________________

property of the result of a measurement or the 

value of a standard whereby it can be related to 

stated references, usually national or international 

standards, through an unbroken chain of 

comparisons all having stated uncertainties
105

SI-Unit 

(definition)

BIPM

Primary calibrator

Secondary calibrator

Mf. ́s working 

(master) calibrator

Mf. ́s product 

calibrator

Secondary reference

measurement procedure

NMI, ACL

NMI, ACL

Mf. ́s reference

measurement procedure

Mf. ́s standing

measurement procedure

End-user ́s routine

measurement procedure

Routine sample

RESULT

NMI, ACL

ACL, MCL

ML

Mf.-> End-user

Mf. and/or End-user

MCL

Traceability

BIPM: 

International Institute 

of Weights and 

Measures

NMI: 

National Metrological 

Institute

ACL: 

Accredited Calibration 

Laboratory

MCL: 

Manufacturer ́s 

Calibration Laboratory

ML: 

Manufacturer ́s 

Laboratory 

Mf: 

Manufacturer

End-user

End-user

Primary reference measurement procedure

ML

μc(y)
106

Organisations and Individuals involved in the Implementation of 

the Concept of Traceability

• Legislative Bodies

• National Metrology Institutes 

• International Institutions issuing Reference Materials (WHO, IRMM, NIST)

• International and National Standardisation Organisations (ISO, CEN, NCCLS, 

DIN)

• International and National Scientific Societies (IFCC)

• External Quality Assessment Organisations

• Reference Laboratories

• Diagnostic Kit Manufacturers

• Clinical Chemical Laboratories

• Physicians

Reference Systems 

in Clinical Chemistry ____________________________________

• Reference Materials

• Reference Measurement Procedures

• Reference Laboratories
107

Suppliers of Reference Materials

__________________________________________

• National Institute of Standards and Technology (NIST), USA 

• Institute for Reference Methods and Materials (IRMM), EU

• World Health Organisation (WHO), Geneva

• National Institute of Biological Standards (NIBSC), U.K.

• National Metrology Institutes

Hierarchy of Laboratories

National Metrology Institutes

Accredited Reference Laboratories

(universities, hospitals, manufacturers)

Manufacturer‘s Standardisation

Laboratories

Routine Laboratories
108

Approval of 

Reference Laboratories and Methods

__________________________________________

• establishing and monitoring of a quality management 

system,

• regular inspections of the reference laboratory,

• regular comparisons using split sample measurements at 

the reference laboratory and the national metrology 

institute.

by:

DGKC Reference Procedures

• Electrolytes:

Calcium

Chloride 

Lithium

Magnesium

Potassium

Sodium

• Metabolites and Substrates:

Cholesterol

Creatinine

Glucose

Total Glycerol

Uric Acid 

Urea

Bilirubine

Lactate

• Enzmes;

AST GGT

ALT AMYLASE

CK

• Hormones:

Aldosterone

Cortisol

Estradiol-17ß

Estriol

Progesterone

17-Hydroxy-progesterone 

Testosterone

Thyroxine

• Drugs:

Theophylline

Digoxin 

Digitoxin

• Total Protein
109

TARGET CONCENTRATIONS IN A CONTROL SERUM

CREATININE μmol/l CHOLESTEROL mmol/l 

------------------------------------------------------------- -----------------------------------------------

Enzymatic/ PAP 151 CHOD - Iodide 4.02

Enzymatic UV system 161 CHOD - PAP 4.30

Jaffe without deproteinisation (Merck) 168 CHOD - Katalase 4.61

Jaffe after deproteinisation (Boehr.) 177 Peridochrom 4.69

Jaffe without deproteinisation (Boehr.) 189 Liebermann-Burchard 5.49

URIC ACID μmol/l TRIGLYCERIDES mmol/l

------------------------------------------------------------- -----------------------------------------------

Fully enzymatic (Boehr./Merck) 457 Fully enzymatic (Behr.) 1.15

UV - system (Boehr.) 476 Fully enzymatic (Merck) 1.34

UV - system (Merck) 539 Fully enzymatic (Roche) 1.30

Phosphotungstic acid (Goed.) 583 Enzymatic (Boehr.) 1.36
110

Target Concentrations for Steroid Hormones in a Commercial Control Serum

__________________________________________________________________

Aldosterone Cortisol Progesterone Estradiol

pmol/l nmo/l nmo/l pmol/l _________________________________________________________________________________________________________

ABBOTT 121.9

AMERSHAM 113.1

BAXTER DADE DIR 104.8 2.16 396.4

BAXTER DADE AG ER 244.1

BAXTER DADE AD EXT 196.0

BIOMERIEUX 2.54 539.6

BIOTEX PREMIX 99.4 70.6 3.72 759.9

CAMBRID ́GE MEDICAL 120.8 0.86

CIBA CORNING 110.3

CLINICAL ASSAYS 99.3

CYBERFLUOR FIAGEN 88.2

DIAGNOSTIC PRODUCTS 113.1 3.12 119.3

EURODIAGNOSTICS 115.8

FARMOS DIAGN. 99.3 4.67 394.9

IMMUNCHEM COV. COAT 110.3 5.41 348.7

LEECO 113.1 2.99 144.2

MALLINCKRODT 96.6

NMS PHARMACEUTICALS 3.18 205.5

PANTEX IMMUNO 118.6 4.13 190.8

PANTEX IMMUNOCOAT 132.4 7.00 154.9

PHARMACIA DELFIA 99.9 790.0

RSL 169.2 4.77 117.4

SCLAVO LISO PHASE 277.4 3.82

SERONO 112.0

SIBAR ELISA 121.3 1.27

SORIN 165.9 68.9 2.86 139.5

SYVA EMIT 137.9

TECHLAND RIA 4.77

VITEK SYSTEMS 110.0
111

PROGESTERONE

MEDIANS IN RING TRIALS

y = 2,98 + 0,87 x

n = 40, r = 0,681

Median (x) = 14,2

Median (y) = 15,75

Median (y-x)% = 10,3

PROGESTERONE

PATIENT SAMPLES

y = 1,84 + 0,96 x

n = 42, r = 0.99

Median (x) = 19,2

Median (y) = 17,9

Median (y-x)% = 17,4

0

10

20

30

40

50

60

70

0 10 20 30 40 50 60 70 IDMS / Progesteron [nmol/l] 

DPC-RIA / Progesteron [nmol/l] 

0

10

20

30

40

50

60

70

0 10 20 30 40 50 60 70

IDMS / Progesteron [nmol/l] 

DPC-RIA / Progesteron [nmol/l] 

PROGESTERONE

MEDIANS IN RING TRIALS

PROGESTERONE

PATIENT SAMPLES

-100

0

100

200

300

400

500

600

700

800

900

1000

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30

Progesteron [nmol/l]

Deviation (%)

-100

0

100

200

300

400

500

600

700

800

900

1000

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30

Progesteron [nmol/l]

Deviation (%)
112

Strategy to establish Reference Systems

for non-SI Traceable Quantities

__________________________________________

• Definition of the analytes

• Development of a Reference Procedure

• Development of a Reference Material

Establishing a Reference System for the

Measurement of the 

Catalytic Concentration of Enzymes

• Decision on a primary reference measurement procedure 

(IFCC working group)

• Establishing of the reference procedure within a network 

of reference laboratories according to stringent 

metrological principles

• Selection of commutable reference material and 

certification by network laboratories
113

94 95 96 97 98 99 100 101 102 103 104 105 106

CFAS / 37°C 

1998, n=12

ASAHI / 37°C 

1998, n=12

Beckman 1 / 37°C 

1998, n=10

Beckman 2 / 37°C 

1998, n=10

CRM 404 / 30°C 

1994, n=8

CRM 404 / 37°C 

1994, n=8

CRM 404 / 37°C 

1999, n=12

 Relative Recovery and 95%-Confidence Interval (mean=100%) [%]

 LD

94 95 96 97 98 99 100 101 102 103 104 105 106

CFAS / 37°C 

1998, n=12

ASAHI / 37°C 

1998, n=12

Beckman 1 / 37°C 

1998, n=10

Beckman 2 / 37°C 

1998, n=10

CRM 299 / 30°C, 

n=9

CRM 608 / 30°C 

1995, n=5

CRM 608 / 37°C 

1999, n=12

 Relative Recovery and 95%-Confidence Interval (mean=100%) [%]

 CK

Estriol

-50

-30

-10

10

30

50

70

Year

Deviation [%]

1980 1981 1982 1983 1984 1985 1986 1987
114

CHOLESTEROL

Method Comparisonin a Set of 

Control Materials

y = -0,19 + 0,82 x

Median of Differences

(y-x)% = -22.8%

CHOLESTEROL

Method Comparison in a Set of

Patient Samples

y = 0.01 + 1.045 x

Median of Differences

(y-x)% = +4.8%

0

2

4

6

8

10

0 2 4 6 8 10

IDMS [mmol/l] 

Routine Test [m m ol/l] 

0

2

4

6

8

10

0 2 4 6 8 10

IDMS [m mol/l] 

Routine Tes t [m m ol/l] 
115

European Reference Materials Activities in the Clinical Field 

H. Schimmel, M. Baptist, N. Kristiansen, T. Linsinger, P. Taylor, J. Pauwels 

European Commission, Joint Research Centre 

Institute for Reference Materials and Measurements, 

Belgium 

Activities on the production of certified reference materials (CRMs) have been 

ongoing for more than 20 years, starting in the BCR programme, which evolved into the 

current M&T programme, of DG Research. Several projects focusing on the production 

of clinical CRMs are continuing. Presently 34 BCR - CRMs related to clinical chemistry, 

or more generally to medical applications, are available (see Table 1). Since 1995, the 

Institute for Reference Materials and Measurements (IRMM) of the Joint Research Centre of the European Commission has taken over the responsibility for the distribution, 

monitoring and replacement of exhausted batches of all BCR CRMs. 

Table 1 : List of BCR Certified Clinical Reference Materials 

PROTEINS 

BCR-148 : lyophilised thromboplastin Bovine 

BCR-149S : lyophilised rabbit thromboplastin 

BCR-393 : lyophilised Apo A I from human serum 

BCR-394 : lyophilised Apo A II from human serum 

BCR-486 : purified alphafoetoprotein 

BCR-457 : human Thyroglobulin 

BCR-470 : 15 serum proteins 

BCR-405 (RM) : glycated haemoglobin in human haemolysate 

BCR-522 : haemiglobincyanide in bovine blood lysate 

BCR-613 : prostate specific antigen in reconstituted material 

ENZYMES 

BCR-299 : creatine kinase BB partially purified, from human placenta 

BCR-319 : γ-glutamyltransferase partially purified, from pig kidney 

BCR-371 : alkaline phosphatase partially purified, from pig kidney 

BCR-404 : human lactate dehydrogenase isoenzyme 

BCR-410 : prostatic acid phosphatase highly purified, from human prostate 

BCR-426 : alanine aminotransferase partially purified, from pig heart 

BCR-476 : pancreatic α-amylase, from human pancreas 

BCR-608 : creatine kinase CK-MB from human heart 
116

HORMONES 

BCR-192 and BCR-193 : cortisol in human serum (2 levels) 

BCR-347 : progesterone in human serum 

BCR-576, BCR-577, BCR-578 : 17β-estradiol in human serum (3 levels) 

ELECTROLYTES and METABOLYTES 

BCR-194, BCR-195, BCR-196 : lead and cadmium in lyophilised bovine blood 

BCR-304 : calcium, magnesium and lithium in lyophilised human serum 

BCR-573, BCR-574, BCR-575 : creatinine in human serum 

BLOOD CELL SIZE REFERENCE MATERIAL 

BCR-165, BCR-166, BCR-167 : latex spheres (2, 4.8 and 9.6 of certified size μm) 

The European "in vitro diagnostic and medical device Directive" (98/79/EC), that 

requires manufacturers of in-vitro diagnostics (IVD) who wish to sell their products in 

the EU demonstrate that they are “traceable to reference systems (i.e. reference materials 

and/or reference methods) of a higher order", will generate an enormous need of internationally recognized certified reference materials (CRMs). On October 30th, 1996, anticipating this need, the IRMM entered into a Co-operation Agreement with the International 

Federation of Clinical Chemistry (IFCC) with the aim of producing joint IRMM/IFCC 

certified reference materials. These reference materials are to be produced in close collaboration with IFCC working groups and committees involving experts in clinical chemistry, reference laboratories and IVD industry. 

In the past, different approaches have been followed for the certification of reference materials. The two extreme approaches have been applied: one focused mainly on 

measurands where SI traceability could be achieved, and the other, based on consensus, 

mainly involved complex measurands where the biological activity was important and SI 

traceablity could not be achieved. Both approaches suffer from drawbacks, if all aspects 

relevant in clinical chemistry, especially for complex analytes, are not taken into 

consideration. By following purely the metrological approach for many clinical analytes, 

no CRMs were produced, because SI traceability could not be achieved or was too costly. 

Consequently, for most of the clinically relevant parameters there are no reference materials available. In addition, the certified quantity may not or not fully have represented the 

clinically relevant form, and the materials may not have been appropriate for direct use 

with IVDs because of the problems with commutability. Commutability is defined as the 

ability of a material to show interassay properties comparable to those of human sera; i.e., 

fresh patient samples and a commutable reference material show the same behaviour 

when submitted to an IVD routine test system. Commutability problems are very common for complex proteins where the matrix and the sample treatment influences the behaviour of the measurand and antibodies usually used in IVD tests. It is evident that 

commutability should be investigated and interpreted with care in order to ensure a 

maximum standardisation of these effects. The scheme for the assessment of commutability should allow a clear separation of true matrix and sample treatment effects due to 

lack of specificity. In case of lack of specificity, the potential standardization effect will 

be reduced due to the varying composition of patient samples. However, the acceptance 

and use of reference materials depends on applicability and commutability. Only com-
117

mutable materials can be used to directly control the various steps in the production of 

IVDs, including the direct calibration of calibrators and performance checks of IVD’s. 

Therefore, commutable CRMs are critical for assuring an unbroken traceability chain. 

The second approach, based on consensus only, was predominantly aiming at 

normalization of various methods to a common reference material. The disadvantage of 

such systems is that they are arbitrary and difficult to reproduce. Often the characterization was done without having a clear concept, using poorly understood methods and 

without assessing the calibration of the methods used for the characterization. If several 

of such CRMs exist for the same analyte, discrepancies between methods can arise if 

normalized to different CRMs. Therefore, it is felt that a more structured and coordinated 

approach is required on an international level involving all relevant parties. This networking approach will also help to better identify the needs and specification requirements for 

new reference materials. 

Currently, joint IRMM - IFCC certification projects attempt to implement a 

widely recognized reference system including methods and CRMs that are based on metrological principles. With reasonable efforts, and current state of the art technical possibilities, the clinically relevant form of an analyte should be identified as enabling traceability to the SI system. The properties, quality and foreseen use of CRMs are defined at 

an early stage, and the production of commutable reference materials is envisaged. International cooperation will help to evaluate all relevant aspects of CRM production and 

will certainly increase the acceptance of the CRMs. IRMM is also supporting international networks on traceability according to IVD directive involving all relevant parties 

such as expert or reference laboratories, IFCC, IVD and CRM producers, accreditation 

bodies and metrology institutes. 

The definition of priorities for projects aiming at the production of reference materials is based on a several criteria. There should be a balance between production efforts, clinical relevance and level of uncertainty to be achieved. The clinical relevance 

and level of uncertainty to be achieved is determined by the significance of the parameter 

in the diagnosis of diseases. The reference ranges of healthy patients, the daily variation, 

and the expected improvement of measurement quality, need to be taken into consideration. The anticipated need for a reference material and the number of analyses needed, as 

well as the impact on individuals and public health care; (e.g. disease prevention) and the 

assessment of cost / benefit ratio, are also important criteria. 

So far, 4 IRMM/IFCC CRMs for enzyme activity and an IRMM/IFCC reference 

panel for cortisol have been certified (see Table 2). 

Table 2 : List of IRMM/IFCC Certified Clinical Reference Materials 

IRMM/IFCC-451 : cortisol reference panel of fresh frozen human sera 

IRMM/IFCC-452 : gamma-glutamyltransferase partially purified, from pig kidney 

IRMM/IFCC-453 : human lactate dehydrogenase isoenzyme 1 

IRMM/IFCC-454 : alanine aminotransferase partially purified, from pig heart 

IRMM/IFCC-455 : creatine kinase CK-MB from human heart 
118

The enzyme CRMs IRMM/IFCC-452, 453, 454 and 455 have been certified for 

catalytic activity in cooperation with the IFCC committee on enzyme activity according 

to newly developed SOPs at 37 oC. These SOPs have been proposed to become IFCC reference methods. Compared to the former IFCC reference methods, crucial parameters in 

the procedure have been optimized for the new temperature, or have been clearly defined 

(examples see Table 3). The reproducibility of results has been improved using the new 

SOPs that could be demonstrated by an approximately 50 % reduction of the 95% confidence interval calculated from the means of means of the data sets from the laboratories 

having carried out the characterization measurements. There was also a move to get 

conditions that are closer to the routine procedures as applied in clinical laboratories. The 

uncertainty of the certified value was calculated according to the GUM and includes possible heterogeneity and instability contributions. The collection of data by the IFCC 

committee for definition of preliminary reference ranges is ongoing and publication of 

the new IFCC reference method is envisaged. Certification has been carried out in collaboration with various Universities, hospitals and IVD producers, including partners 

from the USA and Japan. 

Table 3: Examples for Changes in SOPs for 

Measurement of Catalytic Activity: 

GGT 37°C 37 °C (old) 

pH 7.7 7.9 

Incubation time 180s not specified 

Delay time 60s not specified 

Measurement interval : 180 s 300s 

LD 37°C 30 °C (old) 

pH 9.4 7.2 

Incubation time 180 s 60-300s 

Delay time : 90 s 90s 

Measurement interval : 180 s 60 s 

The serum panel for cortisol has been certified in cooperation with the IFCC 

working group on steroid hormones. A panel consisting 34 individual sera (80 - 770 

nmol/l) has been characterized using a primary method (GC-IDMS) in two independent 

reference laboratories. Uncertainties were calculated according to GUM. The panel is intended to be used by IVD producers directly to calibrate their test kits. The approach using a panel with samples having individual characteristics and concentrations will enable 

the IVD producers to properly assess a calibration curve and the specificity of the test 

system they are producing. The final goal is to establish more reliable reference interval 

ranges, and to improve the inter-assay comparability. Further certification projects are 

being conducted. 
119

Three other CRMs for enzyme catalytic activity will be certified. For two of them 

(aspartate aminotransferase (ASAT) and α-Amylase) the SOPs have been developed for 

certification at 37 oC. The characterization measurements for α-Amylase have been completed and a certification report and certificate are currently being drafted. For ASAT and 

alkaline phosphatase (ALP) new batches of materials need to be produced, and a commutability study for candidate ASAT reference materials is ongoing. 

Work is also underway, in cooperation with the IFCC working group, on cardiac 

markers to produce a myoglobin CRM. Five candidate CRMs have been collected and the 

commutability is currently being assessed. A rigorous protocol has been applied to determine linearity and parallelism of various test systems when using the candidate CRMs. 

In parallel to the commutability study, we intend to develop a reference method for quantification. The project is being carried out in cooperation with IVD industry from US, 

France, Germany, UK and Finland. 

Other ongoing CRM projects are for the replacement of a high level progestorone 

CRM in human serum (freeze dried), and the certification of total proteins in human serum. An important area that the IRMM has entered recently is the production of CRMs 

for genetic markers. 

Further, a variety of projects are currently being discussed that include pure calibrants and a panel for glycated haemoglobin (Hb1Ac), purified human chorion gonadotropin (hCG) and its isoforms, osteocalcin (osteoporosis marker), steroid hormone (progestorone, testosterone, 17β-estradiol) serum panels similar to the cortisol serum panel 

mentioned above and pure progesterone, testosterone, 17β-estradiol and aldosterone as 

primary calibrants in cooperation with the German metrology institute PTB, 

Brauschweig. 

In addition to providing CRMs, IRMM is also providing other means for improving the quality of measurements in the clinical field by organizing IMEP exercises (proficiency testing rounds that are based on traceable reference values). IMEP-17 focuses on 

trace and minor constituents in human serum. Candidate measurands are Na, Cl, Ca, Cu, 

Mg, Li, K, Se, Zn, Fe, glucose, cholesterol, urea, total thyroxine (T4), uric acid and 

creatinine. The test material will be unmodified/modified human serum. The traceable 

reference values will be obtained preferably by primary methods of measurements by experienced reference measurement institutes. The exercise will be executed in 2001-2002. 
120

HEINZ SCHIMMEL 

European Commission, Joint Research Centre 

Institute for Reference Materials and Measurements 

Belgium 

Heinz G. Schimmel, Ph.D., is a Scientific Officer at IRMM. He received his Ph.D. at the 

University of Ulm in 1992 for his work on the analysis and quantitation of polychlorinated dibenzodioxins and dibenzofurans. He then worked at the Community Bureau of 

Reference (BCR) of the Commission of the European Communities, providing technical 

assistance in certification projects on trace organics in environmental matrices. In 1993, 

Dr. Schimmel took over the management of organic trace analysis projects at the University of Ulm. He assumed his current position at IRMM in 1995 and is in charge of the 

management of recertification projects, certification projects, and validation studies in 

the field of environmental and food chemistry, clinical chemistry microbiology and biotechnology. He is head of production control for the analysis of organic and metalorganic compounds. He is also responsible for setting up a new sterile production facility, and a molecular biological and biochemical production control laboratory. Since 

1996, he has been involved in the development of clinical reference materials in cooperation with IFCC. 
121

European Reference Materials Activities in the Clinical Field 

Visuals Presented By 

Heinz Schimmel 

Institute for Reference Materials and Measurements 

Belgium 
122

European Reference Materials 

Activities in the Clinical Field

H. Schimmel

EC - JRC - IRMM, Geel (EU)

Reference Materials Unit

EUROPEAN COMMISSION

JOINT RESEARCH CENTRE

Institute for Reference Materials and Measurements

Geel, Belgium

• Since >20 years production of BCR-CRMs

including clinical CRMs

• Several CRMs currently in the pipeline

• Presently 34 BCR-CRMs available related to 

clinical chemistry or medicine

• IRMM responsible for distribution, monitoring 

and replacement

BCR Programme
123

• PROTEINS

(e.g. thromboplastin, Apo A I, Apo A II, serum 

proteins, PSA)

• ENZYME ACTIVITY

(e.g. CK-BB, CK-MB, ALP, LD, ALAT, α-amylase)

• HORMONES

(cortisol, progesterone, 17β-estradiol)

• ELECTROLYTES and METABOLYTES

(Pb, Cd, Ca, Mg, Li, creatinine)

• BLOOD CELL SIZE

(latex spheres)

Existing BCR CRMs

• Metrological approach (SI tracability)

– For many clinical analytes no CRMs were 

produced because SI traceability could not be 

achieved or was too costly

– The certified quantity may not or not fully 

represent the clinically relevant form

– The materials may not be appropriate for direct 

use with IVDs (problems with commutability)

Approaches for CRM production
124

• Ability of a material to show interassay properties 

comparable to those of human sera

• Not to mixed up with lack of specificity

• Acceptance and use of reference materials depends 

on applicability and commutability

• Only commutable materials can be used for direct 

calibration of calibrators and performance check of 

IVD’s and therefore have large importance to ensure 

unbroken traceability chain 

Commutability

• Consensus approach

– Predominantly aiming at consensus between 

various methods and commutability of materials

– Arbitrary systems and difficult to reproduce

– Often characterisation with not well understood 

methods and without a clear concept

– If several CRMs exist for the same analyte 

discrepancies between methods can arise if 

normalised to different CRMs

Approaches for CRM production
125

• More structured and co-ordinated approach is required 

involving experts in clinical chemistry, IVD and CRM 

producers

• Support to networks of relevant institutions on an 

international level

• Since 1996 direct co-operation with IFCC (International 

Federation of Clinical Chemistry, >70 clinical chemical 

societies and >40 industrial members) 

IVD-harmonisation

• IRMM / IFCC policy: 

– implementation of a widely recognised reference 

system including methods and CRMs

– based on metrological principles

– identification of clinically relevant form of analyte

– definition of properties, quality and foreseen use of 

CRM’s

– production of commutable reference materials 

– acceptance through international co-operation 

IVD harmonisation
126

• Certification of enzyme CRM’s at 37 oC

– Development of optimised SOP’s at 37 oC based 

on official IFCC methods at 30 oC

– Certification of 4 enzymes completed and certified 

according to GUM

• GGT (γ-glutamyltransferase) 

• ALAT (alanine aminotransferase) 

• CK-MB (creatine kinase from human heart) 

• LD (lactate hydrogenase)

Certification of enzyme CRM’s

in co-operation with the IFCC committee on enzymes 

Changes in SOPs:

GGT 37°C 37 °C (old) 

pH 7.7 7.9

Incubation time 180s not specified

Delay time 60s not specified

Measurement interval : 180 s 300s

LD 37°C 30 °C (old)

pH 9.4 7.2

Incubation time 180 s 60-300s

Delay time : 90 s 90s

Measurement interval : 180 s 60 s

Certification of enzyme CRM’s

in co-operation with the IFCC committee on enzymes 
127

• CRM’s available for sale

• Collection of data for re-definition of reference ranges 

by IFCC ongoing

• Publication as official IFCC reference method 

envisaged

• Collaboration of IFCC / IRMM with 

– various Universities and hospitals (including Japan)

– IVD producers (including USA and Japan)

Certification of enzyme CRM’s

in co-operation with the IFCC committee on enzymes 

Certification of enzyme CRM’s

012345

CRM608 CK-MB, 1999

CRM608 CK-MB, 1995

CRM404 LD, 1999

CRM404 LD, 1994

CRM426 ALAT, 1999

CRM426 ALAT, 1992

CRM319 GGT, 1999

CRM319 GGT, 1986

C onfidenc e interval (% )
128

• Preparation and/or certification of 3 enzymes 

at 37 oC ongoing

- ASAT: SOP developed, candidate CRM from 

ASAHI (fresh frozen recombinant material) shows 

best commutability

- ALP: some conflict between most suitable SOP 

and routine procedures - to be solved

- α-Amylase: certification started

Certification of enzyme CRM’s

in co-operation with the IFCC committee on enzymes 

• 34 individual sera (80 - 770 nmol/l) and certified with 

GC-ID-MS measurements according to GUM

• Intended to be used by IVD producers to directly 

calibrate their test kits

• Goal: 

– establishing reference interval ranges

– improve inter-assay comparability

Serum panel for cortisol

in co-operation with the IFCC working group on steroid 

hormones
129

• 5 candidate CRM’s at IRMM

• Currently collection of appropriate serum pools from 

heart infarction patients, later on to be used for 

commutability study

• In parallel development of a reference method for 

quantification

• Cooperation with IVD industry from US, France, 

Germany, UK and Finland 

Myoglobin reference material

in co-operation with the IFCC working group on cardiac 

markers

– Progestorone in human serum (freeze 

dried)

– Total proteins in human serum

CRM projects launched 
130

– Panel for glycated haemoglobin (Hb1Ac)

– Purified hCG and its isoforms 

– Certification of pure progesterone, testosterone, 

17β-estradiol and aldosterone as primary

calibrants in co-operation with PTB

– Progestorone, testosterone and/or 17β-estradiol

serum panels

– Osteocalcin

– Genetic marker CRMs

CRM projects under discussion 

– International network on traceability according to IVD 

directive involving all relevant parties such as expert or 

reference laboratories, IFCC, IVD and CRM producers, 

accreditation bodies and metrology institutes. 

– Projects on SI and non-SI traceable analytes in cooperation with European Metrology institutes, IFCC, 

expert labs and NIST.

CRM activities under discussion 
131

IMEP-17 

trace and minor constituents in human serum

• Test material: unmodified/modified human serum

• Candidate measurands: Na, Cl, Ca, Cu, Mg, Li, K, Se, Zn, 

Fe, glucose, cholesterol, urea, tyroxine T4, uric acid, 

creatinine

• Traceable reference values: obtained preferably by 

primary methods of measurements by experienced 

reference measurement institutes

• Project will be part of the proposed traceability network for 

IVD (EU)

• Execution of the round: 2001-2002

0.39

0.44

0.49

0.54

0.59

0.64

0.69

0.74

0.79

0.84

0.89

0.94

0.99

1.04

1.09

1.14

 c 

 mmol·kg-1

-50

-40

-30

-20

-10

0

10

20

30

40

50

Deviation from middle of certified range in %

Mg

 Results from all laboratories.

IMEP- 7 : Trace Elements in Human Serum 

 Certified range (U=2uc ): 0.766 - 0.812 mmol·kg-1

Amount 

content Uncertainty Deviation 

(in %)

1.340 0.048 70

2.445 0.068 210
132

• Balance between production efforts, clinical 

relevance and level of uncertainty

– clinical relevance and level of uncertainty to be 

achieved determined by

• significance of the parameter in diagnosis

• reference ranges of healthy patients and daily 

variation

• expected improvement of measurement quality

– number of analyses carried out and impact on 

individuals and public health care (e.g. disease 

prevention) i.e. assessment of cost / benefit ratio

Defining priorities
133

The IVD Directive Requirement for Calibrator Traceability: 

Impact on IVD Manufacturers

Neil Greenberg 

Ortho-Clinical Diagnostics a Johnson & Johnson Company 

The new ISO/CEN written standard on traceability of calibrators for in vitro

diagnostic test systems is currently being circulated for comment as a committee draft 

(ISO/CD 175111

) prior to a formal vote. As noted previously in the Regulations and 

Standards column of the July-August, 2000, issue of IVD Technology2

, this new standard 

was developed to support one of the ‘essential requirements’ of the European Union’s In 

Vitro Diagnostics Medical Devices Directive (Directive 98/79/EC3

). The key text of the 

Directive states that... “the traceability of values assigned to calibrators and/or control 

materials must be assured through available reference measurement procedures and/or 

available reference materials of a higher order.” Now that the Directive transition 

period is officially underway, and CE marking is mandatory for IVD products exported to 

Europe by December 2003, global manufacturers are beginning to define strategies to 

achieve compliance, and are recognizing that the word “traceability” is not merely a 

synonym for the word “accuracy.” 

Measurement traceability, as defined under the Directive, means that a 

calibrator’s assigned value is systematically derived from higher order reference 

materials or reference methods, through an unbroken and well-characterized chain of 

comparisons. This chain of comparisons comprises a known hierarchy of reference 

methods and reference materials, all with known uncertainties. Trueness (i.e. closeness of 

agreement between the average value obtained from a large series of test results and an 

accepted reference value4

) is transferred from an international reference system, through 

secondary or intermediate reference materials and methods to product calibrators, which 

are used to standardize routine measurements on human samples. 

With recognition of the state of the art in clinical laboratory methods, 

methodological differences in test results are generally well known and accepted by 

physicians, laboratory practitioners, independent quality assurance organizations, 

regulators, and the IVD industry. Many national and regional proficiency (EQAS) 

programs account for such known differences by employing peer grouping in grading 

laboratory performance. In contrast, European advocates for ‘true’ clinical laboratory 

measurements have succeeded in encoding the objective of measurement truth into 

European Union law, with the approval of the Directive. 

Implications of the metrological traceability requirement for IVD test systems 

calibrators are far reaching. It is expected that some IVD manufacturers may need to 

alter not only the amount of documentation they maintain, but also in some cases their 

procedures and protocols for establishing the calibration of assays they are currently 

selling in global markets. It is also expected that the calibrator traceability requirement 

will re-energize a currently static environment for investment in development of 
134

reference methods and materials. As new reference tools eventually emerge, 

manufacturers will ultimately need to revise procedures and adjust calibration of their 

commercial kits and systems over the long term, to remain competitive. 

ISO/CD 17511 - Scope & Specific Requirements 

ISO/CD 17511 was co-developed by a collaboration among working groups of 

scientists in ISO Technical Committee 212 (Clinical Laboratory Testing and in vitro

Diagnostic Test Systems) and CEN Technical Committee 140 (in vitro Diagnostic 

Medical Devices). In addition to ISO/CD 17511, the same working groups also codeveloped a companion vertical standard, ISO/CD 181535

, Metrological traceability for 

catalytic concentration of enzymes in calibrators and control materials. 

The membership of the working groups represented a broad range of 

constituencies, including laboratory medicine professional societies, national metrology 

institutes, and IVD industry trade associations. Both ISO/CD 17511 and ISO/CD 18153 

apply to product calibrators, for reagent kits or systems, and also apply to some 

“trueness” control materials (i.e., controls intended to evaluate closeness of agreement of 

the test method with accepted reference values.) The primary objective of both of these 

ISO standards and the Directive is to ensure trueness of test results based on available 

internationally recognized reference materials or procedures, when using a routine (lower 

metrological order) test method. ISO/CD 17511 specifies how to ensure traceability, and 

highlights the documentation required to be in the manufacturer’s technical files in 

support of the assignment of particular calibration values to a product calibrator. The 

values assigned to a calibrator must be backed up with a complete, unbroken series of 

linked steps, starting with the highest order available reference methods or materials, and 

working stepwise down the metrological hierarchy to product calibrators. As stated in 

ISO/CD 17511, section 4.1.7, “...the responsibility of the manufacturer for describing the 

traceability chain ...start(s) at the value of the manufacturer's product calibrator and 

end(s) at the metrologically highest reference used by the manufacturer.”

ISO/CD 17511 - Items Excluded from Scope

Manufacturers need to be aware that certain IVD products within the general 

device categories of calibrators or quality control materials are outside the scope of 

ISO/CD 17511. 

Items in this group include: 

• Commercial quality control samples sold without assigned values, and 

used only for assessing precision within a laboratory (precision control 

materials)

• Quality control materials intended for intralaboratory quality control 

purposes, and supplied with intervals of suggested acceptable values 

that were obtained by interlaboratory consensus, and are limited for 

use with only certain methods. 
135

In addition, certain approaches to value assigning IVD calibration materials are 

considered to be non-traceable under the rules of ISO/CD 17511. 

Examples include: 

• Calibrators with values assigned based on “horizontal” or predicate 

method correlation studies, using results for split patient samples tested 

with two different measurement procedures operating at the same 

metrological level. 

• Calibrations derived from correlation between two analytes with 

different physico-chemical characteristics. Example: Total CO2 in 

whole blood, calibrated against a serum bicarbonate reference 

procedure. 

Key Issues and Concerns: The State of the Art in Calibrator Traceability 

Definition of the Analyte.

With IVD measurement devices, a detailed definition of the analyte is an often 

overlooked, but nevertheless important aspect of an analytical reference system. When 

the target analyte is actually a class or group of substances, rather than a single, welldefined chemical species, tracing measurements to metrologically higher order calibrators 

or measurement procedures may be quite challenging. Often, very different forms of a 

particular class of substances may be important in different pathological conditions. An 

example is human chorionic gonadotropin (HCG), which exists in a variety of different 

molecular forms in serum, some of which are predominant in women during pregnancy, 

and others that predominate in certain tumors6

. 

Two General Analyte Classes Defined in ISO/CD 175.

To differentiate those substances that are chemically well defined and present in a 

narrowly definable range of molecular species from those substances that are typically 

found as a range of heterogeneous sub-species, ISO/CD 17511 defines Type A (physicochemically well-defined) and Type B (heterogeneous mixture) analytes. In principle, the 

Type A substances are expected to be traceable to metrologically higher-order reference 

materials and methods, often going back up the hierarchical chain to the fundamental unit 

of measure, as defined according to the System Internationale (SI meter convention). 

Due to the complexities of analyte definition and molecular structures, the Type B 

substances are not expected to be traceable to SI units. Because of their metrological 

complexity, members of the Type B analyte group present the greatest challenges to IVD 

manufacturers from the standpoint of the task of defining the metrological traceability of 

the commercially available measurement systems and test kits. 
136

Some Typical Situations Where Manufacturers May Encounter Difficulty in Defining 

Metrological Traceability, Due to Analyte Complexity and Heterogeneity:

• Analyte chemico-physical complexity is the norm, not the exception: there are 

perhaps up to 1000 different analytes routinely measured in clinical laboratories, less 

than 10 % of which are chemically well-defined (Type A). The Type A substances 

may be more than 80 % of the total volume of procedures actually performed 

annually, but in terms of the available test menu, the vast majority of analytes are 

actually Type B. Examples of some of the items from the extensive menu of type B 

analytes typically offered include tumour markers, coagulation parameters, endocrine 

hormones, immune factors, virology markers, and bacteriology markers. Many of 

these substances are complex proteins, which may be present in blood and tissues as a 

group of various molecular species. 

• Immunoassay methods commonly used in routine measurement systems for many 

complex (Type B) analytes often demonstate antibody selectivity for specific antigens 

and/or epitopes. This may be due to underlying physical attributes of the analytical 

antibodies, owing to their being produced in a biological system that consists of a 

purified antigen/stimulus containing only a limited portion of the spectrum of possible 

epitopes of the antigen existing in nature. 

• The so-called Type A analytes can also display chemico-physical heterogeneity. 

Seemingly ‘simple’ analytes such as serum calcium are present in human serum in a 

chemically broad array of molecular forms, including ionized species, inorganic and 

organic salts, water-complexed, and protein-bound fractions. Other analytes such as 

the most commonly measured serum electrolytes, sodium and potassium, can be 

quantified as either ion concentrations or ion activities, depending on the measurement 

technology chosen. According to strict analytical chemistry interpretation, these 

different chemical entities are considered to be different ‘quantities,’ and therefore, 

their calibrations must follow along separate pathways of traceability. 

Is the Global IVD Reference Systems Infrastructure Sufficiently Mature to Support 

Calibration Traceability?

There are a number of gaps in the present system of reference materials and methods 

for IVDs. In the US, the National Reference System for the Clinical Laboratory 

(NRSCL), a steering committee operating within NCCLS and originally created for the 

purpose of credentialing a US IVD reference systems infrastructure, credentialed 

reference systems for approximately a dozen clinical chemistry analytes over a period of 

activity spanning approximately 10 years. However, NRSCL is not currently actively 

involved in credentialing additional reference systems. 

In the meantime, a number of other organizations have participated in this arena. As 

a result, a variety of clinical laboratory reference methods and materials (but few full 

reference systems) have been certified by groups such as the US National Institute of 

Standards and Technology (NIST), the (US) Centers for Disease Control and Prevention 

(CDC), the European Union - Institute for Reference Materials and Methods (IRMM), the 

World Health Organization (WHO), the International Federation of Clinical Chemistry 
137

and Laboratory Medicine (IFCC), the Japanese Committee for Clinical Chemistry 

(JCCLS), the German Society for Clinical Chemistry (DGKC), and the Scandinavian 

Society for Clinical Chemistry. Across this highly faceted array of organizations, there 

are some program redundancies, including different reference materials and reference 

methods for certain analytes. Patient results will significantly differ, in several of these 

cases, depending on which reference material or reference method is chosen for 

standardization of a particular procedure. 

The current situation is therefore one of having parallel and sometimes redundant 

standards programs around the world. The participants include both regional and 

international organizations (e.g. European Union/IRMM, NIST, IFCC, WHO, DS, etc.), 

many of which are currently, in the wake of the IVD Directive, in the process of initiating 

new projects to develop calibration standards, without the benefit of formal and 

comprehensive global intra-organizational coordination. All of the groups have slightly 

different constituencies, different funding sources, and different (and sometimes even 

competing) objectives. 

Frequently Encountered Deficiencies with Some Existing Reference Materials and 

Reference Methods.

1. Absence of a complete definition of the analyte intended to be calibrated. 

Reference methods and/or materials are often specifically targeted to support 

measurement of an analyte in a particular body fluid (e.g., potassium in serum, creatinine 

in serum, amylase in serum). While this is helpful, in fact the clinical laboratory field 

may be interested in application of routine methods to a broader array of body fluids (e.g. 

urine, cerebrospinal fluid, pancreatic fluid, whole blood, etc.), and reference materials 

and methods simply do not exist to support the other applications. 

2. Non-suitability of available reference materials for their intended purpose.

Many analytes of clinical interest demonstrate chemical instability. Some examples are 

bilirubin, cardiac troponin and cardiac CK-MB. As a result, highly pure preparations of 

these analytes prepared for use as reference materials typically demonstrate similar 

instability. While this problem presents a great technical challenge for the developers of 

reference materials, the absence of stable preparations of reference materials to support 

calibration processes in a manufacturing environment renders such materials relatively 

useless for routine applications. 

3. High uncertainty of assigned values for certain reference materials. Of the 

reference materials that are currently available, some have assigned values that are 

defined using a statistical consensus approach, involving multiple measurements on a 

range of diverse analytical methods that are currently commercially available. Since 

there are no prior calibration standards for these existing methods, their calibrations may 

be arbitrary. The consensus methodology enables all currently marketed devices to meet 

calibration requirements, since the statistical range of uncertainty in the face of often 

highly dispersed data, typically accommodates differences between the methods included 

in these studies. This approach, while expedient and seemingly ‘fair’, does not assist in 

progressing toward the goal over time of achieving ‘true’ values. This approach also 

favours the presently available range of ‘true’ values observed with existing methods, and 
138

could force the calibration of newer technologies and devices against a status quo that 

may be scientifically flawed. 

4. Non-commutability of reference materials. Commutability is an attribute that 

describes the ability of a reference material to emulate actual patient samples in its 

performance characteristics, over a broad range of analytical methods and systems. A 

number of reference materials contain chemical ‘surrogates’ of the analyte of interest, not 

the actual natural compounds, making them useful only in higher order chemical 

measurement systems (e.g. gas chromatography/ mass spectrometry, high-performance 

liquid chromatography, atomic absorption spectroscopy, etc.) Instances of 

incompatibility of many such reference materials with routine field methods, render these 

reference materials significantly less useful, forcing manufacturers to add additional steps 

and layers of complexity to their internal product calibration processes, and contributing 

to increasing uncertainty in the final assigned values for product calibrators. 

5. Lack of portability of reference methods. Some reference methods are 

analytically complex, requiring expensive, rare and sophisticated instruments, as well as 

highly trained technologists to operate these systems. As a result, there are very few or 

even a complete absence of laboratories in most regions of the world capable of 

performing laboratory medicine reference measurements at an attractive cost, in a way 

that ensures access to all interested parties, especially smaller device manufacturers and 

clinical labs. 

6. Recognition and status of existing reference methods and materials. Currently, 

there are some instances where certain IVD reference systems, methods or materials have 

international recognition. However, there are also many cases where recognition and 

status of a particular standard is only at the national or regional level. At the international 

level, there is presently no universally recognized authority charged with ensuring 

harmony among the regional and national bodies involved in accreditation of IVD 

measurement standards. Further, there is no established internationally accepted 

umbrella organization or process to accredit either existing or new reference methods, 

materials or even the reference laboratories that are performing some of the 

metrologically “higher level” measurements. Indeed, some of the organizations 

participating in this arena may have not yet established their scientific credibility at a 

level sufficient to command recognition or status with their peer organizations. Rapid 

resolution of these issues is clearly necessary in order to define the path forward, in the 

direction of progress. 

Additional IVD Industry Concerns 

ISO/CD 17511 Requires That the Uncertainty of Assigned Values of Calibrators be 

Made Available Upon Request.

A fundamental element of the concept of measurement traceability is that each 

link in the chain of measurement must have known uncertainty. Since the ultimate goal 

is ‘accurate’ measurement (i.e. deviations from ‘truth’ no greater than acceptable levels), 

knowledge of the uncertainty at each step in the traceability chain is important, in order to 
139

identify opportunities for improvement7

. However, clinical laboratory personnel and 

IVD manufacturers have little experience with the concept of uncertainty of assigned 

values for reference materials, as it pertains to the calibrators provided for use with 

routine measurement procedures. This demands that manufacturers proceed through 

essentially uncharted waters. From the IVD manufacturers’ perspective, this is a 

completely new and unknown requirement, for which no current models exist in the 

medical device industry. Traditionally, uncertainty calculations have been the concern 

only of higher order metrology institutes. Only in recent years, as part of the quality 

movement, has the measurement uncertainty concept begun to filter down to calibration 

laboratories in the industrial setting. 

The primary resource for manufacturers and others interested in the ‘how to’ of 

determining and expressing measurement uncertainty is an ISO document known as the 

GUM (Guide to the Expression of Uncertainty in Measurement8

). Due to the highly 

complex statistical language and academic discussion presented in the GUM, nonmetrologist readers frequently find the document to be cumbersome, and are left with a 

number of unknowns in terms of how to actually calculate uncertainty. There are no 

readily available interpretations that are applicable to IVD manufacturers, and as a result 

it is expected that different manufacturers will develop a variety of interpretations, 

leading to lack of uniformity in the estimates reported. 

Aside from the concerns about how manufacturers will actually establish their 

uncertainty estimates, the value of the information about uncertainty of assigned values in 

calibrators to routine clinical laboratory customers is questionable. This is especially in 

the case of many typical clinical measurement systems where overall measurement 

variation is only a very small part of the total variation of the analyte, in the context of 

the much greater relative magnitude of the biological fluctuations of the analyte within a 

given individual9

. 

Declaration of New or Improved International Reference Materials and Reference 

Methods – Some Manufacturers May Need to Change Calibrations for Certain Analytes

Development of new or improved reference systems is not a specific requirement 

of the IVD Directive. The Directive states that routine methods need to be traceable to 

“...available reference measurement procedures and/or available reference materials of a 

higher order.” Similarly, ISO/CD 17511 allows for a wide range of scenarios, include 

situations where there is no recognized higher order method or material available to trace 

back to. Under these circumstances, the highest order reference point available to a given 

IVD manufacturer may be a measurement procedure or reference material that is 

uniquely defined, controlled, and maintained by each manufacturer of the various 

commercial assay systems for a given analyte. When new reference methods or materials 

are developed and internationally accredited for such analytes, certain manufacturers will 

inevitably have to make changes in their processes in order to adjust their products to the 

new reference systems. All manufacturers are willing to step up to implementation of 

changes necessitated by customer needs, and which will ultimately be beneficial in 

bringing about improved health care. Hopefully, the changes required are important 

ones, representing needs articulated by a consensus of clinicians and laboratory 

professionals, since all changes are associated with disruption and cost. 
140

Some of the short term costs for manufacturers include: 

1. Labeling changes and inventory obsolescence, 

2. Customer and EQAS program advisory notices, 

3. Filings with regulatory authorities, 

4. Manufacturing process changes, 

5. Training of customers and intracompany personnel, 

6. Reference interval updates requiring additional communications and 

training for clinicians. 

Given these costs, it is especially important that the initiative and the leadership 

for change be customer-focused, originating from a clear expression of need for 

improvement on the part of the end-users, not the manufacturers. 

ISO/CD 17511, in contrast with the Directive, takes a slightly stronger position in 

terms of commitment to the cause of continuous improvement in reference methods and 

materials. As the standard states, “It is the aim of metrology in laboratory medicine to 

improve traceability...by providing the missing reference measurement procedures and 

reference materials, based on international consensus.” Because of this commitment, it is 

expected that many national and international standards organizations, scientific, 

professional, and industry groups, will interpret the Directive’s intent, arriving at an 

interpretation which says that the Directive demands investment in upgrades to the 

international reference system for the clinical lab. What is most important, as new 

reference materials or reference method projects inevitably get underway in the name of 

the Directive, is that adequate scientific support from industry be sought after when 

staffing the working groups. This will help to ensure that a reasonable balance is 

achieved between commercial interests, pragmatic realities of manufacturing materials 

and process limitations, and academic clinical and metrological interests. 

Potential for Mis-Use of the Directive Requirement for Calibration Traceability.

One of the fears occasionally voiced by manufacturers is that once the Directive is 

fully in force throughout the European Union, country level authorities may be inclined 

to overinterpret and mis-use the expectation of calibration traceability, to prematurely 

declare that specific types of IVD products (e.g. test kits for serum troponin) available 

within their own borders, must be traceable to a particular reference method or material, 

which is recognized or preferred only within that particular country. Such actions could 

create protectionism and artificial trade barriers, perhaps favoring the marketing of 

products manufactured only by companies located within certain countries. Such an 

approach could also be used to limit exploitation of new or currently existing technology 
141

options, due to limitations in the capabilities and/or availability of recognized reference 

methods and materials. 

Examples of leading edge IVD technical development where there 

currently exists the potential for barriers to new technologies include:

• Restriction of immunoassay methods to specific epitopes or gene 

products that are currently known markers for a particular disease 

• Selection of defined substrates as standards for enzymes, e.g., serum 

amylase 

• Selection of reference methods or materials, which rely on products 

(equipment or materials) that must be licensed from a single vendor, 

due to patented technology. 

Do IVD Manufacturers Want or Need More Reference Materials And Methods? 

Despite short-term concerns, international calibration standards (reference 

systems) do in fact help IVD manufacturers by providing well-defined market needs and 

customer requirements, clear and universal definition of goals, and tools for objective 

assessment of product attributes. Table 1 provides a more detailed list of a few of the 

global trade-offs that should be considered by all the participating organizations 

(manufacturers and distributors of IVD products, industry associations, user groups, 

customer advocacy and professional groups, government and regulatory bodies, etc.) 

proposing to undertake development of new calibration standards (reference systems) for 

IVDs. It is clearly a matter of cost vs. benefit, and the decision to move forward demands 

careful analysis on a case-by-case basis. 

TABLE 1. Trade-Offs in Decision to Invest in New Reference Systems for IVDs 

BENEFITS 

 Product Quality 

 Interchangeability of data between products 

 Competitiveness – level playing field for competition 

 Defined quality needs 

 Lower long-term costs 

 Clearer pathway to market access 

 Transferable technology 
142

COSTS 

 Availability of appropriate qualified people to participate in 

standards work 

 Risk of investing in standards that, upon completion, are not 

accepted by all stakeholders, especially customers 

 Lengthy cycle time to achieve deliverables 

 Costs of transition (both within manufacturing companies & for 

customers) to make changes to comply with new standards 

 Less variety; fewer alternatives for customers 

 Barriers to innovation 

 Barriers to market entry 

Does Industry Support Initiatives for New IVD Calibration and Reference Systems? 

Public interest requires good quality and safe health care products. It is easy to 

demonstrate that improved standardization contributes to furthering these goals, and all 

IVD manufacturing companies share this interest. Reference materials and reference 

methods for calibration are an important underlying element, contributing to quality and 

safety through the assurance of interchangeability of information across time and space. 

Given these shared goals, it is inevitable that new reference systems projects will 

emerge, and new reference materials and methods will ultimately be defined, which will 

impact the definition of the state of the art. Industry must play a role in this process, using 

its collective wisdom achieved through years of real-world experience, to ensure 

technically sound and practical solutions to the challenges encountered in development 

projects undertaken in the name of the desire for better standards. 

What are the Priorities of IVD Manufacturers for Development of New Calibration 

Standards? 

A short-term objective of many manufacturers is to ensure that adequate tools are 

in place to support priorities established under the IVD Directive. The IVD Directive has 

designated certain IVDs, included within a list known as Annex II, as ones which require 

special certification by independent, notified bodies, as a prerequisite to granting and 

maintaining of the CE mark. Thus, even in the absence of internationally recognized 

calibration standards, manufacturers are likely to have much less latitude in calibrating 

these tests. Evaluation and certification of these products will be performed against new 

de facto standards known as Common Technical Specifications, created especially to 

support conformance assessment of these products, for CE marking under the Directive. 

To ensure fairness and equity in these assessments, common reference materials are 

needed both by the manufacturers and the notified bodies to assist in assessing 
143

performance, and in controlling analytical sensitivity at medical decision limits or cutoff 

values. 

Some specific analytes urgently needing better international standards and 

included in the Directive, Annex II, are infectious disease markers used for blood 

screening and diagnostics for hepatitis, such as HBsAg, anti-HCV, anti HBs; markers for 

immuno-deficiency viral infections, including anti-HTLV and anti-HIV; and blood 

grouping reagents used to ensure donor unit compatibility in markers for Anti D, Anti A, 

Anti B. 

Recommendations 

What factors are necessary to get to the future state? To begin, laboratory 

medicine, scientific, and professional organizations need to provide leadership and 

guidance relative to what standards are needed. In doing so, project priorities should be 

defined with an appreciation that resources are limited, while taking into account factors 

such as (1) the public health significance and disease course, (2) expectations of the 

degree of improvement anticipated in overall clinical effectiveness of a given test if a 

new standard is developed, (3) time and cost estimated to reach a desirable endpoint, and 

(4) the overall likelihood of success. 

High priority should be given to establishing a defined, international, customerfocused, and consensus-based process for setting priorities and contracting projects. 

This process should be led by the major laboratory professional associations (e.g. IFCC, 

AACC, CAP, WASP and others) and should proceed in an atmosphere of open and 

public dialogue, employing decision tools that emphasize quantification and metrics, and 

is inclusive of all key stakeholders (i.e., profession, government, industry, lay public). 

One possible solution is formation of an ‘IRSCL’ (‘I’ standing for international), 

following the NCCLS-NRSCL model. This body must be international in scope and 

responsibility, and could be staffed under contract with an organization such as NCCLS, 

to provide infrastructure needs. 

Sufficient project funding (grants or contracts) must be provided, and will need to 

include government sources (e.g. EU Commission, US Department of Commerce, etc.), 

professional societies (e.g. IFCC, AACC, CAP, WASP), as well as industry groups. 

Projects must be closely managed, with accountability for deliverables and 

schedule. Project teams must be staffed with appropriate clinical and scientific experts, 

coming equally from the professions and industry. 

Conclusions 

Although the EU IVD Directive does not specifically require it, reference 

materials and reference methods development projects will be initiated in the name of the 

Directive. New calibration standards are likely to increase short term costs for IVD 

manufacturers, but this is a minor consideration if there is a clear need for improved 

standards and an expectation of improved quality of health care as an outcome, as 

expressed by a consensus of customers and professional associations. Professional and 
144

customer advocacy groups should take the lead role in advocating for new calibration 

standards, especially in defining where improved standards are needed. Whenever IVD 

calibration and reference systems standards projects are undertaken, whether 

professional, government, or public health groups sponsor them, inclusion of IVD 

industry scientists and experts on the technical team is an absolute prerequisite for project 

success. Creative strategies are needed to ensure adequate project funding, and should 

involve a combination of contributions from public, professional and industry sources. 

References 

1. In Vitro Diagnostic Medical Devices—Measurement of Quantities in Samples of 

Biological Origin—Metrological Traceability of Values Assigned to Calibrators 

and Control Materials, ISO/CD 17511 (Geneva: International Organization for 

Standardization, February, 2000). 

2. Powers, Donald M., Regulations and Standards - Traceability of assay calibrators: The 

EU's IVD Directive raises the bar, IVD Technology, July-August, 2000. 

3. Council Directive 98/79/EC of the European Parliament and of the Council of 27 

October 1998 on In Vitro Diagnostic Medical Devices, Official Journal of the 

European Union L331 (December 12, 1998). 

4. International Standard ISO 3534-1 Statistics – Vocabulary and symbols, Part 1: 

Probability and general statistical terms, International Organization for 

Standardization, first edition, 1993-06-01, Geneva, Switzerland. 

5. In Vitro Diagnostic Medical Devices—Measurement of Quantities in Samples of 

Biological Origin—Metrological Traceability of Values for Catalytic 

Concentration of Enzymes Assigned to Calibrators and Control Materials, 

ISO/CD 18153 (Geneva: International Organization for Standardization, February, 

2000). 

6. Galina Kovalevskaya, Steven Birken, Tatsu Kakuma, John Schlatterer and John F. 

O’Connor, Evaluation of Nicked Human Chorionic Gonadotropin Content in 

Clinical Specimens by a Specific Immunometric Assay, Clinical Chemistry, 1999; 

45: 68-77. 

7. Charles D. Ehrlich and Stanley D. Rasberry, Metrological Timelines in Traceability, J. 

Res. Natl. Inst. Stand. Technol. 103, 93-105 (1998). 

8. International Organization for Standardization, Guide to the Expression of 

Uncertainty in Measurement, Geneva, Switzerland (1993) (corrected and reprinted 

1995). 

9. Fuentes-Arderiu, X. Clin Chem 2000;46:1437-1438, Uncertainty of Measurement in 

Clinical Laboratory Sciences. 
145

NEIL GREENBERG 

Ortho-Clinical Diagnostics a Johnson & Johnson 

Company 

Neil Greenberg, Ph.D., DABCC, is a Project Manager, Regulatory Affairs, OrthoClinical Diagnostics, a Johnson & Johnson Company. He received his doctorate in 

Physiological Chemistry and Clinical Chemistry from The Ohio State University, Columbus, Ohio, and served as a Post Doctoral Fellow in Clinical Chemistry at Hahnemann 

University, Philadelphia. He joined Eastman Kodak in 1978, and has held various 

managerial roles in manufacturing and quality assurance for the Ortho-Clinical Diagnostics Vitros (formerly Ektachem) line of in vitro diagnostic products. As Manager of 

Kodak’s Reference Laboratory, he became actively involved in standardization of measurement procedures and reference materials for laboratory medicine in the early 1980’s, 

and continues to work actively with NCCLS and the American Association for Clinical 

Chemistry on various aspects of measurement standardization for in vitro diagnostics. 

He is a member of Working Group 2, ISO/TC 212, which is developing international 

standards for reference systems in laboratory medicine. Dr. Greenberg also is the liaison for ISO/TC 212 Working Group 2. 
146
147

The IVD Directive Requirement for Calibrator Traceability: 

Impact on IVD Manufacturers 

Visuals Presented By

Neil Greenberg 

Ortho-Clinical Diagnostics a Johnson and Johnson Company 
148

Slide 1

The IVD Directive Requirement 

for Calibrator Traceability: 

Impact on IVD Manufacturers

Neil Greenberg, Ph.D., DABCC

Ortho-Clinical Diagnostics, a Johnson & Johnson Company

Presented at

Workshop on Measurement Traceability 

for Clinical Laboratory Testing

and In Vitro Diagnostic Test Systems

Gaithersburg, MD, November 2, 2000

Slide 2

Traceability: Impact on Manufacturers

Outline

• Overview of ISO/CD 17511

– Scope & Specific Requirements

– Exclusions

– Limitations

– Key Issues and Problem Areas

• IVD Industry Concerns

• Recommendations 

• Conclusions
149

Slide 3

Traceability: Impact on Manufacturers

ISO/CD 17511 - Scope & Specific Requirements

• Primary Objective: Ensure TRUENESS of test 

results based on available internationally recognized 

reference materials or procedures, when using a 

routine (lower metrological order) test method 

• Specifies documentation requirements for assigning 

calibration values in a calibrator (reference material)

• Requires a complete, UNBROKEN series of linked 

steps from highest order available reference methods 

or materials, stepwise “downward” to product 

calibrators

Slide 4

Traceability: Impact on Manufacturers

ISO/CD 17511 - Scope & Specific Requirements

• Applies to calibrators and some control materials sold 

for use with IVDs

• Specifies how to ensure traceability of values assigned 

to calibrators and control materials that establish or 

verify TRUENESS of a measurement method. 

• 4.1.7 “The responsibility of the manufacturer for 

describing the traceability chain ...start(s) at the value 

of the manufacturer's product calibrator and end(s) at 

the metrologically highest reference used by the 

manufacturer.”
150

Slide 5

Traceability: Impact on Manufacturers

ISO/CD 17511 - Items Excluded from Scope 

• Control materials without assigned values, used only 

for assessing precision (precision control materials)

• Control materials intended for intralaboratory 

quality control purposes & supplied with intervals of 

suggested acceptable values (obtained by 

interlaboratory consensus, for particular methods). 

• Correlation between results of two measurement 

procedures at the same metrological level.

– “Horizontal” or predicate correlation studies do not 

provide traceability

Slide 6

Traceability: Impact on Manufacturers

ISO/CD 17511 - Items Excluded from Scope

• Calibrations derived from correlation between two 

procedures at different metrological levels, for 

analytes with different physico-chemical 

characteristics. 

– e.g.: Total CO2 in whole blood, calibrated against a 

serum bicarbonate reference procedure

• Properties defined with ‘nominal’ scales, when no 

magnitude is involved (e.g. blood group).
151

Slide 7

Traceability: Impact on Manufacturers

ISO/CD 17511 - Items Excluded from Scope

• Traceability of routine patient results to a calibrator

• Relationship of routine patient results to medical 

discrimination limits

– Traceability of patient result may be influenced by 

method specificity

– Non-specificity (random interaction between method 

and patient samples) may have little or no impact on 

assigned values of calibrators

Slide 8

Traceability: Impact on Manufacturers

Key Issues & Problems

• Chemico-Physical Nature of Measured Quantities

– Of ~ 1000 analytes, less than 100 are well-defined 

chemical entities

– Analyte heterogeneity

• ‘Total Protein’, enzymes, hormones, etc.

– Bound vs. free analyte forms

– Immunologic method variability

• Selectivity for specific epitopes
152

Slide 9

Traceability: Impact on Manufacturers

Key Issues & Problems

• Reference Systems

– Gaps in reference materials and methods 

– Parallel/redundant programs

• Regional & international organizations (e.g. NIST, 

IFCC, IRMM, WHO, DS) are initiating programs

• Lack of coordination among programs

– Few laboratories capable of performing 

reference measurements at reasonable cost

Slide 10

Traceability: Impact on Manufacturers

Key Issues & Problems

• Reference Systems

– Deficiencies in existing reference materials and 

reference methods

• Non-commutability of many reference materials

• Large uncertainties in assigned values of certain 

reference materials (e.g CK-MB)

• Lack of portability of many reference methods

– No recognized authority to assure harmony or 

accredit (new or existing) reference methods, 

materials or labs at international level
153

Slide 11

Traceability: Impact on Manufacturers

Key Issues & Problems

• Uncertainty of Assigned Values

– Information must be available on request

– Complexity, assumptions, and unknowns in 

calculating combined total uncertainty

• Directly measurable/observed (type A) 

uncertainties

• A priori uncertainties (type B)

– Value to customers in routine applications is 

questionable

Slide 12

Traceability: Impact on Manufacturers

IVD Industry Concerns

• Development of new or improved reference materials

– NOT required by the IVD Directive ...

“...available reference measurement procedures 

and/or available reference materials of a higher 

order.”

– ISO/CD 17511...

“It is the aim of metrology in laboratory medicine to 

improve traceability...by providing the missing 

reference measurement procedures and reference 

materials, based on international consensus.”
154

Slide 13

Traceability: Impact on Manufacturers

IVD Industry Concerns

• Traceability can be mis-used to limit new or existing 

technology options, due to limitations in capability 

and/or availability of recognized reference methods 

&/or materials.

– Restricting immunoassay methods to specific 

epitopes or gene products

– Defined substrates for enzymes, e.g., serum amylase

– Limiting reference (methods or materials) to 

licensed or patented technology

– Limiting development of in vivo techniques

Slide 14

Traceability: Impact on Manufacturers

IVD Industry Concerns

• International recognition status of existing reference 

methods and materials

– international vs. regional recognition of methods 

and materials - How to resolve?

– recognition and scientific status of organizations 

responsible for existing and/or future standards

• Documentation of available reference procedures 

and materials

– Is documentation compliant with ISO/DIS 15193 

& 15194?
155

Slide 15

Traceability: Impact on Manufacturers

IVD Industry Concerns

• Available reference materials suitable for intended 

purpose?

– Analyte clearly defined & stable? 

• e.g. troponin

– Matrix options 

• e.g. whole blood vs. serum

– Low uncertainty of assigned values?

• Reference lab accreditation 

– compliant with ISO/CD 15195?

– Internal company labs vs. outsourcing?

Slide 16

Traceability: Impact on Manufacturers

Do IVD Manufacturers want more IVD 

reference materials & reference methods?

• Standards help IVD companies by 

providing...

−Well-defined market needs 

− Clear, universal definition of goals

−Objective assessment of product 

attributes
156

Slide 17

Traceability: Impact on Manufacturers

• BENEFITS

• Quality

• Interchangeability

• Competitiveness

• Defined quality needs

• Lower long-term 

costs?

• Clearer pathway to 

market access

• Transferable 

technology

• COSTS

• Cost of working on standards 

projects

• Lengthy cycle time to 

deliverables

• Transition to comply with new 

standards

• Initially higher costs?

• Less variety; fewer 

alternatives for customers

• Barriers to innovation

• Barriers to market entry

Standards Trade-Offs

Slide 18

Traceability: Impact on Manufacturers

Does Industry support new IVD standards?

• Public interest requires good quality & safe 

health care products

• Reference materials & reference methods are 

an important underlying element

– Calibration standards will inevitably be defined

– Standards will impact the ‘state of the art’

• Industry must play a role to ensure technically 

sound and practical standards
157

Slide 19

Traceability: Impact on Manufacturers

• IVD Directive ANNEX II Products –

– Draft EU ‘Common Technical Specifications’

(CTSs) call for development of critical biological 

standards (reference materials)

– Common reference materials needed by 

manufacturers and notified bodies for assessing 

performance; controlling analytical sensitivity at 

medical decision limits (cutoff values)

High Priorities for (some) Manufacturers

Slide 20

Traceability: Impact on Manufacturers

•IVD Directive Priorities:

– International standards (IS) for Infectious 

Disease markers, for blood screening and 

diagnostics

• Hepatitis - HBsAg , anti HCV, anti HBs

• anti HTLV & anti HIV

– International blood grouping reagent 

standards 

• Anti D, Anti A, Anti B

High Priorities for (some) Manufacturers
158

Slide 21

Traceability: Impact on Manufacturers

Global Cooperation in Developing Standards

Slide 22

Traceability: Impact on Manufacturers

Recommendations

• Scientific & Professional Organizations should 

provide guidance re: what standards are needed

• Define project priorities with recognition that 

resources are limited, taking into account...

– Public health significance and disease course

– Anticipated improvement in overall clinical 

effectiveness of tests, if a standard is developed

– Time/cost needed to reach a desirable endpoint

– Likelihood of success
159

Slide 23

Traceability: Impact on Manufacturers

Recommendations

• Establish a defined, INTERNATIONAL

consensus-based process for setting priorities and 

contracting projects

– Open, public dialogue & decision process

– All key stakeholders must have a seat (i.e., 

profession, government, industry, lay public)

– Consider formation of an “IRSCL”, following the 

NCCLS-NRSCL model

• Must be International in scope & responsibility

• NCCLS could hold the secretariat role

Slide 24

Traceability: Impact on Manufacturers

Recommendations

• Sufficient project funding (grants or contracts) must be 

provided 

– Government sources (e.g. EU Commission, US Dept of 

Commerce, etc.)

– Professional societies (e.g. IFCC, AACC, CAP, WASP)

– May be supplemented with company grants

• Projects must be closely managed, with accountability 

for deliverables and schedule

• Project teams must be staffed with appropriate clinical 

and scientific experts, from the profession and industry
160

Slide 25

Traceability: Impact on Manufacturers

Conclusions

• Although the Directive does not specifically 

require it, reference materials and reference 

methods development projects will be initiated 

in the name of the Directive

• New standards are likely to increase short term 

costs

• Manufacturers are NOT likely to take a lead 

position in advocating for new standards

– especially which standards

Slide 26

Traceability: Impact on Manufacturers

Conclusions

• Whenever IVD standards projects are 

undertaken (via professional, government 

and/or public health group sponsorship), 

inclusion of manufacturing scientists & 

experts is absolutely required for project 

success

• Significant project funding from the public 

sector is necessary
161 

History of Reference Systems for Clinical Measurements 

John H. Eckfeldt 

University of Minnesota 

Fairview-University Medical Center 

 I must confess at the outset that my "history" will emphasize the U.S. activities, 

particularly the role the College of American Pathologists (CAP) played, because developments in this region are what I know best and have the majority of published reports in 

English. Nevertheless, I am certain that similar concerns by the medical and laboratory 

communities worldwide were being expressed and parallel efforts in other parts of the 

world are progressing. I am sure other speakers at this workshop will mention many of 

these developments from other parts of the world in their presentations. 

 Efforts to improve traceability of 

clinical laboratory measurements in the 

U.S. date back approximately to the middle of the twentieth century. One of the 

most seminal reports that described these 

early efforts was that of a study describing 

a comparison of clinical laboratory test 

results performed in 1946 by the Committee on Laboratories of the Medical Society 

of the State of Pennsylvania (Belk & Sunderman, 1947). At that time only about a 

dozen analytes were routinely measured in 

hospital laboratories. In their study, interlaboratory variability of hemoglobin, glucose, sodium, total protein, albumin, calcium, and urea results were examined. 

Samples were distributed to 59 laboratories in Pennsylvania and surrounding 

states that had expressed willingness to 

participate. Data was collated from two 

different mailings approximately a month 

apart. Figures 1 and 2, which are reproduced from this early report, show the 

results for glucose and hemoglobin. In 

both cases, a number of the reported results were completely discarded, because 

they were so totally out of the range that they would not fit on the graph as presented. In 

their discussion, Drs. Belk and Sunderman concluded, based on the opinions and information expressed by the participating laboratory staff and pathologists, that poorly or inadequately trained technicians, poor equipment, and a variety of miscellaneous factors 

Figure 1. Frequency of results for an 

aqueous glucose control material from 

clinical laboratories in Pennsylvania 

and surrounding states in calendar year

1946. (Reproduced from Belk and Sunderman 1947 by permission). 
162 

contributed to the very high degree of variability between laboratories. The authors concluded that in many cases the clinical laboratories’ results were so discordant from the true 

value that it was difficult to see how any reasonable clinical decisions taking into account 

laboratory information could be made! 

 Dr. Sunderman became a founding 

member of the CAP, guiding many of the 

CAP's early efforts toward investigation and 

improvement of inter-laboratory variability of 

clinical laboratory test results. Interestingly, 

now at age 100, Dr. Sunderman just contributed a brief report in the August 2000 issue of 

CAP Today on his personal experiences with 

today's U.S. medical care system comparing it 

to personal experiences approximately eight 

decades ago. Dr. Sunderman and other CAP 

founders began the CAP Survey’s Program in 

1949. The CAP Survey’s Program was designed, and still is, a proficiency testing (PT) 

program, often called external quality assessment system (EQAS) in other parts of the 

world, whose major goal is to improve interlaboratory comparability of medically important analytes. Over the more than 50 years 

since its inception, the CAP Surveys Program 

has developed into the largest PT program in the world. In the calendar year 2000, it will 

include over 25,000 participating laboratories, testing over 250,000 different samples for 

over 500 different analytes. 

 Shortly after the beginning of the Survey Program, the CAP Standard Solutions 

program began in 1951. This program was designed to provide reference materials to 

clinical laboratories that could be used for calibration of their clinical instruments. In 

many ways this program marked the beginning of the development of a clinical laboratory "reference system" for the U.S. Because of continued inter-laboratory comparability 

problems with hemoglobin measurement, and the extreme importance of this analyte for 

clinical decision-making, the CAP opened a Clinical Standards Laboratory at the Cleveland Clinic, in Cleveland, OH in 1961. This laboratory provided reference measurements 

and the materials for the cyanmethemoglobin determination for clinical laboratories and 

for manufacturers of devices and reagents for hemoglobin measurement through the late 

1970’s at the Cleveland Clinic. In 1970, this laboratory was moved to the United States 

National Bureau of Standards (NBS), as the National Institutes of Standards and Technology (NIST) was then called, under a Cooperative Research and Development Agreement between NBS and CAP. Prior to this laboratory’s move to the NBS, reference 

Figure 2. Frequency of results for 

a whole-blood hemoglobin control 

material from clinical laboratories 

in Pennsylvania and surrounding 

states in calendar year 1946 (Reproduced from Belk and Sunderman 1947 by permission). 
163 

methods for billirubin and cholesterol, two other clinically important analytes, had been 

added to its repertoire. At its peak in the mid-1990’s, under this agreement, three fully 

funded CAP employed Roger K. Gilbert Research Fellows worked side-by-side with 

NIST staff in the Gaithersburg, MD facilities providing high-level reference method 

analyses of NIST and CAP reference materials. 

 At the same time that the CAP’s Clinical Standards Laboratory and Surveys Programs were evolving, another very important program sponsored by the college was being developed. In 1962, the CAP’s Inspection and Accreditation program began conducting on-site inspections of clinical laboratories and issuing accreditation certificates to 

those laboratories that met the written standards designated to insure quality of the work 

they have provided. At approximately the same time, another historically and very important development in clinical laboratory reference systems was developing. Leaders of 

the CAP and the American Association for Clinical Chemistry (AACC), as well as other 

clinical laboratory-related organizations recognized the need for a very broad-based organization. These discussions culminated in the formation of National Committee on 

Clinical Laboratory Standards (NCCLS) as a non-profit corporation in 1968. A critically 

important aspect of this new organization was the recognition that improvement to clinical laboratory quality required a strong partnership between the professional, governmental, and IVD manufacturing sectors. NCCLS has as a major tenet that consensus building 

and a balance of industry, government, and professional representatives at all levels of 

the organization are needed for development of highest quality and workable clinical 

laboratory standards.

 The various programs of these many different organizations continued to evolve 

through the late 1960’s and 1970’s. Clinical laboratory efforts on reference methods and 

materials specifically culminated in 1977 with a conference sponsored by CDC, FDA, 

and NBS, the published proceedings of which give an excellent overview of early clinical 

laboratory standardization activities (Boutwell 1978). This conference led to the (U.S.) 

National Reference System for Clinical Chemistry (NRSCC). The NRSCC was broadened two years later to cover analytes outside of traditional clinical chemistry and was 

renamed the National Reference System for Clinical Laboratory (NRSCL). Over the ensuing twenty-three years, numerous other efforts and organizations became involved in 

improving inter-laboratory comparability of reference methods and materials. As will be 

presented by other speakers, in Europe the clinical laboratory reference system movement 

culminated with the European Union's IVD Directive in 1998. This directive made traceability of clinical laboratory results to higher-level reference methods mandatory for in 

vitro diagnostic devices in European Community countries, and is largely the reason for 

our meeting here today in Gaithersburg. 

 Besides the NRSCL, which includes only about a dozen clinical chemistry analytes, there are numerous other "reference systems" worldwide. Some are limited to providing a pooled reference material (e.g., World Health Organization reference preparations), while others focus on both reference measurement procedures and reference materials. No doubt, most highly developed reference systems are for cholesterol. It has at its 

highest level ("definitive" in NRSCL terminology) an isotope dilution-mass spectrometry 
164 

method, at its mid-level ("reference method" in NRSCL terminology) the Abell-Kendall 

extraction-based, colorimetric method, a NIST-supplied pure cholesterol "certified reference material" (CRM) and a variety of NIST-supplied serum or processed plasma-based 

"standard reference materials" (SRM). An adaptation of this cholesterol reference system 

using panels of patient specimens will be described later. Rather than trying to develop 

an exhaustive list all of the clinical laboratory reference systems, and no doubt missing 

many which individuals in the audience think are important, I thought it might be more 

useful to try to offer some of my personal observations of what factors I think have led to 

significant improvements in inter-laboratory comparability over my 23 years as a hospital-based clinical pathologist, directing progressively larger, and more diverse clinical 

laboratories. 

 I would start by saying that I believe that one of the key factors for improving 

clinical laboratory test accuracy, which is often not formally recognized as part of a clinical laboratory test "reference system," is a uniformly accepted way to measure interlaboratory variability and to make the finding publicly available for scrutiny by the laboratory community. Without a way to measure the current state-of-the-art and its improvement over time, nobody can really know how good or bad things are and whether 

they are getting better or worse. PT/EQAS programs are generally thought to fill this 

role. Certainly Sunderman and Belk’s contributions stated a massive effort to improve 

laboratory test result accuracy in the 

US. Only with publicly available 

data that is accepted as a reliable representation of comparability of actual 

patient clinical test results, are clinical 

laboratories and manufacturers pressured to improve test accuracy. The 

CAP Surveys Program and other 

similar programs worldwide certainly 

yield considerable data on the performance of clinical laboratories. I 

believe this sort of data has been extremely instrumental in driving the 

immense improvement seen since 

Sunderman and Belk’s original 1947 

report. For example, Figure 3 shows 

the performance of laboratories reporting glucose results and Figure 4, 

hemoglobin results in the CAP General Chemistry Survey within the past 

year. These results obviously represent huge improvements over the 

findings of Belk and Sunderman. 

However, there is a subtle weakness of the CAP programs and most other PT/EQAS programs, which relates to the character of the PT/EQAS samples themselves. What are 

typically distributed are samples processed or preserved in some way (e.g., lyophilized 

Figure 3. Frequency glucose results 

from a liquid whole blood control 

material from clinical laboratories 

participating in the CAP C-03 Survey in calendar year 2000. 
165 

serum, gluteraldehyde fixed red 

cell, etc.). This stabilizing process 

can lead to specimens that behave 

somewhat differently on clinical 

analyzers (but much less commonly 

in reference methods) than patient 

materials. 

 These problems have been 

termed "commutability" or "matrix 

effect" problems. The analyte most 

carefully studied for commutability/matrix effect problems is cholesterol. Figure 5 shows an example of comparing results of both 

fresh clinical serum specimens and 

CAP lyophilized Survey PT/EQAS 

samples as measured by a widely 

used, clinical laboratory method and 

the Abell-Kendall high-level reference method (Eckfeldt and Copeland 1993). If one looked only at 

the PT/EQAS samples, clinical method versus reference method results, one would come 

to the conclusion that the clinical 

laboratory method’s results were 10 % 

to 15 % below the true cholesterol 

concentration in the samples. However, looking at the fresh patient results one can see that the clinical 

method shows a very high degree of 

accuracy. There have been examples 

where manufacturers have adjusted 

their IVD devices and reagents to 

make PT/EQAS samples’ results 

comparable to reference method target values, but in doing so unwittingly 

making their results on patient samples less accurate. This situation 

leads to a potential mistrust of 

PT/EQAS results as being representative of the state-of-the-art for clinical 

test result accuracy. Commutability 

issues have created some tension between providers of PT/EQAS programs and the materials they use and 

IVD manufacturers whose clinical 

Figure 4. Frequency hemoglobin results from a liquid whole blood control material from clinical laboratories participating in the CAP HE-03 

Survey in calendar year 1999. 

Figure 5. X,Y-plots of cholesterol results by 

a clinical method and the Abell-Kendall reference method for a set of patient specimens

(•) and for five 1991 lyophilized processed 

human plasma CAP General Chemistry 

Survey specimens (t). 
166 

laboratory methods are being evaluated by such programs. The IVD manufacturers tend 

to claim that that "the problem" lies with "inferior" PT materials that do not accurately 

mimic clinical specimens, while those producing PT/EQAS materials claim it is a lack of 

"robust" clinical reagent and instrument systems. In reality, the "problem" belongs to the 

laboratory community in general. Certainly the early efforts started by Belk and Sunderman and their followers have led to major improvements. However, I believe that for 

many analytes, commutability/matrix effect issues have been a serious impediment to further improvements in inter-laboratory variability for many analytes for approximately the 

past decade. Commutability concerns raise doubts as to the reliability of the PT/EQAS 

data as being reflective to inter-laboratory comparability of patient sample results. Other 

speakers will discuss this topic in more detail. 

 Inter-laboratory comparability continued to improve during the 1970’s and 1980’s, 

but concurrently, commutability/matrix effect issues became a more and more serious 

concern. Several innovative approaches for reference systems developed that circumvented the commutability/matrix effect problem of more typical PT/EQAS materials. 

The earliest and most successful programs within North America, and probably worldwide, has been the U.S. National Heart, Lung, and Blood Institute (NHLBI)/Centers for 

Disease Control and Prevention (CDC) Lipid Standardization Program. Shortly after the 

landmark NIH-funded Coronary Primary Prevention Trial results were published showing that reducing serum cholesterol reduced coronary heart d6isease risk, major efforts 

Figure 6. Strategy for 

transferring the cholesterol accuracy base to 

manufacturers and clinical laboratories (reproduced with permission 

from Myers, et al. 1991 ). 
167 

were begun to improve clinical laboratory cholesterol test accuracy. The National Cholesterol Education Program (NCEP) was formed as a cooperative NHLBI/CDC venture. 

An integral part of the NCEP program was the NCEP Laboratory Standardization Program (NCEP LSP). I think one of the key factors that lead to the immense success of the 

NCEP was that early on it garnered strong support from both the clinical and medical associations such as the American Heart Association and American Medical Association. 

Physicians were asked by these professional medical organizations to inquire whether the 

clinical laboratory that they used had NCEP Lipid Standardization (NCEP LSP) certification for lipid testing (Myers et al. 1991). 

Early on, the NCEP LSP recognized the limitations of using lyophilized reference materials for calibration and PT/EQAS purposes. Two different formats of the program were 

designed. One distributed frozen serum pools that were prepared at CDC and shipped to 

laboratories, primarily those involved in NIH funded research projects on a periodic basis. These pools were frozen quickly, never lyophilized, and stored at -70 oC. As a result, they tended to behave much more like fresh clinical specimens in the field methods 

than the typically lyophilized reference materials for lipids that were available in the 

early 1960's. Abell-Kendall cholesterol, as well as high-level reference method HDLcholesterol and triglyceride values, served as the top of the traceability chain for the 

NCEP LSP. In a slightly different format offered to any manufacturer of cholesterolmeasuring IVDs and to clinical laboratories, fresh patient sample analysis by CDCcertified Network Laboratories became available to document the accuracy of the clinical 

test results. The major shortcoming of the program, particularly for clinical laboratories, 

was the significant expense involved. As a result, although most major manufacturers of 

cholesterol-measuring reagents and instruments became NCEP certified, only a very 

small percentage of clinical laboratories ever went through the process. A parallel initiative begun in the mid-1990 to improve the traceability of lipid values in clinical laboratories in a more cost-effective way was begun by NCCLS, CAP, and NIST. The project's 

aim was to develop a reliable method for producing frozen serum reference materials for 

lipids that was free of commutability/matrix problems. A summary of this project was 

recently published (Myers et al., 1999). 

 A more recent example of another analyte standardization effort that was largely 

motivated by clinical professional society demands is the use of glycohemoglobin. 

Shortly after the publication of the NIH-sponsored Diabetes Control and Complications 

Trial (DCCT) study results in 1993, hemoglobin A1c or glycohemoglobin became a pivotal clinical laboratory test predicting the risk of diabetic complications and managing 

diabetic care of individual patients. The clinical diabetes societies, principally the 

American Diabetes Association (ADA), began proposing numerical glycohemoglobin 

target values for diabetic care, without regard for the method-specific reference ranges 

being used by different clinical laboratories performing the test. Suddenly, interlaboratory comparability of test results became a major issue. In 1994, AACC chartered 

a subcommittee of its Standards Committee to develop a standardization program to assure that clinical laboratories would report glycohemoglobin values equivalent to the 

DCCT method. This subcommittee evolved eventually into the National Glycohemoglobin Standardization Program (NGSP). In the mid-1990's, NCCLS began developing a 
168 

guideline document for this process, which is now nearing finalization. Not only was the 

ADA and AMA giving strong support for reporting NGSP-certified (i.e., DCCTequivalent) results, but also the CAP became involved through its Glycohemoglobin Survey, which began in the early 1990’s. However, this initial Survey (GH Survey) used 

lyophilized samples. Fortunately, it was quickly realized that these materials suffered 

from commutability/matrix effect problems quite analogous to the cholesterol lyophilized 

PT/EQAS materials. In 1996, to circumvent the commutability/matrix effect problems, 

the CAP initiated a new fresh whole blood proficiency testing program (GH2 Survey), 

where blood was collected from diabetics and shipped in a liquid form to clinical laboratories so that it could be tested within a few days of collection. This GH2 Survey allowed 

for direct comparison of various methods that were used in clinical laboratories. By the 

year 2000, glycohemoglobin results reported in the GH2 Survey have seen a greater than 

2-fold reduction in the variability of the method-specific glycohemoglobin results. This 

improvement was possible only through close cooperation of the manufacturers of IVDs 

for glycohemoglobin and the NGSP. I believe that this program's great success resulted 

not only from the previously mentioned important ability to measure and publicly display 

specific methods results on samples that very closely mimic clinical specimens, but there 

was another very important factor. What it had was large, very influential clinical organizations (i.e., the ADA and AMA) that formally and strongly endorsed the need for 

comparable patient results, regardless what IVD manufacturer instruments and reagents 

were being used. 

 Another example of a program aimed at improving inter-laboratory comparability, 

with what I personally view as having had much more limited success is the reference 

system for prostate-specific antigen (PSA). The use of PSA in monitoring and more recently diagnosing prostate cancer goes back about 15 years. However, many reports including the CAP Survey data and others, had shown inter-assay variability of as much as 

two-fold for some samples in the clinically important 4 to 15 ng/mL range. In the mid1990’s there was even a recommendation that the change of PSA, over 2 to 4 years, 

should be used for diagnosis of prostate cancer. This recommendation made interlaboratory variability even more problematic. In an era where patients often change 

health care plans and laboratories every few years, a large change in PSA might be due 

totally to analytical difference in the methods used by two labs, or vice versa, and they 

may mask clinically important changes. Several PSA standardizations conferences were 

held and recommendation for a calibrator with a fixed ratio of free and bound PSA were 

made. The proposal to use free and bound mixtures of PSA in the reference materials 

were based on the fact that some assays were much more sensitive to free PSA than to 

alpha-1-antichymotrypsin bound PSA. The problem was magnified by the fact that many 

of the PT/EQAS materials used semen added to normal human plasma leading to a very 

atypical bound-to-free ratio giving widely discordant results using different clinical 

methods that were not representative of what was seen when clinical specimens were 

analyzed. This well-known problem of PT/EQAS materials not being representative of 

results on clinical specimens, severely hampered progress in improving the inter-method 

variability. In the September 2000 issue of Clinical Chemistry , the "first international 

standards" for PSA are described (Rafferty et al., 2000). However, the serious limitations 

of even these reference materials to reduce inter-method variability without manufactur-
169 

ers significantly changing their method design, was pointed out in an accompanying editorial (Chan and Sokoll, 2000). How this attempt to improve inter-laboratory comparability of a complex protein that is present in different forms in clinical samples succeeds 

is yet to be seen. I would emphasize that I give the PSA standardization example not to 

cast any aspersions on those involved in these efforts, but to highlight the extreme complexity of trying to develop traceability of an analyte that is present in different molecular 

forms in patient materials. Standardization of cardiac troponin methods offers very similar challenges. 

 One could go on describing other clinical laboratory standardization efforts, but I 

believe the examples already presented are reasonably representative of the problems 

faced. It might be instructive to speculate why some clinical laboratory analyte traceability programs have been reasonably successful, and others less so. My observations are 

that key factors leading to improvement in inter-laboratory variability include: 

1) Results of a widely publicized clinical research study comes to the conclusion that 

clinical action should be based on uniform cut-points of some laboratory test and this 

leads to strong pressure from large clinical organizations, 

2) A high-level reference method and/or material must exist or must be developed, 

3) There must be a way to easily and reliably disseminate this accuracy base provided by 

the reference method or material, 

4) There must be a way to reliably evaluate and publicly display inter-method and interlaboratory comparability results. 

Examples of key factor #1 are the CPPT study results and NCEP clinical cut-points as 

well as massive "know your number" (i.e., serum total cholesterol) campaign by the 

NCEP, the DCCT and ADA glycohemoglobin cut-points for good, marginal, and poor 

control of diabetics. The fact that several studies showing utility of serum PSA testing 

have not led to comparable improvement in inter-method variability of PSA results, suggests that key factor #1 while perhaps necessary, is really not sufficient. Traceability 

schemes typically have at the top of the hierarchy a "high-level" reference material. Even 

when there is a high-level reference method or a material as listed in key factor #2 at the 

top of the chain, the major limitation in practical distribution of this accuracy base to the 

field is the lack of commutable reference materials (key factor #3). The NGSP glycohemoglobin and NHLBI/CDC Lipid Standardization Program circumvented this problem 

by direct analysis of patient samples as part of the system. Efforts such as the 

NCCLS/CAP/NIST frozen serum reference material for cholesterol is an example of efforts to address key factor #3. Finally with respect to key factor #4, unless we know how 

good or bad inter-method and inter-laboratory variability are, little progress is typically 

made. Here is where PT/EQAS programs that use commutable reference materials are 

needed. 
170 

 I believe that prior to starting the development or improvement of a reference system, looking at the scattergram comparisons of field method results versus high-level reference method results, using a number of clinical samples from normal healthy subjects 

as well as samples from patients with the diseases the test designed to manage or diagnose, is extremely useful. If all the clinical field methods show very high degrees of correlation (i.e., “r” approaching 1.0 and Sy.x not much larger than the intra-assay SD of the 

methods), the problem is relatively straightforward. Recalibration with some or all of the 

field methods with a commutable calibration material should be relatively easy. However, if the correlation is not good (i.e., r<<1.0 and Sy.x much larger than the intra-assay 

SD of the method) the problem is much more complicated. Recalibration may improve 

the traceability on average, but it will not correct the inter-method comparability of many 

of an individual patient specimen. While recalibration may give the impression that 

traceability has improved, for that individual patient whose result is still two-fold different with one clinical method compared to another, there is still a very serious problem 

when his doctor tries to develop a rational diagnosis or treatment program. This situation 

is still fundamentally where PSA and many other protein/peptide hormones as well as 

several chemically well-define hormone assays (e.g., cortisol) stand. These analytes, 

unfortunately, are those that are most problematic for the clinical laboratories and the clinicians that use them. Frankly, clinical practitioners largely lost interest in interlaboratory variability of sodium and cholesterol more than a decade ago. 

 I believe it is worth examining the reasons for poor correlation between clinical 

field method results versus reference method results more carefully. When results are not 

highly correlated, there are basically two possible causes. For analytes that are well defined chemically, it is likely that the field method has significant analytical interferences 

from substances found in varying concentrations in normal clinical samples. These interfering substances are often not found in reference materials, especially those prepared by 

adding purified preparations of the analyte of interest to normal pooled samples. An example of this situation would be cortisol and other steroid hormones. For many nonextraction or non-chromatographic assays, interfering steroids with similar chemical 

structures can be found at reasonably high concentrations in samples from patients, particularly in those with the very disorders that clinicians are trying to diagnose. Calibrators with increasing amounts of cortisol prepared by adding pure cortisol to pooled normal serum, or even worse an aqueous-based calibration solution, hardly mimic clinical 

samples. Nevertheless, the problem with chemically well-defined analytes is solvable, 

although it may require major changes in the fundamental design of the clinical methods 

to make them analytically more specific. 

 The second general cause that leads to poor correlation between the field and reference methods ((i.e., r<<1.0 and Sy.x much larger than the intra-assay SD of the method) 

is that the analyte itself not well defined or is present in multiple molecular forms that are 

heterogeneously distributed. Examples of this situation are PSA, troponin, and chorionic 

gonadotropin. All of these proteins exist in multiple molecular forms in patient samples. 

Different immunoassays recognizing different epitopes present on the various molecular 

forms. Thereby, the different field methods yield different quantitative results depending 

on what forms are present and in what relative amounts they are present. In such a situa-
171 

tion, it is not at all clear, at least to me, what the approach should be for improving interlaboratory comparability. First, it seems that there must be some agreement on what is 

the clinically most appropriate "analyte" to measure through some national or international consensus process. However, the logistical difficulties and economic incentives 

become real barriers to progress. Various manufacturers often contend that their method 

is clinically superior to others, because it gives different answers. Only with relatively 

large and expensive clinical outcome studies can the question of superiority or inferiority 

of various assay designs be made. In my experience, such studies are extremely difficult 

to design with adequate statistical power and in an unbiased manner. Unfortunately, some 

of the most problematic analytes tend to fall into this category that is very difficult to 

solve. 

 In summary, establishing an on-going documentation of the traceability of field 

methods to reference methods is both difficult and expensive. Clinical studies must 

document a clinical benefit of treating a uniform numerical laboratory result or diagnosis 

or management of a disease process. There must be a widely perceived clinical payback 

for the reduction of inter-method and inter-laboratory variability for the clinical laboratory and manufacturing community to make it a priority. There must be close cooperation of the users, producers, and regulators of laboratory testing for the effort to be a success. Finally, a poorly defined or heterogeneous clinical analyte creates a special problem. Successful inter-laboratory standardization for analytes such as PSA and troponin 

will take new and innovative approaches to successfully improve inter-laboratory comparability. Hopefully, this workshop will provide a sound foundation and workable ideas 

leading to major progress in these efforts. 
172 

References 

1. Belk WP, Sunderman W. A survey of the accuracy of chemical analyses in clinical 

laboratories. Am J Clin Pathol 1947;17:854-61. 

2. Boutwell JH, Editor. A national understanding for the development of reference materials and methods in clinical chemistry: proceedings of a conference. Washington, 

DC: American Association for Clinical Chemistry, 1978. 

3. Chan DW, Sokoll LJ. WHO first international standards for prostate-specific antigen: 

the beginning of the end for assay discrepancies? Clin Chem 2000;46:1291-2 Editorial. 

4. Eckfeldt JH, Copeland KR. Accuracy verification and identification of matrix effects: 

the College of American Pathologists’ Protocol. Arch Pathol Lab Med 

1993;117:381-6. 

5. Myers GL, Eckfeldt JH, Greenberg N, Levine JB, Miller WG, Wiebe DA. Preparation 

and validation of commutable frozen human serum pools as secondary reference 

materials for cholesterol measurement procedures; Approved Guideline. NCCLS 

Document C37-A. Wayne, PA: NCCLS, 1999. 

6. Myers GL, Henderson LO, Cooper GR, Hassemer DJ. Standardization of lipid and 

lipoprotein methods In: Rifai N, Warnick GR , Eds. Methods for clinical laboratory measurement of lipid and lipoprotein risk factors. Washington, DC: AACC 

Press, 1991:101-25. 

7. Rafferty B, Rigsby P, Rose M, Stamey T, Das RG. Reference reagents for prostatespecific antigen (PSA): establishment of the first international standards for free 

PSA and PSA (90:10). Clin Chem 2000:46:1310-7. 
173 

JOHN H. ECKFELDT 

University of Minnesota Medical School 

Department of Laboratory Medicine and Pathology 

John H. Eckfeldt, M.D., Ph.D., is currently a Professor in the Department of Laboratory 

Medicine and Pathology in the University of Minnesota’s Medical School having served 

on the University’s faculty since 1978. His clinical responsibilities include the medical 

direction of the Clinical Laboratories at the Fairview University Medical Center in Minneapolis that is the tertiary care hospital for the Fairview Health System, a seven hospital and thirty clinic health care system which provides care for approximately one-half 

million individuals in Minnesota. He holds a masters degree in physical chemistry from 

Cornell University and an M.D. and Ph.D. in biophysics from the University of Pennsylvania. His primary research interests include risk assessment for cardiovascular disease, hypertension, and iron overload disorders using traditional blood and body fluid 

analyses as well as genetic testing. He has a long-term interest in proficiency testing and 

clinical laboratory method standardization. In the past he has served as a member of the 

College of American Pathologists’ Chemistry and Therapeutic Drug Monitoring and Endocrinology Resource Committees. Presently, he is a member of the College’s Biochemical and Molecular Genetics Resource Committee and chairs their Standards Committee. 

He has also served on the American Association for Clinical Chemistry’s Standards 

Committee since the early 1990’s. Since 1995, he has also served on Working Group 2 

on reference methods and materials of ISO Technical Committee 212. 
174 
175 

History of Reference Systems for Clinical Measurements 

Visuals Presented By 

John H. Eckfeldt 

Department of Laboratory Medicine and Pathology 

University of Minnesota 

Clinical Pathology, Fairview-University Medical Center 
176 

History of Reference Systems for 

Clinical Measurements

John H. Eckfeldt, MD, PhD

Medical Director, Clinical Pathology, 

Fairview-University Medical Center

Professor, Department of Laboratory Medicine and 

Pathology

University of Minnesota

Topics To Be Addressed

• Historical view of traceability efforts with an 

emphasis on US and CAP experiences, 

• Role of PT for assessing traceability/trueness and its 

shortcomings,

• Two examples of working systems within the US to 

establish traceability of important clinical analytes

(NHLBI/CDC Cholesterol Reference Method 

Laboratory Network and National Glycohemoglobin 

Standardization Program),

• Example of program with very limited success (PSA)

• Characteristics of “successful” programs to establish 

traceabilty.
177 

Milestones in US Inter-laboratory 

Comparability Improvement

1947 Belk & Sunderman’s study of clinical 

laboratory chemistry analytes in Pennsylvania 

and surronding states,

1949 CAP Surveys Program begin,

1951 CAP “Standards Solutions” begin,

1961 CAP Clinical Standards Laboratory instituted 

for cyanmethemoglobin,

1977 CDC/FDA/NBS conference on reference 

methods and material --> NRSCC and NRSCL

1998 EC IVD Directive issued 

Eras of Traceability

1940’s and 1950’s discovering how bad things are,

1950’s and early 1960’s better reference methods 

for clinical analytes begin to be developed,

1970’s and 1980’s develop wider variety and 

availability of reference methods and materials 

for clinical analytes, 

1990’s better understanding of matrix effect/ noncommutability problems--more commutable 

reference materials are developed for several 

analytes,

2000’s develop better inter-laboratory traceability

for clinically important analytes???
178 

1947 Glucose PT Results

(Belk and Sunderman. Am J Clin Pathol 1947;17:853-61)

2000 Glucose PT Results

CAP Survey C-03 2000 Glucose

0

500

1000

1500

2000

2500

30 38 46 54 62 70 78 86 94 102 110 118

[Glucose], mg/dL

Frequency
179 

Hemoglobin Proficiency Testing Results

(Belk and Sunderman. Am J Clin Pathol 1947;17:853-61)

1999 Hemoglobin PT Results

1999 CAP HE-03 Survey

0

50

100

150

200

250

7.9

8.2

8.5

8.8

9.1

9.4

9.7

10

10.3

10.6

10.9

He mog lobin, g /dL

Frequency
180 

CAP Surveys PT/EQAS Program

• Evolved from Belk and Sunderman’s 59 labs with 24 

samples testing performance for seven analytes 

distributed in 1947 to >25,000 labs with >250,000 

samples testing >500 analytes in 2000

• Liquid, lyophilized, frozen serum, plasma, whole 

blood, urine, CSF samples sent from 2 to 4 times/yr

• Strengths:

– Largest clinical laboratory performance data base 

in the world

– Largest number of analytes of any PT program

• Weaknesses:

– Large dependence on “peer group” comparisons

Limitations of Lyophilized Reference 

Materials as “Trueness Controls”

and Calibrators

• Non-commutable “trueness controls” can lead to 

erroneous conclusions as to accuracy of a method in 

a PT/EQAS situation,

• Non-commutable calibration materials can lead to 

erroneous patient values if the high-level reference 

method value is used to calibrate a clinical 

instrument. 
181 

Eckfeldt and Copeland. Arch Pathol Lab Med 1993;117:381-6.

Example #1: NHLBI/CDC Lipid 

Standardization Program

• Cholesterol Coronary Primary Prevention Trial and 

shortly thereafter numerous other research studies 

publish results that reducing serum cholesterol 

reduces CHD risk,

• NCEP is formed with CDC/NHLBI Laboratory 

Standardization Program formed as an integral 

part,

• NCEP gets strong support from American Heart 

Association, AMA, etc.

• Shortcomings of lyophilized reference materials 

recognized early as calibrator and PT samples,
182 

Example #1: NHLBI/CDC Lipid 

Standardization Program (Continued)

• Frozen serum pools and fresh patient sample 

comparisons used for accuracy transfer,

• Manufacturers and clinical laboratory certification 

based on sound statistical principles,

• Shortcoming: expensive and time consuming for 

clinical lab participation,

• Led to development of a NCCLS/CAP/NIST frozenserum reference material for lipids (cholesterol, 

triglyceride, HDL-cholesterol, LDL-cholesterol) to 

assist IVD manufacturers calibrate their methods 

better.

Example #2: National Glycohemoglobin

Standardization Program

• In 1993, DCCT and later other research group 

publish results that reducing average blood 

glucose reduces risk of diabetic complications,

• In 1994, AACC Subcommittee of Standards 

Committee begins to develop a standardization 

program for clinical laboratories to enable them 

to report “DCCT-equivalent”

• AACC Subcommittee evolves into the National 

Glycohemoglobin Standardization Program 

(NGSP) and NCCLS draft guideline document,
183 

Example #2: National Glycohemoglobin

Standardization Program (Continued)

• American Diabetes Association and AMA give 

strong support of reporting NGSP-certified (i.e., 

DCCT-equivalent) results,

• In 1997, CAP begins fresh whole blood PT program 

(GH2 Survey) which directly shows comparability 

of field methods to reference methods,

• By 2000, there is a more than two-fold reduction in 

inter-method variability of commercial 

glycohemoglobin results in the CAP Surveys.

Example #3: PSA Standardiztion 

Program

• In the early 1990’s, several studies document the 

usefulness of PSA in tracking prostate cancer 

treatment and to some degree diagnosis in 

combination with digital rectal examination,

• Inter-assay variability is >2-fold for some samples 

in the 4 to 10 ng/mL range,

• In the mid-1990’s, change in PSA over 2 to 4 years 

proposed as more indicative of prostate cancer than 

a single value,

• In the mid-1990’s, the first and second “Stanford 

Conferences” recommend “90:10 calibrator,”
184 

Example #3: PSA Standardiztion 

Program (Continued)

• At the urging of the American Cancer Society, in 

1993, CAP develops a frozen serum-based 

reference material,

• In the late 1990’s free/bound PSA ratios are 

proposed as giving additional diagnostic 

information,

• In 2000, there has been little change in inter-assay 

variability of the commercially available total PSA 

assays.

Why Are Some Traceability Programs 

Successful and Others Not?

• Clinical research study results recommend clinical 

action based on uniform cut-points nationally or 

internationally,

• Strong clinical consensus on treatment or diagnosis 

based on these uniform cut-points was developed,

• A well accepted “higher level” reference method 

exist or are developed to which the clinical study can 

be traced,

• Reference materials are commutable across the 

reference and field methods OR traceability schemes 

using clinical samples are devised and used,
185 

Why Are Some Traceability Programs 

Successful and Others Not? (Continued)

• Although different field methods may be biased 

compared to reference method, they are 

reasonably well correlated to it (i.e., r ~ 1.0 and

Sy.x is the same order of magnitude as analytical 

SD of the methods),

• “Analyte” that is clinically important is 

reasonably well defined

Outcome “A” When Comparing Field

Method to Reference Method Results

0

10

20

30

0 10 20 30 40 50

"Re fe rence" M ethod Results

Field Me thod Results
186 

Outcome “B” When Comparing Field

Method to Reference Method Results

0

10

20

30

0 10 20 30

"Reference" Method Results

Field Method Results

Outcomes When Comparing Field

Method to Reference Method Results

• Outcome A: Results are highly correlated (i.e., r close to 

one, S y.x approximates analytical CV’s of methods)

• Outcome B: Results are not highly correlated (i.e., r 

much less than one, S y.x much larger than analytical 

CV’s of methods)

– Outcome B1: Analyte is well defined, but field method 

has significant analytical interferences from substances 

normally found in typical clinical samples (e.g., 

cortisol)

– Outcome B2: Analyte itself is not well defined or 

heterogeneous (e.g., troponin, PSA, HCG, etc.)
187 

Solutions to Traceability Problems

• Outcome A: Relatively easy to solve by 

recalibration,

• Outcome B1 (well defined analytes): Major 

measurement procedure redesign needed to 

minimize impact of “influence quantities”--for 

example, change to more selective primary 

antibody); change to more selective analytical 

methods (e.g., chromatography, mass 

spectrometry, etc.).

Solutions to Traceability Problems

(Continued)

• Outcome B2 (poorly defined analytes): 

– First there needs to be agreement on what is 

clinically most appropriate “analyte” to measure 

through national/international consensus 

process,

– Potentially major redesign of some or all field 

methods--e.g., PSA, HCG, troponin, etc.
188 

Summary

• Establishing and on-going documentation of 

traceability in field methods is both difficult and 

expensive,

• Therefore, there must be clinical pay-back for 

this to become a clinical laboratory priority,

• Clinical studies must document clinical benefit of 

treating a numerical lab result,

• With poorly defined or heterogeneous clinical

analytes even clinical studies suggesting treating a 

lab number are not in and of themselves sufficient 

to lead to successful inter-laboratory 

standardization (e.g., PSA).
189

NIST Measurement Methods and Standards Reference 

Materials for Health Status Markers: 

Current Program and Future Challenges 

Willie E. May 

National Institute of Standards and Technology 

About NIST 

NIST is an agency of the 

U.S. Department of Commerce’s Technology Administration. NIST’s seven 

discipline-based Measurement and Standards Laboratories work at all stages of the pipeline from advancing 

basic science and pioneering new measurement methods to the development of standard 

test methods, materials, and data to ensure the quality of commercial products. The mission of the NIST Laboratories is to promote the U.S. economy and public welfare by providing technical leadership for the Nation’s measurements and standards infrastructure, 

and assure the availability of essential reference data and measurement capabilities. 

A 1999 study by 

the National 

Academy of 

Sciences Institute of Medicine 

estimates that up 

to 98,000 people 

die annually in 

the United 

States due to 

medical errors. 

While many 

medical errors 

are not due to 

inaccurate measurements, improved measurement accuracy could save lives as well as 

time and money. In addition, International standards organizations are beginning to require traceability of the accuracy of medical measurements to national standards. For example, since the European Union now requires that certain diagnostic medical devices be 

“traceable to standards of a higher order”, we must develop internationally accepted reference methods and certified reference materials to meet this requirement. NIST’s mission is to provide those standards. 

Why NIST and Healthcare? Why NIST and Healthcare?

Problem Magnitude and Scope:

• U.S. Spends ~ $1.5 trillion on Health Care (14% of GDP) (per year)

– ~13% of this amount is associated with measurement ($140B)

– Non-diagnostic measurements cost ~$39B 

– Improvement in accuracy of one marker alone (cholesterol) is 

estimated to have saved $100M per year

New Driver: EU IVD Directive to go into effect 2003

• Worldwide in vitro diagnostic device market is ~$20B;

• >60% of European market is supplied by U.S. based companies
190

Maintaining World-Class 

Metrologically-based Competencies in 

Chemical Measurement Science:

• Analytical Mass Spectrometry

• Analytical Separation Science

• Atomic and Molecular Spectroscopy

• Chemical Sensing Technology

• Classical and Electroanalytical Methods

• Gas Metrology

• Nuclear Analytical Methods

• Microanalytical Technologies

Primary and 

Reference 

Methods

of Both Simple 

and Complex 

Matrices

NIST Analytical Chemistry Division NIST Analytical Chemistry Division

Serves as the Nation’s Reference Laboratory for

Chemical Compositional Measurements

The Analytical Chemistry 

Division is one of five divisions 

in NIST’s Chemical Science and 

Technology Laboratory. Core 

Competencies are maintained in 

a wide range of analytical 

technology areas. The skills and 

knowledge gained from research 

in these areas support the 

development of the internationally recognized reference 

methods and reference materials 

that NIST provides to facilitate 

“traceable” measurements in a 

wide range of application areas from healthcare and food authenticity, nutritional value 

and contamination to chemical monitoring and open path sensing of stack emissions. 

NIST provides about 840 certified 

reference materials that support 

chemical measurement processes; 

those certified for chemical composition and used for instrument 

and/or method calibration, and 

those not certified for chemical 

composition per se, but extensively used to support chemical 

measurements and processes. 

There are also natural samples/matrices with both macroscopic and often trace constituents 

value assigned for use in validating 

the accuracy of chemical measurement methods and processes. 

NIST/CSTL works closely 

with the American Association for Clinical Chemistry 

(AACC), the Centers for 

Disease Control and Prevention (CDC), NCCLS, 

and other organizations interested in health-related 

standards to help prioritize 

our standards activities and 

the development of SRMs. 

NIST’s Chemical Science 

8 IVD Traceability Workshop – 11/00

NIST Standards for Chemical Measurements NIST Standards for Chemical Measurements

constitute 840 of 1,400 NIST SRM Types 

and were 18,361 of 33,347 NIST SRM Units Sold in FY99

• High Purity Neat Chemicals

• Organic Calibration Solutions

• Inorganic Calibration Solutions

• Gas Mixture Standards

• Complex Matrix Standards

- Advanced Materials

- Biological Fluids/Tissues

- Foods/Botanicals

- Geologicals

- Metal Alloys

- Petroleum/Fossil Fuels

- Sediments/Soils/Particulates

• Optical Filter Standards 

• Conductivity Standards 

• Ion Activity Standards

NIST’s Role in Health Care Measurements

NIST

High Accuracy Standard Reference Materials, New Measurement

Methods Data and Other QA Services Tools

Health Care Delivery

Improved Measurement Accuracy Better Diagnoses and Reduced Costs

NIH CDC

FDA

AACC AOAC

CAP NCCLS

Medical/Clinical

Manufacturers

Medical Research

Community
191

and Technology Laboratory serves as the US Reference Laboratory for heath related 

chemical measurements providing high accuracy measurement methods and Standard 

Reference Materials. CSTL is also involved in measurement QA programs and international comparison exercises with other National Metrology Institutes. 

NIST has provided a variety 

of Pure and Artifact Standard 

Reference Materials (SRMs) 

for the clinical community 

for nearly 90 years. First 

released in 1914, SRM 41c is 

still available as is SRM 

915a, and many of the 

standards produced with NIH 

support. At the request of 

clinical laboratories, the pure 

materials have been 

supplemented with the 

analyte of interest in serum, 

for more reliable quality 

control and calibration of 

clinical methods. 

Since the 1980s, NIST 

has developed and refined “Definitive Methods” for several health 

status markers to support 

the national reference 

system for clinical measurements: NIST definitive methods for the 

health status indicators 

have been used to valueassign SRMs and highpriority serum pools used 

to serve as the anchor 

point for CDC developed 

reference methods and by 

the College American 

Pathologists for proficiency testing of more than 20,000 U.S. clinical laboratories. 

History of Clinical Standards Program History of Clinical Standards Program

- Pure Materials & Artifacts (29) Pure Materials & Artifacts (29)

• 1914 SRM 41, Dextrose

• 1945 SRM 1861 & 186 II, phosphates for pH

• 1967 SRM 911, Cholesterol

• 1968 SRM 912, Urea

SRM 913, Uric Acid

SRM 914, Creatinine

SRM 915, Calcium Carbonate

• 1970’s 21 SRMs with NIH support

URL: HTTP://ts.nist.gov/srm 

E Mail: SRMINFO@nist.gov 

National Reference System for Clinical Chemistry

- Hierarchy of Methods for Cholesterol

• Definitive (NIST)

– Highest accuracy and precision

– Thoroughly tested for bias

– Generally not within the capability of clinical laboratories

– Used for primary RMs and validation of reference methods

• Reference (CDC Network)

– Carefully tested vs definitive method; high accuracy and 

precision

– Within capability of most clinical labs, but too time consuming

– For secondary RMs and validation of field methods

• Field (Clinical Labs)

– Routine clinical use

– Sufficient accuracy and precision for reliable diagnosis

– Simple, rugged and cost-effective
192

Cholesterol Reference System - the CDC Network: Improved accuracy facilitated 

by this program has led to 

better diagnosis, treatment 

and reduced health-care 

costs. 

NIST maintains 

and refines definitive methods 

for the twelve 

health status 

markers shown 

in the box to the 

left. Classical 

primary methods are utilized 

as well as isotope dilution 

mass 

spectrometry 

(IDMS). 

Over the last three decades, NIST, in cooperation with the College of American Pathologists (CAP) has developed a series of highly accurate and precise methods, isotope dilution mass spectrometry (IDMS) for a number of clinically important serum constituents, 

including cholesterol (see figure above). These IDMS methods are recognized by the international clinical laboratory community as "definitive" 

The illustration is a schematic representation of ID/MS methodology for organic constituents in a complex matrix. It includes the addition of the isotopically-labeled analog of 

the species to be measured for use as an internal standard, followed by wet chemical procedures for isolation of the analyte and isotopically-labeled analog from the matrix prior 

CDC

Network Labs

NIST/CSTL

Manufacturers

Hospitals and Clinical Laboratories

(thousands)

Certification and QC

Instruments and Reagents Certification and QC

SRMs and Definitive Method

Certification and QC

SRMs and other measurement 

services

A/K Reference Method

Cholesterol Reference System

Definitive Methods for Clinical Definitive Methods for Clinical Analytes

ANALYTE METHOD CONDITION

Calcium ID/MS Cancer, Blood Clotting

Chloride ID/MS, Coulometry Kidney Function

Cholesterol ID/MS Heart Disease

Creatinine ID/MS Kidney Function

Glucose ID/MS Diabetes

Lithium ID/MS Antipsychotic treatment

Magnesium ID/MS Heart Disease

Potassium ID/MS Electrolyte Balance

Sodium Gravimetry, ICP/MS Electrolyte Balance

Triglycerides ID/MS Heart Disease

Urea ID/MS Kidney Function

Uric Acid ID/MS Gout
193

to separation of the analyte species (and coeluting isotope diluent) in 

time from other species 

co-extracted from the matrix via gas chromatography (GC), before introduction into the mass 

spectrometer for determining the relative 

amounts of analyte and 

isotope diluent in the 

sample. Since the absolute of amount of isotope 

diluent is known, the 

concentration of the analyte can be calculated. 

This general approach is 

used for all small molecule organic species, however the wet chemical procedures used 

for assuring that the isotope diluent fully equilibrates with the matrix, for isolation of the 

analyte from the matrix, and for conversion of the analyte to a stable, volatile derivative 

for GC, must be developed specifically for each individual health status marker. 

Typical precisions of the ID-GC/MS definitive methods are shown in the diagram 

Intensity Ratio Measurements

56'

0

-3.0

3.0

10 10 10 10 10 10 10 -9 -8 -7 -6 -5 -4 -3

56'

0

-1.0

1.0

10 10 10 10 10 10 10 -9 -8 -7 -6 -5 -4 -3

Mass Fraction 

Measured Mass Fractions 

of Independently Prepared Samples 

PRECISION OF ID-GC/MS

ility Workshop – 11/00

Schematic representation of 

ID/MS methodology for organic 

constituents in a complex matrix 
194

An example of the 

repeatability of 

the ID/MS procedure for cholesterol over a period 

of time is shown 

in the graph to the 

left. 

A history of serum matrix SRMs available 

from 1979 to the present is listed in the figure. SRMs provide an 

accuracy base for the 

clinical laboratory system. 

The following are currently available 

through NIST’s Standard Reference Material Program: 

SRMs 900, 909b, 956a, 965, 968c, and 966; 

in addition to SRMs 1951a, and 1952a. 

Day 1 - Day 2 Range and Mean of Cholesterol 

Measurements in Serum by ID/MS

6.230

6.240

6.250

6.260

6.270

6.280

6.290

I-1I-2I-3

II-4II-5II-6

III-7III-8III-9

Overall

Set/Sample

mmol/L

History of Clinical Standards Program

- Selected Serum Matrix Materials

• 1979 SRM 900, Antiepilepsy Drug Level Assay Standard

• 1980 SRM 909, Human Serum 

• 1982 SRM 1599, Anticonvulsant Drug Level Assay Standard

• 1988 SRM 1951, Cholesterol in Human Serum (Frozen)

SRM 1952, Cholesterol in Human Serum (Freeze Dried)

• 1989 SRM 968, Fat Soluble Vitamins in Human Serum

• 1991 SRM 956, Electrolytes in Blood

• 1996 SRM 956a, Electrolytes in Frozen Human Serum

• 1996 SRM 965, Glucose in Frozen Human Serum (3 levels)

• 1997 SRM 1951a, Lipids in Frozen Human Serum (2 levels)

• 2000 SRM 966, Toxic Metals in Blood

SRM 2070a, Toxic Elements in Human Urine
195

To the left is an application in 

which calibration control 

would be of benefit. Notice 

that values determined by the 

clinical laboratory on four 

patient samples are all biased 

by a similar amount versus 

the values obtained by NIST 

using the definitive method. 

In three out of the four cases, 

the results provided by the 

clinical laboratory would 

have led to incorrect assessment of heart disease risk 

based on the guidelines of the 

National Cholesterol Education Panel and perhaps, unnecessary medical costs. 

In the area of cholesterol measurements 

alone, it has been estimated that measurement uncertainty was 

on the order of ± 18 % 

relative in 1969, before 

any reference materials 

were available. The 

first pure crystalline 

cholesterol (SRM 911) 

was introduced in 1967. 

Using the definitive 

method, serum cholesterol SRMs were developed in 1981 (SRM 

909) and again in 1988 

(SRMs 1951 and 1952). These SRMs have contributed to a steady decrease in the number 

of false positives and negatives resulting from clinical laboratory results for cholesterol in 

blood to between ± 5.5 % to 7.2 %, relative. 

NIST Contributions to

National Reference

System for Cholesterol

Measurement Improvement 1969 - Present

May Save $100M/year in Treatment Costs

1967 - SRM 911

Pure Cholesterol

1980 - Definitive Method for

Serum Cholesterol

1981 - SRM 909 Cholesterol in

Human Serum

1988 - SRMs 1951 & 1952

Cholesterol in Serum

1994 - Definitive Method for 

Serum Triglycerides

1996-7 - Values for HDL & LDL 

Cholesterol

1949 23.7%

1969 18.5%

1980 11.1%

1986 6.4%

1990 -1994 5.5 - 7.2%*

Unnecessary

Treatments

Untreated

Disease

Correct Value

False

Positives

False

Negatives

Death Wasted $$$ Wasted $$$

* Data from

GAO/PEMD-95-8

Improved Cholesterol Measurement Accuracy Saves Health Care Dollars

Cholesterol Measurements Cholesterol Measurements

Cholesterol in Blood Experiment Cholesterol in Blood Experiment -

Impact of Inaccurate Measurements Impact of Inaccurate Measurements

170

180

190

200

210

220

230

240

250

260

270

012345

Patient

mg/dL

Outside Lab

NIST Result

Watch Diet

Take Drugs
196

NIST conducts privately contracted, formal economic of 

various programs to assess 

the impact of its work. The 

latest impact study quantifies 

a portion of the economic 

benefits associated with the 

Cholesterol Standard reference materials beginning in 

1986. The economic consequences of NIST’s Cholesterol Standards Program are 

experienced at several levels 

of the supply chain from 

manufacturers, to network laboratories, to clinical laboratories that ultimately deliver 

medical services to the consumer. The benefits to industry resulting from the NIST investment have changed over the more than three decades of NIST involvement. However, this analysis timeframe was limited to 1986-1999 covering only part of the program’s life cycle, thus biasing the measured impacts downward. The results indicate that 

NIST has played an important economic role in support of a national effort to monitor, 

measure, and control cholesterol levels, thereby contributing to the reduced level of heart 

disease. This economic impact study estimates a benefit-to-cost ratio of 4.5, and a social 

rate of return of 154 %. The Net Present Value was calculated to be more than $3.6 million. However, this study fails to capture the impact of incorrect clinical measurements 

on patient care, which have been estimated to be approximately $100 M per year. 

The NIST Micronutrients Measurement Assurance Program supports laboratories 

that measure selected fatand water-soluble vitamins 

and carotenoids in human 

serum and plasma. Participants are from more than 

60 laboratories worldwide. 

These include participants 

from Academia, Government and Industry (62% 

domestic and 38% foreign). 

Title: SRMs for Cholesterol Measurements 

Products: NIST SRMs 911, 909, 1951 and 1952 

(Clinical Standards) 

Results (3 levels of supply chain): 

Benefit-Cost Ratio 4.5 

Social Rate of Return 154% 

Net Present Value $3.5 M

NIST Provides: 

• Tools for Comparability Assessment 

• Interlaboratory Comparison Exercises 

• SRMs and Control Materials 

• Performance Database 

• Relevant Research in Measurement Science 

• Workshops, Site Visits, Tutorials, and Consultations

Program Result: 

• Improved accuracy of laboratory measurements resulting in increased interlaboratory 

comparability
197

International Activities 

Key Comparisons in Chemical Metrology

Formal international cctivities in chemical measurements provide a 

metrological infrastructure to facilitate trade. National Metrology 

Institutes (NIST in the U.S.) provide a national traceability structure. 

However, to facilitate international trade, NMIs participate in Key 

Comparisions to establish comparability of measurements. This infrastructure can link field measurements among nations via traceability to their home NMI. NIST/CSTL has taken a leadership role in the International Committee of Weights and Measures (CIPM) Consultative Committee for the Amount of 

Substance (CCQM) and 

has leadership roles in 

many of the Key 

Comparisons. 

CSTL recently lead a Key 

Comparison on 

“Cholesterol in Human 

Serum”. The NMIs that 

agreed to participate were 

sent samples from two 

frozen human serum 

pools. The results of this 

Key Comparison study 

demonstrated that the 

participating NMIs could 

successfully measure 

serum cholesterol for 

normal and elevated levels, using ID/MS-based methods, with interlaboratory expanded 

uncertainties of less than 1%. However, every serum analyte of interest as a health 

marker provides a unique set of challenges. To provide a more comprehensive measure 

of the capabilities of NMIs for measuring well-defined serum analytes, the CCQM is 

conducting pilot studies for the determinations of serum glucose and creatinine that, if 

successful, will be followed by Key Comparisons. 

CSTL supports the IVD Industry 

by continuing to maintain and refine definitive methods for the 

health status markers discussed 

previously. In addition, CSTL researchers are also working to standardize a new and emerging generation of health status markers 

that show great promise from the 

clinical diagnostic perspective. 

CARDIAC TROPONIN I (cTnI)

• Protein found in Heart Muscle 

• Released into the Blood following Myocardial Infarction 

• Useful Clinical Marker for Myocardial Infarction 

• Occurs in Blood as Free cTnI and as 

Complex with Troponins C & T 

Results vary widely among Different Immunoassays 
198

Analytes had been identified as important prior to the EU IVD directive – and preliminary work was already underway 

to develop standards. However, 

these efforts must be accelerated 

in order to meet future industry 

needs in light of the IVD Directive. These analytes offer new 

and more difficult analytical 

challenges for standardization. 

Many of the new markers are 

proteins, peptides, or other large 

biomolecules, usually present at 

very low concentrations. Because 

of the vast market for tests for 

these new markers, many different approaches have been developed and these approaches 

often provide different answers. 

NIST maintains its links to the large clinical measurement community through interactions with a variety of healthcare-related organizations. For example, 

based on input from the American Association for 

Clinical Chemistry, the National Committee for 

Clinical Laboratory Standards, the College of American Pathologists and clinical instrument manufacturers, a number of other health markers have been identified that are medically very important (such as 

markers for anemia and hemochromatosis, nutritional 

health status, risk of heart attack, prostate cancer, 

breast cancer, thyroid function, and neural tube defects) but for which there is high variability among 

clinical laboratory results. 

The international metrological and clinical communities have jointly identified analytes 

that require standardization. They grouped as follows: “A list”consists of approximately 

100 well-defined chemical species, potentially traceable to SI units; “B list” are less well 

defined, potentially not traceable to SI units, and number >500 (for example: method dependent analytes such as liver enzymes) 

CAP 

NCCLS 

Results of Measurements of Troponin-I from 

Same Sample Pool Using Immunoassay Kits 

from Three Different Manufacturers 

 Troponin-I 

 Assay Concentration 

Manufacturer ng/mL # labs reporting 

A 19.9 115 

B 6.7 489 

C 0.85 7 

From G. S. Bodor, Denver Health and Hospitals 

 personal communication 1997
199

Over the past three 

years, we have tripled 

our investment in the 

healthcare measurements area and are 

currently focused on 

the development of 

reference methods 

and materials for the 

health status markers 

shown in the box below. Some information regarding the 

clinical application of 

one of the markers, 

glycated hemoglobin, 

follows. Details of the research for the remaining markers can be found in the extended 

abstracts from the Poster Session (pages 267 – 281). 

 

Diabetes Status 

Glucose accumulates in the blood of those with type II 

diabetes who fail to produce insulin since the cells in their 

bodies are “insulin resistant” and do not respond properly 

to the hormone. Left untreated, diabetes can result in cardiovascular disease, kidney disease, eye diseases, and 

nervous-system maladies. In addition to the daily monitoring of blood glucose levels, the measurement of glycated hemoglobin plays an important role in the diagnosis and treatment of diabetes. 

When blood glucose levels rise above normal, the glucose can react with hemoglobin in 

the blood, forming glycated hemoglobin. Since the 

lifetime of hemoglobin in the bloodstream is approximately 3 months, the measurement of glycated hemoglobin provides a record of the levels of blood glucose 

over the course of 3 months, thus providing physicians 

with information on the efficacy of diabetes treatment 

over a longer period than can the daily blood glucose measurements. While the medical 

utility of glycated hemoglobin measurement has been clearly demonstrated, there have 

been considerable problems with its clinical application due to method-to-method measurement variability. It is clear that there is a strong need measurement standardization. 

The International Federation for Clinical Chemistry (IFCC) has developed two reference 

methods for the determination of HbA1c, one specific form of glycated hemoglobin, in 

blood hemolysates. One method uses liquid chromatography coupled with mass spectrometry (LC/MS) and the other used capillary electrophoresis (CE) for HbA1c determination. Both methods have demonstrated high precision (average inter-laboratory coefficient of variance ≤ 2%) and excellent agreement with each other. These reference methEstablish Reference Systems for New Clinical 

Markers - Highest Priorities 

Troponin-I Myocardial Infarction 

Gylcated Hemo. Diabetes Status 

Homocysteine Risk of Heart Disease 

P53 DNA Breast Cancer 

TSH Thyroid Function 

Speciated Iron Hemochromatosis, Anemia 

Human Serum Alb. Renal Failure 

PSA Prostate Cancer 

Cadmium & Mercury Toxic Metal Poisoning 

Cortisol Endocrine Function 

Thyroxine Thyroid Function 

Folates Neural Tube Defects 

NIST has participated in 

inlaboratory comparison 

studies of IFCC reference 

materials and has provided 

critiques for improvements. 
200

ods have been implemented in a network of laboratories worldwide. Currently the IFCC 

reference methods are being used to value-assign an international HbA1c reference material. 

Future Directions

Expand existing clinical standards program to encompass all major diagnostic markers [perhaps through more effective networking] 

• Expand food standards program to address GMO, adulteration and other issues of US concern 

• Establish program for credentialing commercial instrument 

calibrators 

• Work with the U.S. College of American Pathologists and appropriate EU counterparts to establish trans-Atlantic Clinical 

Laboratory Chemical Measurements Proficiency Demonstration Program 

Preliminary Plan of Action

• NIST would develop and maintain reference methods and 

SRMs for up to 40% of the A list analytes* 

• NMIs in the EU, Japan and Australia would also develop 

methods and CRMs for the remainder of analytes 

• Mechanisms for mutual recognition of reference methods 

and assigned values for CRMs are being discussed [two 

models] 
201

WILLIE E. MAY 

National Institute of Science and Technology 

Willie E. May, Ph.D., is Chief, Analytical Chemistry Division, Chemical 

Science and Technology Laboratory, National Institute of Standards and 

Technology. He received his B.S. degree in chemistry from Knoxville 

College, Knoxville, TN in 1968 and his Ph.D. in analytical chemistry 

from the University of Maryland at College Park, MD in 1977. He 

joined NIST, then the National Bureau of Standards, in 1971. He became Group Leader for Liquid Chromatography, Organic Analytical 

Research Division in 1976, and became Chief of the Division in 1983. 

In 1994, Dr. May became Chief of the Analytical Chemistry (a combination of the Organic Analytical and Inorganic Analytical Chemistry Divisions). As Chief of 

the NIST Analytical Division, Dr. May is responsible for policy development, planning, 

priority setting, fiscal allocation, staffing, and management of the operations of the Division’s 100 scientists and support staff. The Division has an annual budget of approximately $15M. Dr. May has several leadership responsibilities in addition to those at 

NIST. Currently he is Chair, Organic Analysis Working Group, Consultative Committee 

on the Quantity of Material, International Committee for Weights and Measures; Chair, 

Technical Committee, North American Metrology Cooperation; and Chair, Chemical 

Metrology Working Group, Interamerican System for Metrology. In the past he has 

served as President and Executive Board Member for the International Society for Polycyclic Aromatic Compounds; Chair, Physical Sciences Panel, Ford Foundation Predoctoral Fellowship Evaluation Board; Chair, Chemistry Panel, Texas Higher Education 

Coordinating Board; Chair, Chemical Metrology Subpanel, Institute for National Measurement Standards, Canadian National Research Council; and Chair, Committee on Minority Affairs, American Chemical Society. 

Dr. May's personal research activities were focused in the area of trace organic analytical chemistry, with special emphasis on the development of liquid chromatographic 

methods for the determination of individual organic species in complex mixtures (i.e., 

extracts of environmental, food, and clinical samples) and the development of liquid 

chromatographic methods for the determination of physico-chemical properties such as 

aqueous solubilities, octanol/water partition coefficients, and vapor pressures of organic 

compounds. This work is described in more than 100 peer-reviewed publications. He has 

won numerous awards, including the U.S. Department of Commerce Superior Federal 

Service (Bronze Medal) Award in 1981, the U.S. Department of Commerce Meritorious 

Federal Service (Silver Medal) Award in 1985, the Arthur S. Flemming Award for Outstanding Federal Service, in 1987, the Percy Julian Award, NOBCChE, in 1992, the 

Presidential Rank of Meritorious Executive in Federal Service in 1992 and the U.S. Department of Commerce Distinguished Achievement in Federal Service (Gold Medal) 

Award in 1992. 
202
203

NIST Measurement Methods and Standards Reference Materials 

for Health Status Markers: 

Current Programs and Future Challenges 

Visuals Presented By 

Willie E. May 

National Institute of Standards and Technology 
204

Development of Reference Methods and Development of Reference Methods and 

Reference Materials in the U.S. Reference Materials in the U.S.

For further information:

Mail: Analytical Chemistry Division, MS 8390

100 Bureau Dr.,Gaithersburg, MD 20899-8390 USA

E-mail: Willie.May@NIST.gov

Phone: Voice +301-975-3108

FAX +301-926-8671

Websites: ACD http://www.cstl.nist.gov/nist839/

NIST http://www.nist.gov/

Willie E. May

Analytical Chemistry Division

Chemical Science and Technology Laboratory

National Institute of Standards and Technology

Gaithersburg, Maryland 20899 USA

• Mission of NIST Laboratories

• NIST Program in Chemical Measurements 

and Standards

• Why Health-Care?

• Examples of Measurement Problems and 

Impact of NIST Activities

• New Drivers

– Protein-based markers

– EU in vitro Diagnostic Devices Directive

• NIST Response to New Drivers

NIST Measurement Methods and Standards NIST Measurement Methods and Standards 

for Health Care Markers for Health Care Markers
205

3 IVD Traceability Workshop – 11/00

MISSION of the NIST Laboratories: MISSION of the NIST Laboratories:

To promote the U.S. economy and public welfare, 

the Laboratories of the National Institute of 

Standards and Technology 

• provide technical leadership for the Nation's 

measurement and standards infrastructure, and 

• assure the availability of essential reference 

data and measurement capabilities. 

4 IVD Traceability Workshop – 11/00

0LFURHOHFWURQLFV

(QHUJ\6\VWHPV

+HDOWK&DUHDQG

3KDUPDFHXWLFDOV

%LRWHFKQRORJ\

)RRG

1DWLRQDO6HFXULW\ ,QVWUXPHQWDWLRQ 5HJXODWRU\

$JHQFLHV

CSTL Core Competencies CSTL Core Competencies

Support NIST Programs and a Diverse Customer Base Support NIST Programs and a Diverse Customer Base

$XWRPRWLYHDQG

7UDQVSRUWDWLRQ 3HWURFKHPLFDOV

&KHPLFDO3URFHVV

,QGXVWULHV
206

5 IVD Traceability Workshop – 11/00

Maintaining World-Class 

Metrologically-based Competencies in 

Chemical Measurement Science:

• Analytical Mass Spectrometry

• Analytical Separation Science

• Atomic and Molecular Spectroscopy

• Chemical Sensing Technology

• Classical and Electroanalytical Methods

• Gas Metrology

• Nuclear Analytical Methods

• Microanalytical Technologies

Primary and 

Reference 

Methods

of Both Simple 

and Complex 

Matrices

NIST Analytical Chemistry Division NIST Analytical Chemistry Division

Serves as the Nation’s Reference Laboratory for

Chemical Compositional Measurements

6 IVD Traceability Workshop – 11/00

– Standard Reference Materials and Standard 

Reference Databases

– NIST Traceable Reference Materials

– Measurement Proficiency Assessment for Selected 

U.S. Environmental, Health, and Industrial Studies

– Strategic International Comparisons

Tools for Achieving National Traceability and Tools for Achieving National Traceability and 

International Comparability of International Comparability of 

Chemical Measurements Chemical Measurements

SRMs

RMs

NTRMs

Intercomparisons

Proficiency Assessments

Commercial RM

Producers

U.S. Reference

Laboratory

Asian/Pacific

Reference Laboratories

SI

Euporean Reference

Laboratories

APMP EUROMET

NIST

SIM
207

8 IVD Traceability Workshop – 11/00

Why NIST and Healthcare? Why NIST and Healthcare?

Problem Magnitude and Scope:

• U.S. Spends ~ $1.5 trillion on Health Care (14% of GDP) (per year)

– ~13% of this amount is associated with measurement ($140B)

– Non-diagnostic measurements cost ~$39B 

– Improvement in accuracy of one marker alone (cholesterol) is 

estimated to have saved $100M per year

New Driver: EU IVD Directive to go into effect 2003

• Worldwide in vitro diagnostic device market is ~$20B;

• >60% of European market is supplied by U.S. based companies

9 IVD Traceability Workshop – 11/00

NIST’s Role in Health Care Measurements

NIST

High Accuracy Standard Reference Materials, New Measurement

Methods Data and Other QA Services Tools

Health Care Delivery

Improved Measurement Accuracy Better Diagnoses and Reduced Costs

NIH CDC

FDA

AACC AOAC

CAP NCCLS

Medical/Clinical

Manufacturers

Medical Research

Community
208

10 IVD Traceability Workshop – 11/00

Program Objective and Activities Program Objective and Activities

To Serve as the US Reference Laboratory for 

Health-Related Chemical Measurements

– high accuracy measurement methods

– Standard Reference Materials

• pure primary standards

• matrix-based materials

• instrument calibration materials & artifacts

– interactive measurement quality 

assessment/demonstration activities

– international comparison exercises with other 

National Metrology Institutes (NMIs)

11 IVD Traceability Workshop – 11/00

History of Clinical Standards Program History of Clinical Standards Program

- Pure Materials & Artifacts (29) Pure Materials & Artifacts (29)

• 1914 SRM 41, Dextrose

• 1945 SRM 1861 & 186 II, phosphates for pH

• 1967 SRM 911, Cholesterol

• 1968 SRM 912, Urea

SRM 913, Uric Acid

SRM 914, Creatinine

SRM 915, Calcium Carbonate

• 1970’s 21 SRMs with NIH support
209

12 IVD Traceability Workshop – 11/00

National Reference System for Clinical Chemistry National Reference System for Clinical Chemistry

- Hierarchy of Methods for Cholesterol Hierarchy of Methods for Cholesterol

• Definitive (NIST)

– Highest accuracy and precision

– Thoroughly tested for bias

– Generally not within the capability of clinical laboratories

– Used for primary RMs and validation of reference methods

• Reference (CDC Network)

– Carefully tested vs definitive method; high accuracy and 

precision

– Within capability of most clinical labs, but too time consuming

– For secondary RMs and validation of field methods

• Field (Clinical Labs)

– Routine clinical use

– Sufficient accuracy and precision for reliable diagnosis

– Simple, rugged and cost-effective

13 IVD Traceability Workshop – 11/00

CDC

Network Labs

NIST/CSTL

Manufacturers

Hospitals and Clinical Laboratories

(thousands)

Certification and QC

Instruments and Reagents Certification and QC

SRMs and Definitive Method

Certification and QC

SRMs and other measurement 

services

A/K Reference Method

Cholesterol Reference System
210

14 IVD Traceability Workshop – 11/00

Definitive Methods for Clinical Definitive Methods for Clinical Analytes Analytes

ANALYTE METHOD CONDITION

Calcium ID/MS Cancer, Blood Clotting

Chloride ID/MS, Coulometry Kidney Function

Cholesterol ID/MS Heart Disease

Creatinine ID/MS Kidney Function

Glucose ID/MS Diabetes

Lithium ID/MS Antipsychotic treatment

Magnesium ID/MS Heart Disease

Potassium ID/MS Electrolyte Balance

Sodium Gravimetry, ICP/MS Electrolyte Balance

Triglycerides ID/MS Heart Disease

Urea ID/MS Kidney Function

Uric Acid ID/MS Gout

15 IVD Traceability Workshop – 11/00

Organic Definitive Methods Organic Definitive Methods
211

16 IVD Traceability Workshop – 11/00

PRECISION OF ID PRECISION OF ID-GC/MS

Repeated 

Intensity 

Ratio 

Measurements

Mass Fraction

Measured

Mass Fractions

of Independently

Prepared Samples

17 IVD Traceability Workshop – 11/00

Day 1 - Day 2 Range and Mean of Cholesterol 

Measurements in Serum by ID/MS

6.230

6.240

6.250

6.260

6.270

6.280

6.290

I-1I-2I-3

II-4II-5II-6

III-7III-8III-9

Overall

Set/Sample

mmol/L
212

18 IVD Traceability Workshop – 11/00

History of Clinical Standards Program History of Clinical Standards Program

- Selected Serum Matrix Materials Selected Serum Matrix Materials

• 1979 SRM 900, Antiepilepsy Drug Level Assay Standard

• 1980 SRM 909, Human Serum 

• 1982 SRM 1599, Anticonvulsant Drug Level Assay Standard

• 1988 SRM 1951, Cholesterol in Human Serum (Frozen)

SRM 1952, Cholesterol in Human Serum (Freeze Dried)

• 1989 SRM 968, Fat Soluble Vitamins in Human Serum

• 1991 SRM 956, Electrolytes in Blood

• 1996 SRM 956a, Electrolytes in Frozen Human Serum

• 1996 SRM 965, Glucose in Frozen Human Serum (3 levels)

• 1997 SRM 1951a, Lipids in Frozen Human Serum (2 levels)

• 2000 SRM 966, Toxic Metals in Blood

SRM 2070a, Toxic Elements in Human Urine

19 IVD Traceability Workshop – 11/00

CSTL Economic Impact Studies CSTL Economic Impact Studies

Title: SRMs for DNA Profiling

Products: NIST SRMs 2390, 2391

Contract Awarded: July, 1998

expands existing NIJ Study

Comments:

• Web-based survey for data collection

• Representative data collected from state and 

local forensic laboratories

Title: SRMs for Cholesterol Measurements

Products: NIST SRMs 911, 909, 1951 and 

1952 (Clinical Standards)

Results(3 levels of supply chain): 

Benefit-Cost Ratio 4.5

Social Rate of Return 154%

Net Present Value $3.5 M

CDC

Network Labs

(about 10?)

NIST/CSTL

Manufacturers

Hospitals and Clinical Laboratories

(thousands)

Certification and QC

Instruments and Reagents Certification and QC

SRMs and Definitive Method

Certification and QC

SRMs and other measurement 

services

A/K Reference Method

Cholesterol Reference System
213

20 IVD Traceability Workshop – 11/00

NIST Contributions to

National Reference

System for Cholesterol

Measurement Improvement 1969 - Present

May Save $100M/year in Treatment Costs

1967 - SRM 911

Pure Cholesterol

1980 - Definitive Method for

Serum Cholesterol

1981 - SRM 909 Cholesterol in

Human Serum

1988 - SRMs 1951 & 1952

Cholesterol in Serum

1994 - Definitive Method for 

Serum Triglycerides

1996-7 - Values for HDL & LDL 

Cholesterol

1949 23.7%

1969 18.5%

1980 11.1%

1986 6.4%

1990 -1994 5.5 - 7.2%*

Unnecessary

Treatments

Untreated

Disease

Correct Value

False

Positives

False

Negatives

Death Wasted $$$

* Data from

GAO/PEMD-95-8

Improved Cholesterol Measurement Accuracy Saves Health Care Dollars

Cholesterol Measurements Cholesterol Measurements

21 IVD Traceability Workshop – 11/00

Cholesterol in Blood Experiment Cholesterol in Blood Experiment -

Impact of Inaccurate Measurements Impact of Inaccurate Measurements

170

180

190

200

210

220

230

240

250

260

270

012345

Patient

mg/dL

Outside Lab

NIST Result

Watch Diet

Take Drugs
214

22 IVD Traceability Workshop – 11/00

Program Result:

?Improved accuracy of laboratory measurements 

resulting in increased interlaboratory comparability

• to support laboratories that measure selected fat- and water-soluble vitamins 

and carotenoids in human serum and plasma

The NIST Micronutrients Measurement Quality Assurance Program The NIST Micronutrients Measurement Quality Assurance Program

Program Participants

>60 Laboratories Worldwide

Academia

Government

Industry

NIST Provides:

?Tools for Comparability Assessment

•Interlaboratory Comparison Exercises

•SRMs and Control Materials

•Performance Database

?Relevant Research in Measurement Science

?Workshops

?Site Visits, Tutorials, and Consultations

Foreign

38%

Domestic 

62%

Participants

23 IVD Traceability Workshop – 11/00

Quotes from letters ACD has received regarding the impact of the

NIST Micronutrients Measurement Quality Assurance Program:

“We have been participating in the Micronutrients Measurement Quality Assurance Program 

for four years, and we think it is the best quality program for vitamins in the world. 

Moreover, the calculations are performed with the highest scientific and statistical level. I 

do not know any other program which proposes quality assurance for carotenoids.”

Dr. H. Faure

Laboratoire de Biochimie C

La Tronche, France

“Currently, it (NIST Micronutrients Measurement QA Program) is the only QA program 

available for the fat-soluble vitamins and carotenoids. As a CAP/CLIA accredited 

laboratory, we are required to participate in external QA programs when available. We have 

used the data for this QA program at all of our accreditation inspections. A QA program 

from NIST carries a certain reputation that could be a benefit to any laboratory taking 

part in the program.”

I. Osberg

The Children’s Hospital

University of Colorado Health Sciences Center
215

24 IVD Traceability Workshop – 11/00

Quotes from letters ACD has received regarding the impact of the

NIST Micronutrients Measurement Quality Assurance Program: (continued)

“Obviously the Program benefits us by providing the independent assessment of our 

ability to measure antioxidant nutrients. In addition, the fact that we are registered in the 

scheme and can show independent documentary evidence of our analytical rating

helps in our applications for funding to various bodies including the Ministry of Agriculture 

Fisheries and Food. I would also point out that as sub-optimum intakes of antioxidant 

nutrients are increasingly implicated in many diseases including coronary heart disease and 

cancer, there is a need for a QA scheme to monitor the accuracy of results of the growing 

number of epidemiological and clinical trials undertaken in this areas. As far as I am aware, 

no other scheme such as the [NIST] Micronutrients Measurement Quality Assurance 

scheme exists, certainly not in Europe.”

Dr. G. Duthie

Rowett Research Institute

Scotland, UK

“The addition of vitamin A to my list of analyses would have been essentially impossible

without SRM 968 and the assistance of the capable staff at NIST. The truth was then 

(and still is) that there are no other programs, no other reference materials out there for 

those of us who measure these analytes...the availability of the Micronutrients Program 

(SRM and round robin samples) has enabled me to provide service to patients”...

P. Radmacher

Department of Pediatrics

University of Louisville

25 IVD Traceability Workshop – 11/00

Health

• clinical diagnostic markers 

(cholesterol/heart disease, diabetes/glucose,

creatinine/kidney function, trace hormones) 

• electrolytes (Na, K, Ca)

• Pb in blood

• Anabolic steroids in urine

Food

• pesticide residues

• antibiotics in meat

• growth hormones in meat

• vitamins and minerals 

• drinking water (EPA List)

Environment

• air (EPA HAPs List)

• soil/sediments

• biological tissues

• waste water (EPA List)

CCQM Key Comparison Areas:

Advanced Materials

• semiconductors

• metal alloys

• polymers and plastics 

Forensics

• drugs of abuse

• explosive residues

• breathalyzer (ethanol-in-air)

• DNA profiling

Commodities

• emissions trading (SO2 in stack emissions)

• sulfur in fossil fuels

• natural gas

• sucrose

• cement (Ca, Si, Al, S, Ti, Na, Mg )

• source of origin/adulteration 

Pharmaceuticals

Biotechnology
216

26 IVD Traceability Workshop – 11/00

CCQM K6 Matl B

1.68

1.7

1.72

1.74

1.76

1.78

1.8

1.82

NIST

234567

mean +/- U (mg/g)

CCQM K6 Matl A

2.1

2.12

2.14

2.16

2.18

2.2

2.22

2.24

2.26

2.28

2.3

NIST

234567

mean +/- U (mg/g)

K-6 Cholesterol 6 Cholesterol 

in Human Serum in Human Serum 

(Frozen) (Frozen)

 Cholesterol: 

1.5 to 2.5 mg/g 

(physiological range)

 Study contained two 

samples; both natural 

levels

27 IVD Traceability Workshop – 11/00

CCQM Cholesterol CCQM Cholesterol 

Pilot Study Pilot Study vs Key 

Comparison Comparison

P-7 (1999) K-6 (2000) (mg/g)

Mean 1.694 1.726

U 0.032 0.014

No. of labs 6 (same 6 for both studies)

1.60

1.62

1.64

1.66

1.68

1.70

1.72

1.74

1.76

NIST-P

NIST-K

2-P

2-K

3-P

3-K

4-P

4-K

5-P

5-K

6-P

6-K

mean +/- U (mg/g)
217

28 IVD Traceability Workshop – 11/00

Health Care Measurements: Health Care Measurements: 

... supporting the national reference system and the IVDD industry

v Maintain and Refine Definitive 

Methods for 12 Health Status Markers

Calcium Cancer, Blood Clotting

Chloride Kidney Function

Cholesterol Heart Disease

Creatinine Kidney Function

Glucose Diabetes

Lithium Antipsychotic Treatment

Magnesium Heart Disease

Potassium Electrolyte Balance

Sodium Electrolyte Balance

Triglycerides Heart Disease

Urea Kidney Function

Uric Acid Gout

v Establish Reference Systems for New 

Clinical Markers - Highest Priorities

Troponin-I Myocardial Infarction

Gylcated Hemo. Diabetes Status

Homocysteine Risk of Heart Disease

P53 DNA Breast Cancer

TSH Thyroid Function

Speciated Iron Hemochromatosis, Anemia

Human Serum Alb. Renal Failure

PSA Prostate Cancer

Cadmium & Mercury Toxic Metal Poisoning

Cortisol Endocrine Function

Thyroxine Thyroid Function

Folates Neural Tube Defects

Measurement Challenges for new markers:

• High molecular mass proteins (>20,000 daltons)

• Heterogeneity of the protein

• Separation of different forms of the proteins

• Serum matrix complex; analyte level low

• Stability of analytes

• Standardization necessary before medical 

diagnostic benefit can be realized

Characteristics of current markers:

• Relatively small well-defined molecular or 

elemental species

29 IVD Traceability Workshop – 11/00

Assessing Needs and Setting Priorities Assessing Needs and Setting Priorities
218

31 IVD Traceability Workshop – 11/00

Heart Disease Heart Disease

Heart disease is the number one cause of death in the 

United States and accounts for 1/3 of all deaths

Myocardial infarction (AMI) is responsible for 30% of 

these deaths

Approximately 6 million people visit ERs annually for 

chest pain

Approximately 3 million are admitted for possible AMI

Of these, 2 million are not diagnosed as having AMI 

[FALSE POSITIVE ⇒ UNNECESSARY MEDICAL COSTS!!!]

Of those not admitted , 2-8 % actually had an AMI

[FALSE NEGATIVE ⇒ DELAYED TREATMENT ⇒ DEATH??]

32 IVD Traceability Workshop – 11/00

CARDIAC TROPONIN I (cTnI) CARDIAC TROPONIN I (cTnI)

Protein found in Heart Muscle

Released into the Blood following Myocardial Infarction

Useful Clinical Marker for Myocardial Infarction

Occurs in Blood as Free cTnI and as Complex with Troponins C 

& T

Results vary widely among Different Immunoassays

♥

♥

♥

♥

♥
219

33 IVD Traceability Workshop – 11/00

Results of Measurements of Troponin-I from Same Sample Pool

Using Immunoassay Kits from Three Different Manufacturers

Troponin-I

Assay Concentration

Manufacturer ng/mL # labs reporting

A 19.9 115

B 6.7 489

C 0.85 27

From G. S. Bodor, Denver Health and Hospitals -- personal communication 1997

Example of Variability of Results Among Example of Variability of Results Among 

Currently Used Clinical Immunoassays Currently Used Clinical Immunoassays

“Failure to use effective treatments . . . for acute myocardial infarction for all 

patients who could benefit from these interventions may lead to as many as 18,000 

preventable deaths each year in the United States.”

- JAMA, 280, 1000, September 16, 1998.

34 IVD Traceability Workshop – 11/00

Time (min)

5 10 15 20 25 30 35 40

0

2000000

4000000

6000000

8000000

10000000

12000000

14000000

recombinant human cTnI

LC/MS of

m/z 600 800 1000 1200

0

20

40

60

80

100

120

Max: 91563

820.7

849.0

794.2

879.3

769.5

911.8

946.8

746.3

724.3

984.8

1025.7

703.5

1070.1

1118.6

1230.8

1172.1

684.3

1367.1

1295.3

24500 24550 24600 24650 24700

0

100

200

300

400

500

600

700

800

24,592

deconvoluted Mr



peak 
220

35 IVD Traceability Workshop – 11/00

Time (min)

5 10 15 20 25 30 35 40

0

500000

1000000

1500000

2000000

2500000



ó

ì

LC/MS of

Extracted Human cTnI

36 IVD Traceability Workshop – 11/00

NIST Program Vision: NIST Program Vision:

–Expand existing clinical standards program to encompass all 

major diagnostic markers [perhaps through more effective 

networking]

–Expand food standards program to address GMO, 

adulteration and other issues of US concern

–Establish program for credentialing commercial instrument 

calibrators

–Work with the U.S. College of American Pathologists and 

appropriate EU counterparts to establish trans-Atlantic 

Clinical Laboratory Chemical Measurements Proficiency 

Demonstration Program
221

37 IVD Traceability Workshop – 11/00

Preliminary Plan of Action Preliminary Plan of Action

• NIST would develop and maintain reference 

methods and SRMs for up to 40% of the A list 

analytes

• NMIs in the EU, Japan and Australia would also 

develop methods and CRMs for the remainder of 

analytes

• Mechanisms for mutual recognition of reference 

methods and assigned values for CRMs are 

being discussed [two models]

38 IVD Traceability Workshop – 11/00

Assessing Needs and Setting Priorities Assessing Needs and Setting Priorities 

for Health Status Markers: How do we for Health Status Markers: How do we 

Proceed?? Proceed??
222

Acknowledgement: Acknowledgement:

Health-Care Markers Team Care Markers Team

✰ David Bunk

✰ Joe Dalluge

✰ Yolanda Davidson

✰ Polly Ellerbea

✰ Steve Long

✰ Bryant Nelson

✰ Chad Nelson

✰ Curtis Phinney

✰ Lorna Sniegoski

✰ Susan Taia 

✰ Bob Vocke

✰ Michael Welch

a College of American Pathologist Research Associate
223

Importance of Commutable Reference Materials and Patient Test 

Distributions for Assay Calibration 

George G. Klee 

Department of Laboratory Medicine and Pathology 

Mayo Clinic and Mayo Foundation 

Introduction 

 

The development of integrated health care systems and consensus agreements of 

practice guidelines has increased the requirements of uniformity and standardization for 

clinical laboratory measurements. Small charges in analytic set-points produce major 

charges in the number of patients having test values which cross critical decision thresholds. Most practice guidelines and many clinicians assume that all test methods produce 

equivalent test values and therefore provisions for laboratory differences seldom are included in practice decisions. Therefore, it is important for laboratorians, industries producing laboratory diagnostic reagents, and laboratory regulatory agencies, to develop systems for minimizing these differences in laboratory test values. This paper illustrates the 

effects of analytical bias on the changes in the number of patients crossing key decision 

thresholds, and shows that certified reference materials by themselves are not sufficient 

to assure uniform laboratory practice. A novel protocol is proposed for calibration adjustment that utilizes a combination of reference analytic methods, commutable control 

materials and statistical processing of the distributions of test measurements from welldefined patient groups to minimize the variation of reported test values. 

Effect of Analytic Bias on Medical Decisions 

 

There are multiple sources of errors in laboratory measurements, but one helpful 

way of classifying these errors is to divide them into random and systematic errors. Random errors, as the term implies, occur sporadically and can be evaluated in terms of the 

test imprecision (standard deviation (SD) or coefficient of variation (CV)). In clinical 

decisions, assay imprecision combines with patient biologic variability to produce a total 

variability that is larger than either of these two components. Therefore, analytical imprecision only significantly increases the total variability when it is large relative to the 

biologic variation. This relationship had formed the basis for laboratory precision performance goals (1-5). If the analytical imprecision (SD) is less than one-quarter of the 

biological variation, the total variation (SD) is only about 3 % larger than the biological 

variation. On the other hand, systematic errors directly effect all clinical decisions without the buffering effect of biological variance. The effect of these bias errors can be 

evaluated by examining shifts in the distribution of test measurements. Figure one illustrates this concept using the Gaussian distribution. Although most laboratory test distributions are not Gaussian, the general effects of analytic shifts for most laboratory test distributions is similar to the effects on Gaussian distributions (6). In the figure, the clinical 

decision threshold is set at +2.0 SD. When the laboratory test is operating with the cor-
224

correct set-point, about 2.3 % of 

the patients without the disease 

in question have values which 

exceed this threshold (falsepositives). However, if the analytic test is biased 1.0 SD high 

(as illustrated by the dotted line) 

then 15.8 % of the patients 

without disease have test values 

which cross this threshold. This 

causes an almost seven-fold increase in false-positives. Obviously, most clinical decisions 

are not based solely on laboratory test values, but for some 

key analytes especially those 

used for screening, laboratory 

tests are important factors for 

medical decisions and analytical 

bias can cause major problems (7,8). Since many of these medical decisions involve expensive second and third level clinical investigations and/or the prescribing of expensive 

drugs, small differences in laboratory test calibrations can have major effects on downstream medical costs. 

 

Table-one illustrates the 

effect of analytical bias 

on the number of patients 

having cholesterol or thyrotropin (TSH) test measurements which exceed 

the 200 mg/dL or 5.0 

mIU/L decision thresholds. An assay for cholesterol that is biased 1 % 

high results in a 3.0 % 

increase in the number of 

patients having “high” 

cholesterol values (using 

actual test distributions 

rather than Gaussian 

models). Similarly, a 3 % bias causes a 8.8 % increase, and a 10 % bias causes a 27.8 % 

increase. These numbers are based on the average distribution of cholesterol test values 

for out-patients seen at the Mayo Clinic (7). Overall, the change in the percentage of patients crossing this 200 mg/dL threshold increases about three fold greater than the percentage change in the analytical shift due to the multiplier effect of the distribution 

curves. Similar effects are found at other decision levels for cholesterol, such as 240 

Table 1 

Effect of Analytic Bias on Clinical Decisions

Analyte Analytic Bias Patients Multiplier

Cholesterol +1% +3.0% 3.0X 

@200 mg/dL +3% +8.8% 2.9X 

 +10% +21.8% 2.8X 

TSH +6% +10.8% 1.8X 

@5.0 mIU/L +10% +21.7% 2.2X 

 +20% +50.8% 2.5X 

STANDARD DEVIATES

-4 -2 0 2 4

0.5 

0.4 

0.3 

0.2 

0.1 

 0 

FREQUENCY 

Original Positives 2.3% 

With Bias 15.8% 

+2.0 SD Biol

Figure 1: Effect of Analytic Bias on Decisions

Illustration of effect of analytic bias on the percentage of a Gaussian distribution that crosses 

the +2 SD decision threshold. Reproduced and 

modified with permission of Clinical Chemistry 

(1993:39:1514-1518) 
225

mg/dL, and for other tests. Table one illustrates that the multiplier is about 2 to 2.5 fold 

for comparing the effect that changes in analytical bias for the TSH assay have on the 

changes in the percentage of patients with TSH values greater than 5 mIU/L. Again, 

similar effects are found at other decision levels. 

Analytic Differences Across Assays Utilizing the Same Standard Reference Material 

 

Most assays for human chorionic gonadotropin (hCG) currently are standardized 

to the WHO 3rd International Reference Preparation (WHO IRP 75/537); however even 

with the use of this common standard, major differences are found between test values 

measured with different assays (9). Table 2 illustrates the differences across theses assays 

found with a recent College of American Pathologists (CAP) Survey (10). 

This survey utilizes intact hCG spiked into human serum. Matrix effects of the 

survey material may cause part of these differences, but major between-assay differences 

also are seen with patient samples. Even within the same manufacturer, different assays 

often do not give harmonious results. Even larger differences have been seen when proficiency specimens which include both intact and free beta subunits of hCG were evaluated (11,12). Several factors may contribute to these differences in hCG assays: 

1) Differences in the immunoreactivity of the antibodies used in the assays 

for specific epitopes on the hCG molecule which may be differentially expressed in the standard compared to the control or patient specimens. 

Table 2 

Human Chorionic Gonadotropin (hCG, mIU/L) 

(Who Third IRP) 

College of American Pathologist K-A Survey 2000 

Method K-02 K-05 K-01

Abbott Architect 7.36 176.84 841.36 

Abbott Axsym 9.87 229.28 969.64 

Beckman Access 13.54 158.20 667.14 

Biomerieux Vidas/Mini - 247.08 1213.04 

Chiron ACS:180 11.02 241.22 946.92 

Chiron ACS:Centaur 11.21 240.76 923.22 

Dade Dimension HM 0.76 238.88 1093.49 

DPC Immulite 9.12 253.78 1060.02 

DPC Immulite 2000 12.14 270.82 1269.82 

Roche/BMC Elecsys 10/2010 1.56 212.73 968.82 

Technicon Immuno-1 5.97 197.01 916.01 

Tosoh AIA-Pack BHCG 5.42 230.02 1094.46 

Vitros ECI 6.04 222.79 949.76

ALL METHODS 9.37 224.22 953.53 
226

2) Cross-reactivity and/or inhibition with alternate molecular forms or metabolic fragments of the hCG molecule which may be present in the standards, controls and/or specimens tested. 

3) Matrix differences between the standards and the specimens tested. 

4) Differences in the glycosylation and (or protein nicking) between the standard and the specimens. 

5) Lack of true traceability of the assay to the reference standard. (The first 

generation of the assay may have been directly standardized and subsequent generations may be only indirectly linked by comparison studies) 

Biochemical, antigenic and metabolic differences in complex glycoproteins such 

as hCG make standardization and harmonization difficult; however, even simpler molecules like triiodothyronine (T3), which have homogenous crystalizable standards, have 

major analytical differences among assays. Table 3 illustrates the differences found in T3 

measurements across methods for the CAP K-A, 2000 survey. Again, major differences 

are found even between assays from the same manufacturer. It is assumed that these differences are related to matrix issues and/or differences in standardization protocols and/or 

large tolerances in manufacturing specifications. 

Significant assay differences also have been found between reagent lots and 

across time within the same assay system, especially when key components are changed. 

Even when no components in the reagents are changed, significant lot-to-lot variations 

frequently occur do to large tolerance specifications in reagent manufacturing and large 

tolerances in the assignment of calibration set-points for production instruments. 

Table 3 

Triiodothyronine (T3, ng/dL) 

College of American Pathologist K-A Survey 2000 

Method K-02 K-05 K-01

Abbott Architect 115.9 288.5 490.6 

Abbott Axsym 108.0 259.1 519.3 

Beckman Access 131.4 309.1 660.2 

Chiron ACS:180 107.2 260.8 700.2 

Chiron ACS:Centaur 124.5 286.7 733.0 

DPC Immulite 74.5 218.8 458.7 

DPC Immulite 2000 67.9 199.1 418.6 

Roche/BMC Elecsys 10/2010 168.6 438.5 - 

Technicon Immuno-1 89.8 243.2 497.6 

Tosoh AIA-Pack BHCG 101.6 294.0 655.5 

Vitros ECI 131.6 301.2 658.1 

ALL METHODS 109.0 268.1 574.0
227

Proposal for improving Assay Calibration Adjustment 

 

This proposal for assay calibration adjustment has five basic parts: 1) certified 

reference materials, 2) reference measurement systems capable of measuring both the 

reference material, controls and human specimens, 3) commutable control specimens 

which have values traceable to reference analytical systems, 4) human specimens from 

patients with well characterized clinical states, and 5) real-time data processing systems 

for processing both the control specimen data and the patient test values. A combination 

of statistical rules for monitoring the traceable control results and mathematical algorithms for tracking the patient test value distributions are used to adjust assay calibration 

to assure uniform test performance. 

The term “commutable” is defined by Webster as “capable of being exchanged or 

interchanged” (13). The phrase “commutable control specimens” refers to control specimens that perform identically to patient specimens in both the reference system and the 

production assay system. “Commutability” means that measurements of these control 

specimens on the production system that recover the same set-points that were assigned 

to the controls by the reference system, will assure that the production system also will 

recover the same population set-point as the reference system for a group of patient 

specimens. A potential protocol for testing this definition of commutability of a control 

would be to measure the control 10 times on both the reference assay and the production 

assay. Then compare the difference in the averages of the control values on the two systems with the average difference found by testing 10 patient samples on both systems. If 

the average of the controls can be used to harmonize the patient values, then the control is 

commutable. When these commutability requirements are met, statistical factors can be 

developed to provide real-time calibration adjustments, which can be used to correct for 

inaccuracies in production measurement systems, caused by manufacturing variations 

and/or inaccuracies in assay calibration assignment. 

Figure two outlines a protocol 

for calibration adjustment using a 

combination of 

three commutable 

control sera and 

the statistical distributions of patient test values. 

The control test 

values are evaluated relative to 

absolute warning 

and action limits 

assigned to the 

controls using es-
228

tablished reference methods. Multi-rule tracking algorithms help to evaluate these 

changes (14). Patient test values are normalized for demographic and clinical differences, then evaluated relative to clinical decision thresholds. Again, statistical algorithms 

help to track trending differences (14-16). If both systems indicate that the same analytical bias shift has occurred, the assay calibration is adjusted statistically to bring it back to 

the prior level. After adjustment, the calibration is verified by replicate (5 to 10) measurements of reference calibrators. If the two tracking systems (controls versus patient 

specimens) differ, then the controls and the patient demographics, as well as preanalytical factors and instrument performance are evaluated. 

 

For large homogeneous groups of patients, central distribution parameters such as 

the median, 25th and 75th percentiles are remarkably constant. When analytical systems 

are tightly controlled the variation of these parameters is less than the usual reagent 

manufacturers tolerance specification for many key analytes. Consequently, tighter reagent control could translate directly into smaller short-term variation in key analytical 

parameters used in clinical practice. Some laboratory tests have large variations in patient test value distributions due to demographic and pathophysiologic differences. When 

these differences are well characterized, mathematical corrections can be used to make 

the distributions of patient test values more homogeneous. Examples of these corrections 

are: factors to correct for gender differences in hemoglobin concentrations; age adjustment for some hormone measurements; and renal function corrections for compounds 

cleared by the kidney. Further normalization can occur by sub-setting test distributions 

into separate tracking groups according to ordering location and/or time of the request. 

Examples of this type of normalization is to subset patients from pediatric and/or oncology areas, or to exclude patients from intensive care units or burn units, or to exclude 

weekend or late-night specimens in the calibration adjustment factor calculations. 

Conclusions 

 

Lack of laboratory test harmonization can cause major problems in patient care 

and assay calibration with standard reference preparations does not assure harmonization 

of patient test results. There are multiple reasons for this lack of harmonization, but nonhomogeneity of analytes and matrix effects could be major factors. A combination of statistical rules using commutable control material with traceable reference values and 

mathematical algorithms using distributions of normalized patient test values could be 

used for calibration adjustment. A preliminary protocol for this form of calibration adjustment is proposed. 

Disclaimer 

 

All conclusions and interpretations in this article with respect to the College of 

American Pathologists’ data base are those of the author and not those of the College. 
229

References 

1. Analytical Goals in Clinical Chemistry. College of American Pathologists’ Conference, Aspen, CO, 1976, F.R.Elevitch, ed., College of American Pathologists, 

Northfield, IL 60093-2750. 

2. Fraser CG, Petersen PH, Ricos C, Haeckel R: Proposed quality specifications for the 

imprecision and inaccuaracy of analyitical systems for clinical chemistry. Eur J 

Clin Chem Biochem. 30;311-317, 1992. 

3. Fraser CG, Petersen PH: Desirable performance standards for imprecision and bias in 

alternate sites. Arch Pathol Lab Med. 119:909-913, 1995. 

4. Fraser CG, Petersen PH, Libeer JC, Ricos C: Proposals for setting generally applicable quality goals solely based on biology. Ann Clin Biochem 34:8-12, 1997. 

5. Fraser CG: General strategies to set quality specifications for reliability performance 

characteristics. Scand J Clin Lab Invest. 1999;59:487-490. 

6. Klee GG: Tolerance limits for short-term analytical bias and analytical imprecision 

derived from clinical assay specificity. Clin Chem 39(7):1514-1518, 1993. 

7. Klee GG, Schryver PG, Kisabeth RM. Analytic bias specifications based on the analysis of effects on performance of medical guidelines. Scand J Clin Lab Invest 

59:509-512, 1999. 

8. Petersen PH, de Verdier CH, Groth T, Fraser CG, Blaabjerg O, Horder M: The influence of analytical bias on diagnostic misclassification. Clin Chim Acta 260:189-

206, 1997. 

9. Cole LA: Immunoassay of human chorionic gonadotropin, its free subunits, and 

metabolites. Clin Chem 43(12):2233-2243, 1997. 

10. 2000. K/KN-A Ligand (general). Participant Summary Report. 2000, College of Surveys American Pathologists, Northfield, IL. 

11. Klee GG: Serum HCG Assays: Wide across-method differences. College of American Pathologist. Therapeutic Drug Monitoring/Endo Resource Committee, March 

1994. 

12. Klee GG: Human Chorionic Gonadotropin. Mayo Clin Proc 69:391-392, 1994. 

13. Woolf HB, ed. Websters New Collegiate Dictionary: Springfield, Massachusetts: G. 

& C. Merian Company, 1979:229. 

14. Westgard JO, Klee GG. In: Burtis CA, Ashwood ER, eds. Tietz Textbook of Clinical 

Chemistry 3rd Edition. WB Saunders. Philadelphia. 1999:384-418 

15. Klee GG: A conceptual model for establishing tolerance limits for analytic bias and 

imprecision based on variations in population test distributions. Clin Chim Acta 

260:175-188, 1997 

16. Smith FA, Kroft SH: Optimal procedures for detecting analytic bias using patient 

samples. Am J Clin Pathol 108:254-268, 1997. 
230

GEORGE G. KLEE 

Mayo Clinic and Mayo Foundation 

George G. Klee, M.D., Ph.D., is a clinical pathologist with over 20 years experience 

working with clinicians to provide quality laboratory support. He is a medical consultant 

at the Mayo Clinic with appointments in both the Department of Laboratory Medicine 

and Pathology and in the Department of Health Sciences Research. He holds the title of 

Professor in the Mayo medical School and is on graduate faculty of both Mayo Graduate 

School and The University of Minnesota. He has published over 100 papers related to 

endocrine testing, quality assurance, and health care outcomes. He is an active member 

of numerous societies including CAP, AACC, ACLPS, ATA, CLMA, and the AMA. 
231

Importance of Commutable Reference Materials and Patient Test 

Distributions for Assay Calibration 

Visuals Presented By 

George G. Klee 

Department of Laboratory Medicine and Pathology 

Mayo Clinic and Mayo Foundation 
232

Importance of Commutable Human Specimens and 

Test Value Distributions for Assay Calibration 

Adjustment

George G. Klee, M.D., Ph.D.

Mayo Clinic

Rochester, Minnesota, USA

Clinical Necessity for Clinical Necessity for 

Standardization & Harmonization Standardization & Harmonization

• Differences and changes in analytic setpoints cause major discrepancies in 

clinical decisions.

• Practice guidelines with standardized 

action limits require better 

standardization of laboratory tests.

• Most clinicians assume all test methods 

are equivalent.
233

Two Analytic Control Issues Two Analytic Control Issues

• Precision (scatter)

• Accuracy (bias)

Tolerance Limits for Precision Tolerance Limits for Precision

• Analytic precision is clinically filtered by 

biologic variation

• If analytic SD <1/4 biologic SD, total SD 

only increases by 3%
234

Tolerance Limits for Analytic Bias Tolerance Limits for Analytic Bias

• Bias directly affects test values

• Small analytic changes can produce 

major shifts in frequency distributions of 

clinical test values

Original Positives 2.3%

With Bias 15.8%

+2.0 SDBiol

STANDARD DEVIATES

F

R

E

Q

U

E

N

C

Y

Effect of Analytic Bias on Effect of Analytic Bias on 

Decisions Decisions
235

Frequency Distribution of Cholesterol Frequency Distribution of Cholesterol 

Values

Cumulative Frequency Distribution of Patient 

Data for Total Cholesterol

0

50

100

150

200

250

50 100 150 200 250 300 350

Cholesterol, mg/dL

Number of patients

# of patients

Cutpoint

±L1

±L2

±L3

Frequency Distribution of TSH Values Frequency Distribution of TSH Values

Cumulative Frequency Distribution of Patient 

Data for TSH

0

100

200

300

400

500

600

700

800

900

0 5 10 15

TSH, mUI/L

Number of patients

# of patients

Cutpoint

±L1

±L2

±L3
236

Effect of Analytic Bias on Clinical Decisions Effect of Analytic Bias on Clinical Decisions

Analyte Analytic Bias Patients Multiplier

Cholesterol +1% +3.0% 3.0x

@200 mg/dL +3% +8.8% 2.9x

+10% +21.8% 2.8x

TSH +6% +10.8% 1.8x

@5.0 mIU/L +10% +21.7% 2.2x

+20% +50.8% 2.5x

Problems with Current Laboratory Problems with Current Laboratory 

Standardization Standardization

• Lack of consistency between analytic 

methods even within the same diagnostic 

reagent companies

• Standardization with the same reference 

material does not assure consistency of 

patient results across methods

• Reagent and manufacturing processes can 

not economically maintain adequate 

consistency of analytic set-points.
237

Two Examples of Tests With Standards Two Examples of Tests With Standards 

That Still Have Wide Method Differences That Still Have Wide Method Differences

• Human Chorionic Gonadotropin (hCG)

• Triiodothyronine (T3)

Human Chorionic Gonadotropin ( Human Chorionic Gonadotropin (hCG) 

(Who Third IRP) (Who Third IRP)

College of American Pathologist K-A Survey 2000

Method K-02 K-05 K-01

Abbott Architect 7.36 176.84 841.36

Abbott Axsym 9.87 229.28 969.64

Beckman Access 13.54 158.20 667.14

Biomerieux Vidas/Mini - 247.08 1213.04

Chiron ACS:180 11.02 241.22 946.92

Chiron ACS:Centaur 11.21 240.76 923.22

Dade Dimension HM 0.76 238.88 1093.49
238

Human Chorionic Gonadotropin ( Human Chorionic Gonadotropin (hCG)

(Who Third IRP) continued (Who Third IRP) continued

College of American Pathologist K-A Survey 2000

Method K-02 K-05 K-01

DPC Immulite 9.12 253.78 1060.02

DPC Immulite 2000 12.14 270.82 1269.82

Roche/BMC Elecsys 10/2010 1.56 212.73 968.82

Technicon Immuno-1 5.97 197.01 916.01

Tosoh AIA-Pack BHCG 5.42 230.02 1094.46

Vitros ECI 6.04 222.79 949.76

ALL METHODS 9.37 224.22 953.53

Triiodothyronine (T3) Triiodothyronine (T3)

College of American Pathologist K-A Survey 2000

Method K-02 K-05 K-01

Abbott Architect 115.9 288.5 490.6

Abbott Axsym 108.0 259.1 519.3

Beckman Access 131.4 309.1 660.2

Chiron ACS:180 107.2 260.8 700.2

Chiron ACS:Centaur 124.5 286.7 733.0
239

Triiodothyronine (T3) Triiodothyronine (T3)

continued continued

College of American Pathologist K-A Survey 2000

Method K-02 K-05 K-01

DPC Immulite 74.5 218.8 458.7

DPC Immulite 2000 67.9 199.1 418.6

Roche/BMC Elecsys 10/2010 168.6 438.5 

Technicon Immuno-1 89.8 243.2 497.6

Tosoh AIA-Pack BHCG 101.6 294.0 655.5

Vitros ECI 131.6 301.2 658.1

ALL METHODS 109.0 268.1 574.9

Proposal for Improving Proposal for Improving 

Standardization Standardization

• Develop robust reference methods standardized 

with certified reference materials.

• Develop commutable control materials for 

comparison of routine methods with reference 

methods.

• Develop mathematical algorithms for tracking 

patient test value distributions.

• Use the combination of the traceable results and 

the patient test distribution to adjust assay 

calibrations to homogeneity.
240

Tandem Mass Spectrometry Tandem Mass Spectrometry

• Can have a dual role as a standard reference 

method and a production assay system

• Faster, cheaper and higher quality

• Can simultaneously measure multiple forms of 

an analyte to allow better determination of 

physiologically active forms which could 

potentially lead to better diagnoses and 

treatments

Calibration Adjustment Protocol

Measure 

Control 1

Measure 

Control 3

Measure 

Control 2

Evaluate instruments & 

patient demographics

Evaluate relative to 

absolute warning & 

action limits

Evaluate trending and 

multi-rule algorithms

Measure Specimen 2

Measure Specimen 1

Measure Specimen N

Calibration 

verification

Calibration 

adjustment

Both 

high or

Both 

low

Both OK

Call Supervisor

Evaluate relative to 

decision thresholds

Evaluate trending and 

multivariate algorithms

One 

high 

or

One 

low

Controls vs Specimens
241

Definition of Commutable Definition of Commutable

Webster’s definition for Commutable.

• “Capable of being exchanged or interchanged”

Specific definition for Commutable 

Controls.

• “Controls that perform the same as patient 

samples”

• “Recovery of traceable values with controls 

assures recovery of traceable values with 

patient samples”

Constancy of Normalized Constancy of Normalized 

Distributions of Patient Values Distributions of Patient Values

• For large homogenous groups of 

patients the median and key percentiles 

of test values are remarkably stable.

• For smaller non-homogenous groups 

mathematical corrections for 

demographic and pathophysiologic 

differences produces greater stability.
242

Mathematical Corrections for Mathematical Corrections for 

Demographics Differences Demographics Differences

• Age normalization (Inverse of 

Regression)

• Gender correction

• Corrections for Race and Ethnic groups

• Geographic correction

Mathematical Corrections for Mathematical Corrections for 

Pathophysiology Pathophysiology Differences Differences

• Normalize by creatinine

• Elimination of values from patients in

- End Stage Renal Failure/Dialysis

- End Stage Hepatic Failure

- ICU/CCU

• Subset Ambulatory versus Hospitalized 

Patients. 
243

Illustration of Stability of Patient Test Distributions, Illustration of Stability of Patient Test Distributions, 

Influenced Mainly by Analytic Changes Influenced Mainly by Analytic Changes

• Collected calcium test values 20 days during a 

time when laboratory QC was acceptable.

• Median, 25th, 75th percentiles for each day 

(>1000 pts/day) tracked over time.

• Shift in these percentiles by about 0.1 mg/dL 

occurred after about 10 days.

• Shift in patient distributions corresponded to 

shift of time-averaged QC data.

Patient Calcium Values at the Patient Calcium Values at the 

50th Percentile 50th Percentile
244

Patient Calcium Values at the Patient Calcium Values at the

25th Percentile 25th Percentile

Calcium Pool Calcium Pool - CHEM 1
245

Calcium Pool Calcium Pool - CHEM 3

Mean Analysis of Calcium Pool Mean Analysis of Calcium Pool

CHEM 1
246

Mean Analysis of Calcium Pool Mean Analysis of Calcium Pool

CHEM 3

Conclusions Conclusions

• Lack of laboratory standardization can cause 

major problems in patient care.

• Assay calibration with standard reference 

preparations does not assure harmonization of 

patient test results.

• A combination of commutable control material 

with traceable reference values and 

mathematical algorithms using adjusted 

patient test values could be used for 

calibration adjustment.
247

Reference Systems Models from Other Industries 

Joan Walsh Cassedy 

Executive Director of The American Council of Independent Laboratories 

Measurement traceability is applicable to a variety of industry segments. These 

segments have defined reference systems models that may be adapted to other business 

activities, such as clinical laboratory operations. 

The ACIL membership performs calibration and testing services that require the 

implementation of measurement traceability. The ACIL membership includes industry 

sectors such as civil engineering, environmental sciences, microbiology testing, food testing, mineral analysis, structural evaluation and conformity assessment. These industries 

have adopted measurement traceability as a result of international quality system requirements, the need for comparable data and as a basis for sound scientific practices. 

The foundation for the traceability of measurements is fundamental whenever a 

test result or measurement is performed. Measurement traceability during calibration activities ensures the comparability of data among manufacturers, regulators and users of 

the test results. Calibrations performed without traceability result in questionable, noncomparable and unsubstantiated results. Sound scientific practice is assured by selecting a 

laboratory that is competent. Laboratory competency is defined by international quality 

system standards such as ISO/IEC Guide 25 and ISO/IEC 17025. 

The extent and degree of traceability necessary during calibration is defined by 

each industry. No two industries require the same degree of traceablity for similar activities. Every industry segment develops standards that define the requirements for traceability of the measurement to a national or international reference standard. 

Laboratories adopting conformity assessment standards are implementing the 

ISO/IEC Guide 25 and its recent replacement ISO/IEC 17025. These standards require 

operations that trace the measurement of test items (samples) to the standard used for 

making the measurement. These ISO standards also define the elements required for a 

laboratory to be uniformly defined as competent. 

The international definitions for traceability and calibration define traceability as 

a property and calibration as a set of operations. These terms go hand in hand to identify 

the measurement system. 

Traceability -"property of the result of a measurement or the value of a standard 

whereby it can be related to stated references, usually national or international 

standards, through an unbroken chain of comparisons, all having stated uncertainties". 

Calibration - "set of operations that establish, under specified conditions, the relationship between values of quantities indicated by a measuring instrument or 

measuring system, or values represented by a material measure or a reference 

materials, and the corresponding values realized by standards". 
248

During the calibration activity, the laboratory must establish values realized by 

standards. The source and identification of these standards provide the basis for the 

measurement. If traceability of the measurement is achieved, the documentation from the 

laboratory provides the chain of comparisons to national or international standards. It 

must be noted that the chain must include stated uncertainties. In many cases the standards used for the calibration do not include the uncertainty of the measurement or metrological specification. This results in measurements that do not meet the definition of 

traceability. 

In many industry sectors measurements are sometimes assumed to be traceable. 

Many industries requiring comparability of measurements made by testing laboratories 

identify uniformity of measurements through conformity assessment requirements (also 

termed standards). These requirements define the specific activities for the product or 

program requiring testing. In order for the conformity assessment requirements to be 

completed in a consistent manner many of these requirements specify that laboratories 

perform work within quality system models defined for laboratories. This requirement 

helps to ensure that the testing is performed in a consistent manner and by a laboratory 

that can demonstrate it is competent to perform such testing. 

The quality system models found in the ISO/IEC Guide 25 and its recent replacement, ISO/IEC 17025, are cited and recognized by ACIL member laboratories. 

These quality system models define the quality system elements required by any testing 

and calibration laboratory. They are applicable to any facility or body that calibrates or 

tests. These models provide specific requirements for the laboratory to document the 

measurement calibration and verification activities. 

ISO/IEC Guide 25 requires laboratories to calibrate or verify equipment used in 

testing or calibration. The laboratory defines the extent of the calibration and verification. The extent of the activities including the documentation of the calibration or verification is based on client requirements, data used for meeting product specification or the 

standards cited by the client as required for testing or calibration. The requirement for 

the client to specify the calibration or verification needed is specifically cited in ISO/IEC 

17025. 

Calibration of equipment is often performed using standard reference materials 

(SRM) from a nationally or internationally recognized body. These bodies produce standards that may be certified as a reference material since they are traceable to International 

System of Units (SI) (Système international d'unités) of the primary standard. These 

bodies perform their testing in conformance to the ISO/IEC Guide 25 or ISO/IEC 17025 

requirements. The report or Certificate of Analysis of the reference standard must include all the information from cited requirements for calibration laboratories as the Certificate of Analysis provides the details of the certified reference materials (CRM). 

In the United States, calibration standards are often traced to the National Institute 

of Technology (NIST). NIST provides standard reference materials for a variety of 

measurement activities. These SRMs provide a stated uncertainty and the laboratory uses 

this material as a national or reference standard for calibrating test equipment. Whenever 

NIST reference materials are not available, the laboratory often explores the use of stan-
249

dard reference materials from other internationally recognized bodies that are comparable 

to NIST. 

If these materials are not available from international bodies, then the laboratory 

must review the testing requirements with the client to determine the process to ensure 

traceability to a defined specification. The calibration certificates indicate the traceability 

to national standards with either the uncertainty of the material used for calibration, or 

provide the statement of compliance with an identified metrological specification 

In all of these cases, the laboratory must maintain the certificate of analysis or 

other documentation. This documentation provides the information required for traceability of the measurement to a recognized standard. The documentation must include 

the statement of uncertainty and provide an unbroken chain of documents that show the 

basis for the final reported result. 

Whenever materials for traceability are not available from the national or international bodies, ISO/IEC Guide 25 requires that proficiency testing be conducted. This is 

sometimes referred to as round robin testing. This round robin testing provides the data 

users with information on the comparability of the data between laboratories. 

Laboratories must ensure that reference standards are used for calibration only. 

Reference standards used by laboratories for routine testing may find that the standard 

deteriorates, is destroyed or altered during routine testing use. The laboratory must 

evaluate the use of the reference standards to ensure that it does not mishandle the standard. This mishandling may invalidate the standard or its performance as a reference 

standard. 

The laboratory must ensure that a program for calibration and verification of the 

reference standards exists. Testing may become suspect if a reference standard that was 

calibrated 10 years ago is not recalibrated or verified as to its continuing suitability for 

the testing or calibration being performed. The reference standard certification body (i.e.; 

NIST) or the accredited calibration laboratory sometimes provides information on the 

handling of the material and defines when the reference standard requires calibration or 

verification. This information is presented on the certificate of analysis presented with 

the reference standard. It is important to note here that the reference standard or the container in which the standard resides should be clearly linked to the certificate of analysis. 

This is often identified through a unique serial number or identifier. 

In many testing activities, the laboratory verifies its testing by using one or more 

of the national, reference or working standards. The primary standard is the standard 

used by the national certification body (NIST) to compare the national standard to the 

working standard. In many cases the laboratory routinely uses a working standard that is 

one that may or may not be traceable to a national standard certification body. This 

working standard(s) is verified by analyzing the reference standard to ensure that the 

measurement is comparable to the national standard, which is demonstrated in Figure 1. 
250

ISO/IEC Guide 25 has always required the traceability of measurements 

made by an accredited 

laboratory to national or 

international standards. In 

some industry sectors this 

has not always been possible due to the lack of reference materials available 

from national or international certification bodies. 

In lieu of these reference 

materials all accreditation 

bodies require the use of 

proficiency testing or 

round robin testing to ensure the comparability of 

data. 

With the recent updating of ISO/IEC Guide 25 by ISO/IEC 17025, laboratories 

performing calibrations are specifically required to document and have procedures that 

ensure traceability by defining reporting and operational requirements. 

ISO/IEC 17025 requires the laboratory to have procedures for selecting, using, 

calibrating, checking, controlling and maintaining measurement standards, reference materials used as measurement standards, and measuring and test equipment used to perform 

tests and calibrations. This includes equipment used for ancillary measurements such as 

environmental conditions, when these conditions effect the result of the measurement. 

ISO/IEC 17025 defines separate criteria for calibration and testing laboratories. A 

laboratory providing calibration services must design and operate its program to ensure 

traceable measurements to the International System of Units. This is done by the calibration laboratory through an unbroken chain of calibrations or comparisons to the relevant 

primary standards of the SI units of measurement. 

The international system of units is based at the present on the following seven 

base units. These are length (meter), mass (kilogram), time (second), electric current 

(ampere), thermodynamic temperature (kelvin), amount of substance (mole) and luminous intensity (candela). The unbroken chain of calibrations or comparisons may be 

achieved in several steps carried out by different laboratories that can demonstrate traceability. 

Figure 1: Left shows a testing operation without 

traceability. 

Right identifies a testing operation with all the appropriate traceability items identified. 
251

The following is an example of traceability to mass using a spring scale. Figure 1

presents a pictorial presentation of the use of primary, national or reference and working 

standards. As an example, a load cell calibrates the spring scale to an accuracy of 1% 

with an uncertainty of ± 0.25%. The load cell is certified by dead weights with an uncertainty of ± 0.0625%. The dead weights are a reference standard with a certificate from an 

ISO/IEC 17025 accredited laboratory that includes the dead weight value and the uncertainty. 

The load cell is a working standard that has a certificate from an accredited secondary laboratory that indicates the value and uncertainty for the load cell. The secondary laboratory determines the uncertainty of the load cell using the reference standard of 

the dead weights. The testing laboratory uses the load cell to evaluate the spring scale on 

a quarterly basis to verify the measurements. The uncertainty of the load cell is greater 

than the reference standard of the dead weights. 

As presented in Figure 2, the 

uncertainty is greater when the 

testing is performed using the 

working standards. Metrologists 

and laboratories often use the 

standard of 4:1 to perform measurements. This 4:1 ratio attempts to minimize the uncertainty associated with any measurement. The laboratory must 

attempt to control the measurement in order to maintain this 

uncertainty ratio. As illustrated 

in Figure 2 the uncertainty 

grows very rapidly even when 

this is maintained. Users of testing laboratories accept this uncertainty for costs, ease of performing the measurement and 

final data requirements or specifications. 

The laboratory program for using the spring scale requires the operators to verify 

the spring scale every quarter. This requirement may be more or less frequent depending 

on the scale’s stability and reproducibility over time. The record of the measurement 

identifies the load cell and the reading obtained from the spring scale. The personnel are 

instructed to notify a supervisor when the load cell measurement is not within the specification for the spring scale and determined by the data user laboratory. 

In order to verify the value of the load cell, the laboratory program requires the 

spring scale calibration by the secondary laboratory every quarter. The secondary laboratory follows its procedure for measuring the load cell results using the dead weights. The 

secondary laboratory program may require the replicate results to be averaged and calculated for the standard deviation. This information and the uncertainty of the measurement 

made by the secondary laboratory are compared to the reference standard. 

Figure 2
252

In addition the quarterly readings from the load cell are compiled. The average 

and standard deviation are compared to the secondary standard criteria. The records must 

include the serial number of the reference standard, working standard and all raw and 

calculated values. These records provide the traceability of the results reported by the 

laboratory. A process using a calibrated spring scale measures the results. The scale is 

calibrated using reference standards with a stated uncertainty. The spring scale laboratory determines the uncertainty budget if requested by the client. The records from the 

primary calibration and secondary verification provide an unbroken chain with stated uncertainties for the measurements performed by the laboratory using the spring scale.

A calibration laboratory must design and operate a process to ensure that the 

equipment calibrations are traceable to the International System of Units. A calibration 

laboratory establishes traceability of its own measurement standards and measuring instruments to the SI by means of an unbroken chain of calibrations or comparisons linking 

them to relevant primary standards of the SI units of measurement. The link to SI units 

may be achieved by reference to national measurement standards. National measurement 

standards may be primary standards, which are primary realizations of the SI units or 

agreed representations of SI units based on fundamental physical constants, or they may 

be secondary standards, which are standards calibrated by another national metrology institute. 

When using external calibration services, traceability of measurement are assured 

by the use of calibration services from laboratories that can demonstrate competence, 

measurement capability and traceability. The calibration certificates issued by these laboratories present the measurement results, including the measurement uncertainty and/or a 

statement of compliance with an identified metrological specification. The term "identified metrological specification" means that it must be clear from the calibration certificate which specification the measurements have been compared with, by including the 

specification or by giving an unambiguous reference to the specification. 

Calibration laboratories meeting the requirements of ISO/IEC 17025 are considered to be competent. A calibration certificate bearing an accreditation body logo from a 

calibration laboratory accredited to ISO/IEC 17025 or Guide 25, is sufficient evidence of 

traceability of the calibration data reported. Accredited calibration laboratories maintain 

their own primary standard or representation of SI units based on fundamental physical 

constants. 

When the terms "international standard" or "national standard" are used in connection with traceability, it is assumed that these standards fulfill the properties of primary standards for the realization of SI units. 

There are certain calibrations that currently cannot be made strictly in SI units. In 

these cases calibration includes traceability to appropriate measurement standards such 

as: 
253

• The use of certified reference materials provided by a competent supplier to give a reliable physical or chemical characterization of a material. A competent supplier is often defined as a supplier with registration to an ISO 9000 or equivalent standard. This type of registration provides the purchaser with the assurance that a documented 

quality system is operating within the supplier organization; 

• The use of specified methods and/or consensus standards that are 

clearly described and agreed to by all parties. 

• Participation in a suitable proficiency testing or round robin testing 

program, where possible. 

Testing laboratories must meet the requirements of calibration laboratories unless 

it has been established that the associated contribution from the calibration contributes 

little to the total uncertainty of the test result. When this situation arises, the laboratory 

shall ensure that the equipment used can provide the uncertainty of measurement needed. 

If calibration is the dominant factor in the testing, the testing laboratory must follow the 

requirements of a calibration laboratory. 

Where traceability of measurements to SI units is not possible and/or not relevant, 

the same calibration laboratory requirements are required for the testing laboratory. 

The laboratory shall have a procedure for the calibration of its reference standards. A body that can provide traceability shall calibrate reference standards. Such reference standards of measurement are used for calibration only and for no other purpose. 

The laboratory must demonstrate that the reference standard is not affected or invalidated 

if used for other purposes. Reference standards shall be calibrated before and after any 

adjustment. 

Reference materials shall, where possible, be traceable to SI units of measurement, or to certified reference materials. Internal reference materials shall be checked as 

far as is technically and economically practicable. Checks needed to maintain confidence 

in the calibration status of reference or working standards are performed according to defined procedures. 

The laboratory shall have procedures for safe handling, transport, storage and use 

of reference standards and reference materials in order to prevent contamination or deterioration and in order to protect their integrity. 

Why is this traceability important to both the users of the equipment and test data 

and the laboratory? 

The importance of traceability is founded not only in the quality system program 

requirements for international standards, but it is also founded in sound scientific practices. Worldwide comparability of data is based on the use of the same standards for defining the measured result. One prime example is “time.” Our time standard worldwide 
254

allows comparability to a single atomic clock. Everyone in the world uses this international standard when critical time measurements are required. The extent of the use of 

the international standard for “time” is based on the user requirements. Without this 

common reference point for “time,” both local and international travel, meetings and 

other activities, would be chaos. 

The demonstration of traceability for “time” occurs when documentation identifies and provides the information necessary to trace the recorded time back to this international standard. (i.e., time measurements by the atomic clock) 

The assurance that the measurement is traceable allows the user to compare data 

although the same principle of operation is not used to generate the data. A clock, no 

matter what size, shape or mechanism, measures time. The extent of agreement is based 

on the user’s need to measure time to a defined reference standard. 

For another example, in environmental monitoring, air sampling is performed using a personal air-sampling pump in the field. This pump is a precision rotameter that is 

calibrated using an electronic bubble meter. The following summarizes the laboratory 

activities for traceability of the measurement: 

• Personal air sampling pump set at 2 L/min. 

• Precision rotameter sets this flow rate in the field 

• Measurement uncertainty of the rotameter is maintained by a calibration curve in laboratory using an electronic bubble meter. 

The electronic bubble meter is calibrated by an accredited calibration laboratory and is 

traceable to the ampere or electronic signal generated by the bubble meter. A reference 

standard material is used and the electronic meter has a certificate of calibration, which 

indicates the uncertainty of the measurement. Since the electronic meter calibrates the 

rotameter, the air-sampling rate of the pump is traceable to a reference standard. Through 

measurement and calculations the uncertainty budget may be determined for the flow rate 

measurement performed in the field using the rotameter. 

Any valid certification of measurement has all the information necessary to trace 

standards used to NIST or other internationally recognized calibration body. Recertifying 

calibration equipment is ongoing and not a one time activity. The frequency is determined by manufacturer’s recommendation, but the user’s company quality assurance 

program is paramount. Frequency of use is one determinant, as is the difference in measurement uncertainty between the calibration equipment and the standard. 

Periodic recertification, traceable to NIST, is the heart of any quality assurance 

program. Without stringent recertification, product quality suffers, resulting in product 

unreliability detrimental to public health and safety and the cost of doing business. 

Uncertainty measurements are tools for the calibration house and should not be 

compared between companies. However, because the accredited calibration laboratory 
255

documents the uncertainty and standards used in the measurement, the traceability of the 

measurement is known, which allows the comparison of test data based on sound scientific principles while meeting the needs of the data user 

References 

1. “International Vocabulary of Basic and General Terms in Metrology” (VIM) - BIPM, 

IEC, IFCC, ISO, IUPAC, IUPAP, OIML, 1993. 

2. “New National Standard for Expression of Uncertainty in Measurement”, SI/NCSL 

Z540-2-1997. 

3. NIST, Reference on Constants, Units and Uncertainty at 

http://www.physics.nist.gov/cuu/Uncertainty/bibliography.html 

4. “General Requirements for the Competence of Calibration and Testing Laboratories,”

ISO/IEC Guide 25, 1990. 

5. "General requirements for the competence of testing and calibration laboratories" 

ISO/IEC 17025:1999, First edition 1999-12-15. 

6. “General Methods and Instrumentation”, Annual Book of Standard, Volume 14.02, 

1996. 

7. “General Requirements for Bodies Operating Product Certification Systems”, 

ISO/IEC Guide 65, 1996. 

8. “Guide for General Criteria Used for Evaluating Laboratory Competence”, ASTM 

E548-94, Volume 14.02, 1996. 

9. "Clause 9: Measurement Traceability and Calibration" J. Horlick, NIST, October 

1997. 

10. “The Occupational Environment - Its Evaluation and Control” Edited by Salvatore R. 

DiNardi, Copyright AIHA Press 1997, p. 156, Chapter 8, Principles and Instrumentation for Calibrating Air Sampling Equipment, Peter F. Waldron, MPH, IHIT. 
256

JOAN WALSH CASSEDY 

American Council of Independent Laboratories 

Joan Walsh Cassedy, M.B.A., is the Executive Director of the ACIL, formerly the American Council of Independent Laboratories. She received her M.B.A. from the Wharton 

School of the University of Pennsylvania in 1982 while serving as Director of Information and Staff Director of Studies at the National Petroleum Council from 1972 to 1984. 

In 1984, she served as the Chairman and Co-Founder of the International Management 

Group, Inc. representing clients, including the Society of Toxicology, the International 

Association for Energy Economics and the Transportation Research Forum, in government and public affairs activities. In 1994, she became the President and Chief Executive 

of the King Publishing Group and King Communications Group, both award-winning 

business information services companies. During this time, she was the Associate Publisher of The Energy Daily, Defense Week, New Technology Week, High Performance 

Computing & Communications Week, Military Space, BMD Monitor, Navy News & Undersea Technology, The Brownfields Report and White House Weekly. In addition, she 

was the Associate Producer of White House Chronicle, a weekly public affairs talk show 

broadcast on commercial and public radio and television stations. In 1999, she took her 

present position at ACIL, providing testing, certification and accreditation services to 

U.S. and international companies and governments in such areas as civil engineering, 

environment, microbiology, food testing, mineral analysis, conformity assessment, and 

structural evaluation. 
257

Reference Systems Models from Other Industries 

Visuals Presented By 

Joan Walsh Cassedy 

The American Council of Independent Laboratories 
258

“Reference Systems Models 

from Other Industries”

SUHVHQWHGWR

1,67:RUNVKRSRQ

0HDVXUHPHQW7UDFHDELOLW\

1RYHPEHU

Importance of Traceability

• Foundation of a quality program

• Absence of traceability leads to 

misleading results--chaos
259

Traceability and Uncertainty
260

Same principle of operation not 

necessary

• Example: A spring scale with an 

accuracy of ± 1% could be certified by 

a load cell with an uncertainty of ± 

0.25%, which in turn could be certified 

by dead weights with an uncertainty of 

± 0.0625%

• The load cell is a secondary standard

• The calibrated weights are a primary 

standard

• Secondary standards are used for 

convenience, but must be traceable to 

primary standards.
261

Air Sampling Model

• personal air sampling pump set at 2 

L/min.

• precision rotameter sets this flow rate 

in the field

• measurement uncertainty of the 

rotameter maintained by a calibration 

curve in laboratory using an electronic 

bubble meter

Bottom line

Any valid certification of measurement 

has all the information necessary to 

trace standards used to NIST.
262

Recertifying calibration equipment

• Frequency determined by manufacturer’s 

recommendation, but company’s quality 

assurance program is paramount.

• Frequency of use is one determinant, also the 

difference in measurement uncertainty 

between the calibration equipment and the 

standard.

• Periodic recertification, traceable to 

NIST, is the heart of any quality 

assurance program.

• Without stringent recertification, product 

quality suffers, resulting in product 

unreliability detrimental to public health 

and safety and the cost of doing business.
263

WARNING:

• Uncertainty measurements are tools for 

the calibration house and should not be 

compared between companies.

Resources on uncertainty calculations:

• ANSI/NCSL Z540-2-1997, “New 

National Standard for Expression of 

Uncertainty in Measurement”

• NIST, Reference on Constants, Units 

and Uncertainty at 

http://www.physics.nist.gov/cuu/Uncert

ainty/bibliography.html
264

For more information about ACIL, 

please contact: 

Joan Walsh Cassedy

Executive Director

American Council of Independent Laboratories

1629 K Street, NW, Suite 400

Washington, D.C. 20006

Tel: (202) 887-5872 Fax: (202) 887-0021

E-mail: jcassedy@acil.org

www.acil.org

Traceability

• property of the result of a 

measurement or the value of a 

standard whereby it can be related to 

stated references, usually national or 

international standards, through an 

unbroken chain of comparisons, all 

having stated uncertainties.
265

Calibration

• set of operations that establish, under 

specified conditions, the relationship 

between values of quantities indicated by 

a measuring instrument or measuring 

system, or values represented by a 

material measure or a reference 

materials, and the corresponding values 

realized by standards.

• Source: “International Vocabulary of Basic and General Terms 

in Metrology” (VIM) - BIPM, IEC, IFCC, ISO, IUPAC, IUPAP, 

OIML, 1993.

• Ideal minimum accuracy = 4:1

• 10:1 not uncommon
266
267

Poster Session 

Michael J. Welch, Chair 

National Institute of Standards and Technology, USA 

Held November 2, 2000 

1. SRMs TO SUPPORT ACCURACY AND TRACEABILITY IN HEALTHRELATED MEASUREMENTS 

Jennifer Colbert and Michael Welch 

 NIST 

Background: Inaccuracy in health-related measurements raises overall health 

care costs, results in misdiagnoses, and leads to inaccurate conclusions in clinical studies. 

In addition, lack of certified reference materials (CRMs) hampers traceability, and with 

the EC IVD directive requiring traceability to recognized national standards, there is an 

increased need for CRMs for many analytes.

NIST Activities: NIST has a long history of providing reference materials to support accuracy in health-related measurements. Initial efforts in the 1960’s focused on 

providing pure reference materials to serve as primary calibrators. A series of crystalline 

materials of stated purity and uncertainty were developed for many of the most commonly measured analytes in human blood, such as cholesterol, glucose, uric acid, 

bilirubin, creatinine, and urea. Most such organic compounds cannot be directly assayed 

for purity. The determination of the purity of these materials is carried out at NIST and 

generally involves using a variety of analytical techniques to measure impurities and then 

to subtract the total mass of these impurities from 100 % to determine the purity. 

In the 1970’s, NIST began developing very accurate and precise isotope dilution 

mass spectrometric methods for important serum analytes, both organic and inorganic. 

These methods are described in another section of this document. These methods were 

then applied to the determination of these analytes in human serum-based SRMs. SRM 

909 Human Serum and its successors, 909a and 909b, are lyophilized serum materials 

with certified concentrations of approximately 12 analytes. Other lyophilized serum materials were developed specifically for serum lipids (SRM 1952) and vitamins and carotenoids (SRM 968). 

Because some routine methods demonstrate strong matrix effect biases between 

lyophilized serum and fresh serum, recent SRM activity has focused on providing frozen 

human serum, stored at –80 ° C. Extensive studies using a large variety of routine methods have shown little or no matrix effect problems with frozen serum. SRMs using frozen human serum matrices include SRM 956, certified for many electrolytes, SRM 965, 

certified for glucose, and SRM 1951a, with certified and reference values for cholesterol, 

HDL- and LDL-cholesterol and triglycerides. 

Further information about NIST SRMs can be found at: 

 http://ts.nist.gov/ts/htdocs/230/232/232.htm 
268

2. ID/GC/MS METHODS FOR HEALTH-STATUS MARKERS 

Michael Welch, NIST 

Diagnostic Markers for Diabetes, Cardiovascular Risk, and Other Conditions 

Well-defined organic species in blood have been measured for many decades to 

determine a variety of conditions. Among the most widely measured substances (and 

condition) are glucose (diabetes), cholesterol (cardiovascular disease risk), uric acid 

(gout), urea (kidney function), creatinine (kidney function), and triglycerides (cardiovascular disease risk). 

Measurement Challenge: 

Routine clinical measurements have been developed by a large variety of manufacturers, often using different approaches for a given analyte. Results from these different methods may vary considerably; with no clear-cut means of determining which is 

more accurate, unless reference systems are in place to provide an accuracy base. 

NIST Research Activities: 

Over the last twenty years NIST has developed highly accurate and precise isotope dilution/gas chromatography/mass spectrometry-based methods for the six analytes 

listed above. These methods are recognized as definitive methods by the National Reference System for Clinical Laboratories, and as such, sit atop the measurement hierarchy 

for providing traceability for routine clinical measurements. 

These methods involve careful gravimetric measurement of all critical quantities, 

addition of a known mass of a stable isotope-labeled analog of the compound of interest 

(as an internal standard) to a known mass of serum, equilibration of the resulting solution, 

a series of chemical steps to isolate the compound of interest from the serum matrix, a 

chemical conversion of the compound to a thermally stable derivative, injection of the 

derivatized sample into a specially designed gas chromatograph/mass spectrometer system, and careful measurement of the ratio of the ion signals from the native compound 

and the isotopically labeled internal standard. The ratio is compared with ratios measured 

for standards with known weight ratios of pure native compound and the isotope-labeled 

analog to determine the quantity of native analyte in the serum sample. 

These methods are not intended for use in routine applications. The primary function of such methods is to certify high-level reference materials that can then be used in 

evaluating the performance of routine methods and the accuracy of the value assignment 

of lower order reference materials and controls. To facilitate this transfer of accuracy to 

manufacturers and users of routine clinical measurement methods, NIST has developed a 

series of Standard Reference Materials® with concentrations of important analytes, such 

as those listed above and all of the major electrolytes, in human serum matrices. These 

materials are described in more detail in an accompanying section. 

For a review of the development and use of isotope dilution mass spectrometric 

methods for clinical analytes, both organic and inorganic, see: Bowers, G.N., Fassett, J. 

D., and White V, E., Anal. Chem., 65, 475R-479R (1993). 
269

3. DEVELOPMENT OF A REFERENCE MATERIAL FOR 

 CARDIAC TROPONIN I 

David Bunk, Joseph Dalluge, Michael Welch, NIST 

Robert Christenson (U. of MD), Show Hong Duh (U. of MD) 

Diagnostic Marker for Heart Attack

Cardiac Troponin I is a diagnostic marker for cardiac injury, including myocardial 

infarction. This protein, together with the Troponin T and C subunits, make up the Troponin CIT complex, which regulates heart muscle contraction. After damage to heart 

muscle tissue, Troponin I is released into the bloodstream and can be detected within 

hours and remains at an elevated blood level for days. Compared to other diagnostic 

markers for myocardial infarction, Troponin I has the advantage of being heart-specific; 

it is normally found only in heart muscle tissue. Therefore, its presence in serum is always an indicator of heart muscle tissue damage. 

Measurement Challenge:

Cardiac Troponin I is present in serum after myocardial infarction in very low 

levels (0.1 – 50 μg/L). A variety of post-translational modifications, including phosphorylation and proteolytic cleavage, make Troponin I very heterogeneous in structure. 

In heart muscle tissue, Troponin I is complexed to the Troponin T and C subunits. When 

Troponin is released from the damaged heart tissue after a heart attack, it is not known to 

what degree the I subunit is still complexed to the other subunits. In order to produce an 

effective reference material, the structure of the protein in the reference material has to 

reflect the structure of Troponin I found in serum. The challenges are to identify the 

relevant forms, isolate them from the serum matrix, concentrate them, and measure their 

concentration. 

NIST Research Plan/Activities: 

• Structural characterization of Troponin I standards using LC/MS and LC/MS/MS 

• Round-Robin Study of Troponin I standards and their compatibility with commercial clinical immunoassays 

• Develop methods for measurement and characterization of Troponin I in serum 

using LC/MS 

• Issue Troponin I reference material (purified preparation) 

• Develop serum-based Troponin I reference material 

Collaborators: 

American Association for Clinical Chemistry (including affiliations with Bayer 

Corporation, Beckman Coulter, Abbott Laboratories, Byk-Sangtec Diagnostica, Spectral 

Diagnostics, Ortho-Clinical Diagnostics, Biosite Diagnostics, Dade Behring, Inc., the 

University of Maryland, and the University of Miami). 
270

4. DETERMINATION OF THYROXINE AND CORTISOL IN SERUM BY 

ISOTOPE DILUTION LIQUID CHROMATOGRAPHY-MASS 

SPECTROMETRY WITH ELECTROSPRAY IONIZATION 

Susan Tai, Lorna Sniegoski, and Michael Welch 

NIST 

The determination of thyroxine, a thyroid hormone, is widely used for the diagnosis and therapy control of thyroid disorders. A new method based on isotope dilution 

/liquid chromatography-mass spectrometry using electrospray for ionization (LC/MSESI) has been developed for the determination of total thyroxine in serum. An isotopically labeled compound, thyroxine-d5, was used as an internal standard. Protein precipitation, ethyl acetate extraction and solid-phase extraction were used to prepare samples for 

LC/MS analyses. LC was performed on a C18 column with an isocratic mobile phase. 

(M+H)+

 ions at m/z 778 and 783 for thyroxine and its labeled internal standard were 

monitored for positive ions and (M-H)-

 ions at m/z 776 and 781 for negative ions. Samples of frozen serum pools were prepared and measured on three separate sets. 

Excellent precision was obtained for three levels of serum samples with relative 

standard deviations of the mean ranging from 0.1 % to 0.6 %. Positive and negative ion 

measurements agreed within 0.8 %. The detection limit is estimated to be 30 pg for positive ions, and 20 pg negative ions. The results of the LC/MS-ESI method and field methods compared well with an average difference of 5% for all three levels. 

Cortisol is a steroid hormone that is a diagnostic marker for endocrine function. 

ID/GC/MS methods for cortisol have produced results that are about 20 % to 30% lower 

than field methods. GC/MS is not a robust method for this molecule as it must be derivatized at several positions, resulting in a large molecule that is difficult to get into the 

gas phase. Reproducibility of a GC/MS method developed at NIST was poor. LC/MS 

appears to be a much better alternative. It is polar enough that it should work with polar 

solvent systems and it should have good sensitivity in electrospray ionization. An approach similar to that used for thyroxine will be investigated. 
271

5. CHARACTERIZATION OF THYROID STIMULATING HORMONE AND 

RELATED PROTEINS USING MASS SPECTROMETRY 

Maura Donohue, David Bunk, and Joseph Dalluge 

 NIST 

Diagnostic Marker for Thyroid Function 

Human Thyroid Stimulating Hormone (hTSH) is a glycoprotein hormone secreted 

by the pituitary gland. Its role is in the stimulation of the thyroid gland to produce triiodothyroxine (T3) and thyroxine (T4). Serum TSH levels are used as a diagnostic tool for 

assessing thyroid function. Thyroid conditions such as hyperthyroidism can arise if the 

thyroid gland fails to recognize the presence of TSH, and are characterized by increased 

serum hTSH. 

Measurement Challenge 

TSH is structurally heterogeneous and present at very low levels in human serum, 

and therefore presents a significant measurement challenge. It is measured clinically using immunoassays, which are confounded by differences in antisera binding specificities, 

lack of internationally accepted standards, and the heterogeneity of TSH. These limitations often lead to significant variability in measurements of this analyte using different 

FDA-approved immunoassay kits. 

NIST Research Plans/Activities 

• Development of analytical chemistry-based methodology for the structural characterization and measurement of hTSH in serum based primarily on directly combined liquid chromatography-mass spectrometry (LC-MS). 

• Characterization of a variety of hTSH standards at the molecular level to understand further, the heterogeneity of the analyte, arising predominantly from differential glycosylation of the hormone. 

• Determination of a glycoprotein mass map for hTSH, and the nature and extent of 

site heterogeneity. 

• Production of preliminary well-characterized standards for improvement of between-method variations in calibration of immunoassay measurements. 

• Determination of specific forms of hTSH in patient samples to understand better 

the clinically relevant forms of this compound. 

• Address the challenge of analyte enrichment in patient samples using nonimmunological methodology including combinatorial production of aptamers with 

high binding affinities for hTSH. 

• Development of a suite of clinically relevant standards. 

Collaborators: 

The Mayo Clinic 
272

6. CHARACTERIZATION OF PROSTATE-SPECIFIC ANTIGEN USING 

LIQUID CHROMATOGRAPHY/MASS SPECTROMETRY 

Chad Nelson, Joseph Dalluge, David Bunk, and Michael Welch 

 NIST 

Diagnostic Marker for Prostate Cancer 

Prostate-specific Antigen (PSA) is a single chain glycoprotein (appproximately 7 

% to 8 % carbohydrate) present in the prostate and seminal fluid. The presence of PSA in 

the serum is measured as a diagnostic indicator of prostatic cancer. Clinical laboratories 

currently measure PSA by immunoassays, which are confounded by lack of common 

standards, and the heterogeneity of the antisera and antigen being measured. 

Measurement Challenge 

Measurement of PSA using analytical methods is a significant challenge due to 

the structural heterogeneity of PSA in serum, it’s low concentration in this complex matrix (< 10 ng/mL), and lack of knowledge regarding what compound is actually being 

measured in clinical PSA tests. 

NIST Research Plans/Activities 

• Characterization of a variety of PSA standards at the molecular level to understand further the heterogeneity of the analyte, arising predominantly from differential glycosylation of the protein. 

• Production of preliminary well-characterized standards for improvement of between-method variations in calibration of immunoassay measurements. 

• Determination of specific forms of PSA in patient samples to understand better 

the clinically relevant forms of this compound. 

• Address the challenge of analyte enrichment in patient samples using nonimmunological methodology including combinatorial production of aptamers with 

high binding affinities for PSA. 

• Development of LC/MS and LC/MS/MS methods for the structural characterization and measurement of PSA in serum. 

• Development of a suite of clinically relevant standards. 

Collaborators: 

The Mayo Clinic 

The Johns Hopkins University 
273

7. EVALUATION OF LIQUID CHROMATOGRAPHY/MASS 

SPECTROMETRY FOR THE DETERMINATION OF HOMOCYSTEINE 

IN HUMAN PLASMA 

Joseph Dalluge, Bryant Nelson, Lorna Sniegoski, and Sam Margolis 

 NIST 

Diagnostic Marker for Risk of Heart Disease 

Total homocysteine (tHcy) has emerged as an important independent risk factor 

for cardiovascular disease, as an indicator of nutritional cofactor deficiency (folate and 

cobalamin), and as a contributing factor in the pathogenesis of neural tube defects. 

Measurement Challenge 

tHcy is measured clinically using a variety of methods including immunoassays 

and LC-based methods requiring extensive derivatization procedures and frequent analysis of quality control samples. Because clinical laboratories use a variety of different 

methods for its measurement, interlaboratory comparisons of tHcy measurements are 

poor. 

NIST Research Plan/Activities 

• Develop collaborative effort between NIST and Mayo Clinic to develop 

LC/MS/MS reference method for the measurement of tHcy toward the production 

of standards that will be used to calibrate and validate this and other methods currently used in the clinical community. 

• Development of the method will include the addition of a chromatographic separation prior to MS/MS measurement of tHcy to improve the selectivity and precision of the measurements without significantly compromising analysis time. 

• Investigate development of GC/MS methodology for measurement of tHcy in 

plasma/urine as a second method for confirmation. 

Collaborators: 

The Mayo Clinic 
274

8. PRE-COLUMN EXTRACTION AND LIQUID CHROMATOGRAPHY/ 

ELECTROSPRAY-IONIZATION MASS SPECTROMETRY FOR THE 

QUANTITATION OF FOLIC ACID DERIVATIVES IN HUMAN PLASMA 

Bryant Nelson, Joseph Dalluge, Sam Margolis, Dawit Bezebeh, and Lane Sander 

NIST

Diagnostic Marker for Neural Tube Defects: Folic acid is a water-soluble B vitamin 

that plays a significant role in the development of the central nervous system during the 

early weeks of gestation. A deficiency of folic acid during this time period prevents the 

neural tube from forming correctly, which leads to spina bifida, anencephaly and other 

serious fetal malformations. Folic acid supplementation reduces the risk of recurrent 

neural tube defects (NTDs) and recently, the Food and Drug Administration (FDA) issued a regulation “requiring that all enriched grain products be fortified with folic acid to 

reduce the risk of neural tube defects in newborns.” 

Folic acid is also directly involved in reducing the levels of homocysteine in the 

body. High levels of homocysteine have been linked to the increased risk of cardiovascular disease, cerebrovascular disease, nutritional cofactor deficiency and neural tube abnormalities. Folic acid supplementation reduces the level of homocysteine in the body 

through the transfer of a methyl group from 5-methyl tetrahydrofolate (a folic acid vitamer) to homocysteine to form the amino acid methionine. 

Measurement Challenge: The ability to determine folic acid status accurately in segments of the population at risk for NTD births is a significant challenge. Determination 

of folic acids (folates) in biological samples is extremely difficult due to the existence of 

numerous folic acid metabolites (folic acid exists in as many as eight different biologically important forms), their extreme instability (folic acids are easily degraded by air, 

light, temperature and pH and they frequently interconvert among forms) and their low 

levels in plasma and serum (≤ 10 ng/ml for total folates). 

NIST Research Plans/Activities 

• Develop liquid chromatographic separation methods combined with electrochemical 

(coulometric), fluorescence and electrospray ionization mass spectrometric detection for 

the determination of the five most biologically important folic acid vitamers in plasma 

and serum. 

• Step 1 - Develop the separation and detection methods (completed). 

• Step 2 - Stabilize the folates (on-going). 

• Step 3 - Quantitate the folates in spiked plasma/serum and in control plasma/ serum samples (on-going). 

• Apply the developed LC/EC, LC/Fluor and LC/MS methods to the determination of folic 

acid vitamers in food matrices. 

Collaborators: 

The current collaborators are the Centers for Disease Control and Prevention 

(CDC).
275

9. METHOD DEVELOPMENT FOR THE DETERMINATION OF 

GLYCATED HEMOGLOBIN (HbA1c) 

David Bunk and Joseph Dalluge, NIST

Diagnostic Marker for Control of Diabetes 

Glycated hemoglobin is a diagnostic marker for diabetes and the effectiveness of 

diabetes care. When the blood glucose levels of diabetics rise in an uncontrolled fashion, 

some of the glucose binds to blood proteins. This binding is known as glycation and affects many blood proteins including hemoglobin. Because the lifetime of hemoglobin in 

the blood is several months, monitoring the amount of glycated hemoglobin gives an indication of how well blood glucose levels have remained controlled over the long term. 

Long term control of blood glucose levels is important in order to alleviate the complications that are associated with diabetes. 

Measurement Challenge: 

The challenges are to modify and improve existing methods involving LC/MS 

and CE to more accurately measure the degree of glycation of hemoglobin and to develop 

a reference material that will help to harmonize results among the more than forty glycated hemoglobin assays that are used in clinical labs. 

NIST Research Plan/Activities: 

• Participate in the evaluation of the two reference methods developed by the International Federation for Clinical Chemistry (IFCC) which use LC/MS and LC/CE 

• Modify methods as needed to achieve accuracy and precision goals 

• Prepare and evaluate glycated hemoglobin reference material 

Collaborators: 

International Federation for Clinical Chemistry (including participation by the 

Center for Disease Control and Prevention). 
276

10. METHOD DEVELOPMENT FOR THE DETERMINATION OF 

ALBUMIN IN HUMAN SERUM 

David Bunk, NIST 

Diagnostic Marker for Nutritional Health Status 

Human serum albumin is a diagnostic marker for nutritional health status (in serum), and kidney functions (in urine). Patients who suffer from severe wasting, such as 

those with cancer or advanced HIV, generally have low serum albumin concentrations 

from malnutrition or poor adsorption of nutrients. When albumin is found in urine, it 

generally means that there is kidney damage, as filtration of the blood by healthy kidneys 

excludes serum proteins from urine. 

Measurement Challenge: 

The challenge of serum albumin measurement is dealing with the extreme structural heterogeneity of this protein. The structural heterogeneity combined with the high 

molecular mass of albumin (approximately 67 kDa) makes it difficult to find an internal 

standard for accurate quantitation. In contrast to other protein markers, albumin is present at high levels in serum. 

NIST Research Plan/Activities: 

• Evaluate purified albumin preparation for use as reference materials 

• Develop analytical methods for serum-based quantification of albumin by LC/MS 

Collaborators: 

University of Wisconsin Medical College 

11. METHOD DEVELOPMENT FOR THE DETERMINATION OF 

SPECIATED IRON IN SERUM 

David Bunk and Stephen Long 

 NIST 

Diagnostic Marker for Anemia and Hemochromatosis 

In the anemic patient, a low level of iron in the bloodstream results in reduced 

blood oxygen levels as oxygen is bound to the iron in hemoglobin. In hemochromatosis, 

the body absorbs too much iron from food; this iron can accumulate in tissues such as the 

liver, resulting in damage over the long term. Both anemia and hemochromatosis and 

other iron-related diseases result in an altered level of serum iron concentration, however, 

the distribution of iron among the various iron-binding and iron storage proteins in serum 

provides more clinical information than just the traditional measurement of total serum 

iron. 
277

Measurement Challenge: 

Total serum iron, as currently measured in metrology laboratories using isotope 

dilution mass spectrometry, is generally higher than that measured in clinical laboratories 

using spectroscopic techniques. The cause of this measurement bias is unknown, although it has been speculated that heme contamination in serum is the cause. The iron 

bound to heme is not detected by spectroscopic techniques but does contribute to total 

serum iron values obtained my mass spectrometry. 

The two most prevalent iron-carrying proteins in serum are transferrin and ferritin. In the case of transferrin, the clinical quantity of interest is the degree of transferrin 

iron saturation, i.e., the percentage of transferrin iron-binding sites that are occupied by 

iron. The measurement of transferrin saturation requires both the measurement of the 

amount of iron bound to transferrin and the amount of transferrin in serum. Additionally, 

transferrin measurement is hindered by the structural heterogeneity of this protein due to 

glycosylation. But the characterization of transferrin glycoforms has an addition benefit 

as increased glycoform heterogeneity of transferrin has been linked to chronic alcoholism 

as well as a genetic disease called carbohydrate deficient glycoprotein syndrome. 

Ferritin measurement presents much more of a challenge. Ferritin is present in 

very low levels in serum (10 μg/L to 20 μg/L) and is an extremely large protein. Ferritin 

is made up of 24 subunits and has a molecular mass of approximately 450 kDa. Up to 25 

ferritin isoforms are thought to exist, composed of various combinations of the two primary subunits. Each ferritin molecule is capable of storing up to 4500 iron atoms. 

NIST Research Plan/Activities 

• Discover the source of bias in total serum iron measurements when using mass spectrometry (ICP-MS) as compared to the spectroscopic methods used by clinical laboratories. 

• Characterize the heterogeneity of transferrin standards 

• Develop analytical methods to quantify transferrin in serum by LC/MS 

• Develop analytical methods to quantify the iron bound to transferrin by LC/ICP-MS 

• Prepare a serum-based reference material for transferrin iron saturation. 

• Investigate glycoform heterogeneity in transferrin as a potential reference material for diagnosis of chronic alcoholism 

• Characterize serum ferritin standards and access the degree of structural heterogeneity 

present. 

• Develop analytical methods for isolating and quantifying ferritin from serum using 

LC/MS 

• Prepare serum-based ferritin reference material 

Collaborators: 

The College of American Pathologists 

The University of Minnesota 
278

12. METHOD DEVELOPMENT FOR THE DETERMINATION 

OF BILIRUBIN IN SERUM 

Yolanda Davidson, Lorna Sniegoski, and Michael Welch 

NIST 

Diagnostic marker for liver function 

Bilirubin is a tetrapyrrole produced as a metabolic product that is processed by the 

liver and excreted into the bile. Elevated levels of bilirubin are an indication of impaired 

liver function, such as is caused by hepatitis. In newborns, jaundice is often observed 

from build up of blood levels of bilirubin, caused by hemolysis of erythrocytes exceeding 

the rate at which the liver can conjugate bilirubin and excrete it into the bile. Conversely, 

a recent study reported that the antioxidant properties of bilirubin are beneficial in reducing heart disease. 

Measurement challenge

Bilirubin is a large, thermally labile molecule that is subject to degradation in 

light. There are at least three known isomers and it exists in blood in both conjugated and 

non-conjugated forms. Gas chromatography/mass spectrometry, the methodology of 

choice for determination of many small organic molecules, has not been feasible because 

no thermally stable derivatives have been found. The currently accepted reference 

method is a spectrophotometric procedure, Doumas, et.al., Clin. Chem., 31, 1779-89 

(1985). 

NIST activities 

Initial NIST efforts were focused on development of a crystalline bilirubin material of known purity. This led to the development of SRM 916 and its successor 916a, 

certified for purity and isomeric composition. Reference values are provided for the molar extinction coefficients measured under certain conditions. The certification of this 

material is described in detail in Cohen, et. al., Fresenius J. Anal. Chem., 338, 426-29, 

(1990). 

Recent activities have focused on implementing the spectrophotometric reference 

method at NIST so that it can be used to value assign bilirubin concentrations in serumbased SRMs. Plans are to organize a small round-robin of laboratories that have experience with this method to perform measurements on SRMs with appropriate levels. 

The new technologies now available in liquid chromatography/mass spectrometry 

instrumentation appear to make it possible to measure serum bilirubin levels with unsurpassed specificity and accuracy. NIST will be investigating this approach as an independent means of determining bilirubin. If the two methods provide results that agree, 

the confidence in the accuracy of the result will be very high. 

Collaborators: 

University of Wisconsin Medical College 
279

13. CERTIFICATION OF Pb AND Cd IN SRM 966, TOXIC METALS 

 IN BLOOD, BY ISOTOPE DILUTION ICP-MS AND RADIOCHEMICAL 

NEUTRON ACTIVATION 

Stephen E. Long, Michael S. Rearick, Robert D. Vocke and Elizabeth A. Mackey 

NIST 

Background: A need was identified in the late 1990’s for a whole blood reference material that contained normal as well as elevated levels of selected toxic heavy 

metals, specifically Cd, inorganic Hg, methyl Hg (MeHg) and endogenous Pb. This need 

originated from continued interest in workplace and public health monitoring for Cd and 

MeHg in blood and the large number of participants in proficiency testing programs. In 

addition, new OSHA workplace monitoring standards now require measurement of blood 

Cd. Measurement methods have been developed at NIST for the certification of Pb and 

Cd in SRM 966, a two level bovine whole blood SRM. These are based on isotope dilution inductively coupled plasma - mass spectrometry (ID-ICP-MS), a primary method, 

and for Cd only, radiochemical neutron activation analysis (RNAA). 

The base material for SRM 966 was prepared at a USDA licensed facility under 

the direction of the Centers for Disease Control and Prevention (CDC). Cows were dosed 

with lead nitrate and then bled in units of 500 mL. Multiple pools of blood were prepared. 

One pool was spiked with MeHg (as iodide), inorganic Hg and inorganic Cd. At each 

concentration level, a minimum of 2000 plastic containers were filled with at least 2 mL 

of blood each and frozen at -20°C. The vials were subsequently shipped to NIST for 

value assignment. 

Processing and Measurement: For ICP-MS, approximately 2 g of blood were 

removed from each vial and spiked with a calibrated mixture of 206Pb and 111Cd separated 

isotopes. The samples were digested with HNO3 / HClO4 and analyzed by quadrupole 

ICP-MS. 

For RNAA measurement of cadmium, each vial was freeze-dried at -20 °C and a 

pressure of 20 mtorr, for five days. The dry powder was transferred to linear polyethylene 

bags, and heat-sealed. Samples were placed in a polyethylene rabbit, irradiated for 8 

hours at a neutron flux of 3 x 1013 n.cm

-2

.s

-1 and allowed to decay for 48 hours to permit 

the level of 24Na to decrease. Samples were then digested with HNO3 / HClO4. After digestion was complete, zinc-diethyl-dithiocarbamate (DDC) in chloroform was added and 

the Cd back extracted from the organic DDC fraction into HCl. The Cd fraction was then 

transferred to a polyethylene bottle for γ-ray spectrometry. 

SRM 966, Toxic Metals in Blood 

(Concentrations of Pb and Cd, μg/dL) 

Element Certified Value Information Value

Level 1 Pb 1.56 μg/dL ± 0.05 

Cd <0.4 μg/L 

Level 2 Pb 25.27 μg/dL ± 0.22 

Cd 5.22 μg/dL ± 0.16 
280

Certificate Values: The certified values for Cd were determined by combining 

data from ID-ICP-MS and RNAA following the ISO GUM procedure. The expanded uncertainty is a 95 % confidence interval and reflects the combined effects of measurement 

uncertainty, variability in concentrations between vials, blanks and any systematic differences between the two techniques. 

The certified values for Pb were determined by the primary method, ID-ICP-MS. 

The expanded uncertainties are 95 % confidence intervals and reflect the combined effects of measurement uncertainty, variability in concentrations between vials, and blanks. 

14. NIST STANDARD REFERENCE MATERIALS FOR 

 HUMAN DNA ANALYSIS 

Barbara Levin, Haiyan Cheng, Margaret Kline, Jan Redman, Dennis Reeder, 

Kristy Richie, Catherine O’Connell, and Lois Tully 

 NIST 

The National Institute of Standards and Technology (NIST) has developed three 

Standard Reference Materials (SRM 2390, SRM 2391 and SRM 2392) and is working on 

two more SRMs to provide quality assurance in the analysis of human DNA. SRMs 2390 

and 2391- the DNA Profiling SRM and the Polymerase Chain Reaction (PCR)-based 

DNA Profiling standard - are intended for use in forensic and paternity identifications, for 

instructional law enforcement, or non-clinical research purposes. SRM 2392 is for standardization and quality control when sequencing the entire or any segment of human mitochondrial DNA (mtDNA) for forensic identification, disease diagnosis or mutation detection. It can also be used for quality control when performing PCR and sequencing any 

DNA. SRM 2393, under development, is for p53 mutation detection and consists of 12 

plasmid clones (one wild-type and 11 with single point mutations in exons 5-9). Since 

the p53 gene is the most commonly mutated human cancer gene, this SRM will be useful 

to clinical laboratories examining tumors and laboratories involved in mutation detection. 

A heteroplasmic mtDNA SRM (SRM 2394 also under development) consists of mixtures 

of various concentrations of two mtDNA templates differing in one base pair. This SRM 

is designed for medical, forensic and toxicological scientists who wish to determine the 

limit of detection of low frequency mutations, polymorphisms or heteroplasmic DNA 

sites. 

This work was funded in part by the U.S. National Institute of Justice through the 

NIST Office of Law Enforcement Standards and the NIST Standard Reference Materials 

Program. 
281

15. THE MICRONUTRIENTS MEASUREMENT QUALITY ASSURANCE 

PROGRAM 

Jeanice Brown Thomas, NIST 

The Micronutrients Measurement Quality Assurance (QA) Program was organized to support measurement technology for selected fat- and water- soluble vitamins and 

carotenoids in human serum and plasma. This program was initiated in 1984 as part of 

investigations supported by the National Cancer Institute (NCI) Division of Cancer Prevention and Control to study the possible role of these analytes in reducing the risk of developing certain types of cancers and diseases. The program initially supported a core of 

approximately 10-NCI grantee laboratories and has since expanded to include more than 

60 laboratories worldwide. Today, the program is 100 % fee-supported and is “the only

QA program available internationally for the fat-soluble vitamins and carotenoids.” 

One of the services that the QA program provides laboratories is that of measurement comparability assessment. NIST provides laboratories with the tools for comparability assessment through use of interlaboratory comparison studies, Standard Reference Materials (SRMs) and control materials, and methods development and validation. 

Serum-based samples with assigned values for the target analytes and performanceevaluation standards are distributed by NIST to laboratories for analysis. NIST staff provides the laboratories with technical feedback concerning their performance and suggestions for methods development and refinement. The results from the comparison studies 

are used to establish a laboratory performance database, which is used to help laboratories to improve their measurement comparability and to obtain reliable data needed to 

make accurate clinical and health-care decisions. 

As a result of the QA program, the accuracy of laboratory measurements resulting 

in increased interlaboratory comparability for retinol, ∀-tocopherol, and ∃-carotene has 

improved substantially over time. The average estimated coefficient of variation for retinol and ∀-tocopherol has been approximately 5%, and about 10% for ∃-carotene, for the 

past five years. 

Future plans for FY 2001 include the continuation of the round robin exercises for 

selected vitamins in serum, including the measurement of coenzyme Q10 and vitamin K1. 

As an effort to continue the program beyond FY 2001, a two-phase plan is currently underway to expand the program. Beginning November 2000, an external QA web page 

will be available. Phase I will allow customers to enroll into the program via the internet. 

They will also be able to register on-line for the upcoming QA workshop, which will be 

held as a symposium at the Experimental Biology conference on April 4, 2001 in Orlando, Florida. Phase II (pending resources) will provide electronic data entry and feedback with security capabilities. 

Publications/Outputs:

Brown Thomas et al. “Preparation and Value Assignment of Standard Reference Material 968c: Fat-Soluble Vitamins, Carotenoids, and Cholesterol in Human Serum,”

accepted by Clin. Chim. Acta, September 2000. 
282

Duewer, D.L., Kline, M.C., Sharpless, K.E., and Brown Thomas, J., “NIST Micronutrients Measurement Quality Assurance Program: Characterizing Individual Participant Measurement Performance Over Time,” Anal. Chem.,72, 15, 3611-3619 

(2000).

Duewer, D.L., Kline, M.C., Sharpless, K.E., and Brown Thomas, J., “NIST Micronutrients Measurement Quality Assurance Program: Characterizing the Measurement 

Community’s Performance Over Time,” Anal. Chem., 72, 17, 4163-4170 (2000). 

16. ANABOLIC STEROID CRMS FOR IN VITRO ANALYSIS 

Steven Westwood and Bernard King 

Australian Government Analytical Laboratory, Australia 

The Pure Substance Reference Materials team at NARL has produced a collection 

of anabolic steroid metabolites, conjugates and deuterates. The initial motivation for their 

production was for use as traceable Reference Materials in sports drug analyses, but they 

have the potential for use in general clinical diagnostic, veterinary residue, toxicology 

and forensic testing applications. Key compounds, particularly those used to provide 

quantitative estimations of steroid concentrations in vivo, are available as full Certified 

Reference Materials. 1

 Over one hundred steroids will be produced and characterised 

when the project is completed. 

We have developed a flexible but rigorous approach to establishing the key properties of identity and chemical purity for these materials that in principal confers traceability to the SI units the mole and kilogram, respectively. 2

 

Chemical identity is established by combining information from the synthetic 

route used to prepare the material, comparison with established literature data and firstprinciples examination of the spectroscopic properties of the material using modern instrumental techniques. High-field 1

H and 13C NMR is especially valuable in this regard, 

particularly for establishing with acceptable confidence what can be quite subtle questions regarding the three-dimensional configuration of a molecule. 

Chemical purity is assessed by summation of impurity analysis and subtraction 

from 100% to give the purity estimate for the compound with an accompanying uncertainty budget. Analysis techniques utilised include GC, HPLC, differential scanning calorimetry, thermogravimetric analysis, 1

H NMR, elemental microanalysis and Karl-Fischer 

titrimetry. Isotopic purity is also determined in the case of labeled compounds. Key components of the quality system recommended in ISO Guide 34 3

 for reference material 

producers have been implemented and we have also recently achieved accreditation to 

ILAC G12:2000 for the production of pure substance organic Reference Materials and 

Certified Reference Materials in the areas of anabolic steroids, illicit drugs and agrochemicals. 
283

References: 

1. “The Role of Reference Materials” (ISO Publication 2000-01/1000) 

2. Bernard King and Steven Westwood; “GC-FID as a Primary Method for Establishing the Purity of Organic CRMs Used for Drugs in Sport Analysis”. Accreditation 

and Quality Assurance, submitted.

3. ISO Guide 34:2000 “Quality system requirements for reference materials producers”

17. OBSERVATIONS ON WHO CONSULTATION MEETING ON 

INTERNATIONAL BIOLOGICAL STANDARDS FOR IN-VITRO 

DIOAGNOSTIC PROCEDURES 

(SEPTEMBER 14-15, 2000; GENEVA) 

Rudolf M. Lequin 

Diagnostics Consultancy, The Netherlands 

Attended By: 

International Professional Societies: IFCC, ISTH, ICSH, WASP, IABS, IUPAC, IUIS 

Regulatory Bodies: FDA (CBER / CDRH); EU IVD Taskforce 

Notified Bodies: Japanese Regulatory Agency 

WHO Collaborative Centers: NIBSC, London; CLB, Amsterdam; CBER, Washington 

IVD Industry: EDMA; HIMA (AdvaMed) 

The report of this meeting will be presented to the WHO Expert Committee on 

Biological Standardization (ECBS) meeting to be held first week of November 2000. 

The ECBS has to approve the recommendations! 

Terminology

Biological substance For control of vaccines or therapeutics 

Analyte For in-vitro diagnostic/monitoring procedures 

Note: the analyte may or may not have a defined biological activity 
284

Traceability 

1. Clearly define the analyte in relation to medical needs

2. TYPE A analytes TYPE B analytes

 SI traceable Not SI traceable, but arbitrary units e.g. 

 WHO IUs 

 

 Defined Compounds Poorly defined analytes, especially 

 in biological fluids 

 

 Reference Measurement International Conventional Reference

 Systems (RMS) * 

 

 Measurement Systems ( ICRMS )* 

 

 consisting of consisting of

 

 Reference Measurement International Conventional Reference 

 Procedure ( RMP ) Procedure ( ICRMP ) 

 

 Reference Material ( RM ) International Conventional Reference 

 Material ( ICRM ) 

 

 For many in place For virtually all, not in place 

 * Independent of routine measurement procedures 

3. A network of reference measurement laboratories is needed to perform the measurements 

employing the RMSs or ICRMSs 

4. For Type B analytes, value assignment is proposed as follows: 

Value Assignment

1ST ICRM Arbitrary Figure e.g. 1,10,100 

IU/ampoule (note: medical application) 

ICRMP in place 

2ND ICRM Calibration in terms of 1st ICRM.

Value plus Uncertainty

ICRMP in place 

3RD ICRM Calibration in terms of 2nd ICRM

Value plus Uncertainty 

etc 
285

5. For the purpose of labeling, the following is proposed: 

1ST, 2ND, 3RD etc., WHO (?) International Conventional Reference Material (ICRM) 

of analyte “X”, for in-vitro diagnostic procedures: “Y” IU per ampoule. 

18. THEMATIC NETWORK PROJECT PROPOSAL ON TRACEABILITY 

ISSUES RELATED TO THE EU DIRECTIVE 98/79/EC 

IN VITRO DIAGNOSTIC MEDICAL DEVICES 

Annarita Baldan, Ed de Leer 

Nederlands Meetinstituut, The Netherlands

Background 

This Thematic Network project proposal is under submission to the European 

Commission and it is the response to the dedicated call for proposals that appeared in 

April 2000, under the specific programme Competitive and Sustainable Growth of the 5th

Framework Programme. 

This Network proposal is based on the traceability requirements reported in Annex I A.3 of the EU directive 98/79/EC In Vitro Diagnostic Medical Devices: 

"The traceability of values assigned to calibrators and/or control material must 

be assured through available reference measurement procedures and/or available reference materials of a higher order". 

It aims at the creation of an international network to promote the understanding 

and knowledge transfer of concepts such as traceability, comparability and measurement 

uncertainty in the field of In Vitro Diagnostics. 

Relevance of a Network on IVDs 

• to give support to the essential requirements introduced by the EU directive 

98/79/EC, which will come into force in the year 2003 

• to provide the proper infrastructure (network) and tools (deliverables) for building 

up an IVD traceability chain and for transferring knowledge about metrological 

issues (dissemination) 

• to contribute to the improvement of the quality, accuracy and comparability of 

IVD medical devices' measurement results 

• to set up an international co-operation between a variety of organisations with different backgrounds that are dealing with IVDs 
286

World-wide spirit of the Network 

The strength of this Network is its international character. In order to create a 

traceability network in the IVD area, and to establish new policies and trends that will 

have world-wide effect, it is necessary to cooperate on a global scale. 

The Network project will appoint an international advisory board, the IIGTH, "International Initiative on Global Traceability in Healthcare". This board, composed of 

IVD experts from all continents, has the task of supervising the Network, and to setting 

the main guidelines for the creation of a global consensus.

Furthermore, the Network’s broad composition will provide complete coverage of 

the IVD community: 

• IVD Manufacturers 

• Professional Scientific organisations 

• Certified Reference Materials producers (Europe and United States) 

• National Metrology institutes (Europe and United States) 

• Reference Measurement Laboratories 

• Accreditation bodies (ILAC) 

• Standardization institutes (ISO,) 

• IVD experts 

Objectives and Deliverables 

The Thematic Network is structured in work packages, which will focus on: 

1⁄2 Understanding, and knowledge transfer of metrological issues in the IVD area, realised by the preparation of a booklet on IVD quantities traceable and not traceable to SI, by the creation of a network web site and by making available on 

internet an IVD reference systems data-base; 

1⁄2 investigation of the needs in IVD for new certified reference materials (CRMs) 

and new reference measurement procedures, realised by the preparation of guides; 

1⁄2 global collaboration to harmonise measurement procedures and to mutually recognise reference materials, for the reduction of trade barriers (especially Europe 

and United States); 

1⁄2 identification of measurement areas that need strengthening, and support the organisation of interlaboratory comparison exercises. 
287

19. DEVELOPMENT OF REFERENCE SYSTEMS OF MEASUREMENT 

FOR THE IMPLEMENTATION OF THE IVD DIRECTIVE FOR 

CHEMICALLY DEFINED MEASURANDS 

A European Project Proposal 

Detlef Schiel 

Physikalisch-Technische Bundesanstalt, Braunschweig, Germany

Motivation and Objectives: 

To provide the metrological basis for the implementation of the IVD-Directive by 

means of : 

1⁄2 reliable reference measurement procedures and reference materials for some 

measurands for which traceability to the SI units can be established 

1⁄2 improvement of the accuracy and comparability of routine measurements in clinical chemistry for an enhancement of the quality of heath care through: 

- reliable diagnoses and therapies, 

- strengthening of the confidence in clinical measurements 

- avoiding superfluous repeat measurements that will aid in the reduction of 

health care costs 

On the basis of the Mutual Recognition Arrangement of the national metrological institutes (NMI) to contribute to the preservation of the competitiveness of the EU reference material manufacturers. 

Measurands:

b) Digitoxine, Digoxine Cardiac insufficiency, therapeutic drug monitoring 

c) Testosterone, Estriol Endocrine disorders 

d) PH Acid-base disorders 

e) Potassium Electrolyte balance disorders 

R&D Work:

1⁄2 develop reference measurement procedures (primary methods) linked up to the SI 

system 

 - Isotopic dilution mass spectrometry (except for pH) 

 - Characterization of pure materials of the measurands (a, b) 

 - Development of separation methods as GC or IC (a, b, d) 

1⁄2 international comparison measurements to validate the procedures 

1⁄2 a study to develop the ability to certify matrix-based higher order reference materials 
288

Participants:

The work is divided among eight participants of five EU countries and two collaborating partners from Australia and USA. The consortium is composed of NMI’s, universities, reference institutes for clinical chemistry and other RM producers and manufacturers of diagnostic devices. The partners are highly experienced and recognized in the 

field of chemical analysis in clinical chemistry. 

20. IN-VITRO DIAGNOSTIC MEASUREMENT in LABORATORY MEDICINE 

An EU Proposal 

Rudolf M. Lequin 

Diagnostics Consultancy, The Netherlands 

In a joint effort with IFCC and IRMM a proposal has been developed on In-vitro 

Diagnostic procedures in diagnosis and monitoring of Thyroid Disease entitled: 

“Feasibility to Develop Reference Measurement Systems for Thyrotropin 

(TSH) and for Free Thyroxine (FT4), and Validation of Reference Measurement 

Systems (procedure and material) for Thyroxine (T4) and Triiodothyronine (T3) in 

Human Serum”

Main Deliverables of this project: 

1. Preparation, purification and characterisation of a subgroup of serum TSH glycoforms (possibility to develop an International Conventional Reference Material) 

2. A possible solution to an international agreement of the procedure to separate free 

from bound T4 in human serum without disturbance of the equilibrium. This is 

the first and crucial step to a future reference measurement system for FT4 in human serum, independent of routine measurement procedures for FT4. 

3. Validation of published primary reference measurement procedure (RMP) for total T4 in human serum, and establishment of a primary reference material for T4. 

4. Validation of published primary reference measurement procedure (RMP) for total T3 in human serum, and establishment of a primary reference material or T3.

Note 1 : the results of # 3 and # 4 should be endorsed by an international advisory 

group 

Note 2 : the results of # 2 and # 1 will be communicated to an international advisory 

group who then may recommend to proceed with development of a reference measurement system and an international conventional reference measurement system, respectively. 
289

BREAKOUT SESSIONS: Reports and Discussion 

Held November 3, 2000 

Breakout Session One 

Neil Greenberg, Ph.D., D.A.B.C.C. 

Priorities for National and International Investments in New or Improved 

Reference Systems in Support of Clinical Laboratory Measurement 

Breakout Session Two

Don Powers, Ph.D. 

Reference Materials and Reference Measurement Procedures to Support 

Traceability Requirements of the IVD Directive 

Breakout Session Three

Greg Miller, Ph.D. 

Impact of Method-Material Matrix Interactions on Calibration Traceability 

Protocols for Successful Harmonization of Patient Results 

Breakout Session Four

William F. Koch, Ph.D., F.A.C.B. 

Development of International Consensus on the Credentialing Process for 

Reference Systems 

Breakout Session Five 

Gary L. Myers, Ph.D., F.A.C.B. 

Creating and Sustaining Reference Method Laboratory Networks 
290
291

Breakout Session Reports: Summaries and Discussion 

Willie E. May, Moderator 

Edited for Clarity by Ellyn S. Beary 

National Institute of Standards and Technology 

DR. MAY: Now comes the difficult part of the 

meeting where we have to try to synthesize and 

assimilate all of the hard work of this morning. As 

you know, we are going to do this by having 

reports from the five breakout groups. 

During this morning’s session, I walked 

through and spent some time with most 

of the breakout groups. In reviewing the 

subject matter, and listening to the 

discussions, I think there are some 

common themes. So, what I propose is 

that the groups report back in the 

following order -- group one, group four, 

group five, group two, and group three. 

Groups one, four and five were focused on reference systems, and priority setting. Groups 

two and three were focused on reference materials, reference methods, and they are tools to 

be used as a part of the reference systems. So I think it may be easier to collate the feedback 

from the separate groups if we collect the information in that order. So, if no one objects, 

that's how we'll proceed. 

We will begin with the reports from each of the working group leaders, restricting that 

report to no more than 25 minutes per group. We will allow questions for clarification 

only. I think that we can have a more coherent discussion after all the reports are given. 

During the discussion period, we ask that you use the microphones on either side of the 

room: please state your name, your affiliation, so that we can maintain an accurate 

record. Before we proceed, are there any questions? If not, I'll 

introduce the first speaker, Neil Greenberg. 

Report from Breakout Session One: Priorities for National 

and International Investments in New or Improved 

Reference Systems in Support of Clinical Laboratory 

Measurement, Neil Greenberg, convener. 

Neil Greenberg, Ph.D., DABCC, is a Project Manager, 

Regulatory Affairs, Ortho-Clinical Diagnostics, a Johnson & 

The Discussion came to order at 

1:15 p.m. at NIST in the Green 

Auditorium, Building 101, 100 

Bureau Drive, Gaithersburg, MD 

Willie May, Ph.D., presiding. 

... in some other housekeeping 

rules, this entire session is being 

transcribed. I hope no one has a 

problem with that. The purpose is 

to help us to put together the final 

report of the meeting. 

Transcribed by Neal Gross and Co, Inc. 
292

Johnson Company. He received his doctorate in Physiological Chemistry and Clinical 

Chemistry from The Ohio State University, Columbus, Ohio, and served as a Post 

Doctoral Fellow in Clinical Chemistry at Hahnemann University, Philadelphia. He 

joined Eastman Kodak in 1978, and has held various managerial roles in manufacturing 

and quality assurance for the Ortho-Clinical Diagnostics Vitros (formerly Ektachem) line 

of in vitro diagnostic products. As Manager of Kodak’s Reference Laboratory, he 

became actively involved in standardization of measurement procedures and reference 

materials for laboratory medicine in the early 1980’s, and continues to work actively with 

NCCLS and the American Association for Clinical Chemistry on various aspects of 

measurement standardization for in vitro diagnostics. He is a member of Working Group 

2, ISO/TC 212, which is developing international standards for reference systems in 

laboratory medicine. Dr. Greenberg also is the liaison for ISO/TC 212 Working Group 2. 

DR. GREENBERG: There were about 20 of us in this breakout session, with an excellent 

cross section of people from industry professionals, the people from standards institutes, and 

regulatory bodies. We had a very healthy discussion. We decided to break out our 

discussion in terms of what it is that we needed and then further deal with some of the whos

and the hows. So, to start off with, I’d like to just highlight some of the key points. 

Breakout Session One 

Editor’s Note: Prior to the Meeting Dr. Greenberg provided the following details about 

his discussion group. 

Title: Priorities for National and International Investments in New or Improved 

Reference Systems in Support of Clinical Laboratory Measurement 

Objective: To identify and rank the factors that are necessary for determining priorities 

for development of new reference systems. 

Summary of Topics/Discussion Points: 

• Who determines when there are measurement problems with IVDs? Who are the 

stakeholders? (e.g., government/regulators, professional organizations and/or 

proficiency providers, manufacturers, laboratory personnel, physicians, or 

patients?) How should we balance these perspectives? 

• What metrics are useful in assessment of process capability (PCI) for clinical 

laboratory measurements? What are the right measurement specifications 

(e.g., CLIA performance criteria), and how does the clinical laboratory 

industry perform (generically) relative to an overall objective of "design for 

six-sigma"? 

• What needs have been identified by industry? This topic will include more 

detailed review and discussion of findings from AdvaMed’s (formerly HIMA) 

industry needs/gap analysis (just getting underway). 

• What factors should be considered in a "performance scorecard" approach for 

evaluation of current suitability of available measurement systems for each 

analyte in laboratory medicine? (e.g., volume of testing performed, 

criticality/clinical significance factors, interlaboratory proficiency, etc.) 
293

First of all there was a consensus 

among the participants that we needed 

global reference systems, and these 

reference systems needed to 

harmonize the clinically reported 

results. We also felt it was important 

to minimize redundancies in 

development and reference systems. 

In addition, we want to assure that 

barriers to new technology are not 

created. So, these reference systems 

certainly need to encourage new 

technologies. 

 

Setting Priorities: Key points are as follows: 

We felt it was very important that the numeric results in diagnosis and or management of 

disease states in terms of the degree of overlap between diseased and non-diseased 

populations needs to be taken into account as a factor in determining whether or not there 

was any improvement that could be had by improved standards and improved reference 

systems. 

Participant Affiliation

Dave Armbruster Abbott Labs

David Bunk NIST/CSTL

Mary Burritt Mayo Clinic

Jennifer Colbert NIST/SRMP 

Sylvia Crush-Stanton Boston Biomedica, Inc. 

Yolanda Davidson NIST/CSTL 

Ed de Leer NMi, Delft 

Eleanor Ling Beckman Coulter

John Eckfeldt CAP 

Glenn Ehlers Ortho-Clinical 

Adolfas Gaigalas NIST/CSTL 

Patricia Garrett Boston Biomedica, Inc. 

Farrah Kamalian Bio-Rad Labs 

Rudy Lequin Diagnostics Consultancy 

Bryant Nelson NIST/CSTL 

Max Robinowitz FDA 

John Thompson Dade Behring, Inc. 

Willard Tuthill UL, Inc.

Emil Voelkert Roche Diagnotics 

NIST Workshop on IVD Calibration

Traceability - Breakout Session I

To be successful at a global level, the establishment of a clearly defined, 

credible IVD reference systems project selection and prioritization 

process is required. The process...

• should be an open and public one, that actively seeks out and welcomes 

input from all the relevant stakeholders. 

• should engage industry to participate in project prioritization.

• should use consensus as a decision-making tool

• should be data-driven, and systematically examine a series of key questions 

and factors prior to recommending and/or undertaking a given reference 

systems development program. 

• would engage active support and participation from the laboratory medicine 

community/industry in the selected standards development programs.

The (clinical laboratory medicine) professional sector should play the 

most significant role in setting the priorities.

“Strawman” position/viewpoint...

The figure to the left 

is a summary of the 

features of a globally 

recognized reference 

system, and an 

identification of who 

should be engaged in 

priority setting 

activities. 
294

Another important factor was the clinical outcome for the patients as a result of incorrect 

diagnosis or patient management decisions. If indeed there was going to be some change 

in clinical outcomes envisioned as a result of improved reference systems, that factor had 

to be part of the equation. In addition, demographic factors like disease prevalence and 

the volume of the tests and the number of labs performing the tests should be considered. 

For example, if there are only a small number of laboratories that are conducting a 

specific test, then the need for standardization might not be as great. 

Then we considered success factors such as the “treatability” of the disease, especially in the 

case of broadly used screening tests. If the disease “treatability” is high, then of course 

standardization efforts would be encouraged. 

In addition, current inter-method variability must be 

taken into account as a determining factor. For that, 

we felt that it was important to leverage experience 

from manufacturers. That coupled with data from 

some of the PT (Proficiency Testing) and EQAS 

(European Quality Assessment Scheme) type 

organizations could be extremely valuable in terms of 

helping us to differentiate the tests that demonstrated 

larger amounts of method-to-method variability. 

One of the manufacturers 

present was from a quality 

control product 

manufacturer, and they felt 

that their data from method 

to method variability would 

be extremely valuable. 

NIST Workshop on IVD Calibration

Traceability - Breakout Session I

• The project prioritization process should (1) recognize that there are 

limited resources available to do the work, and (2) take into account 

decision factors such as...

– technical feasibilty and likelihood of success 

– impact (both positive or negative?) on the clinical utility of the 

measurement, if a new or improved standard were to be made 

available 

– public health significance of disease states toward which the given 

measurement is targeted.

– costs 

– estimated time needed for the project to deliver the desired output 

(e.g., materials, methods, or both) 

Strawman Position/viewpoint (continued)...

We talked a great 

deal about their 

project prioritization 

strategy, and this is 

really, I think, the 

heart of the 

discussion, and the 

primary intent of this 

particular breakout 

session. 
295

Identifying stakeholders and 

reference systems needs: 

There is an immediate need 

to inventory and 

communicate information 

about current reference 

systems. There seems to be a 

lack of that information 

available to the general 

public. I know that this is one 

of the projects that has been 

proposed under the IIGTH 

consortium in Europe. We 

have a couple of 

representatives here. 

As we make the transition to new reference systems as we inevitably will, we will need to 

develop new standards. 

 

The following priority lists were excerpted from the draft report on 

Reference Materials and Systems Priorities identified at WHO 

Consultation on International Biological Standards for in vitro

Diagnostic Procedures. 

September 14 –15, 2000, Geneva 

1. International Federation of 

Clinical Chemistry and Laboratory

Medicine 

HCG Isoforms, Lp(a), Myoglobin & 

Troponins, HbA1c, PSA, 

Homocysteine, ATIII, Osteocalcin 

TSH, T4/T3, FV Leiden DNA, 

Alkaline Phosphatase, CRP, S-Transf. 

2. U.S. Food & Drug 

Administration’s (FDA) Center for 

Biologics Evaluation and Research

 

HCV, HbsAG 

Anti-HBs, HIV-1, HIV-2, HIV subtype 

Group M, HIV subtype Group O, HTLV 

3. EU Notified Bodies 

HBsAG reference panel, HCV Ag, 

Anti-HCV genotype reference panel, 

Anti-HIV subtype reference panel, 

HCV-RNA, HBV-DNA, HIV-RNA 

(geno-, subtypes) 

EU Notified Bodies are organizations 

designated by the national governments 

of member states as being competent to 

make independent judgments about 

whether or not a product complies with 

the protection requirements laid down 

by each CE marking directive. 

NIST Workshop on IVD Calibration

Traceability - Breakout Session I

• Defining the stakeholders in the process,e.g....

• government/regulators, professional organizations and/or 

proficiency providers, manufacturers, laboratory personnel, 

physicians, or patients?

• How should we balance or weight the stakeholder 

perspectives?

• What IVD reference system needs have been 

identified by industry? professional associations? 

government authorities?

Some background questions & points of discussion...
296

WHO Priority lists, continued 

Training: In addition, there is an important need to develop tools for training and 

increasing the awareness. That training has to occur in a number of areas. The company’s 

manufacturers, of course, need to train their people, but the clinical laboratories, the 

customers, and the clinicians also need to be trained. We felt that some of those tools could 

be developed by organizations such as the professional societies, IFCC (International 

Federation of Clinical Chemistry and Laboratory Medicine), AACC (American Association 

for Clinical Chemistry), and so on. They can do that in collaboration with some of the 

manufacturers groups such as EDMA (European Diagnostic test Manufacturers Association) 

and AdvaMed (Advanced Medical Technology Association). And in fact, that could build 

on some information, which is already being developed within EDMA to provide guidance 

specifically for manufacturers. 

Global Consortiums: Now, a bit more about the whos and the hows. We talked a lot about 

global consortiums. In fact, we had some very interesting inputs from some of the people in 

the room about other models in some of the other industries, such as the plasma industry, 

where manufacturers consortia have gotten together. 

Those consortia have happened essentially 

through the manufacturing associations for 

those particular industries. They have been 

instrumental in terms of helping those 

particular industries develop standards. 

A couple of examples of this type of 

consortium would be the IIGTH, which I 

have already mentioned, that has formed in Europe to implement the ISO 17511 standard 

(In vitro Diagnostic Medical Devices – Measurement of quantities in samples of biological 

We felt that it was important to 

offer a proposal here that we 

establish a global consortium of 

IVD manufacturers, professional 

societies, NMIs, and regulatory 

bodies. 

4. FDA’s Center for Devices and Radiological Health (CDRH) 

CDRH concurs with statements and suggestions made by the International Council for the

Standardization of Hematology (ICSH) and other Hematology groups about a need for standards in

general hematology, hemostasis, thrombosis and for clotting factor assays. Included in this would

be d-dimers and Leiden Factor V. Development of reference materials are encouraged for any of the

tumor markers being used clinically such as CA 15-3 and CA 125. International standards are also

needed for FISH assays and immunohistochemical assays for molecular markers such as HER2/neu

and other molecular oncology markers. Standards for in vitro allergy testing are needed, and

international reference materials for Lp(a), bone markers, telopeptides and anti cardiolipins. 

5. National Institute of Health, Japan 

Development of viral safety markers applied to the virological safety testing of blood, and blood 

products. 
297

origin – Metrological traceability of values 

assigned to calibrators and control materials). 

There is another consortium, already existing, 

named SOGAT (WHO International WG on the 

Standardization of Gene Amplification Tests) that 

is concerned with the infectious disease marker 

devices. They seem to be having a great deal of 

success with this model. 

Funding sources need to be explored and 

pursued by this proposed consortium (or could be 

multiple consortia - depending on the focus). In fact, we felt that perhaps we ought to leave 

the infectious disease marker activity to the existing organization, SOGAT. Perhaps we 

could pursue expansion of 

the IIGTH model for full 

implementation of some of 

the other so called type A 

analytes, and type B 

analytes and the infectious 

disease markers could be 

left into a special class. 

The funding sources that 

these consortia could 

pursue would certainly be 

the industry type funding 

pools that could be 

brought together through 

industry associations such 

as AdvaMed. Also some 

tax dollar sources, of 

course, because IIGTH is already applying to the European Commission for support. 

Government organizations that have a long history of this type of funding are: U.S. 

Department of Commerce, the National 

Institutes of Health, and Centers for Disease 

Control and Prevention. Active nongovernment foundations include: the 

American Diabetes Association, American 

Heart Association, American Cancer 

Society. This is all sort of American in 

flavor, but these were examples that we 

could think of off the tops of our heads, and 

we know that similar organizations exist in 

Europe and other parts of the world. 

We need to consider approaching 

those organizations in the interest 

of their mission to support 

programs in various aspects of 

health care, to consider funding 

some of these programs for 

reference systems. 

The function of the 

consortium would be to agree 

on projects, channel funding, 

also they would apply for 

funding to various sources 

such as government agencies, 

or even private or 

commercial sources, or nongovernmental sources. 

NIST Workshop on IVD Calibration

Traceability - Breakout Session I

• In determining where there is need for improved 

standardization, which metrics may be useful in assessment of 

process capability (PCI) for clinical laboratory measurements? 

• Can EQAS data be used to define the present state? 

 What are the right measurement specifications (e.g., US 

CLIA performance criteria,German Federal Medical 

System requirements, etc.) ? 

• What international and/or national organization(s) should... 

• Participate in or be represented in the priorization process? 

• Manage the project selection/prioritization process? 

• How should IVD reference systems projects be funded?

Some Additional Concepts and discussion points...
298

Advocacy: We felt that professional advocacy is another important piece of the overall 

program. We felt that this advocacy - even some leadership - could be solicited from 

organizations like the American Medical Association and other medical societies around the 

world. We could have those groups highlight the 

importance of standards to assure quality in 

health care. The smaller professional groups 

such as CAP (College of American Pathologists), 

and AACC, and IFCC, and so forth could follow 

along and support statements that were 

developed in some of the more powerful 

organizations such as AMA. 

Industry advocacy for the investment in standards is extremely important, particularly with 

respect to facilitating funding through organizations or government agencies like NIST. 

We also look to trade organizations such as AdvaMed and EDMA to develop statements 

and advocacy plans. 

These conclude the recommendations from Breakout Session One. 

Breakout Session Four 

Report from Breakout Session on the Development of International Consensus on 

the Credentialing Process for Reference Systems, William F. Koch, convener. 

Dr. William F. Koch is the Deputy Director of the Chemical 

Science and Technology Laboratory at the National Institute 

of Standards and Technology. He began his career at NIST as 

a research chemist in the Center for Analytical Chemistry at 

NBS, where he expanded his research interests to include pH, 

electrolytic conductivity and ion chromatography. He has 

had direct involvement in the development and certification 

of over 70 Standard Reference Materials, and has authored 

over 70 scientific publications. 

Dr. Koch has been an elected member of the Board of Directors of the National 

Committee for Clinical Laboratory Standards (NCCLS) since 1990, and is currently the 

The industry must articulate 

to Congress and the 

Department of Commerce 

how important the standards 

activities are to our success ...

Process Proposed by this Discussion Group: 

• Define a future state: Where would we like to be (in terms of process, 

organization & infrastructure for setting priorities) 3 years from now? 

• Define gaps: Where are we now vs. where do we want to be in the future 

state? 

• Recommend ways to close gaps and get to future state. 
299

Immediate Past President of this organization. He is on the Governing Board of the 

Council for Chemical Research and chair of the CCR Science Education and Human 

Resources Committee. He was recently elected as a Fellow in the National Academy of 

Clinical Biochemistry. He is an active member of the Directors of Industrial Research - 

Analytical Group, the American Chemical Society, the Society for Electroanalytical 

Research, and the American Association for Clinical Chemistry. 

DR KOCH: Group four was charged with developing an international consensus on the 

credentialing process for reference systems. 

There was some hang up on the 

use of the words “credentialing 

process”. Apparently that does 

not translate well for our 

European friends, and so they 

would like to strike the word 

“credentialing”, and call it an 

“approval process for 

reference systems”. 

Breakout Session Four 

Editor’s Note: Prior to the Meeting Dr. Koch provided the following details about his 

discussion group. 

Title: Development of international consensus on the credentialing process for reference 

systems. 

Objective: To develop an international consensus on the credentialing process for 

reference systems. 

Discussion Points: 

Is there a need for an international reference system for clinical laboratory testing 

(IRSCL)? 

 Who are the main players and how would an IRSCL be funded? 

 Who should maintain the IRSCL? 

 Are Mutual Recognition Agreements among nations necessary? 

 Does a process for accreditation or conformity assessment need to be established?

We quickly agreed that 

international consensus 

was necessary and that 

the reference systems 

were necessary. 

Participant Affiliation

Mara Caler Beckman Coulter

Kim Carneiro Danish Inst. of Metrology

Donna Chapman Bio-Rad

Ellen Chen FDA

Nancy Dubrowny Becton Dickinson 

Rene Dybkaer H:S Frederiksberg Hospital 

Bernard King LGC 

Alfred Hartman CAP

Jean-Michel Lacroix Visible Genetics, Inc 

Barbara Levin NIST 

Stanley Lo Med. College of Wisconsin 

Richard Miller Dade Behring

Ronald Ng Abbott Labs 

Mauro Panteghini Azienda Ospeoaliera 

Richard Ross Beckman Coulter

Lothar Siekmann University of Bonn 

Ken Slickers Roche

Lorna Sniegoski NIST 

Beth Ann Wise NCCLS 
300

We spent some time 

deciding or trying to 

figure out whether we 

were talking about 

reference methods, 

reference materials, 

reference laboratories, 

and we went down several 

paths. But I think we 

ultimately came back to the agreement that a reference system was going to be needed, 

and that it was comprised of lots of components. We focused on those components that 

were considered essential elements of a reference system. So, here are our 

recommendations. One, is we need an international reference system that’s composed of 

reference materials, reference methods, and a mechanism – key word mechanism – for 

demonstrating the competence and equivalence for making reference measurements. 

This reference system is 

then realized through 

reference laboratories. 

So, the reference 

laboratories become a 

way of realizing our 

reference system, since 

these reference 

laboratories would have 

demonstrated traceability 

to the international 

references that were 

compatible with the Meter Convention, realizing that not all analytes are going to be 

directly applicable to the convention. 

However, all the things that we want to do 

for the reference lab have to be at least 

compatible with the Meter Convention. 

The reference labs must demonstrate 

competence through accreditation and we 

would charge that it be compatible with 

ILAC. Reference labs will need adequate 

compensation in order to carry this off. 

In addition to compensation incentives, 

they may need other incentives, which 

were left undefined. I think the reference 

labs and the manufacturers will know what 

these incentives are. We have to realize 

An International Reference System, composed of:

• Reference Materials

• Reference Methods

• Mechanism for Demonstrating Competence and 

Equivalence for making Reference measurements

This reference system is realized through 

Reference Laboratories

• Demonstrated Traceability to International 

References compatible with Meter Convention

• Demonstrated competence through 

Accreditation compatible with ILAC)

• Compensation

• Incentives

• Regulation

A bit of history...

Convention du Metré

As far back as 1875 when the Convention du

Metré was signed in Paris, the necessity for

international collaboration and agreement on

metrological issues was recognized. The United

States, while not a world power at the time was

one of the seventeen original signatory nations.

The Convention remains the basis of all

international agreement on units of

measurement. Currently, there are forty-nine

Member States, including all the major

industrialized countries. 
301

that regulation is also a factor that the reference labs have to take into account. We would 

like IFCC to be the main conductor, if you will, of this orchestra. IFCC could use a 

consensus process (perhaps the NCCLS process) to achieve consensus on a reference 

system. Remember that the reference 

system consists of the materials, the 

methods, and the mechanisms to 

ensure equivalency. Other stakeholders 

would necessarily involve WHO, 

CCQM, ILAC, and probably a host of 

other organizations. Unlike breakout 

session one that advocated creating a 

whole new organization, we spent 

some time thinking that we shouldn’t 

have a whole new organization 

because industry was already stressed with the number of memberships they must 

maintain in existing organizations. 

So, we wanted to use existing structures, and IFCC 

seemed to be the most viable of those organizations. 

But we agree on who’s going to pay for this. 

(Laughter) And that would be AdvaMed, MDMA, 

EDMA, AACC, and governments. We didn’t go into 

detail of which government agencies were going to do 

it. They would have to be all the governments, not 

just the U.S. The European governments would have 

participate. 

The next step in our recommendation would be to engage IFCC. Professor Siekmann and 

Dr. Panteghini agreed to talk with the leadership 

of IFCC as soon as they return to Europe to try 

to get this dialogue started, at least to find out if 

IFCC is willing to take this on. IFCC would 

then be in the driver seat. They would have to 

consult the stakeholders, pull together a meeting 

to develop a formal proposal in order flesh out 

the reference system. 

With that proposal, we could go to funding agencies and say, “Here is our proposal. How 

about up fronting some money for it?”

Our group proposes to have a second traceability meeting 

in about a year to discuss the progress in this endeavor. 

This concludes the report from Breakout Session Four. 

Questions? 

Who is going to pull this 

international reference system 

off? 

Perhaps IFCC using consensus 

process (eg. NCCLS-like) 

in cooperation with WHO, 

CCQM, ILAC 

Funded by Members of: 

• AdvaMed 

• MDMA 

• EDMA 

• AACC 

• Governments 

Next Steps: 

• Engage IFCC 

• Consult stakeholders 

• Formal proposal (6 

months) 

• Acquire funding 

“Son of Traceability” 

Meeting to discuss 

progress (one year) 
302

PARTICIPANT: Why can AACC can be a funding organization, and not CAP, for 

example? 

DR. KOCH: We can expand this list. If IFCC takes the lead, and puts the proposal 

together, the stakeholders will decide where they think they can get money. We’ll go to 

as many mortgage companies as we can and let them all pay for it. 

DR. LEQUIN: Lequin, the Netherlands. 

Bill, you mentioned international reference systems. I, and many people now regard that 

we have international reference systems for the type A, well-defined compounds and 

where you have a reference material and method. However, for the type B compounds, 

which were discussed at length at a WHO meeting about six weeks ago, we came to the 

conclusion on the basis of the standard ISO 17511, to call the related reference system an 

international conventional reference system. As I said, this may be playing with words, 

but I think we should start to make this distinction right from the beginning. In that case 

you call the material an international conventional reference material. 

DR. KOCH: So noted. We will take that into consideration. 

DR. DOUMAS: If we can hold other specific questions until later. 

Breakout Session Five 

Report from Breakout Session on Creating and Sustaining Reference Methods 

Laboratory Networks, Gary L. Myers, convener. 

Gary L. Myers, Ph.D. is the Chief, Special Activities 

Branch, Division of Laboratory Sciences at the Centers 

for Disease Control and Prevention (CDC). Dr. Myers 

is responsible for CDC's activities in laboratory 

standardization and methodology research to assure the 

comparability and accuracy of blood lipid and 

lipoprotein measurements performed in support of 

coronary heart disease research worldwide. In this 

capacity he also serves as the Scientific Director of the 

World Health Organization Collaborating Center for 

Reference and Research in Blood Lipids established at 

CDC. He represents the CDC on various national and 

international committees dealing with laboratory 

measurement issues. Currently he serves as Vice-chair of the NCCLS Area Committee on 

Clinical Chemistry and Toxicology. He is a consultant for lipid and lipoprotein 

measurement issues to the College of American Pathologists’ Chemistry Resource 

Committee. He is a member of the ISO/TC 212/Working Group 2 on reference systems. 

Dr. Myers also serves on the Steering Committee for the National Glycohemoglobin 
303

Standardization Program and is a member of the International Federation of Clinical 

Chemistry’s Working Group on HbA1c Standardization. Dr. Myers has received 

numerous awards for his efforts in laboratory standardization and laboratory 

improvement. In 1991 Dr. Myers received the Public Health Service's Special 

Recognition Award for his leadership in improving the measurement of cholesterol in the 

nation's laboratories. 

 

DR. MYERS: Group five was 

charged with discussing 

“creating and sustaining 

reference lab networks”. We had 

a good cross section of 

representatives. We actually had 

representatives from some of the 

networks that are already 

currently in operation. 

We also had representatives from 

manufacturers, and from NMIs 

(such as NIST and IRMM). We 

had some very good discussions 

on some key points concerning 

reference lab networks. 

I think you will see that as I go through just a few of 

the recommendations that we have, there is some 

overlap with some of the other groups that you have 

already heard from, particularly in the area of funding. 

Breakout Session Five 

Editor’s Note: Prior to the Meeting Dr. Myers provided the following details 

about his discussion group. 

Title: Creating and sustaining reference method laboratory networks. 

Objective: To develop criteria necessary for the implementation and 

maintenance of reference laboratory networks 

Discussion points: 

• When is a network needed? 

• How many labs does a network make? 

• Resources needed to establish and maintain a network. 

• How much does a network really cost to operate? 

• Performance requirements (intra- and interlaboratory performance). 

• Function of network in validating traceability. 

• Administration of network: How will this be accomplished? 

• How will manufacturers use the reference laboratory networks? 

Participant Affiliation

Ellyn Beary NIST

Patrick Caines Ortho-Clinical

Joan Cassedy ACIL

Stanley Cernosek Beckman Coulter

Basil Doumas Med. College of Wisconsin 

Mark Manak BBI Biotech

Mary Kimberly CDC 

Masakazu Nakamura Osaka Med. Center 

Paula Radmacher Univ. of Louisville

Curt Rohlfing Univ. of Missouri 

Heinz Schimmel IRMM 

David Secombe Canadian External Q/A 

Michael Welch NIST

Ruth Wu-Wong Abbott Labs 

Funding was obviously a 

key component across all 

of the Breakout Sessions. 
304

One thing that we discussed was that we needed 

to determine where reference lab networks are 

necessary. For this we need to have a needs 

assessment performed of manufacturers to find 

out what they think their needs are, so we can 

concentrate resources in those specific areas. We 

need to know immediately where those 

resources should be focused. 

Another recommendation was that a network 

would require a minimum of three laboratories. 

One lab isn’t sufficient. Two labs are close, but 

you need three labs to have that odd man out. If 

there are discrepancies, you can use the three 

labs to help validate which labs or which 

methods or which results are correct. 

 Then we discussed where the network 

labs actually fit in to the traceability 

chain. I think this is very important, 

because that is based on how the 

manufacturers and other clients might 

use these reference laboratories. 

 

Obviously, one area is to value assign 

manufacturer’s calibrators. That fits 

into the traceability chain. Also, 

another way to support the 

manufacturers is to provide reference values on PT samples for PT providers. Again, we 

are assuming that these are either commutable materials or fresh frozen patient samples. 

These services will help close the loop for manufacturers to determine whether their 

value assignment in the traceability chain is being transferred down to the routine 

laboratory. So, the reference network laboratories can provide input into the overall 

process by providing assistance to PT providers. 

These reference laboratories should provide leadership to the community in reference 

method improvement. Some of the current reference methods have been in existence for 

around 15-20 years. Reference laboratories may be able to provide leadership in coming 

up with improvements in those reference methods. 

Our group thought that it is very important to have an oversight body for operations of 

the network labs to help develop SOPs for the network. I think this fits with what you’ve 

heard from the first two working groups as far as having an oversight body for these 

We recommend that a needs 

assessment be performed, 

and we recommended that 

this be done by AdvaMed. 

We think that this should be 

done immediately before any 

more resources are 

committed in any areas. 

Based on experience we 

felt that three labs per 

network would be a 

minimum requirement. 

Where does the network lab fit in the 

traceability chain? Does it: 

• Value assign manufacturer 

calibrators 

• Provide reference values for PT 

providers 

• Provide leadership in Reference 

Method improvement 
305

reference systems. One example of this would be to design and develop the minimum 

requirements for a transfer protocol to help educate manufacturers. If the manufacturers 

know the requirements when they 

go a reference network laboratory 

or a group of reference network 

laboratories, they may say, "I want 

my value assignment done on two 

days in duplicate." 

When looking at the overall situation, a more appropriate protocol may be developed 

having input from statisticians and so on. So, this is a minimum developed protocol, and 

we feel there should be an oversight group that could help develop these type of SOPs 

which the network labs then would follow, and would also transfer that information to the 

manufacturers. 

Finally, the one area that probably we spent as much time as any on, was the 

development of mechanisms of support for the network labs. Network labs, reference 

labs are expensive. Networks of reference labs are expensive to operate. No doubt about 

that. So, we have to have some ways to fund these laboratories. Creating network labs 

requires a certain amount of developing the infrastructure if it’s not already there. So, one 

suggestion was to approach the Hughes Foundation for funding to create infrastructure. 

Apparently, the Hughes Foundation has money that they invest in the medical and public 

health area. The representative from ACIL said that she would help facilitate this 

approach to looking at the Hughes Foundation to come up with possible funding to help 

build the infrastructure for 

developing reference network 

laboratories. However funding to 

get started is only one part of it, the 

second part is sustaining those 

laboratories. We thought we could 

establish an advisory group of the 

stakeholders for developing 

mechanisms of sustaining these networks. This might be tied into a workshop that 

IRMM is going to have in the spring of 2001. 

The purpose of this meeting would be to develop business plans. We thought that it was 

important to develop a business plan, not only by the individual manufacturers, but by the 

industry in general. AdvaMed might be the group that would facilitate this. 

The network labs could also develop a business plan. What does it cost to operate a 

network laboratory on a sustained basis? To come up with this plan, we felt it would be 

good to include representatives from: CDC and the NGSP who operate networks; and, 

NIST who develops reference and definitive methods. Professor Siekmann, would also 

provide valuable input to this group since he has some experience globally in operating a 

reference laboratory and reference networks in Europe. Stakeholders have to buy into 

An oversight body for operations of the 

network labs would develop SOPs for 

the network 

for example: minimum requirements 

for a transfer protocol 

It is relatively easy to create and 

develop networks, the hard part is 

coming up with the funding to sustain 

those laboratories over one, five, ten 

years or more. 
306

these things. But I think that we have 

to have a very defined approach to 

going and getting this money and to 

deciding how it has to be distributed. 

These decisions must be based on a 

very sound business plan. 

These conclude the recommendations from Breakout Session Five. 

DR. MAY: Clarifications? 

DR. KOCH: My question, Gary, had to do with the number of labs. I’m assuming that 

what you mean is you need to have three labs for any particular measurand, and those 

three labs could be totally different for a different measurand. 

DR. MYERS: They could be. For each measurand there should be at a minimum of 

three network labs in a network. Obviously, each network labs could provide reference 

services for a variety of measurands. It would be most cost effective if one network 

provides services for many measurands. We should try not to create individual reference 

lab networks for one or maybe two measurands, but rather try to have as many as 

possible within that reference lab network. 

 

DR. DOUMAS: Maybe it was one before that there was something where you described 

the three laboratories and what they were going to do. 

DR. MYERS: Where they fit into the traceability chain? 

DR. DOUMAS: No - I got the impression that you thought that these laboratories should 

somehow be accredited, right? Their capability should somehow be accredited, and it was 

not clear to me which process you were thinking about. 

DR. MYERS: The oversight body? 

DR. DOUMAS: No, that’s another one. It was before that. 

DR. MYERS: I think that may have been part of Bill’s. 

DR. DOUMAS: No, because I was at Bill’s session. 

 (Laughter) 

It was part of Bill's, but not in that sense. But anyway, so you didn't treat the mechanism 

part of it. The mechanism for say an accreditation or something like that. 

DR. MYERS: No, we didn't actually talk about the oversight body as far as selecting and 

establishing network labs. That would be done by an oversight body based on the needs. 

Again, the overriding area that we 

discussed is funding. You have to have 

funding to put these reference systems 

into place. You have to have funding to

sustain these reference labs. 
307

Accreditation of the reference labs would fit in more to the discussion on the reference 

system that Bill’s group had. We just recommended that there be an oversight body to 

help develop the SOPs under which those network labs would operate, and also the 

criteria for selecting those reference laboratories. That there needs to be a series of SOPs 

for operation and selection of those reference labs. 

DR. DOUMAS: Do you really mean that that could be a mechanism for selecting the 

laboratories? It would not be open field, so to speak, where the laboratory could offer its 

services and ...

DR. MYERS: Oh, certainly. Certainly. I mean, we would not exclude that. But if there 

is a need and no one volunteers, there still has to be a mechanism possibly to go out and 

select laboratories to fill that need. It is the old idea of looking for champions when you 

talk about developing reference methods. Same thing -- looking for laboratories that 

want to serve as part of a reference lab network. That can go two ways. A lab can 

volunteer, but if there are no volunteers, then there must be a process of solicitation, 

trying to find labs that we feel like are adequate to provide the service. Recruitment. 

DR. KESSNER: Art Kessner from Beckman Coulter. 

I just wanted to understand your mentioning the IRMM meeting in the spring. 

And where was that going to be? Are we talking about an American network with 

American manufacturers meeting overseas to talk about this or do we need to have a 

domestic site to talk about a network system? 

DR. MYERS: No, the purpose there was to have this stakeholders group meet in 

conjunction with an IRMM workshop and I may have to refer to Dr. Schimmel to give 

more explanation on that. We discussed that stakeholders could meet at that time to look 

at a business plan on developing funding mechanisms to sustain the proposed network. 

MR. KESSNER: I was just concerned about the logistics of shipping everybody from 

here over there to talk about it. 

DR. MYERS: Well, it wouldn't be shipping everyone. Stakeholders could send a 

representative. 

DR. MAY: It seems that that's an issue, and perhaps we can discuss that a bit later after 

we've heard all. 

DR. LASKY: Fred Lasky, Ortho Clinical. 

I had a couple of questions. One is just for clarification. You proposed a whole series of 

things that a reference laboratory might do, and I presume that is “wish to do”, and that it 

would be up to the laboratory. One of the things that has come up in discussion, and I 

haven't heard any objections, was that manufacturers laboratories are able to join this 

network and get certified or accredited or whatever. My guess, based on our own 

experience, is that the manufacturer’s laboratory would not be interested in picking up 

any additional services because it is working in a production mode. So, I just wanted to 
308

get clarification that your group agreed with that sort of approach, or if it was discussed 

at all. 

DR. MYERS: Actually, we didn’t discuss that particular aspect of it, but that is 

something that we certainly could consider in our further discussions. 

DR. LASKY: And the second was you mentioned that it would take three labs for a 

viable network, and I do not disagree with your estimate. But I can also envision places 

where there might be a specialized analyte or an orphan analyte. This was discussed in 

the working group I was in. Did your group talk about alternatives if indeed there’s only 

one laboratory, or if there are only two laboratories, and you don’t hit the magic three? 

What then? 

DR. MYERS: Well, that’s where I think you would have to recruit additional laboratories 

to make a network. I mean, our feeling was one lab does not make a network. 

DR. LASKY: Okay. Well, obviously I don’t mean to put you on the spot, but I think 

that’s one of the things that perhaps we should discuss afterwards. 

DR. MYERS: Yes. 

DR. LASKY: This is an important concern, because for instance, if a manufacturer 

comes up has an analyte that they think is going to change the course of health care in 

any particular area, they are the prime source for traceability of the highest order from 

that standpoint. So, we have to consider that also in our discussions. 

DR. MAY: This isn’t the time to do that, but I know there are differing opinions of what 

traceability means, and perhaps we can discuss that later when we have the entire group 

back. Because I think we have to come closer to a unified agreement of what we are 

talking about when we talk about traceability. 
309

Breakout Session Two 

Report from Breakout Session Reference System Infrastructure to Support 

Traceability Requirements, Donald M. Powers, convener. 

Donald M. Powers, PhD is President of Powers Consulting 

Services, providing advisory and training support to the global 

IVD industry in the areas of regulatory compliance, product 

licensing and quality management systems, and an affiliate of 

Quintiles Consulting. Dr Powers served 22 years in the IVD 

industry, most recently as Director of Regulatory Affairs for 

Ortho-Clinical Diagnostics, a Johnson and Johnson company, 

where he had responsibility for worldwide regulatory affairs for 

the Vitros chemistry and immunochemistry systems. 

He previously held various scientific and management positions 

in Quality Assurance, Customer Technical Support, Clinical 

Evaluation and Research and Development at Eastman Kodak’s 

Clinical Diagnostics Division. After earning his PhD in biochemistry from Cornell 

University, he was a research scientist at the National Institutes of Health in Bethesda, 

MD and a clinical chemist at Hahnemann University and Hospital in Philadelphia, PA. 

Dr. Powers is a member of the US Delegation to ISO/TC212 and Convenor of the 

Working Group on IVD product standards. He recently completed two terms on the 

NCCLS Board of Directors and is past chairholder of the Area Committee on Evaluation 

Protocols. He is an active member of RAPS, ASQ and AACC. 

Breakout Session Two

Editor’s Note: Prior to the Meeting Dr. Powers provided the following 

details about his discussion group. 

Title: Reference Materials and Reference Measurement Procedures to 

Support Traceability Requirements of the IVD Directive. 

Objective: To identify practical ways to close important gaps in available 

reference measurement procedures and materials, and to explore ways to 

maximize cooperation among existing organizations developing and 

providing reference systems. 

Discussion Points: 

• Does industry need or want additional reference materials and 

measurement procedures? 

• What elements of the network are in place today? Who are the players? 

• How can they work together to provide the missing elements? 

• When are they needed? When can they be in place? 
310

DR. POWERS: Our group looked at 

the reference system infrastructure to 

support traceability requirements, the 

reference measurement procedures 

and reference materials. 

As others have mentioned, there is a 

lot of overlap with the other reports. 

I think it’s all converging very 

nicely. We took a bit of time in the 

morning figuring out what the future 

states should look like. We did some 

brainstorming, and captured some 

visions or ideas of what should be 

elements of the future state. 

We kept reminding ourselves that 

the goal of traceability is to end up 

clinically relevant reproducible 

results over time and space. And 

at times, we found ourselves 

focusing perhaps a bit too much on 

the analytical or the metrological 

requirements. And we kept bringing 

ourselves back to the ultimate goal. 

We felt a global infrastructure 

needs to be in place yesterday. We 

would take it by 2003, but everyone recognized that it’s really needed now in order to 

allow manufacturers to meet the 2003 requirements of the EU Directive. 

Global kept coming up 

everywhere. There is 

nobody thinking about 

establishing a parallel 

structure on this continent 

to what’s already going 

on in Europe. It needs to 

be collaborative. A lot of 

effort needs to be put into 

making sure that the 

overlaps are minimized, 

and whatever we do 

needs to be cost effective 

in terms of process and 

the end results. 

Participant Affiliation

Nina Chace FDA

Robert Chi Taiwan Committee for 

Clin. Lab. Standards 

William Childress FDA

Sae Choo First Medical

Lou Dunka LifeScan 

Ronald Elin CAP

Patrick Ford Ortho-Clinical 

Theresa Gratiano BBI Technologies 

Chandra Jain Beckman Coulter

Andy Jaunzemis BD Biosciences 

Carolyn Jones AdvaMed 

Patricia Klimley Bio-Rad Labs 

Manfred Kratzer Roche Diagnostics GMBH

Fred Lasky Ortho-Clinical 

Nate Lawrence BBI Technologies 

Ed Levine Diagnostic Products Corp. 

Randie Little U. of Missouri 

Roy Marcus Bayer Corp. 

Laurence Potter Dade Behring, Inc. 

Terry Pry Abbott Labs 

R. Sridharan Instrumentation Lab 

Charles Tonkin UL International, Ltd. 

Steven Westwood Australian Govt. Labs 

Richard White Dade Behring

Thomas Yager Visible Genetics, Inc.

Future State

• Reproducible, clinically relevant results 

over time and space (goal of traceability)

• Global infrastructure in place by 2003 (but

needed earlier)

• Collaborative, not overlapping

• Cost-effective process and results
311

We need reference materials available for at least the medically important analytes. So, 

the prioritization is going to be extremely important. The process needs to be robust. 

I’m talking now about 

the overall global 

transferability process. 

It needs to be easily 

integrated into what the 

industry does today. It 

could be extremely 

disruptive if 

manufacturers have to 

go back and redo 

processes for the 

literally dozens or 

hundreds of analytes. 

There needs to be a 

reasonable transition into the world of traceable values. It’s not going to be an easy thing 

to do for some manufacturers. Also the impact goes beyond manufacturers. It goes to 

the clinical lab and ultimately the physicians. 

They’re going to see changes. If the laboratories change, the physicians are going to see 

change. Those are not going to be welcome. Nobody we’ve heard has been talking about 

what kind of transition is allowed. Two thousand and three seems to be a drop-dead time 

that may not be reasonable. 

In addition, we need accuracy-based proficiency testing as a way to verify that the 

traceability is working as implemented. Along with that goes things like commutable 

survey materials and so on. So, this is not going to be easy. But we feel it’s something 

that’s important. 

We ended up with a number of recommendations, which were consolidated, like I think 

group one. We were favoring the international consortium that was discussed 

yesterday as the foundation for whatever is done. It said “form”, but we really mean 

“expand” what's already in place. The industry is represented at least in Europe. The 

IFCC is representing the professions. A number of metrology institutes, including NIST, 

are represented. So, why not just build on what already exists? 

Stakeholders: We felt like IFCC 

should be a player, but no one 

suggested that they would actually 

be the lead or coordinate the whole 

thing. We also felt medical 

professionals should be involved in 

this, because prioritization of the 

Future State

• Reference materials available for medically 

important analytes

• Robust process, easily integrated into 

industry development processes

• Reasonable transition to traceable values 

• Accuracy-based proficiency testing

Form/expand International 

Consortium within 6 months to 

include all major stakeholders in the 

major geographic areas (eg, medical 

and laboratory professional 

organizations, industry trade 

associations, reimbursement agencies)
312

analytes to be worked on is very important. Also reimbursement agencies since cost is going 

to be a huge factor here. We felt they should play a role. We weren’t sure exactly what, but 

someone is going to pay for this. I don’t think the list was long enough to fund everything 

that needs to be done. 

We see this consortium being developed for the purpose of developing global 

consensus, and then for coordinating the activities. We lumped most of our 

recommendations into this. 

They need to prioritize the 

analytes based on medical 

needs. We did not attempt to 

specify who would do this. So, 

I think it’s understood that this 

would be IFCC and the 

medical professional groups 

working together. 

We were reluctant to specify 

individual organizations here 

because we felt that was the 

purpose of the consortium. 

The seeds of the consortium 

are already there. Let’s expand it, and let them decide among themselves who is in the 

best position to do these things. 

Obviously the development of new reference materials and methods needs to be very 

well coordinated to avoid the redundancies. We didn’t really dwell on that. 

One thing that kept coming up is the need for interpretive guidelines. Manufacturers 

certainly need them. The traceability standards are not exactly something you can sit 

down with and figure out immediately what to do. Someone needs to provide guidelines, 

and we know that EDMA is producing a set of guidelines. It seemed obvious to us that 

the trade associations would probably get this assignment. But again, the consortium 

ought to decide who should do that. 

For NMIs, there needs to be guidelines and procedures in place that they would follow. 

Perhaps they already exist. We didn’t have that knowledge in the room. Also clinical 

laboratories and practitioners are going to need some guidelines on how to cope with the 

changes they’re going to see. 

We talked a bit about clinical traceability and where that fit in. We decided it was 

probably outside the scope. But I think it’s important to recognize that there are analytes 

that have very well established values in the clinical community, and they are not going 

to want to change. 

Summary of Recommendations 

This global consortium will be formed for the 

purpose of developing global consensus and 

coordinating the following: 

• Prioritize analytes based on medical needs 

• Coordinate development of reference 

materials/methods 

• Develop interpretive guidelines for 

manufacturers, NMIs, clinical laboratories 

and medical practitioners to assist 

implementation of traceability requirements 
313

There are clinical trials that have been done over many years, epidemiological studies and 

so on. So, the laboratories are going to figure out how to deal with this, because they will 

be obligated to conform to the analytical traceability requirements. 

Again, we wanted to make sure 

that as we go through all of 

this, the clinical and medical 

communities are very well 

represented, because they are 

going to bear the brunt of a lot 

of these changes. 

This consortium also would be 

charged with establishing and 

maintaining an information 

network. This was a pretty 

important element that kept 

coming up again and again. 

We need a data bank of what materials, methods, reference laboratories are available. 

 

We would see the consortium establishing its 

own web site. They can identify somebody 

who can actually run it for them. There are a 

lot of possibilities there. And then all the 

stakeholders would establish links to this 

web site.

Developing and overseeing processes for approving reference materials and methods, and 

accrediting reference laboratories need to be established. We knew those were being 

covered by other groups, and we didn’t dwell on them. 

We need to make sure there is a global distribution process in place so that reference 

materials are available everywhere in the world to whoever needs them. There are 

systems already in place that 

could be used, and there are 

probably new ones that need to 

be invented. In the end, 

somebody needs to verify the 

effectiveness of what we are 

doing, the processes. Also, the 

impact. Did all this traceability 

activity make an impact on the 

clinical use of these tests? 

It is going to cost a lot of 

money, and it needs to be 

Summary of Recommendations, continued 

 

• Establish and maintain an information 

network to inform users of available reference

materials, methods and laboratories. 

• Develop and oversee processes for 

credentialing reference materials/methods 

and accrediting reference laboratories. 

• Establish and maintain an information 

network to inform users of available reference

materials, methods and laboratories. 

... we could use a web site 

as a way to capture what is 

already available and what 

needs to be done. 

Summary of Recommendations, continued 

 

• Oversee a global distribution process for 

reference materials. 

• Verify effectiveness of processes and impact 

on clinical outcomes. 

• Establish procedures for arbitration of 

differences. 

• Follow up with additional conferences to 

monitor progress and educate stakeholders in 

6-month timeframe. 
314

justified in terms of the impact it has on medicine. We also brought up the possibility of 

differences. How would they be arbitrated if there were disagreements between a 

manufacturer and another party. 

There ought to be some way that these type of differences can be acknowledged and 

arbitrated. We think the consortium ought to be able to deal with these things if it’s made 

up of the right organizations. 

We also felt that a follow up conference to this conference is essential to monitor 

progress. There should be a series, but we were looking more toward the six-month time 

frame, and having at least some people that are here meeting again to see how things are 

going. 

This concludes the report from Breakout Session Two. 

DR. MAY: Questions? Clarifications? 

In reference to your second from the last slide: I remember that it said that the consortia 

should oversee the distribution of reference materials. Can you explain what that means? 

DR. POWERS: Yes. Again, we don’t see this consortium actually doing things, but 

making sure they have the right organizations involved so that everything gets done. We 

would see them overseeing the distribution processes in place today, making sure they’re 

effective, and recognizing gaps and creating new ones. 

DR. WEINMEYER: Hsiao-Mei Weinmeyer, from the University of Missouri, Columbia. 

I think that your working group came out very clearly that the involvement of the medical 

profession in decision making is a very, very important step in this whole process of 

establishing of traceability, and that the clinical outcome is also a very important. 

DR. POWERS: Exactly right. That is the way we felt. 
315

Breakout Session Three 

Report from Breakout Session on the Impact of Method-Material Matrix 

Interactions on Calibration Traceability Protocols for Successful 

Harmonization of Patient Results, Greg Miller, convener. 

Greg Miller, Ph.D. is a Professor in the Pathology 

Department at Virginia Commonwealth University where 

he has been on the faculty since 1977. He serves as 

Director of Pathology Information Systems, Co-Director of 

Clinical Chemistry and Director of Pathology Quality 

Management for the Medical College of Virginia Hospitals 

of the Virginia Commonwealth University Health Systems 

Authority. He received a Ph.D. degree in Biochemistry 

from the University of Arizona in 1973; did post-doctoral 

training in Clinical Chemistry at the Ohio State University; 

and became a Diplomat of the American Board of Clinical 

Chemistry in 1976. Current professional activities include 

Chair of the NCCLS Area Committee on Clinical Chemistry and Toxicology and 

Consultant to the College of American Pathologists Chemistry Resource Committee. 

Past positions include Chair of the AACC Standards Committee and Chair of the AACC 

Lipids and Lipoproteins Division. Research and publications have been in the areas of 

instrument and method performance, quality control, proficiency testing, laboratory 

standardization, and fiber optic immunosensors. 

Breakout Session Three 

Editor’s Note: Prior to the Meeting Dr. Miller provided the following details about his 

discussion group. 

Title: Impact of method-material matrix interactions on calibration traceability protocols for 

successful harmonization of patient results. 

Objective: To develop a statement of limitations in the commutability of reference 

materials and the appropriate use of such reference materials for traceability and 

harmonization of results. 

Discussion Points: 

• Reference material matrix alterations, non-native analyte components and 

commutability with patient specimens 

• Role of authentic patient specimens in traceability validation 

• Commutable reference materials: how to validate performance; role in traceability 

documentation 

• Non-commutable materials: role in method calibration; how to value assign 

• Requirements for commutability between reference materials and patient specimens,

and between method calibrators and patient specimens 

• Impact of matrix modified reference materials on traceability for new technology 

applications 

• Impact of method non-specificity on traceability validation 
316

DR. MILLER: Session three took 

on the topic of “method, material 

and matrix interactions for noncommutability, and its effect on 

traceability protocols”. 

The entire focus of the group was on 

reference materials that have 

assigned values. We didn't consider 

other situations. One key 

recommendation was that it is 

important to validate the 

commutability of the material for all 

the methods for which it is intended 

or designed to be used. 

There is a document from NCCLS EP14, is a matrix evaluation protocol that gives some 

statistical tools, selection of patients, and so on, which can be used to accomplish this. 

We spent most of our 

time dealing with what 

happens if the material is 

found not to be 

commutable. There are a 

number of specific 

recommendations around 

this area. First of all, that 

a patient specimen panel 

can be used to transfer 

accuracy to working 

calibrators. 

There are a couple of 

aspects of using a patient specimen panel. The first is that method specific target values 

can be assigned to reference materials using an appropriate algorithm that allows that 

material to become commutable for a panel of patient specimens. 

Participant Affiliation

Caryl Antalis Roche

Sabine Arends Roche, GmbH 

Edward Ashwood CAP

Steven Binder Bio-Rad

D. Bozimowski Abbott Labs

Joe Bugler Abbott Labs

F. Ceeriotti H.S. Raffaele 

Cynthia Coady Diagnostic Chemicals, Ltd. 

Sherb Edmondson Abbott Labs

F-Javier Gella Biosystems 

Monica Giguere BD Biosciences 

Isabelle Giuliani Bio-Rad 

Omer Guzel Biruni Laboratories

Tuija Halonen PE Life Sciences 

Art Kessner Beckman Coulter 

George Klee Mayo Clinic 

Peter Maxim FDA 

Joseph Murnane UL, Inc. 

Chad Nelson NIST 

Donald Parker Bayer Corp. 

Barbara Pizza Genzyme Corp. 

Nancy Ring Diametrics Medical, Inc. 

R. San George Inverness Med.Tech. 

Nick Smit i-STAT Canada

H-M Wiedmeyer U. of Missouri-Columbia

We really focused on the

reference materials 

themselves, some of 

their limitations, and 

had some specific 

recommendations to 

how to deal with these 

problems. 

If a material is not commutable patient specimen

panel can be used to transfer accuracy to 

working calibrators 

• Method specific target values can be 

assigned to a reference material using an 

algorithm that achieves commutability for a 

patient panel 

• Method specific target values can be 

assigned to a transfer material 

• Method specific target values may be 

reagent lot specific 
317

In addition, method specific target values can be assigned to a transfer material, which 

may be some type of intermediate calibration product or other reference material used 

typically by a manufacturer for an internal process to maintain the transferability process. 

It is important to recognize that when method specific target values are assigned to a 

reference material, they may be specific to a particular reagent lot. Additional work 

needs to be done to demonstrate that the correction factors or the target values can be 

used on subsequent lots in the manufacturing process. 

Second point is that a 

reference material may be 

incompatible with the method. 

By incompatible, we mean 

that it may not actually 

measure the analyte that we 

think it’s measuring. A 

typical example here would 

be the different epitopes in 

immunoassay assay. 

We also discussed an example of 

different lead isotopes ratios and their 

geographic distribution. It may be a 

similar situation in a non-immunoassay 

example. This can look like a noncommutability condition when you do a 

typical commutability validation where 

you compare the reference material 

against a panel of patient specimens. 

A group such as the NCCLS might be an 

appropriate place to undertake the important 

task of developing guidelines for assigning 

method specific target values to reference 

materials. In addition to specific guideline 

development, such a project might include tools, 

statistical procedures, etc. to do these processes. 

 

Patient panels then take on a very important role in the transferability process. We also 

felt it would be important that guidelines be developed on how you select patient panels 

for a variety of purposes. 

Some of the key attributes that we felt should be included in this approach would be that 

the patient panel represent typical measurand conditions and not have unusual molecular 

Reference material may be incompatible with 

the method used in a clinical setting 

• This condition can look like noncommutability 

1⁄2 For example: epitopes in immunoassay; 

 lead isotopes by mass spectrometry 

• In this case, that reference material should

not be used 

... it is important to investigate 

why a non-commutability issue 

exists before jumping to the 

conclusion that a method specific 

target value will in fact deal with 

the situation appropriately.

Guidelines should be 

developed for assigning 

method-specific target values

to reference materials 
318

forms. So, for purposes of 

traceability and calibration, the 

patient panel needs to represent 

the type of analytes that are 

typically encountered in an 

average mix of patients. 

The problem of a particular 

method not reacting correctly 

with epitopes, is considered a 

method problem. Or if in 

some type of pathology, a very 

unusual variant of a molecule 

might be present, that’s a 

method specificity issue that doesn’t need to enter into the situation in which you are 

trying to establish accuracy or calibration traceability. 

For commutability evaluation, the 

patient panel needs to have values that 

are near the values of the reference 

material. But for a transferability 

purpose, the patient panel needs to cover 

the measurement range. 

Other points that must be addressed are: pre-analytical collection storage issues; the 

availability of the panels to people who need them; and, demographics. Different patient 

demographics and ethnic backgrounds can, in fact, influence analyte content and method 

specificity. These are just some of the considerations that we think are important to 

consider before moving forward. 

In addition, patient pools can be used to simulate individual patient specimens if they 

have the appropriate attributes. This obviously represents a material, then becomes a 

reference material of sorts. They can, in fact, substitute for patient panels under the right 

circumstances. 

The NCCLS C37

document, which addresses 

the development of patient 

pools for cholesterol and 

lipid measurements, does 

have protocols on how to 

develop a fresh frozen 

material, and particularly 

contains a protocol how to validate that that pooled material is in fact commutable among 

all the methods for its intended use. We recommend that this be a starting place for the 

Patient Panels; guidelines should be developed

• Represent typical measurand conditions 

• Not have unusual molecular forms for 

traceability use measurand content 

• For commutability: near values of the reference 

material 

• For transferability: cover measurement range 

• Collection and storage issues 

• Availability 

• Demographics 

... patient panels can be used in 

different ways, and it's important 

that, as this guideline is developed 

and clarified, these situations are 

distinguished from each other to 

avoid confusion. 

Patient Pools 

• Intended to simulate individual patient specimens

• NCCLS C37 document provides a procedure for 

preparing and validating pool commutability 

• Stability - reevaluate on a regular basis 
319

validation process. Again, stability of patient pools, just as stability of any other type of 

reference material, is quite important and needs to be reevaluated regularly. 

We felt that harmonization

was another very important 

goal, independent of reference 

methods development or 

traceability issues. However, 

we felt that it was important 

when harmonization exercises 

are performed, that reference 

materials be included. Even if 

they are only candidate 

reference materials, and 

harmonization process is based on patient specimens for any given analyte or measurand, 

it would be useful to make sure reference 

materials are included in the process. In that case, 

when it’s all over, we may have a validated 

reference material that can then be value assigned 

by an appropriate mechanism, either by a 

reference method or a type of consensus process 

of available methods. This would utilize resources 

in the most efficient manner. 

Another very important point 

we discussed was that 

reference materials and 

proficiency testing materials 

are not the same thing. This is 

an important distinction. It is 

desirable for PT materials to be 

commutable. In a perfect 

world, they would be 

commutable, and they could in 

fact play a key role in evaluating the success of accuracy transfer right into the field 

methods. In practice, this is difficult to achieve and is not in fact mandatory for a 

successful proficiency testing program. However, the critical recommendation is that PT 

materials cannot be used as reference materials, and we must not harmonize methods to 

PT results. Due to the frequency of non-commutability problems with these materials, 

you can easily make a mistake in using them. We felt that it was important for education 

on this point to be made available through whatever appropriate mechanisms are out 

there, because people’s experience was that there are laboratories that rely on PT results 

as a way to adjust calibration or make changes, and that would be inappropriate. 

Harmonization is important clinically in 

absence of reference method

• Include reference materials in harmonization 

process 

• Validate reference material commutability in 

process 

• Consensus value can be assigned without a 

reference method 

It is also important to 

harmonize the 

measurement units in the 

process of doing 

harmonization exercises. 

Reference Material versus Proficiency 

Testing (PT; EQAS) material

• Commutability for PT is desirable but difficult 

and not mandatory 

• Do not use PT as a reference material 

• Do not harmonize to PT results 

• Education is needed 
320

There are situations where a stable patient 

population can in fact provide traceable 

calibration for very select analytes. One 

example is for red blood cells the mean 

cell volume of a large population is very 

stable and can in fact be used as a 

calibration point. However, this approach 

must be based on clinical validation 

exercises. 

Also, the role of reference 

materials in new technology 

development is important. As new 

technology to measure analytes or 

measurands is developed, it’s 

important to validate the 

commutability of reference 

materials for that particular 

methodology. 

We also suggest that wherever possible, multi-analyte materials be used because it is 

quite expensive to have a different reference material for each measurand in our 

laboratories. 

 

There are probably many examples where 

existing materials contain the analyte for 

which we’re measuring a new test. 

Somebody just needs to go look for it and 

see if in fact that material can be used rather 

than developing an entirely new one. 

Finally, in the process of reference method 

development, we recommend that in all 

cases, reference materials be developed as a 

parallel process with the reference method

so that we don’t get into a situation where 

we have a method, but suddenly no 

reference material that can be used to 

transfer that to other methods. 

This concludes the reports from the five Breakout Session Conveners. 

Patient Population 

• Stable patient population can 

provide traceable calibration for 

some analytes (e.g. RBC mean cell 

volume) 

• Based on clinical validation exercise 

New technology development 

• Reference Material - validate commutability 

• Encourage use of multi-analyte material 

• For new analyte consider an existing material 

• Must requalify for new analyte 

...when new analytes are 

developed, and the technology to 

measure new analytes is 

developed, it is important to 

consider using an existing 

material if possible. 

... reference materials should 

be developed as a parallel 

process with the reference 

method ...
321

DR. MAY: We will have a coffee break in a few minutes, but I wanted you to know that 

we will re-convene afterwards to discuss the proposals and recommendations. We need 

that input from you because, this evening, a group of us will be sitting down together 

trying to synthesize these recommendations and make them more coherent. I’m sure 

there are a number of issues that need discussion further, and get even further 

clarification. 

I would like to share with you a few things that I jotted down. I’ll give you something to 

talk about when you go out for the coffee break. 

In listening to this discussion, I 

thought it was important to 

differentiate between a certified 

reference material - that is, one that 

is going to be produced in very large 

batches for a distribution world 

wide, and a manufacturer calibrator 

that will be developed primarily to 

support one type of instrumentation. 

I see some heads shaking. Well, 

that’s my perspective. 

It wasn’t clear to me in the discussions that there was a clear distinction between the role 

of national metrology institutes and the role of reference laboratories. Or can they be 

interchangeable? Perhaps they can be. But I think we need to discuss that. 

Who provides CRMs? Are we talking about providing CRMs through the reference 

laboratories? Providing them through NMIs? Providing them through whomever wants 

to provide them? I think that’s important. We have a lot of confusion out there as to what 

a certified reference material means. This is based on what I was getting from the 

discussion, I think we can have further clarification. 

What do we mean by traceability? In my view, certainly this is a complicated area, we’re 

talking about linkages to some stated reference. I’m not going to sit in an ivory tower and 

say it has to be the SI, but I think it has to be an internationally accepted stated reference. 

Then there has to be some agreement on what we mean by linkages to that stated 

reference. When we say an unbroken chain of intercomparisons, what are we talking 

about? There needs to be some consensus on that. 

In much of the discussions, the appropriate lead organization was not indicated. 

Certainly we can’t identify all of the lead players because not everyone in the universe is 

here. But I think we need to be careful to talk about the types of organizations, if not 

specifically who would be appropriate to take the lead in some of these areas. 

Some Points to Consider 

• Difference between CRMs and 

manufacturer calibrators 

• Role of NMIs and role of Reference Labs

• Who should provide CRMs? 

• What do we mean by: 

1⁄2 Traceability 

1⁄2 Linkages to stated reference 

• Identify lead players for action items 
322

Bill (Koch), any housekeeping? I think the coffee 

is ready and it is now slightly past 2:30. We 

should come back at 3:00 for discussion and 

wrap-up. 

DR. MAY: The next order of business, from my perspective anyway, is to have some 

discussion on where do we go from here in terms of getting together again. There have 

been some recommendations that we get together in roughly six months. There have 

been others that say a year. I heard 

some people say two years. I think 

that’s probably too far out. So what’s 

your pleasure? Should we look to try to get together someplace say six to nine months 

from now, or should we wait and think about this -- say the fall of 2001? What’s 

appropriate? I really think we need to hear from the manufacturers here. 

PARTICIPANT: Six months. 

DR. MAY: I think the manufacturers see that 2003 is coming in a hurry. 

 (Laughter) 

PARTICIPANT: I actually think there is an important point to consider. If you want to 

get funding, you need to get into manufacturer’s budget cycles, and you want to make 

sure you do that as early as possible so six months would be advisable. 

DR. MAY: Let me clarify one thing. Certainly from the NIST perspective, we don’t 

expect to have our activities funded based on manufacturer contribution, because 

realistically, they will not be able to fund all the things that we need to do. Certainly 

having some funds from a consortium of the manufacturers certainly shows good faith, 

and that you believe that this is an important issue. 

Six months is sort of on the table. 

Where should the meeting be? There are some people who feel that perhaps looking at 

the needs of the U.S. based manufacturers, we need to have a meeting here. There have 

been other recommendations that perhaps we should alternate this between the U.S. and 

Europe with the thought that if this series continues, maybe it should move around the 

world. 

What is our pleasure? 

PARTICIPANT: Gaithersburg is centrally located. It is not too expensive for 

Europeans, or people from the West Coast. 

(Whereupon, the foregoing 

matter went off the record 

at 2:33 p.m. and went back 

on the record at 3:05 p.m.) 

When should we get together again? 
323

DR. MAY: That is the first time I’ve ever heard that Gaithersburg was the center of the 

world. 

 (Laughter) 

When I came here about 30 years ago, I had this T-shirt, “Where in the heck is 

Gaithersburg, Maryland?”

 (Laughter) 

MR. LEQUIN: I think we should alternate between U.S. and Europe. 

DR. MAY: He's giving me instructions. It's always you, Rudy. 

 (Laughter) 

Rudy knows I like him. That's why I can pick on him. 

DR. DOUMAS: May I suggest that we have an alternating schedule. I think what we 

have done yesterday and today was a marvelous cooperation between people in the 

United States and Europe. I suggest strongly that we would have the next session in 

Europe so we can receive and we can welcome our American friends there. The next 

meeting, as you say, might be even at another place in the world. Just to let everybody 

have the feeling that we work indeed globally. Thank you. 

 

DR. GREENBERG: Yes, I just heard a suggestion that we have the next ISO TC212 

meeting I believe it's in Ireland in June. So, that conceivably is a venue that might be 

appropriate. We could have it a day or two before or a day or two after. So, let's think 

about that. 

DR. MAY: Yes, I guess there are really two competing forces here. One is the 

realization that this truly is a global issue. This is not a U.S. issue alone, so it makes 

sense to think about moving the 

meeting around. However, the other 

consideration is purely financial. I 

think in order to get the right people to 

attend, the costs have to be reasonable. 

Everybody has lots of meetings to go 

to, and we want them to choose the 

next meeting or the next gathering of 

this group as one of the meetings they choose to attend. 

Having it in conjunction with the meeting that Neil mentioned, does that meet the need? 

Is that a meeting that most of you are going to attend anyway? 

PARTICIPANT: I think I want to second that because IFCC meeting follows the TC212 

meeting. So, if you want to get IFCC involvement, have it in between. If people want to 

go to the IFCC meeting, that might work. 

There are two competing forces 

for deciding a meeting place: 

• Global nature of the issue on 

the one hand, and 

• Finances on the other 
324

DR. MAY: Now, who would be willing to organize it? Any volunteers? 

The dates of the meetings are the 6th through the 8th of June. We are talking about 

Dublin as the location. I guess once we get together, we can talk about whether or not we 

need to expand the Steering Committee to accommodate this. So we have decided that 

the follow on meeting will be about seven months from now, and it will be in Dublin. I 

guess that takes care of that issue. 

Just one other general comment in listening to some of the discussions during the coffee 

break, I think there are a number of people who see producing these reference materials 

(because the need is great) as a viable business enterprise. I’ll be quite frank with you. 

We’ve been doing this for a long time, and you do not make a profit at it. To do it 

correctly, it’s going to cost you a lot more than you’re going to be able to charge 

customers. Now, you don't have to take my word for it ... Making reference materials 

in a metrologically sound way costs a lot of money, and you need to be subsidized by 

someone. 

DR. DOUMAS: I just would like to confirm your statement as the second largest 

reference material producer. You will never earn the money you are investing in making 

reference materials if you do it in the proper way. 

DR. MAY: Now we would like to have some further discussions on the reports from the 

working group chairs. Are there any burning questions that you have of any of the 

presenters, and that will sort of trigger the order of the discussion. Or, I can begin 

drafting people. 

DR. SEMERJIAN: I don't remember now 

which group it was, but there was a 

recommendation for a needs assessment. I 

think that this is very important, especially 

since funding must be generated. Also if 

various companies need to be convinced 

that this is going to work, some attempt 

must to be made to quantify what the need is, what the impact will be. I think that's a very 

important activity. I'm not quite sure who would do it or how it would be done. It's not an 

easy task since it's such a diffuse activity, very broad based. I think that this is something 

that we should pay special attention to make sure that it gets done early enough to have an 

impact. 

DR. MAY: I'll just second what Dr. Semerjian just said, and I want to mention 

something about quantifying the impact of the work we do. I think we shared with some 

of you that we have recently done some impact studies (one was for the cholesterol 

standards program), and we found also that it is probably best to think about commercial 

impact in two parts, if you will. That is: 1. What is the impact on commerce of not doing 

certain things; and 2. What would be the benefit of doing certain things on health care 

decision-making based on clinical results. 

... there was a recommendation 

for “needs assessment” ... I 

think that this is a very 

important activity. 
325

DR. MYERS: I just have three clarifications that came up during the discussion at coffee 

break. One was before –

DR. MAY: For the recorder, that's Dr. Gary Myers from CDC. 

DR. MYERS: I'm sorry. You don't recognize me? 

You made a comment before we broke earlier about the difference between the NMIs and 

the reference labs. One thing we discussed in our breakout session and that is the national 

metrology institutes are not set up on a cycle to provide real time reference services. They 

are to anchor these reference systems. The reference methods that we're talking about and 

the reference lab networks are the ones 

that can interact with the manufacturers to 

provide real time reference services as 

needed when they're needed. We don't see 

the organizations like NIST and IRMM 

and so on as being able to provide those 

types of reference services. That's what 

our group defined as one of the 

differences. 

To clarify a comment from Dr. Myer’s report. 

A question was raised during the break concerning the comment I made that one area in 

which the reference network labs could interact in the traceability cycle was to provide 

value assignment to manufacturer’s calibrators. That is not something that the reference 

labs are going to go out and seek. They are providing those services to the manufacturers 

as the manufacturers need those services. Rudy (Lequin) asked during the break if there 

will be quality assurance within the reference lab networks to make sure they’re providing 

the highest quality possible. Yes, we talked about that in great detail. We saw that as 

being one of the operations of the oversight body that develops the SOPs for the 

reference lab networks. One of their 

functions would be to provide QC or PT 

for the network labs themselves, which 

is in addition to what they would 

normally provide as reference services for the manufacturers. This is to make sure 

internally the reference lab networks are providing the best analytical services possible. 

DR MAY: I’d like to ask one question, though. I think we need to be careful and not be 

too prescriptive, and not say 

certainly there are two different 

roles that an NMI has and that a 

reference laboratory has. I don’t 

think you mean to say that an 

NMI might not choose to wear 

both hats. For example, I think 

PTB is wearing both hats in 

Germany, and they are actually 

One difference could be that 

NMIs anchor reference systems 

 while 

Reference Labs provide real 

time reference services 

... laboratory networks would 

have QA procedures in place ...

 NMIs “Wearing Two Hats”

... until these other networks get 

formed, it might be most feasible in 

the short term to have some NMIs 

fulfill the role of “reference lab” if 

they choose to do so.
326

providing some credentialing (I’ll use that word we shouldn’t use) for some of the 

calibrators ... 

 

DR. MYERS: No, my earlier comment was not an exclusion statement. During our 

discussion of reference labs and reference lab networks, that was just one of the ways we 

differentiated between NMIs and the secondary reference labs. 

DR. MAY: Sure. 

DR. MYERS: Looking at more of the real time type services that are needed. 

DR. MAY: I think I understood that, but I was just trying to clarify it so that everyone 

understood what we were talking about. 

MR. LASKY: Fred Lasky, Ortho Clinical Diagnostics. 

There are a couple of points that I discussed with Don Powers during the break that came 

up during our breakout session that I just want to emphasize and be sure it gets on the 

record. 

One was the issue about guidelines and 

guidances. And I think that is going to be 

a key function to give an example, to 

illustrate why it's so important. I think this 

standard on traceability is extremely 

detailed and frankly, very difficult 

standard to understand. It is not for the 

casual reader by any stretch of the 

imagination. In our experience, we passed 

this draft document over to our biochemists in development and they came back and said, 

"Well, it's nice of you to pass it on, but I didn't understand how we're going to deal with 

these things like uncertainty." And it's not a trivial exercise. So, that was just as an 

example to emphasize why we feel that guidelines are so important. Particularly for new 

terms and words for us to deal with as we're rubbing elbows with our friends, the 

metrologists. 

DR. MAY: Let's provide a response to that. Help, Bernard. I think you are planning to 

do that as a part of the consortium that we're putting together, that actually will provide 

some explanatory notes, if you will, regarding traceability in clinical measurements. 

DR KING: 

Bernard King, LGC, UK. 

I'm happy to respond to that. I think there are some developments and some new 

terminologies such as how do you actually achieve traceability, what is this thing called 

measurement uncertainty, which is relatively new to some people. There is a conference 

going on in another part of Washington at the moment - it is the Laboratory of Accreditation 

people who are meeting. One of the two big topics on their agenda is this issue of 

... the guidelines are really 

going to be needed in order 

for a manufacturer to comply 

with the traceability 

requirement in a cost 

effective, reasonable, and 

timely fashion. 
327

traceability and measurement uncertainty. They have just published a new Laboratory of 

Accreditation Standards 17025, which requires the 

use of traceability and measurement uncertainty in 

a fairly firm way for the first time. There have been 

many discussions on this topic, for five years it’s 

been in development at least, and you can see its 

origins further back than that. There seems to be a 

recognition that perhaps this is not such a fearful 

and enormous task as we first thought if we go 

about it in a pragmatic and sensible fit for purpose 

way. I’m not downhearted about these things, and I 

suspect in a year or two, we will not see them as being such big hurdles and big obstacles as 

perhaps some see them now. 

To help that along, and to respond to the 

question you asked, is that in the 

European project that we hope will be 

funded, we see one component of that as 

being able to produce some user 

friendly guides on how to achieve these 

things, and how to understand and 

interpret them for both industry, the 

people who go around doing 

accreditation, and the people in the 

laboratories who have to deal with them 

on a routine basis. I think this task is 

doable. 

DR. MAY: Sure. I think that there are 

other documents that might be useful as 

well. I think it is helpful to realize that 

traceability is not an end unto itself. But 

it’s actually a pathway or tool to be used 

in improving the comparability of 

measurements. That’s really what we’re 

after. To make measurements more comparable, and certainly by using this concept of 

traceability, we believe that that will provide that end. 

DR. LASKY: Thanks. I’m feeling better already. 

 (Laughter) 

DR. MAY: Do you promise? 

DR. LASKY: I work in regulatory affairs. I know that what we need to do is already 

specified; now I’m really concerned about how we do it. I’m encouraged, but I really 

want to emphasize that a year is probably the maximum that we could probably tolerate 

... people all around the world in 

all sectors are asking themselves 

this question: 

How do we establish traceability, 

how do we determine 

measurement uncertainty, and 

how do we do it in a cost effective,

affordable, manner to fit the 

purpose without going way 

overboard on it. 

In the last year, I’ve seen 

quite a substantial move 

from the sort of denial and 

fear – “we can't cope with 

these things” - to 

recognizing that perhaps it

does make sense. 

... traceability is not an end unto 

itself ... it is a pathway or tool 

to be used in improving the 

comparability of measurements.
328

in order to meet the 2003 deadline. However, that’s beside the point. That’s just an 

editorial. I know people understand the urgency.

The other comment that I wanted to make (this also came up in our session) was this idea 

of reference laboratories. Just to bring to light an idea that had surfaced was something 

similar to what NCCLS used to have with their reference system for clinical laboratories. 

Essentially, it was a proposal to reintroduce that concept as an international reference 

system for the clinical laboratories. From my own 

perspective, I think that that was a good start. It 

dealt mostly with papers and methods that are 

certainly invaluable. I don’t want to diminish that. 

But we also need real laboratories to take those 

proposed reference methods and actually work through them and demonstrate that in fact 

they do come up to the standard -- the standard for being a reference method. 

DR. MAY: Are you as a manufacturer willing to provide resources to compensate 

laboratories to do that? I think there are a lot of people that would be more than willing 

to do that work. 

PARTICIPANT: We discussed that extensively today. I want to answer a question that 

you raised before first. Yes, industry addressed their desire to have reference laboratories. 

However, the question is how 

those reference laboratories 

will be financed, and how they 

will survive. I don’t think they 

can depend on grants, and I 

don’t think they can depend on 

some kind of retainer. They 

need volume of work to 

survive. My question is: “Is 

industry willing to support it?” 

There must be a long-term commitment to the laboratory, because a laboratory cannot 

start and invest a lot of money and train people, and then after six months, "Sorry, we 

cannot send you anything anymore." 

DR. MAY: Other concerns? 

DR. SEMERJIAN: I think this meeting in nine months in Dublin is fine, but I also sense 

another urgency: The 2003 deadline will be 

here before we know it. As a result, I think we 

need to do some parallel processing. Just 

because the meeting is in nine months does not 

mean that nothing can be done in between. 

I think the prioritization process has to happen. 

It may not be the final priorities or it may not be 

In terms of starting up new reference 

lab, we should think very, very carefully 

about finances. It’s the volume of work 

that keeps reference labs in business. 

We must keep the people busy and 

guarantee employment. 

A prioritization process must

happen now ... I think there 

is an urgency here, and we 

cannot wait for each little 

step to fall in place. 

I don’t think we should lose

sight of all the good work 

that has gone on before. 
329

the final word, but I think we must attempt to make some decisions, develop some 

consensus on what are the most important analytes. We cannot afford to wait for the 

completion of one step to do the next. 

Even if there is some overlap or maybe some redundancy, I think we need to get going on 

the needs assessment and prioritization, et cetera, even though it may be more efficient in 

the long run to wait for the other step. I don’t think we can afford it time-wise. I would 

like to know if I’m wrong about that. 

 

DR. MAY: I think what Dr. Semerjian is saying, and this is from the U.S. perspective, 

that maybe we need to think about having a meeting to get our house in order prior to the 

next meeting of this group. And I think there has been some discussion bubbling up in 

that regard. 

DR. POWERS: Don Powers. 

I think you’re right. I think we need to move forward with a lot of urgency. But it just 

occurred to me that we ought to keep in mind that the directive, which is imposing the 

2003 deadline, only requires that traceability be made to available reference materials. 

Now may not be the right time to go map out all the other analytes that need reference 

materials, but those that exist, we ought to be using those to figure out how to put a 

process in place that can accommodate the manufacturers, including the reference 

laboratories. I would defer spending time trying to identify all the other analytes that 

don’t yet have reference materials. 

 

DR. MAY: I would like to bring 

up an issue. There are no reference 

materials right now, as I 

understand it, for troponin, and 

glycated hemoglobin, however, 

those are tests are already being 

used by physicians. We know that 

there are measurement problems. 

If we just stop our efforts on that, 

all of a sudden, come 2003, 2005, 

or whenever, we’ve got this big 

gap. So, again, I think you almost 

have to do some parallel 

processing. 

DR. COLLINS: Yes, I agree with that. 

That is a different question, though. They 

are not driven by the 2003 timeline. You do 

need those, but for manufacturers who are 

trying to meet the IVD directive 

requirements, they need to identify those 

materials that are available today that they 

I think we need to think about how 

we can use the existing tools to put 

together reference systems, but I 

think part of our activities have to 

be looking forward, also 

developing those next set of tools to

be used to put together reference 

systems for analytes that are 

becoming more and more clinically

relevant. 

For the purpose of the directive, 

manufacturers need to identify 

those materials that are available 

today that they can show 

traceability to and focus on those.
330

can show traceability to and focus on those. Those driven by medical needs ought to 

continue anyway, but on a different timeline. 

DR. MAY: I really think that exactly what you said probably should be the focus of a 

meeting a few months from now with the manufacturers that are present here. That 

should probably be one of the points of emphasis for that meeting. 

DR. COLLINS: Sure. 

DR. BINDER: Steven Binder from Bio-Rad. 

I would like to agree with the previous 

speaker that when we talk about prioritization, 

we are trying to meet several different needs 

here: regulatory and medical. The medical 

need really has two parts. One is the need to 

produce better numbers, which are useful to 

the patients. I agree with you completely, that is something we should all focus on. But 

another medical necessity, is this idea of eliminating repeat testing by making methods 

that have the appearance of being more reliable to physicians, and that’s a very, very large 

goal. I feel that if we focus on that goal of trying to give physicians a higher level of 

confidence, we may well end up with a two-tiered system, where we have a very small 

number of analytes that have this glow of certification around them and a large other 

number of analytes that do not have that glow of certification around them. I think that 

this will create even more confusion for the physician. I think we really need to 

prioritize regulatory needs first, urgent medical decisions like cholesterol levels 

second, and improving the physician view of our industry as more of a moderate, or 

long-term, and much more challenging goal. 

DR. MAY: Should something like that be the focus of a meeting in the very near future 

prior to the meeting in Dublin? I am hearing a few people say yes, but I don’t know that 

this is a majority. 

 

DR. MILLER: Greg Miller from Virginia Commonwealth University. 

It seems to me that the parallel processing is the way to succeed here. What we need to 

do is drive this process by clinical requirements, not necessarily what we know how to 

measure. It seems to me that somebody has got to take responsibility to get it done. So, 

some organization needs to be assigned a task. It seems to me if a formal request is made 

of the College of American Pathologists, who basically has lab directors that run the 

pathology labs in this country, they could be asked for a priority list of analytes that are 

causing problems from a medical interpretation, or a medical use point of view. We 

could get that list together in a relatively 

short order. I think the College would 

be happy to oblige, and they seem a 

logical group to ask. But someone has 

to do that. Someone has to be assigned 

As a manufacturer, I have the 

regulatory need as the highest 

priority since otherwise 

products cannot be sold. 

We have to find a path forward with

specific assignments for specific 

people or organizations to do. 
331

the job of doing that. If we just talk about how nice it would be to do it, we’ll be talking 

about it again in June, and it won’t have happened yet. A second point that I would like to 

make is, during the last two days, I have not heard anybody ask for somebody to specify 

what is the highest metrological level that exists for a list of analytes. It seems to me, if 

we had a list of what is considered the highest metrological order, whether it be a 

material or a method, then manufacturers would know exactly what they have to trace to, 

and they could just get on with doing it. I don’t know how to achieve that. I don’t know 

what that is, but I think if that could be done, it would be quite helpful. 

DR. MILLER: Rick Miller from Dade Behring. 

Just to key on Greg’s statements and a couple of 

other things I have heard now, and something that 

Art Kessner touched on earlier. Should we be, in the 

short-term, developing a resource web site or 

resource list to help us understand where we are 

with some of these things, and help us determine 

what milestones we have to make, and even perhaps 

start putting together some of the already existing standards in such a way that they are 

more identifiable for each of us? 

DR. MAY: Okay. I’m writing these things down. 

DR. KESSNER: Art Kessner from Beckman Coulter. 

It sounds like we’re developing an incredibly 

complex list of questions and a very short 

list of answers. I think there is a need not 

only for manufacturers to understand the 

process but we need to have a process in 

order to understand it. Right now I think we 

have a lot of pieces. So, I would think that 

the next meeting in the short-term that you 

just mentioned should really focus on, like 

you say, putting your house in order and understanding or sketching out what is the 

process and what are the pieces that need to be fulfilled now, not in some prioritized 

future. We need an action plan so by the time we get to next June we at least understand 

what it is we are trying to accomplish in some areas, and which areas are more important 

than others. It is incredibly complex, and there are a lot of priorities to be set, but right 

now the priority seems to be to know what to do tomorrow when I go to work. I’d like to 

see a very focused type of a meeting to understand what the American situation is relative 

to the global situation. 

DR GUZEL: Omer Guzel, World Trade, from Istanbul. 

I think most of us were thinking before we attended this meeting that these items were all 

discussed, and most of the problems might be solved. Suddenly we realize that at the end 

of these two days meetings there is a lot to be done. I second Dr. Semerjian’s proposal 

that we should not lose time. There must be an official governing body to keep things 

Should we develop a 

website and perhaps start 

putting together what 

resources are already 

available? 

... in the reality of today, we 

should ask: 

1. what do we have? 

2. what do we need?, and 

3. how do we make the 

information available? 
332

moving. Perhaps the organizing committee of this meeting can continue until the next 

meeting and it can be delivered later to some other institutions or some other people 

willing to do it. It should be done in organized way. At least to coordinate the 

information gathered from here .... Thank you. 

DR. KING: I would just like to make a 

suggestion. I was sitting there trying to think of 

who has the information that the manufacturers 

need about which reference materials are 

available, that is, reference materials of a higher 

order. It strikes me that NIST probably has that 

information. You certainly know which ones 

that you offer. Through the MRAs you should 

know what others are available in Europe and so on. 

DR. MAY: We had actually put down to develop a web site. I had asked Dr. Koch here 

if we would be willing to do this? We would be willing to perhaps take a shot at it for a 

while and then maybe pass that off to someone else after we get the information there. I 

have to give credit to the other folks. The European Consortium has already put together 

a lot of that information, and I am fairly sure that they would be willing to work with us. 

So, we would get that information together in partnership with them. 

DR. KING: Okay. All we need is a date. 

DR. MAY: Before our next meeting. It really shouldn't take that long. I think we have 

all the information at hand to do that. 

PARTICIPANT: It seems that most manufacturers, if not all, do not understand what 

exactly the traceability document is saying. I have a suggestion, at least for this country, 

to translate it for lay people: 

maybe have a one-week workshop, 

or maybe ask AACC to put it on 

the web. But translate it, tell an 

example of an analyte for which 

we have a reference method and a reference material, and then go down the line and tell 

them this is how it is done, how one thing is linked to the other, graphically, in very 

simple words. 

DR. MAY: Okay. Just stay there a second. 

Bernard, you don't have to go to the microphone, but I assume that something like this is 

what you had in mind; is that not true? 

DR. KING: That is exactly what we had in mind. 

DR. MAY: There are plans to do that. Bernard has planned to do this as one of his 

activities, and we at NIST have already agreed to work with him on that project. This will 

We need to translate the “traceability 

document” for lay people. 

Would it be reasonable for 

perhaps NIST to establish 

a page on its web site that 

would list these reference 

materials. 
333

be a document that essentially translates the traceability requirement section of the IVD 

directive. 

PARTICIPANT: Well, then what timeline are you thinking of there? 

DR. MAY: Bernard, what timeline did you have in mind for that? 

DR. KING: Well we’re to seek funding from the European Union to help bring together 

the players that will contribute to this. 

DR. MAY: How much of an effort do you need? I’m sure you don’t mind where you get 

the resources from, so how much will it cost? 

DR. KING: I’ve not gotten to that. 

DR. MAY: Roughly. 

DR. KING: It is not a single document. It is a series of documents about a series of 

related issues. 

DR. MAY: If it is not proprietary, what you were expecting to get from the EU in order 

to do this? 

DR. KING: What we were envisioning is that the project would take place in two years. 

We plan to find documents that already exist and material that already exists, and then 

provide some user-friendly resources. 

DR. MAY: Essentially, what you would need would be some resources to support travel 

and a bit of time for certain key people. 

DR. KING: Certainly. We’ll be bringing together the people who need to have an input, 

and then also some more people who will sit down and put the material together. 

DR. MAY: That seems to be something that is truly needed, so if you can put together a 

document, let’s say, what it would take to have this done in draft form by the Dublin 

meeting. Is that impossible? For this time next year? 

DR. KING: I’ll think about it. It is easy for me to say, yes, oh, yes. The reality is can we 

have it done at the appropriate cost? 

DR. MAY: I guess the statement from Basil (Doumas) here would be a translation for 

lay people not the “higher order”. ... Basil is shaking his head. Is it important, Basil, for 

lay people, that is for patients, to understand it or is it only important to people in the 

clinical laboratory and the manufacturers? 
334

DR. DOUMAS: It needs to be in plain English, simple with examples for all people in 

the industry who are assigned this to be able to understand it. 

 (Laughter) 

DR. MAY: Now we have our marching orders. 

DR. POWERS: Willie, may I make a related comment? 

I was going to ask if Emil is still here. If so, I’ll let him answer my question. I 

understood that EDMA was working on a guidance document. I am kind of concerned 

about creating a document for lay people. I think we need something for manufacturers 

written in the language manufacturers understand. I would see the trade associations as 

being the logical owners of that specific document. I think we have enough people who 

have been working on the traceability standards who could get together and knock it off 

in a week. 

 

DR. MAY: Okay. 

 

DR. POWERS: Or two. 

DR. MAY: Is that true, Emil? 

DR. VOELKERT: I’m Emil Voelkert from EDMA. 

In fact, I would like to pour some wine into the water. I think we have more than the 

discussion really shows. First of all, I think we do have lists of reference materials, NIST 

for one, and there are several others. Secondly, I am also aware that there have been 

descriptions and publications of reference methods. The only thing I can say right now is 

that if you look back at the German quality regulation system (I don’t know whether it’s 

updated) it lists, I think, about 20 different reference methods. So, you might take a look 

at that. 

Concerning these traceability documents that we are working on, we have tried in a small 

working group in EDMA, to highlight the positions which we think from the industry 

side are relevant when transposing the standard into actual life. From my point of view, I 

think it is understandable. 

But I come to another problem I 

perceive. I am not sure whether we 

really mean the same thing when we 

talk about reference methods and 

reference materials. I perceive from 

some of the discussions there are still 

different meanings around the world. 

This is something we have to address. 

I think, also from my point of view, that it would be useful to have another meeting in 

between so that we can work out the subtle differences. I will be willing to make the 

So, when we talk across the Atlantic,

so to say, we have to be aware that 

there may be differences in 

semantics. 
335

EDMA document available to all manufacturers that are interested. It will be available 

through the EDMA sites, and I will also distribute it to the AdvaMed. I think that it will 

also be available to other interested parties. 

If you are asking for the timeline, actually, the document is in the last stages of 

development. I anticipate that it will be available by end of this year at the latest. 

However, we would welcome any documents that would provide other perspectives, 

particular from the point of view of the manufacturers, because that is mainly whom we 

are working for. 

DR. MAY: All right, Dr. Voelkert, is that document drafted so that perhaps some of the 

manufacturers could review it to ask for clarification, if you will, or to make sure that 

things are stated clearly enough that they can understand the intention of the directives? 

DR. VOELKERT: This is a very difficult question. It depends on the level of 

understanding of the particular person who reads it. From my point of view, if somebody 

has dealt with the IVD directive and has a rough idea about traceability, it helps 

understanding. I think it is up also to this global initiative really to come up with 

additional tools and educational material really to translate this idea of traceability. So, I 

would not put the expectations too high, but from my point of view and from my 

perspective, it is a useful document. 

DR. MAY: Let’s take this position: We should take a look at the EDMA document. If 

by some chance it does not meet your needs, then maybe we can take that as the basis of 

a document, and then modify it so that sufficiently meets your needs. 

DR. SMIT: Nick Smit with i-STAT, Canada. 

Nine months seems awfully far away, as far as I’m concerned at i-STAT. I have a couple 

of suggestions that could be an outcome of this workshop. A question: Is there going to 

be something that resembles a project plan that in a draft form or as an outline, or an 

interim progress report that so that in nine months when we get together - everybody here 

has an expectation, and you have your expectation - but when we get together in nine 

months, we don’t disconnect at the other side of the world in Dublin? 

DR. MAY: Well, project plan? I am not sure that we have defined a project. What we 

will provide for you would be the 

proceedings of this meeting. Regarding 

the next meeting, I think we’ve already 

agreed that we need to have a meeting 

before the nine months, and the earlier 

meeting would be one here in the United 

States. I don’t know whether Carolyn 

Jones is still here or not, but I jotted on 

my notes that perhaps we would ask 

AdvaMed to convene that meeting. 

Then we would discuss many of the issues that we have been discussing this afternoon to 

We will provide: 

• a copy of the presentations 

that were made yesterday, and

• an editorial version of the 

discussions that we’ve had this 

afternoon. 
336

prepare ourselves to the international meeting in Dublin. I think it’s at that meeting that 

we can sort of clarify some of the issues that are bubbling up here. 

 

DR. SMIT: All right. Just in terms of letting you know what we will be doing - I think 

that a lot of the industry (including us), will be drafting their own documents and their 

own process and procedures on how to implement this. 

DR. MAY: Sure. 

DR. SMIT: I know it’s a chicken and egg kind of situation, who is waiting for who to 

give who direction. However, i-STAT is going to move ahead, and we’re going to figure 

it out. Whether the reference laboratories are in place or not in place, we need to have 

some kind of documented resolution to traceability. Nine months just seems far away. 

DR. MAY: Again, that’s certainly something that you have to do. I think you have 

realized that certainly from a NIST perspective we are here to provide the tools for you, 

but, obviously, you have to do the work to develop what is required to get your products 

into the European Union. We are here to try to facilitate that process, and we will 

commit to do that. 

Let me share with you what I have 

heard here, and we can see if this 

sort of covers most of the holes we 

had. Certainly, we had talked about 

the necessity of perhaps having a 

meeting sooner than nine months, 

and we will talk to some people, 

perhaps AdvaMed about convening 

that meeting to address some 

pressing concerns, primarily of the 

IVD manufacturers. 

We talked about requesting that CAP provide a list of high priority analytes based on 

results that they have from their proficiency testing trials. 

We talked about the necessity of communicating information, perhaps having someone 

volunteer at NIST to look at putting together a web site that would list the tools that are in 

already in place, both the reference methods, and the reference materials. Perhaps on 

that, we could pull from the IVD directive the listing, as it stands now, of the “A list”

analytes and the “B list” analytes so that you have as much information as we have, 

basically. We would work with the European Union on putting this information together. 

DR. CARNEIRO: This is Kim Carneiro from DFM, Denmark. 

I would certainly be happy to do that, because exactly if the European project gets 

funded, one of my tasks to put up the web site and the information. 

Plan of Action: 

• Meeting of US IVD manufacturers 

before next international meeting 

• CAP provide priority list of analytes 

• EU’s “A list” and “B list” analytes 

• Web dissemination of information 
337

DR. MAY: Okay. 

DR. CARNEIRO: But let me be very clear about ...

 

DR. MAY: NIST would just link to it then. 

DR. CARNEIRO: The project that we have -- that you have taken very kindly on board - 

is a proposal that has not even been reviewed yet. So, we will be into the new year 

before that can start. Then, of course, we will be going full steam. But it would be 

wrong for anybody to rush, and I think that's partly why Bernard is rather reluctant to 

publish anything within the next few weeks. 

We really are not ready yet. We do have a 

project plan, and it has been nicely received 

here, but it will take us into the new year 

before it gets started. 

DR. MAY: Sure. 

DR. CARNEIRO: In that respect, I think it's very timely to meet in Dublin in June, 

because there will be some progress, and we can continue the dialogue on a much more 

specific level than we have been able to do here. We've had very good dialogue on a 

general level. We all need to go back and think a bit about it. Let's do this well, and 

still think about timeliness. 

DR. MAY: Can we do this? I guess what I am hearing from the manufacturers, from 

what they say and by reading their faces, is they need this maybe a bit sooner ... That's 

why I carefully said that maybe NIST would take a crack at this now and maybe pass it 

off to someone later. I wonder we could work together to try to get something out there 

now so that, basically, we can provide as much information to the manufacturers as we 

have now. Then as your (Dr. Carneiro’s) project moves forward, you could take that over 

and perhaps put a lot more resources than we would put into this initial effort to just put 

the information there without a lot of bells and whistles. 

DR. CARNEIRO: I'll be very happy to discuss that and do our utmost. As soon as the 

project gets going, I think we can easily arrange that everybody in this room is kept 

electronically informed about what goes on. I think that is the most efficient way of 

doing it. 

DR. MAY: Sure. 

DR. CARNEIRO: I was certainly saying to have a similar meeting to this in June next 

year would be fine, because some work can be done along the lines that we've been 

proposing. Having another meeting in between is fine. We need to keep this going. We 

may arrange a similar meeting in Europe to keep closer to the European-based industry. 

Then we'll meet in June and have a good next round. A meeting before then I think 

would be too hastily done. 

... let me quote you a Danish 

proverb that “what gets hastily 

done gets poorly done”. 
338

DR. MAY: Great. 

DR. ECKFELDT: John Eckfeldt, CAP in University of Minnesota. 

I actually know AACC Standards Committee put together a compilation of reference 

methods and materials that was and I think it’s still on the web site somewhere at 

AACC. That would be a starting place. It is not that comprehensive, but it is one place 

and is already on a web site, and worth looking at. As I remember, there was a brief 

description of the method, literature references, materials, who you got them from, and 

who the contact people were. There was a format that was already devised. Jean Rhame 

was involved in putting that together for AACC. It is just some place to start, we could 

look at it and maybe use it as a template or something. I know some of it is out of date. 

Another place that might be able to provide some information is CAP. When we 

looked at our Standards and Reference Materials Program, we looked at what other 

materials or methods were available internationally, particularly materials, just to see if 

there were things that we were going to discontinue that were not available somewhere 

else. There was virtually no analyte for which we provide the material that you couldn’t 

get somewhere else. I think we have that on a database somewhere. I could probably 

share that with you on the material side. 

DR. MAY: We’d certainly appreciate that. 

Continuing to go down the list that I had, translate the IVD traceability document was 

one of the things that was 

discussed. It appears that EDMA 

is well on the way to doing that, 

and I think what we agreed to do 

was to take a look at what they 

provided and go from there. If that 

meets our needs, the problem is 

solved. If we need further 

translation, then we can take that aboard and have it defined who "we" is. Then finally, I 

think we’ve agreed that this international group would meet next in Dublin in June. Any 

other questions, comments, concerns? Not putting you on the spot, Rene, but I know at 

our coffee break you had some issues on your mind. Do you feel comfortable sharing 

them with us? I know you aren’t shy. 

DR. DYBKAER: Rene Dybkaer, H:S Frederiksberg Hospital, Denmark. 

First of all, I should like to say that it should be quite clear that although it is nice to be 

hasty with something that must be done, you shouldn’t be too hasty. The facts are that the 

European Directive will come into force at the end of 2003, and it does not include a 

demand that everything that the manufacturers provide will suddenly have to be traceable 

to the SI. As was said a little while ago, the directive says available reference materials 

and procedures. So, that means that if nothing is available, you can go on with what 

you have until something comes along. That’s very important issue. The other thing I 

Plan of Action, continued: 

“Translate” IVD traceability document 

– it appears that EDMA is well on the 

way to doing that ...
339

would say is that just as Dr. May is a great admirer of Rudy Lequin, so am I. But that 

does not mean that I always agree with him. 

 (Laughter) 

 

DR. MAY: Me either. 

 

MR. DYBKAER: And earlier on he made a plea that we should look differently at the 

so-called A&B situations. The “A list” are analytes for which there was obviously 

existing a possibility of traceability through SI, and the “B list” is the group where this 

traceability does not exist. I would submit that it 

is more important to say we should always use 

the same procedure in principle, and that is we 

should identify the problem at the bottom in the 

routine laboratory, and then we should say how 

we can obtain traceability for the results we 

deliver. Then we go up the existing chain as far as we can. If we can go to SI, fine. If 

not, then, as Rudy was saying, we will have to rely on some international conventional 

reference material or reference procedure. It is all traceability. We just go as far up the 

chain as is technically possible. I think that was most of what I wanted to submit. Thank 

you. 

 

DR. MAY: I think what he is saying is that metrology is essentially metrology, and it is 

our responsibility to make the system as 

metrologically sound as we possibly can 

and not say that we're going to look at 

some measurement problems in a sound 

metrological basis and the others, well, 

just forget about metrology. We really should be striving, if you will, to convert all of the 

“B list” analytes to “A list” if we can. Obviously, we know that that's not possible, but 

that should be our quest. 

Well, it's getting late, and some people have to 

leave. Let me say that perhaps a year or so ago, 

not even that, Basil and Bill decided that we were 

going to have this meeting, and many of us were 

cynical. We weren't sure if we could pull it 

together so quickly, whether anyone would come 

other than the folks that work here and the Steering Committee. We've been pleasantly 

surprised. 

I think that we all agree that this certainly is an 

important issue that we need to discuss. And we 

thank all of you for taking time out of your very busy 

schedules to come here and share your thoughts on 

these issues with us. 

It is our responsibility to make 

the system as metrologically 

sound as possible. 

... thank you to Basil 

Doumas and Bill Koch for 

the vision ...

... thanks to our 

many participants 

for your insights ... 

... we should start at the 

bottom and then go as far 

up the traceability chain as 

possible ... 
340

So, on behalf of the Steering Committee, the NIST staff 

that worked on this, and in particular Lyn Beary, Donna 

Sirk, Jeanice Brown-Thomas, Amy Grafmuller, Mike 

Welch, who worked behind the scenes to make this a 

success, and certainly on behalf of Bill Koch and Hratch 

Semerjian, and myself, we really thank you for your 

efforts. We are sure that this won’t be the last time we’ll be seeing you and talking to you 

about these issues. Thank you very much. 

 (Applause) 

 

 

Whereupon, the Workshop was 

concluded at 4:00 p.m. 

... thanks to all who 

worked to make this 

workshop a success. 
341

Workshop on Measurement Traceability for Clinical 

Laboratory Testing and In Vitro Diagnostic Test Systems 

A Report on the Workshop 

William F. Koch 

National Institute of Standards and Technology 

In early November of the year 2000, the footings for a new bridge were laid in 

Gaithersburg, Maryland, just outside Washington, DC. Representatives of government, 

the IVD industry, and the medical professions from 15 nations representing four continents gathered at the National Institute of Standards and Technology to participate in the 

“Workshop on Measurement Traceability for Clinical Laboratory Testing and in vitro

Diagnostic Test Systems”. Their goal: to develop recommendations regarding the needs 

for measurement traceability for health status markers to (1) address IVD industry needs 

for compliance with international standards (e.g., EU IVD Directive) and (2) improve 

comparability of clinical measurement data to facilitate better decision making by medical professionals. 

As these 135 scientific experts and stakeholders from around the globe (25% were 

from outside the United States) entered the NIST “Green Auditorium” for the start of this 

intense two-day workshop, their eyes were drawn to the huge screen displaying the icon 

for the meeting - a bridge superimposed on a map of the world. This bridge symbolized 

three fundamental concepts related to the workshop. First, a bridge is a means to get 

from one place to another, often across a formidable gap. The first day of the Workshop 

was filled with 10 talks delivered by experts providing the background and current status, 

thereby anchoring one end of the bridge. The second day was spent in breakout sessions 

and general discussions intended to plan the size and direction of the path forward. Secondly, just as a bridge is a critical part of the infrastructure for the transportation system 

that facilitates free and open commerce, so too is traceability a vital piece of the measurement infrastructure for the global healthcare system that supports equitable and open 

trade, as well as assuring improved comparability and reliability of clinical testing. And 

finally, the bridge on this Workshop’s logo spans the waters of the world. Healthcare is 

now more than ever a global issue and a global commodity, demanding global solutions. 

Before recounting the details and conclusions of the meeting, it is essential to understand the driving force behind the need for this workshop. Traceability to internationally recognized and accepted standards is an important component in assuring the accuracy and comparability of clinical laboratory measurements. In addition, the global marketplace is presenting new demands for measurement traceability. NIST has a long history of providing certified measurement standards in several fields including the clinical 

laboratory disciplines, and is continuing its efforts to develop new reference methods and 

materials for important health-status markers to meet on-going and future needs for traceability. NCCLS is a globally recognized, voluntary consensus standards-developing organization that enhances the value of medical testing within the healthcare community 
342

through the development and dissemination of standards, guidelines, and best practices. 

NCCLS has the Secretariat responsibility for the ISO Technical Committee 212 (ISO/TC 

212) on Clinical Laboratory Testing and In Vitro Diagnostic Test Systems, and is the 

home of the National Reference System for the Clinical Laboratory (NRSCL), a collection of broadly understood reference systems intended to improve the comparability of 

test results, consistent with the needs of medical practice. Recently, an important opportunity has emerged that applies new pressure to the quest for traceability and the demand 

for reference systems. Prompted by the European Union’s In Vitro Diagnostics Directive 

(IVDD), the European Committee for Standardization’s Technical Committee 140 

(CEN/TC 140), in vitro diagnostic systems, began drafting a standard on metrological 

traceability. By working closely together, CEN/TC 140 and ISO/TC 212 will develop 

identical European and ISO standards on this topic. Full implementation of the IVD Directive is expected by December 2003 and will require that calibration of all IVD assays 

be traceable to available reference materials or methods of higher metrological order. 

(See Don Powers Article in IVD Tech, July, 2000, http://www.devicelink. 

com/ivdt/archive/00/07/003.html).

During the first day of lectures, the attendees learned directly from the leading authorities in Europe about the European IVD Directive. Dr. Kim Carniero, Danish Institute of Fundamental Metrology and representing the European Commission, presented a 

succinct summary of the Directive, its basis, purpose, implementation, and implications. 

Dr. Emil Voelkert, Roche Diagnostics GmbH and chair of CEN/TC140, described the 

standards activities within CEN and ISO, focusing on CEN/TC 140 and ISO/TC212, in 

support of the IVD Directive and with the goal of harmonized standards within Europe. 

Professor Lothar Siekmann, University of Bonn and representing the IFCC, discussed the 

concept of traceability as applied to the field of clinical chemical analysis, remarking that 

traceability provides probably the most important strategy to achieve standardization in 

laboratory medicine. He went on to illustrate the process for credentialing reference 

laboratories, methods and materials in Germany. Rounding out the European viewpoint 

was Dr. Heinz Schimmel, Institute for Reference Materials and Measurements (IRMM), 

who informed the audience of the EU’s current and planned clinical reference material 

development activities. 

To segue back to the United States, Dr. Hratch Semerjian, National Institute of 

Standards and Technology, related the important and increasing role that national metrology institutes (NMI), such as NIST, PTB and DFM, have in establishing mutual recognition of measurements and tests between nations. In October 1999, the NMI Directors of 

the 38 member states of the Meter Convention signed “the mutual recognition arrangement (MRA) on national measurement standards and calibration and measurement certificates issued by national metrology institutes.” In addition, the US and EU have entered into an MRA to facilitate bilateral trade between the US and European Community 

realizing that mutual recognition of conformity assessment activities is an important 

means of enhancing market access. Continuing with efforts at NIST, Dr. Willie May, 

NIST, explained how NIST serves as the primary US reference laboratory for healthrelated chemical measurements through: the development of high-accuracy measurement 

methods; the development, certification and distribution of Standard Reference Materials; 

interactive measurement quality assessment activities; and international comparison exercises with other NMIs. Highlighting a subtle but significant change in his title, Dr. Neil 
343

Greenberg, Ortho-Clinical Diagnostics, presented the perspective held by the global 

(originally the US) IVD industry of the Directive’s requirement for calibrator traceability. 

He made a special note that the IVD Directive does not specifically call for the development of new or improved reference materials, but only that available reference materials 

and measurement procedures of higher metrological order be used for traceability purposes. Nevertheless, ISO/CD 17511 states that it is the aim of metrology in laboratory 

medicine to improve traceability by providing the missing reference measurement procedures and materials, based on international consensus. As a result, reference material and 

methods development projects will be initiated in the name of the Directive. Whereas

IVD companies value standards because they help define market needs and provide a 

clear universal definition of goals, as well as an objective assessment of product attributes, it is unlikely that they will take a lead position in advocating for new standards at 

this time. However, the inclusion of manufacturing scientists and experts in IVD standards projects led by professional, government and public health groups is absolutely 

necessary for project success. 

Adhering to the philosophy that one should know where one has been in order to 

better know where to go next, Dr. John Eckfeldt, Fairview-University Medical Center 

and representing the College of American Pathologists, presented a historical review or 

reference systems for clinical measurements. Evolving from the Belk and Sunderman’s 

study in 1947 involving 59 clinical laboratories, using 24 samples, and testing for 7 analytes, the CAP PT/EQUAS Program today encompasses over 25,000 laboratories with 

over a quarter of a million samples and testing for over 500 analytes. He cited several examples to illustrate the role of performance testing for assessing traceability, noting that 

some programs were more successful than others, often due to limitations of the analyte 

being tested (e.g., non-commutability, poorly defined or heterogeneous analyte). He concluded that establishing and maintaining the documentation for traceability in field methods is both difficult and expensive. Dr. George Klee, Mayo Clinic, expanded on the need 

for commutability, noting that reference materials, even when available and used in conjunction with established reference methods, do not necessarily assure harmonization of 

test values on patient samples. He concluded that a combination of commutable control 

materials (based on panels of commutable human specimens) with traceable reference 

values and mathematical algorithms using adjusted patient test values could be used for 

calibration adjustment. This approach will complement the role of reference materials in 

improving patient care. And finally, Ms. Joan Walsh Cassedy, ACIL, presented models 

of reference systems used by other industries, emphasizing that measurement traceability 

forms the foundation of a quality program, and that absence of traceability leads to chaos. 

Armed with this information, the participants reconvened the next morning in five 

separate breakout sessions led by: Neil Greenberg (Priorities for National and International Investments in New or Improved Reference Systems in Support of Clinical Laboratory Measurement); Don Powers (Reference Materials and Reference Measurement Procedures to Support Traceability Requirements of the IVD Directive); Greg Miller (Impact 

of Method-Material Matrix Interactions on Calibration Traceability Protocols for Successful Harmonization of Patient Results); William F. Koch (Development of International Consensus on the Credentialing Process for Reference Systems); and Gary L. 

Myers (Creating and Sustaining Reference Method Laboratory Networks). Four hours of 

spirited discussions and consensus-building within each group resulted in five reports that 
344

were presented to the re-assembled audience in the afternoon. Although each group 

brought forward unique perspectives and recommendations based on the topical area, 

there was general agreement on some critical attributes of the traceability bridge. The 

need for global reference systems composed of reference methods, reference materials 

and a mechanism for demonstrating competence and equivalence was of paramount importance. Internationally recognized and accepted reference laboratories should implement these reference systems, using a networked approach. In order to meet the immediate requirements of the IVD Directive, a catalog of the available reference methods and 

reference materials must be communicated to the IVD manufacturers. There was concurrence that when properly implemented, traceability is a value-added exercise that will 

improve patient care, testing accuracy, reliability and availability, market access, and, in 

the long run, reduce costs. However, it was emphasized that efforts undertaken must be 

designed to minimize redundancy and barriers, encourage new technologies and facilitate 

global collaborations. 

And now that we know where to build our bridge and have some idea of what the 

bridge will look like, what are the next steps? The closing session of the workshop 

achieved concurrence on the following actions: 

• Develop a web-enabled database of currently available reference methods 

and internationally recognized CRMs. NIST will undertake this activity while 

realizing that the EU has submitted a proposal to be reviewed in several 

months for a similar exercise. If the EU proposal is funded, NIST will coordinate with the EU. 

• Organize an internationally accepted “oversight” group that is industry-led 

(but composed of government experts and medical professionals as well as 

industry experts) whose task will be to: 

> Set priorities for new reference methods and materials based on medical 

importance, need, and commutability. 

> Identify sources and secure funding for developing reference methods 

and materials. 

> Develop the process for international consensus on reference systems 

• Develop interpretive guidelines on traceability requirements. 

• Develop a global process for approving reference methods and for distributing certified reference materials (CRMs). 

• Identify alternative approaches for those analytes for which SI traceable reference methods and/or materials are impossible, at this time. 

• Convene a follow-up meeting in Dublin, preceding the ISO/TC212 Meeting in 

June 2001. Progress in the above bulleted action items will be reviewed and 

further design and construction of the bridge will proceed. 

It is clear that the Traceability Bridge is far from complete. This Workshop was 

only the beginning, but we should be encouraged by the broad representation, both geo-
345

graphic and sector, and by the level of effort, enthusiasm and commitment exhibited at 

this groundbreaking in Gaithersburg. The waters under the bridge need not be troubled if 

international consensus on measurement traceability can be achieved. 

WILLIAM F. KOCH 

National Institute of Standards and Technology 

Dr. William F. Koch is the Deputy Director of the Chemical Science and Technology 

Laboratory at the National Institute of Standards and Technology. He began his career 

at NIST as a research chemist in the Center for Analytical Chemistry at NBS, where he 

expanded his research interests to include pH, electrolytic conductivity and ion chromatography. He has had direct involvement in the development and certification of over 70 

Standard Reference Materials, and has authored over 70 scientific publications. 

Dr. Koch has been an elected member of the Board of Directors of the National Committee for Clinical Laboratory Standards (NCCLS) since 1990, and is currently the Immediate Past President of this organization. He is on the Governing Board of the Council for 

Chemical Research and chair of the CCR Science Education and Human Resources 

Committee. He was recently elected as a Fellow in the National Academy of Clinical 

Biochemistry. He is an active member of the Directors of Industrial Research - Analytical 

Group, the American Chemical Society, the Society for Electroanalytical Research, and 

the American Association for Clinical Chemistry. 
346
347

Workshop on Measurement Traceability 

for Clinical Laboratory Testing 

and In-Vitro Diagnostic Test Systems 

November 2-3, 2000 

Participants List

Ahnell, Joseph 

BD Biosciences 

7 Loveton Circle, MC 622 

P.O. Box 999 

Sparks, MD 21152-0999 

USA 

410/316-4078 

410/316-4046 

Joseph_E_Ahnell@ms.bd.com 

Antalis, Caryl 

Roche Diagnostics 

9115 Hague Rd. 

Indianapolis, IN 46250 

USA 

317/576-3120 

317/576-3085 

caryl.antalis@roche.com 

Arends, Sabine 

Roche Diagnostics GmbH 

Bahnhofstrasse 9-15 

Tutzing 

Germany 

82327 

49/81 58224349 

49/81 58224152 

sabine.arends@roche.com 

Armbruster, David 

Abbott Laboratories 

1920 Hurd Drive, MS 1-9 

Irivng, TX 75038 

USA 

972/518-6016 

972/753-3295 

david.armbruster@abbott.com 

Ashwood, Edward 

College of American Pathologists 

ARUP Laboratories 

University of Utah 

500 Chipeta Way 

Salt Lake City, UT 84108 

USA 

801/583-2787 

801/584-5207 

ashwood@med.utah.edu 

Beary, Ellyn 

NIST 

100 Bureau Drive, MS 8300 

Gaithersburg, MD 20899-8300 

USA 

301/975-8307 

301/975-3845 

ellyn.beary@nist.gov 

Binder, Steven 

Bio-Rad Laboratories 

4000 Alfred Nobel Dr. 

Hercules, CA 94708 

USA 

510/724-7000 

steve_binder@bio-rad.com 

Blessum, Cynthia 

Beckman Coulter, Inc. 

200 S. Kraemer Blvd. 

Brea, CA 92822 

USA 

714/993-8980 

714/961-3759 

crblessum@beckman.com 
348

Bozimowski, Dennis 

Abbott Laboratories 

1920 Hurd Dr. 

Irving, TX 75038 

USA 

972/518-7546 

972/753-3233 

dennis.bozimowski@add.ssw.abbott.com 

Bugler, Joe 

Abbott Laboratories 

MediSense Products 

Unit 14/15 Eyston Way 

Abingdon, Oxon 

OX14 1TR 

United Kindom 

44/1/235-542-310 

44/1/235-555-705 

joe.bugler@abott.com 

Bunk, David 

NIST 

100 Bureau Drive, MS 8392 

Gaithersburg, MD 20899-8392 

USA 

301/975-5071 

301/977-0685 

david.bunk@nist.gov 

Burritt, Mary 

Mayo Clinic 

200 1st St., SW 

Rochester, MN 55905 

USA 

507/284-3954 

507/266-4176 

mburritt@mayo.edu 

Caines, Patrick 

Ortho-Clinical Diagnostics 

100 Indigo Creek Dr. 

MC 24149 B 642 

Rochester, NY 14626 

USA 

716/453-5089 

716/453-5170 

pcaines@ocdus.jnj.com 

Caler, Mara 

Beckman Coulter, Inc. 

P.O. Box 269006 

San Diego, CA 92196 

USA 

858/621-4583 

858/621-4752 

mcalerchapman@beckman.com 

Canouse, Patricia 

Abbott Laboratories 

5440 Patrick Henry Dr. 

Santa Clara, CA 95054 

USA 

408/567-3377 

408/567-3568 

patricia.canouse@add.ssw.abbott.com 

Carneiro, Kim 

Danish Institute of Fundamental Metrology 

Building 307, Anker Engelunds Vej 1 

DK-2800 Lyngby 

Denmark 

Tel: +45 4525 5867 

Fax: +45 4593 1137 

kc@dfm.dtu.dk 

Cassedy, Joan Walsh 

ACIL 

1629 K St., NW – Suite 400 

Washington, DC 20006 

USA 

202/887-5872 

jcassedy@acil.org 

Ceriotti, Ferruccio 

H.S. Raffaele 

Via Olgettina 60 

Milan 20132 

Italy 

39/2 26432282 

39/2 26432640 

ceriotti.ferruccio@hsr.it 
349

Cernosek, Stanley 

Beckman Coulter, Inc. 

200 S. Kraemer Blvd. 

Brea, CA 92822 

USA 

714/993-8805 

714/961-3740 

sfcernosek@beckman.com 

Chace, Nina 

FDA 

2098 Gaither Road 

Rockville, MD 20850 

USA 

301-594-1293 

Chapman, Donna 

Bio-Rad Laboratories 

9500 Jeronimo Rd. 

Irvine, CA 92618 

USA 

949/598-1280 

949/598-1555 

donna_chapman@bio-rad.com 

Chen, Ellen 

Food and Drug Administration (FDA) 

12725 Twinbrook Pkwy. 

FDA/CDRH/DMMS, Rm.22 

Rockville, MD 20852 

USA 

301/827-4716 

301/827-4877 

exc@cdrh.fda.gov 

Chi, Robert 

Taiwan Committee for Clinical Laboratory 

Standards 

Department Clinical Path 

Tri-Service General Hospital 

Taipei 

Taiwan 

886/2-3653825 

886/2-23655582 

chin422@ndmctsgh.edu.tw 

Childress, William 

Food and Drug Administration 

109 Holton St. 

Winchester, MA 01890 

USA 

781/729-5700 

781/729-3593 

wchildre@ora.fda.gov 

Choo, Sae 

First Medical 

530 Logue Ave. 

Mountain View, CA 94043 

USA 

650/903-5974 

650/903-9040 

saechoo@firstmedical.com 

Clark, Douglas 

Dade Behring, Inc. 

P.O. Box 6101, MS 707 

Building 700 

Newark, DE 19714-6101 

USA 

302/631-7545 

302/6316998 

clarksdp@dadebehring.com 

Coady, Cynthia 

Diagnostic Chemicals Ltd. 

16 McCarville St 

Charlottetown, PE 

C1A 2A6 

Canada 

902/566-1396 

902/566-2498 

lwoold@dclchem.com 

Cockerham, Kenneth 

Bio-Tech Imaging, Inc. 

5711 Industry Lane 

Unit 31 

Frederick, MD 21704 

USA 

301/695-9900 

301/695-0030 

krcham@aol.com 
350

Colbert, Jennifer 

NIST 

100 Bureau Drive, MS 2321 

Gaithersburg, MD 20899 

USA 

301/975-2579 

jennifer.colbert@nist.gov 

Crush-Stanton, Sylvia 

Boston Biomedica, Inc. 

375 West St. 

West Bridgewater, MA 02379 

USA 

508/580-1900 

508/580-0250 

scrush-stanton@bbii.com 

Dalluge, Joseph 

NIST 

100 Bureau Drive, MS 8392 

Gaithersburg, MD 20899-8392 

USA 

301/975-3651 

301/977-0685 

joseph.dalluge@nist.gov 

Davidson, Yolanda 

NIST 

100 Bureau Drive, MS 8392 

Gaithersburg, MD 20899-8392 

USA 

301/975-3116 

301/977-0685 

yolanda.davidson@nist.gov 

de Leer, Ed W.B. 

NMi Van Swinden Laboratorium 

Department of Chemistry 

PO Box 654 

2600 AR Delft 

The Netherlands 

31/15-269-1712 

31/15-261-2971 

edeleer@nmi.nl 

Doumas, Basil 

Medical College of Wisconsin 

Plenary Session Chair 

2430 Whipple Tree Lane 

Brookfield, WI 53045 

USA 

262/786-1038 

262/786-4171 

bdoumas@core.com 

Dubrowny, Nancy 

BD Vacutainer Systems 

1 Becton Drive, Mail Code 310 

Franklin Lakes, NJ 07417 

USA 

201/847-4409 

Dunka, Lou 

LifeScan 

1000 Gibraltar Drive 

Milpitas, CA 95035 

USA 

408/942-5762 

 

Dybkaer, Rene 

H:S Frederiksberg Hospital 

Nordre Fasanvej 57 

Department Standards Laboratory Medicine 

Frederiksberg, DK-2000 

Denmark 

45/3816 3870 

45/3816 3879 

Eckfeldt, John 

Fairview University Medical Center 

420 Delaware St., SE 

Minneapolis, MN 55455 

USA 

612/626-3176 

615/273-6994 

eckfe001@tc.umn.edu 

Edmondson, Sherb 

Abbott Laboratories 

5440 Patrick Henry Dr. 

Santa Clara, CA 95054 

USA 

408/567-3424 

408/982-4882 

sherb.edmondson@add.ssw.abbott.com 
351

Ehlers, Glenn 

Ortho-Clinical Diagnostics 

100 Indigo Creek Dr. 

Rochester, NY 14626 

USA 

716/453-5025 

716/453-5188 

gehlers@ocdus.jnj.com 

Elin, Ronald 

College of American Pathologists 

University of Louisville 

7015 Windham Pkwy. 

Prospect, KY 40059 

USA 

502/852-4464 

502/852-1771 

rjelin01@gwise.louisville.edu 

Files, Nancy 

Immunicon Corp. 

3401 Masons Mill Road, Suite 100 

Huntingdon Valley, PA 19006 

USA 

215/830-0777 

Fish, Terri 

U.S. House of Representatives 

Science Committee Staff 

2319 Rayburn House Bldg. 

Washington, DC 20515 

USA 

202/225-8844 

teresa.fish@mail.house.gov 

Fitzgerald, Marc 

American Chemical Society 

1155 16th St., NW 

Washington, DC 20036 

USA 

202/872-6296 

202/872-6060 

m_fitzgerald@acs.org 

Ford, Patrick 

Ortho-Clinical Diagnostics 

Cardiff Laboratories 

Forest Farm Estate 

Cardiff CF47YT 

South Wales 

United Kindom 

44/029 20521693 

44/029 20521632 

pford@ocdgb.jnj.com 

Gaigalas, Adolfas 

NIST 

100 Bureau Drive 

Gaithersburg, MD 20899 

USA 

301/975-2873 

301/975-5449 

adolfas.gaigalas@nist.gov 

Garrett, Patricia 

Boston Biomedica, Inc. 

47 West Street, Suite 1 

Portland, ME 04102 

USA 

860/225-1900 

Gella, F-Javier 

Biosystems 

Costa Brava 30 

Barcelona 08030 

Spain 

34/9331 10811 

Giguere, Monica 

BD Biosciences 

7 Loveton Circle, MC 614 

Sparks, MD 21152 

USA 

410/316-4287 

410/316-4499 

mgiguere@bd.com 
352

Giuliani, Isabelle 

Bio-Rad 

UMR CNRS 5094 

Faculte de pharmacie 

15 Avenue Charles Pl 

Montpellier 34060 

France 

33/4 67 54 86 07 

33/4 67 54 86 10 

isabelle.giuliani@ibph.pharma.univmontp1.fr 

Gratiano, Theresa 

BBI Biotech Research Labs., Inc. 

217 Perry Pkwy. 

Gaithersburg, MD 20877 

USA 

301/208-8100 

301/208-8829 

tgratiano@bbii.com 

Greenberg, Neil 

Ortho-Clinical Diagnostics 

100 Indigo Creek Dr. 

Rochester, NY 14626 

USA 

716/453-3768 

716/453-3982 

ngreenbe@ocus.jnj.com 

Guzel, Omer 

Biruni Laboratory 

Buyukdere Cad Gokfil 

Ishani Mecidiyekoy 

Istanbul TR80310 

Turkey 

90/21221 74106 

90/21221 74106 

oguzel@biruni.com.tr 

Halonen, Tuija 

PerkinElmer Wallac 

Customer Support Laboratory 

P.O.Box 10 

20101 Turku 

Finland 

358/2 2678111 

358/2 2678357 

tuija.halonen@wallac.fi 

Hartmann, Alfred 

College of American Pathologists 

Physicians Laboratory 

1000 E. 21 St. 

Suite 4100 

Sioux Falls, SD 57105 

USA 

605/322-7190 

605/322-7222 

al.hartmann@mckennan.org 

Hawes, Katherine 

Immunicon Corp. 

3401 Masons Mill Road, Suite 100 

Huntingdon Valley, PA 19006 

USA 

215/830-0777 

Hughes, John 

LifeScan, Inc. 

1000 Gibraltar Dr 

Milpitas, CA 95035 

USA 

408/956-4064 

408/942-5906 

jhughes@lfsus.jnj.com 

Jain, Chandra 

Beckman Coulter, Inc. 

200 South Kraemer Blvd. 

Brea, CA 92822 

USA 

714/961-4116 

714/961-3740 

cpjain@beckman.com 

Jaunzemis, Andris 

Becton Dickinson 

7 Loveton Circle, MS 622 

Sparks, MD 21152 

USA 

410/316-4860 

Jones, Carolyn 

AdvaMed 

200 G Street NW, Suite 400 

Washington, D.C. 20005-3814 

USA 

cjones@advamed.org 
353

Kahn, Stephen 

Loyola University Medical Center 

Rm. 0122 Clinical Lab 

2160 S. 1st Ave. 

Maywood, IL 60153 

USA 

708/216-4725 

708/216-4146 

skahn@lumc.edu 

Kamalian, Farrah 

Bio-Rad Laboratories 

9500 Jeronimo Rd. 

Irvine, CA 92618 

USA 

949/598-1319 

949/598-1553 

Farrah_kamalian@bio-rad.com 

Kessner, Art 

Beckman Coulter, Inc. 

200 South Kraemer Blvd. 

Brea, CA 92822 

USA 

714/993-8999 

714/961-3740 

akessner@beckman.com 

Kimberly, Mary 

Centers for Disease Control and Prevention 

4770 Buford Hwy., NE, MS F25 

Atlanta, GA 30341 

USA 

770/488-4683 

770/488-4192 

mmk1@cdc.gov 

King, Bernard 

LGC 

Queens Rd 

Teddington 

Middlesex TW11 0LY 

United Kindom 

44/208 943 7667 

44/208 943 2767 

bmjk@lgc.co.uk 

Klee, George 

Mayo Clinic 

200 1st St., SW 

Stabile 104C 

Rochester, MN 55905 

USA 

507/284-8213 

507/266-4341 

klee.george@mayo.edu 

Klimley, Patricia 

Bio-Rad Laboratories 

4000 Alfred Nobel Drive 

Hercules, CA 94547 

USA 

510/741-6263 

Koch, William 

NIST 

100 Bureau Drive, MS 8300 

Gaithersburg, MD 20899-8300 

USA 

301/975-8301 

301/975-3845 

william.koch@nist.gov 

Kratzer, Manfred 

Roche Diagnostics GmbH 

Nonnenwald 2 

Penzberg 82372 

Germany 

49/8856-603032 

49/8856-602130 

manfred.kratzer@roche.com 

Lacroix, Jean-Michel 

Visible Genetics, Inc. 

700 Bay St. 

Suite 1000 

Toronto, Ontario 

M5G 1Z6 

CANADA 

416/813-3240, x4883 

416/813-3250 

lacroix@visgen.com 
354

Lasky, Fred 

Ortho-Clinical Diagnostics 

100 Indigo Creek Dr., MC 00881 

Rochester, NY 14626-5101 

USA 

716/453-3880 

716/453-3113 

flasky@ocdus.jnj.com 

Lawrence, Nate 

BBI Biotechnology 

217 Perry Pkwy. 

Gaithersburg, MD 20877 

USA 

301/208-8100 

301/208-8829 

nlawrence@bbii.com 

Lequin, Rudolf 

Diagnostics Consultancy 

P.O. Box 4963 

Eindhoven CD 5604 

The Netherlands 

31/40-290-8620 

31/40-290-8621 

r.m.lequin@wxs.nl 

Levin, Barbara 

NIST 

100 Bureau Drive, MS 8311 

Gaithersburg, MD 20899-8311 

USA 

301/975-6682 

barbara.levin@nist.gov 

Levine, Edward 

Diagnostic Products Corporation 

5700 West 96th Street 

Los Angeles, CA 90045-5597 

USA 

310/645-8200 

310/645-9999 

elevine@dpconline.com 

Ling, Eleanor 

Beckman Coulter 

1000 Lake Hazeltine 

Bloomington, MN 55318 

USA 

952/368-1208 

Little, Randie 

University of Missouri-Columbia 

One Hospital Dr, M767 

Department of Child Health 

Columbia, MO 65212 

USA 

573/882-1257 

573/884-4748 

LittleR@health.missouri.edu 

Lo, Stanley 

Medical College of Wisconsin 

9000 W. Wisconsin Avenue 

Children’s Hospital 

Milwaukee, WI 53226 

USA 

414/266-2508 

Long, Stephen 

NIST 

100 Bureau Drive, MS 8391 

Gaithersburg, MD 20899-8391 

USA 

301/975-4140 

Stephen.long@nist.gov 

Manak, Mark 

BBI Biotech 

217 Perry Pkwy. 

Gaithersburg, MD 20877 

USA 

301/208-8100 

301/208-8829 

mmanak@bbii.com 

Marcus, Roy 

Bayer Corporation 

511 Benedict Ave. 

Tarrytown, NY 10598 

USA 

914/524-2687 

914/524-3323 

roy.marcus.b@bayer.com 

Maxim, Peter 

FDA/CDRH 

2098 Gaither Road 

Rockville, MD 20850 

USA 

301/594-1293 
355

May, Willie 

NIST 

100 Bureau Drive, MS 8390 

Gaithersburg, MD 20899-8390 

USA 

301/975-3108 

301/926-8671 

willie.may@nist.gov 

Miller, Greg 

Virginia Commonwealth University 

Department of Pathology 

P.O. Box 980286 

Richmond, VA 23219 

USA 

804/828-0375 

804/828-0353 

millerg@hsc.vcu.edu 

Miller, Richard 

Dade Behring, Inc. 

P.O. Box 6101, M/S 1 

Glasgow Business Community 

Newark, DE 19711 

USA 

302/631-6116 

302/631-6122 

rrmiller@ix.netcom.com 

Murnane, Joseph 

Underwriters Laboratories, Inc. 

1285 Walt Whitman Rd. 

Melville, NY 11747 

USA 

631/271-6200 

631/439-6042 

Joseph.P.MurnaneJr@us.ul.com 

Myers, Gary 

Centers for Disease Control and Prevention 

4770 Buford Hwy., NE 

Atlanta, GA 30341 

USA 

770/488-4603 

770/488-4192 

glm1@cdc.gov 

Nadel, Debbie 

AACC 

2101 L St., NW 

Washington, DC 20037 

USA 

202/835-8704 

202/887-5093 

dnadel@aacc.org 

Nakamura, Masakazu 

Osaka Medical Center for Cancer 

3 Nakamichi 1-chome 

Higashinari-ku 

Osaka 537-8511 

Japan 

81/6-6972-1181 

81/6-6972-7749 

xnakamur@iph.pref.osaka.jp 

Nelson, Bryant 

NIST 

100 Bureau Drive, MS 8392 

Gaithersburg, MD 20899-8392 

USA 

301/975-8540 

301/977-0685 

bryant.nelson@nist.gov 

Nelson, Chad 

NIST 

100 Bureau Drive, MS 8392 

Gaithersburg, MD 20899-8392 

USA 

301/975-3389 

301/977-0685 

chad.nelson@nist.gov 

Ng, Ronald 

Abbott Laboratories 

4A Crosby Dr. 

Bedford, MA 01730 

USA 

781/276-6212 

781/271-1417 

ng@medisense.com 
356

Pankratz, Thomas 

Dade Behring, Inc. 

Glasgow 707 

P.O. Box 6101 

Newark, DE 19714 

USA 

302/631-7608 

302/631-7487 

pankratj@dadebehring.com 

Panteghini, Mauro 

Azienda Ospeoaliera 

’Spedali Civili’

Rm. 25125, Brescia 

Italy 

39/030 3995 470 

39/030 3995 365 

panteghi@bshosp.unibs.it 

Parker, Donald 

Bayer Corporation 

1884 Miles Ave. 

Elkhart, IN 46514 

USA 

219/264-8962 

219/262-7826 

don.parker.b@bayer.com 

Pizza, Barbara 

Genzyme Corporation 

One Kendall Square 

Bldg. 200, 4th Floor 

Cambridge, MA 02139 

USA 

617/252-7953 

617/374-7470 

barbara.pizza@genzyme.com 

Potter, Laurence 

Dade Behring, Inc. 

P.O. Box 6101, M/S 514 

Newark, DE 19714-6101 

USA 

302/631-0441 

302/631-6299 

potterla@dadebehring.com 

Powers, Don 

Powers Consulting Services 

One Wandering Trail 

Pittsford, NY 14534 

USA 

716/381-7575 

425/952-1450 

powers@frontiernet.net 

Pry, Terry 

Abbott Laboratories 

1920 Hurd Dr. 

MS 1-10, D2252, LC-1 

Irving, TX 75038 

USA 

972/518-6046 

972/518-7692 

terry.pry@add.ssw.abbott.com 

Quear, Mike 

U.S. House of Representatives 

Science Committee Staff 

2319 Rayburn House Bldg. 

Washington, DC 20515 

USA 

202/225-8844 

mike.quear@mail.house.gov 

Radmacher, Paula 

University of Louisville 

511 South Floyd St. 

Rm. 107 MDR Bldg. 

Neonatal Nutrition Research Laboratory 

Louisville, KY 40292 

USA 

502/852-5532 

502/852-8826 

paula.radmacher@louisville.edu 

Ring, Nancy 

Diametrics Medical, Inc. 

2658 Patton Rd. 

Roseville, MN 55113 

USA 

651/638-1250 

651/368-1060 

nring@diametrics.com 
357

Robinowitz, Max 

FDA HFZ-440 

2098 Gaither Road 

Rockville, MD 20850 

USA 

301/594-1293 

Rohlfing, Curt 

University of Missouri-Columbia 

One Hospital Dr., M772 

Columbia, MO 65212 

USA 

573/884-2385 

573/884-4748 

rohlfingc@health.missouri.edu 

Ross, Richard 

Beckman Coulter, Inc. 

P.O. Box 8000 

Brea, CA 92822-8000 

USA 

714/961-4912 

714/961-4123 

RTRoss@Beckman.com 

San George, Richard 

Inverness Medical Technology, Inc. 

Two University Office Park 

51 Sawyer Road 

Waltham, MA 02453 

USA 

richardcsg@aol.com 

Sander, Lane 

NIST 

100 Bureau Drive, MS 8392 

Gaithersburg, MD 20899-8392 

USA 

301/975-3117 

301/977-0685 

lane.sander@nist.gov 

Schiel, Detlef 

PTB 

Bundesallee 100 

Braunschweig 

Germany 

49/5315 923220 

49/5315 923045 

detlef.schiel@ptb.de 

Schimmel, Heinz 

Institute for Reference Materials and 

Measurement (IRMM) 

Geel B-2440 

Belgium 

32/014-571-720 

32/014-590-406 

schimmel@irmm.jrc.be 

Seccombe, David 

Canadian External Quality Assessment 

Laboratory 

307-2083 Alma St. 

Vancouver BC 

V6R 4N6 

Canada 

604/222-1355 

604/222-1373 

dseccombe@ceqal.com 

Semerjian, Hratch 

NIST 

100 Bureau Drive, MS 8300 

Gaithersburg, MD 20899-8300 

USA 

301/975-8300 

301/975-3845 

hratch.semerjian@nist.gov 

Siekmann, Lothar 

University of Bonn 

Bonn 

D-53127 

GERMANY 

49/228-287-5911 

49/228-287-5033 

siekmann@mailer.meb.uni-bonn.de 

Sieberman, Steven 

Bio-Tech Imaging, Inc. 

5711 Industry Lane, Unit 31 

Frederick, MD 21704 

USA 

301/695-9900 

301/695-0030 

silb@biotechimaging.com 
358

Slickers, Kenneth 

Roche Diagnostics Corporation 

9115 Hague Road 

P.O. Box 50457 

Indianapolis, IN 46250-0457 

USA 

317/576-7196 

317/576-4249 

ken.slickers@roche.com 

Smit, Nick 

i-STAT Canada 

436 Hazeldean Road 

Kanata, Ontario, KsL 1T9 

CANADA 

613-831-2725, x381 

Sniegoski, Lorna 

NIST 

100 Bureau Drive, MS 8392 

Gaithersburg, MD 20899-8392 

USA 

301/975-3133 

301/977-0685 

lorna.sniegoski@nist.gov 

Sridharan, Ramamurthi 

Instrumentation Laboratory 

113 Hartwell Ave. 

Lexington, MA 02421 

USA 

781/861-4246 

781/861-4452 

ssridharan@ilww.com 

Tai, Susan 

NIST 

100 Bureau Drive, MS 8392 

Gaithersburg, MD 20899-8392 

USA 

301/975-3127 

301/975-0685 

susan.tai@nist.gov 

Thomas, Jeanice Brown 

NIST 

100 Bureau Drive, MS 8392 

Gaithersburg, MD 20899-8392 

USA 

301/975-3120 

301/977-0685 

jeanice.brownthomas@nist.gov 

Thompson, John 

Dade Behring 

P.O. Box 6101 

707 

Newark, DE 19714 

USA 

302/631-7584 

302/631-7487 

thompsjc@dadebehring.com 

Tonkin, Charles 

UL International Ltd 

Old Portsmouth Rd. 

Wonersh House 

Guildford GU3 1LR 

United Kindom 

44/1483 302130 

44/1483-302230 

tonkinc@ul.com 

Trahey-Bale, Nancy 

NIST 

100 Bureau Drive 

Gaithersburg, MD 20899 

USA 

301/975-2021 

nancy.trahey@nist.gov 

Tuthill Jr., Willard 

Underwriters Laboratories, Inc. 

1285 Walt Whitman Rd. 

Melville, NY 11747 

USA 

631/271-6200 

631/439-6132 

Willard.E.TuthillJr@us.ul.com 
359

Vocke, Robert 

NIST 

100 Bureau Drive, MS 8391 

Gaithersburg, MD 20899-8391 

USA 

301/975-4103 

robert.vocke@nist.gov 

Voelkert, Emil 

Roche Diagnostics GmbH 

Dept. GH 

Sandhofer Str. 116 

D-68305 Mannheim 

GERMANY 

49/621/759-1656 

49/621-759-4583 

emil.voelkert@roche.com 

Wadhera, Ashwani 

Health Canada 

Main Stats Building, Room 1601 

Ottawa, Ontario K1A 0L2 

CANADA 

613/954-0387 

ashwani.wadhera@ha-sc.gc.ca 

Wahed, M.A. 

Nutritional Biochemistry 

ICDDR, B 

GPO Box 128 

Dhaka-1000 

BANGLADESH 

Welch, Michael 

NIST 

100 Bureau Drive, MS 8392 

Gaithersburg, MD 20899-8392 

USA 

301/975-3100 

301/977-0685 

michael.welch@nist.gov 

Westwood, Steven 

NARL, AGAL 

P.O. Box 385 

Pymble NSW 2073 

AUSTRALIA 

61/2-9449 0111 

61/2-9449 1653 

steven.westwood@agal.gov.au 

White, Richard 

Dade Behring Corporation 

20400 Mariani Ave. 

Cupertino, CA 95014 

USA 

408/366-3730 

408/996-9216 

rjwhite@jps.net 

Wiedmeyer, Hsiao-Mei 

University of Missouri-Columbia 

School of Medicine 

1 Hospital Drive, Rm. M767 

Columbia, MO 65212 

USA 

573/882-2705 

573/884-8823 

wiedmeyerh@heath.missouri.edu 

Wielgosz, Robert 

BIPM 

Pavillon De Breteuil 

Sevres Cedex F-92312 

FRANCE 

33/1 45 07 62 51 

33/1 45 34 20 21 

rwielgos@bipm.org 

Williams, Christopher 

Abbott Laboratories 

1401 Sheridan Rd. 

D-736/M4 

North Chicago, IL 60064 

USA 

847/938-9124 

847/937-4634 

christopher.williams@ln.ssw.abbott.com 

Wise, Beth Ann 

NCCLS 

940 West Valley Rd. 

Suite 1400 

Wayne, PA 19087 

USA 

610/688-0100 

bawise@nccls.org 
360

Wise, Stephen 

NIST 

100 Bureau Drive, MS 8392 

Gaithersburg, MD 20899-8392 

USA 

301/975-3112 

301/977-0685 

stephen.wise@nist.gov 

Wu-Wong, Ruth 

Abbott Laboratories 

5440 Patrick Henry Dr. 

Santa Clara, CA 95054 

USA 

408/567-3298 

408/588-2912 

ruth.r.wuwong@abbott.com 

Yager, Thomas 

Visible Genetics, Inc. 

700 Bay St., Suite 1000 

Toronto, Ontario 

M5G 1Z6 CANADA 

416/813-3240 

416/813-3250 

tom@visgen.com 

Onsite Registrants:

Campbell, Michael 

DiTuillo, Robert