97-2

Planning 

Report

Economic Evaluation 

of Radiopharmaceutical

Research at NIST 

U.S Department of Commerce

Technology Administration

Prepared by:

Albert N. Link

September 1997

Prepared for the

Ionizing Radiation Division

Physics Laboratory

National Institute

of Standards

and Technology
Economic Evaluation of Radiopharmaceutical

Research at NIST

Prepared for

Ionizing Radiation Division

 Physics Laboratory

 National Institute of Standards and Technology

by

Albert N. Link

Professor of Economics

University of North Carolina at Greensboro

September 1997
Table of Contents

Acknowledgments i

I. Introduction 1

II. Radiopharmaceutical Research at NIST 3

A. Incidence of Radiation Treatment 3

B. Research Mission of the Radioactivity Group 3

C. Research Activities of the Radioactivity Group 5

III. An Economic Analysis of Radiopharmaceutical Research at NIST 8

A. The Conceptual Framework 8

B. Survey Data and Analysis 8

C. Calculation of Economic Evaluation Metrics 11

IV. Interpretation of Findings 13

A. Internal Rate of Return 20

B. Adjusted Internal Rate of Return 14

C. Benefit-to-Cost Ratio 14

V. Conclusions 15

References 16
Acknowledgments

This study has benefited from the comments and suggestions of Bert Coursey, Dan Golas,

Robin Hutchinson, and Brian Zimmerman, all of the Radioactivity Group within the Ionizing

Radiation Division of the Physics Laboratory at NIST. It has also benefited from the comments

of Greg Tassey of the Program Office at NIST and John Scott in the department of economics at

Dartmouth College. In addition, it is a pleasure to acknowledge various individuals from the

member companies of the Measurement Assurance Program and elsewhere in the medical

community for their input and expert opinions regarding the economic benefits considered in this

report.
Economic Evaluation of Radiopharmaceutical Research at NIST

I. Introduction

Nuclear medicine is the branch of medicine that relies on radiation emitting substances,

called radioisotopes or radionuclides, for the diagnosis or treatment of a disease or condition.

The field of nuclear medical and biological research can be traced to the 1895 discovery by

Wilhelm Conrad Roentgen of a ray that could penetrate the human body and display bone

structures.1 Soon thereafter, in 1896, Henri Becquerel demonstrated that uranium in its normal

state gave off X-rays. Following these two path-breaking discoveries came the radioactivity

discoveries by Marie Curie in 1897. Then, in 1925, Herrman Blumgart and Otto Yens used

radioactive substances in human patients for the first time. In 1936, John H. Lawrence and

Joseph Gilbert independently undertook the first therapeutic clinical applications of a radioactive

substance to treat leukemia.

The use of radiochemicals or radiopharmaceuticals increased slowly during the pre-war

period and then more rapidly in the decades following World War II. Finally, in the early 1970s,

radiopharmaceutical manufacturers realized the importance of calibrations for the short-lived

radionuclides being used in nuclear medicine. According to Golas [1996], in the early 1970s

Standard Reference Materials (SRMs) were available in unsuitable physical forms and activity

levels. Also, there were no standard methods for measuring the activity of a radionuclide, even

ignoring the imprecise decay information that was available. As a result, no method existed for a

pharmacy to assure the accuracy of the radiation in a prescribed dose.

In the early 1970s, there was not an established research program on radiopharmaceutical

measurement at NIST [then the National Bureau of Standards (NBS)], or at any other federal

facility. In 1972 the Atomic Industrial Forum (AIF) responded to industry needs and appointed a

subcommittee of Manufacturers of Radioactive Reference Standards, with the objective of “...

obtaining a high degree of consistency and reliability in commercially available radioactive

reference standards and their accompanying Certificates of Calibrations” [Seidel and Hutchinson,

1976 (p. 1)].2

 The NBS was represented on this committee, as were all major commercial

manufacturers of radiochemicals.

In 1973, following the formation of this subcommittee, a symposium was held at NBS to

address the concerns of radionuclide manufacturers regarding the proposed standards. Speakers

at the symposium emphasized the need for such standards, citing:

1. the lack of standards for about 75 percent of the more than 100 radionuclides then

produced by industry;

 

1

 This information draws directly from Society of Nuclear Medicine [1996a].

2

 This committee became subcommittee N42.2 of the American National Standards Institute (ANSI).
2

2. the unusable physical form and activity levels of the few standards that were available;

and,

3. the failure to have industry-wide adopted decay-scheme data upon which to base

derived standard instrument calibrations.

It was also noted at the symposium that the lack of standards was an obstacle to both the

assurance of accuracy in the administration of radioisotopes, and to the approval of new drugs by

the federal Food and Drug Administration (FDA).

In response to industry-wide concerns expressed at this symposium, NBS entered into a

cooperative research agreement3 with the Atomic Industrial Forum (AIF) [now the Nuclear

Energy Institute (NEI)]. “... whereby NBS will supervise and administer on behalf of AIF a

measurements technology quality assurance program which caters more specifically to the needs

of the radiopharmaceutical industry” [Collé, 1976 (p. 71)].4 The NBS also entered into an

interagency agreement with the FDA to “ensure the continuous availability of national

radioactivity standards at appropriate levels of activity for use by the radiopharmaceutical

industry, and thus to establish a degree of uniformity in the measurements throughout the

industry” [Collé, 1976 (p. 71)].

Today, radiopharmaceutical Standard Reference Materials are produced within the

Radioactivity Group of the Ionizing Radiation Division of the Physics Laboratory at NIST

through a CRADA with NEI, and sold to the members of the NEI/NIST Radioactivity

Measurement Assurance Program (MAP) and to the public. The purpose of this report is to

document and quantify selected broad-based economic impacts associated with this program’s

research activities.

The report is outlined as follows. In Section II, the research activities of the Radioactivity

Group are discussed in detail. In Section III, a conceptual framework for evaluating the

economic effects of radiopharmaceutical research at NIST is described and evaluated. In Section

IV, the findings from the economic evaluation are interpreted. Finally, concluding remarks are

offered in Section V.

 

3

 Such agreements are now known as Cooperative Research and Development Agreements or CRADAs.

4

 Today, this program is known as the NEI/NIST Radioactivity Measurements Assurance Program.
3

II. Radiopharmaceutical Research at NIST

A. Incidence of Radiation Treatment

In 1996, over 3,900 hospital-based nuclear medicine departments in the United States

performed over 10 million nuclear procedures; about 90 percent of these procedures were

diagnostic and about 10 percent were therapeutic.5 The number of such procedures has increased

from 7.7 million just a decade ago. Experts expect these numbers to increase between 5 percent

and 10 percent per year over the next 5 years.6 Patients undergoing a nuclear medical exam

receive a prescribed dosage of a radiopharmaceutical. These radiopharmaceuticals produce

radioactive emissions specifically formulated to be collected temporarily in the organ being

examined. Gamma cameras and PET scanners are used to detect and transfer these emissions to

film or to a computer image for study.7

Data from the Health Physics Society [1996] show that in 1987, 26 percent of the

7,690,000 diagnostic tests conducted in the United States were bone studies, 18 percent were

heart studies, and 17 percent were lung studies.8

If a patient receives too little radiation the procedure will often have to be redone,

particularly for a therapeutic procedure. If the patient receives too much radiation, injury or death

could result. In either case, there is an economic cost to imprecise measurement. It is therefore

incumbent upon the nuclear pharmacist to deliver as close to the prescribed dosage as measurably

possible, that is to ensure that the patient receives x amount of radiation, where x is measured in

terms of number of atoms of the radioactive substance decaying per unit of time.

B. Research Mission of the Radioactivity Group

The Ionizing Radiation Division, one of six divisions in the Physics Laboratory at NIST,

provides national leadership in:9

“... fundamental research and measurement standards for all types of

ionizing radiation, including X-rays, gamma rays, electrons, neutrons,

energetic heavy charged particles, and radioactivity. The Division’s

 

5 Health Physics Society [1996]. See also, Institute of Medicine [1995].

6 This forecast was provided by experts within the Radioactivity Group at NIST. See also Institute of Medicine

[1995].

7

Society of Nuclear Medicine [1996b].

8

 These are the most recent study-specific data available.

9

 Physics Laboratory [1996].
4

extensive domestic and international collaborations in applied research help

to make ionizing radiation effective in industrial and health applications and

safe in the environment and the workplace.

The Radioactivity Group is one of three research groups within the Division. Its mission

is to develop and maintain U.S. radioactivity standards and to confirm the standards

internationally; to perform research on radioactivity decay scheme characteristics; to develop and

apply radioactivity measurement techniques; and to disseminate results using standard reference

materials, calibrations and testing services, and cooperative industrial measurement traceability

programs.

This mission complements NIST’s overall responsibilities. The NBS was established in

the Treasury Department under the Organic Act of March 3, 1901:

Be it enacted by the Senate and House of Representatives of the United

States of America in Congress assembled, That the Office of Standard

Weights and Measures shall hereafter be known as the National Bureau of

Standards ... That the functions of the bureau shall consist in ... the

determination of physical constants and the properties of materials, when

such data are of great importance to scientific or manufacturing interests

and are not to be obtained of sufficient accuracy elsewhere.

The NBS was transferred to the Department of Commerce and Labor in 1903 and to the

Department of Commerce in 1913.10 All responsibilities were transferred to NIST when NBS

was re-named under the Omnibus Trade and Competitiveness Act of 1988. As described in the

United States Code, Title 15, Chapter 7:

[The] National Institute of Standards and Technology [shall] enhance the

competitiveness of American industry while maintaining its traditional

function as lead national laboratory for providing the measurement,

calibrations, and quality assurance techniques which underpin United States

commerce, technological progress, improved product reliability and

manufacturing processes, and public safety ... [and it shall] prepare, certify,

and sell standard reference materials for use in ensuring the accuracy of

chemical analyses and measurements of physical and other properties of

materials.

 

10 Between 1903 and 1934 it was called the Bureau of Standards.
5

C. Research Activities of the Radioactivity Group11

To date, NIST has produced the 28 radiopharmaceutical standards listed in Table 1.12

Also shown in Table 1 is the primary diagnostic use of each of the radionuclides.

A radiopharmaceutical standard produced at NIST is typically accurate to within ±1% at

the one standard deviation level; that is all standards produced at NIST are calibrated within ±1%

of the actual decay rate of the substance. Or stated alternatively, NIST’s calibration error on

these SRMs is ±1%. These calibrated SRMs are provided to the radiopharmaceutical

manufacturing companies that belong to the NEI/NIST MAP for an approximate fee of $600.00

per vial.13,14 The current members of the assurance program are listed in Table 2.

Not all radiopharmaceutical manufacturers are members of the MAP. Non-member

companies can obtain standards from commercial laboratories for a similar fee; however, the

standards produced and sold by commercial laboratories are less accurate than those produced at

NIST. On average, commercial laboratories are within ±5% of the actual decay rate of the

substance, but these laboratories are traceable to NIST.15

Ten SRMs are produced at NIST each year, one each month following a schedule agreed

upon by MAP members. The months of May and November are so-called “open months.”

During these two months, members of the MAP can submit their own samples to NIST for

calibration as long as it contains a radionuclide already standardized by NIST. This allows

companies that produce radioactive substances to have calibrated samples of radionuclides that

are important to them, but that are not on NIST’s current production schedule to obtain

traceability.

Table 3 shows NIST research costs related to the production of the radionuclide SRMs

since 1990. While NIST initially became active in providing these SRMs in 1974, the cost data

relevant to this study, as discussed below, begin with 1990. These cost data include all NIST

capital, labor, and materials expenses.

 

11 This section draws directly from Golas [1996].

12 Some of these standards are no longer produced or are produced only rarely for a user to verify response rates.

The letter at the end of the SRM number indicates how many times the substance has been produced (e.g., A=1,

B=2, and so on).

13 In general, a vial is a standard NIST glass ampoule that contains 5 milliliters of solution.

14 These revenues are divided between NIST and NEI; $108.00 to NIST to underwrite, in part, research and the

production of the SRM, and $492.00 to NEI.

15 This information was provided by experts within the Radioactivity Group at NIST, and it was verified during the

interview stage of the study.
6

Table 1. Radiopharmaceutical Standard Reference Materials (SRMs) Produced at NIST

Radionuclide SRM Number Diagnostic/Therapeutic Use or Relevant Organ

Chromium-51 4400 N Diagnosis of blood cell survival

Iodine-131 4401 V Thyroid and other organs

Tin-113-Indium-113m 4402 C [Replaced by Indium-111]

Strontium-85 4403 B Bone studies

Thallium-201 4404 S Cardiac studies

Gold-198 4405 B Synovectomy, treatment for rheumatoid arthritis

Phosphorus-32 4406 N In vitro traces/biotechnology

Iodine-125 4407 T In vitro clinical chemistry

Cobalt-57 4408 F Standard for calibration of imaging instruments

Selenium-75 4409 D Biochemical tracer

Technetium-99m 4410 V Imaging of multiple organs

Iron-59 4411 B Hemoglobin studies

Molybdenum-99 4412 U [Parent of Technetium-99m]

Mercury-197 4413 A In vitro tracer studies

Iodine-123 4414 C Thyroid and other organs

Xenon-133 4415 T Lung imaging studies

Gallium-67 4416 Q Gastrointestinal tract imaging

Indium-111 4417 P Radioimmunotherapy

Mercury-203 4418 A In vitro tracer studies

Ytterbium-169 4419 C Brachytherapy

Lead-203 4420 B In vitro tracer studies

Gold-195 4421 A Tracer for new therapeutic agents

Chlorine-36 4422 A Beta-particle standard

Strontium-90 4423 A Beta-particle standard

Sulfur-35 4424 A Cell kinetics and metabolism; biochemical tracer

Samarium-153 4425 B Bone palliation agent

Strontium-89 4426 A Bone palliation agent

Yttrium-90 4427 A Radiolabelled antibody therapy

Source: Institute of Medicine [1995] and information provided by experts within the Radioactivity Group

at NIST.
7

Table 2

Current Participants in the NEI/NIST Radioactivity Measurement Assurance Program

Bristol-Meyers Squibb Company

DuPont Merck Pharmaceuticals Company

Institute of Nuclear Energy Research (Taiwan)

Mallinckrodt Medical, Incorporated

Medi+Physics, Incorporated

Nordion International, Incorporated

Packard Instruments, Incorporated

Syncor International Corporation

U.S. Food and Drug Administration

Source: Golas [1996] and the Radioactivity Group at NIST.

Table 3. NIST Costs Related to the Production of Radiopharmaceutical SRMs

(in $1,000)

Year NIST Costs

1990 $210

1991 $218

1992 $218

1993 $265

1994 $226

1995 $384

1996 $364

Source: The Radioactivity Group at NIST.
8

III. An Economic Analysis of Radiopharmaceutical Research at NIST

A. The Conceptual Framework

The approach selected for evaluating the economic impacts associated with the research

activities of the Radioactivity Group at NIST has been adopted from previous evaluations of other

laboratory research programs at NIST and from evaluations of other public investments in

infrastructure technology.16 The actual NIST costs related to the production of

radiopharmaceutical SRMs (Table 3) are compared to estimates of the economic benefits received

by SRM users derived from a hypothetical counterfactual experiment. The experiment assumes

that the first-level economic benefits associated with the Radioactivity Group’s research can be

approximated in terms of the additional costs that radiopharmaceutical manufacturers and patients

would incur in the absence of NIST’s research.

The hypothetical counterfactual experiment is used because this case study lacks

comparable baseline observations. In other words, it is not the case that some members of MAP

rely on NIST for only selected SRMs. Were that the case, then a comparison of diagnostic/

therapeutic efficiency could in principle have been conducted between calibrated and noncalibrated substances.17

It is important to keep in mind that this study focuses on only one part of the production

and distribution chain—the radiopharmaceutical manufacturer. The radiopharmaceutical itself

requires a radionuclide that is produced either in a nuclear reactor or a particle accelerator. Some

manufacturers operate their own reactors or accelerators, but many do not. Therefore, the

radionuclides must be purchased from other laboratories such as those operated by the U.S.

Department of Energy or by universities. Once the radiopharmaceuticals are prepared at the

manufacturer, they are shipped to the clinic or hospital. At this point, it is the responsibility of the

radiopharmacist to dispense the bulk shipment into individual doses for the patients. Finally, the

radiopharmaceutical is administered to the patient by the nuclear medicine technician. At each of

these steps in the distribution chain, it is important to maintain good measurement and quality

assurance practice because any errors introduced, especially early in the distribution chain, are

quickly multiplied before the drug is given to the patient.

B. Survey Data and Analysis

Previous experience with the collection of information related to the economic benefits

associated with NIST’s laboratory research suggests that the most efficient, and presumably the

most accurate means to collect data is through semi-structured, interactive telephone interviews.

Accordingly, the Radioactivity Group identified a contact person in each of the seven

manufacturing companies in the MAP listed in Table 2. In effect, this group defined the domestic

industry of radiopharmaceutical manufacturers for this study.

 

16 See Link [1996a, 1996b] and Link and Scott [1997].

17 Data could not be obtained from manufacturers of radiopharmaceuticals that are not members of the MAP.
9

The radionuclide industry is of modest size. Based on data from the Department of

Commerce’s Bureau of the Census in its 1992 Census of Manufacturers, SIC industry 2835 is

Diagnostic Substances. In 1992, the value of product shipments from this four-digit industry was

$6,177.1 million. The five-digit SIC industry 28352, In Vivo Diagnostic Substances, had a value

of product shipments of $1,051.8 million in 1992. The value of product shipments of In Vivo

Radioactive Reagents (diagnostic and therapeutic), SIC 2835220, was $323.2 million in that year,

and there were seven companies with shipments over $100,000.

Prior to the telephone interview, each identified participant was sent an electronic mail

message from the Radioactivity Group introducing the study, soliciting participation in the study,

and assuring confidentiality of individual responses. Each identified participant was asked to think

about two general topics prior to the telephone interview. The first topic related to what their

company would do under the hypothetical counterfactual situation where NIST ceased to produce

SRMs. The second topic related to how the industry would adjust to such a situation and what

the consequences would be.

Each identified manufacturing individual was interviewed. The following conclusions are

based on these interviews.

1. Absent NIST’s Radioactivity Group’s involvement in SRMs, it would take between 5

and 10 years for some industry group or association to form and become accepted as a

de facto standard setting body. The total (summed over the seven companies)

expected transaction costs during this transition period would be at least $1.3 million

per year. This amount represents the additional labor costs expected to be required on

the part of manufacturers to resolve measurement disputes between manufacturers and

customers, and the associated additional measurement equipment needed by

manufacturers. The expressed expectation was that these costs would increase

between 4 percent and 10 percent per year until a steady state situation was reached.

2. During this 5 to 10 year transition period, the level of accuracy at the manufacturing

stage would decrease from the current ±3% to between ±5% and ±10% owing to the

lack of accurate reference materials and measurement methods. At the hospital, the

accuracy of dosages would fall from the current ±10% to at least ±15%.

3. Reliance on foreign national laboratories for SRMs has never been considered due to

the measurement quality at NIST. Thus, no respondent could reasonably assess the

likelihood that the industry would turn to a foreign national laboratory for SRMs in the

hypothetical counterfactual situation.

These findings are summarized in Table 4.

Table 4. Summary of Findings from Interviews with the MAP Manufacturing Members
10

Hypothetical Counterfactual Situation Range of Responses

Absent NIST’s radiopharmaceutical research, how long

would it take an industry group or association to form and

become accepted as a de facto standard setting body?

Between five and ten years

During this five to ten year transition period, what would

happen to the accuracy of radiopharmaceuticals at the

manufacturing stage?

Accuracy would decrease from ±3%

to between ±5% and ±10%

During this five to ten year transition period, what would

happen to the accuracy of the radiopharmaceuticals

administered by hospitals?

Dosage accuracy would decrease

from ±10% to at least ±15%

The findings from these interviews with radiopharmaceutical manufacturers are, that

should NIST’s current efforts regarding SRMs be absent, an industry association would over a 5

to 10 year period evolve to replicate NIST’s current role and level of accuracy in SRMs. During

this transition period manufacturers would experience sizable transaction costs and patients would

receive less accurate dosages of radionuclides. The economic costs to manufacturers during this

transition period are estimated to be at least $1.3 million per year, to increase at a rate not less

than 4 percent per year; and patient dosages of radioactive substances will fall by at least

5 percentage points, from ±10% to ±15%.18

To quantify, in dollar terms, the economic cost of decreased accuracy, the Radioactivity

Group identified three recognized experts at renowned U.S. medical centers as candidates for

telephone interviews. Each was interviewed, and as a group they expressed the following

opinions:

1. A decrease in the accuracy of a radioactive substance from ±10% to ±15% would

require that, on average, 1 percent of all diagnostic procedures be re-done due to too

low a dosage (e.g., imprecise imaging) at an estimated average cost of between $500

and $750 per diagnostic procedure.

2. A decrease in the accuracy of a radioactive substance from ±10% to ±15% would

require that, on average, 3 percent of all therapeutic procedures be re-done due to too

low a dosage (e.g., ineffective treatment) at an estimated average cost of between

$1,500 and $2,500 per therapeutic procedure.

3. The health implications (e.g., organ damage) of too high a dosage could not be

estimated owing to the fact that such consequences would not be known immediately.

 

18 Certainly, manufacturers and patients received benefits from NIST’s research during the 1990 to 1996 period.

However, as shown in Table 6, these benefits were not quantified owing to the counterfactual methodology adopted

for estimating avoided costs if NIST were to cease its related research activity.
11

However, it is probable that substantial long-term economic costs would be associated

with this type of misadministration.

Table 5 summarizes these economic benefits.

Table 5. Summary of Economic Benefits from Radioactivity Research at NIST

Category of Economic Benefit Economic Benefit

Reduced transaction costs between radiopharmaceutical

manufacturers and their customers Saved $1.3 million

Increased accuracy of diagnostic procedures Saved re-doing 1% of all

procedures at an average cost of

$500 to $750 per procedure

Increased accuracy of therapeutic procedures Saved re-doing 3% of all

procedures at an average cost of

$1,500 to $2,500 per procedure

C. Calculation of Economic Evaluation Metrics

Table 6 replicates in the second column the NIST research costs from Table 3 from 1990

through 1996. The cost datum for 1997 is the 1997 budgeted cost amount; estimated costs

(rounded) for 1998 through 2001 are based on a 5.5 percent annual increase, the actual increase

from 1996 to 1997. The five year forecast period is based on the opinion of radiopharmaceutical

manufacturers that, absent NIST, the industry would incur transactions costs and decreased

accuracy for a period of five to ten years. For the sake of conservativeness, the lower bound of

the forecasted range is used—five years.

The third column in the table contains manufacturers’ estimates of the total industry cost

savings from NIST continuing its SRM activity as estimated by the transaction costs (labor and

equipment costs) that manufacturers would have had to incur in the absence of NIST’s

radiopharmaceutical research. The 1997 estimate of $1.3 million is a direct interview aggregate.

The range of opinion on the growth rate of this amount was 4 percent to 10 percent per year.

Again for the sake of conservativeness, the lower bound of the forecasted range is used—4

percent.

The fourth column contains the estimated value of the cost savings to patients from

avoiding having to have procedures re-done because of an underdose of radiation. With respect

to diagnostic procedures, the base estimate of 1 percent of 9,000,000 diagnostic procedures per

year was used, valued at the lower-bound estimate of $500 per procedure. For therapeutic

procedures, the base estimate of 3 percent of 1,000,000 therapeutic procedures per year was

used, valued at the lower-bound estimate of $1,500 per procedure. Coincidentally, the estimate

of patient costs avoided given NIST’s research involvement is $45,000,000 for each procedure, or

a total annual economic benefit estimate of $90,000,000. Based on expert opinion that the
12

number of such procedures will increase between 5 percent and 10 percent per year over the next

five years, a conservative lower-bound rate of increase of 5 percent per year is imputed to the

$90,000,000 estimate. Absent opinion on the annual rate of increase in procedure costs, it was

assumed, again for the sake of conservativeness, that per procedure costs will remain constant

over the next five years.

Finally, the fifth column in Table 6 contains total net benefits, defined, by year, to be the

sum of manufacturer and patient benefits less NIST costs. Net benefits are negative in value in

years 1990 through 1996 owing to the fact that economic benefits are not realized until 1997.

Table 6. Actual and Projected NIST Costs and Economic Benefits

(in $000)

Year NIST

Costs

Manufacturer

Benefits

Patient

Benefits

Total Net

Benefits

1990 $210 — — - $210

1991 $218 — — - $218

1992 $218 — — -$218

1993 $265 — — - $265

1994 $226 — — - $226

1995 $384 — — - $384

1996 $364 — — - $364

1997 $384 $1,300 $90,000 $90,916

1998 $405 $1,352 $94,500 $95,447

1999 $427 $1,406 $99,225 $100,204

2000 $450 $1,462 $104,186 $105,198

2001 $475 $1,521 $109,395 $110,441

IV. Interpretation of Findings

The data in Table 6 are used for the calculation of three evaluation metrics; an internal rate

of return, an adjusted internal rate of return, and a benefit-to-cost ratio. Each of these is an
13

accepted metric for quantifying the net economic impact associated with a public research

program like that within the Radioactivity Group.

A critical issue related to the calculation of the evaluation metrics is the choice of a base

period and the choice of a terminal period for the analysis. The choice of the terminal period was

discussed previously with regard to a five year forecast. However, some judgment must be

exercised in determining the base period. The relevant question is the following: the current level

of research expertise within the Radioactivity Group represents the culmination of previous

efforts dating back to what year? Certainly, knowledge depreciates over time. Based on

discussion with the Radioactivity Group, current economic benefits are traceable to research that

began four to six years ago. Again, for the sake of conservativeness, a six-year period is used,

and thus the cost data in Table 3 and Table 6 begin with 1990.

A. Internal Rate of Return

By definition, the internal rate of return (IRR) is the value of the discount rate, i, that equates

the present value (PV) of a net benefit stream to zero. Mathematically, the IRR is the rate of discount,

i, that satisfies the equation:

(1) PV = [(B0-C0)/(1+i)

0

] + ... + [(Bn-Cn)/(1+i)

n

] = 0

where (Bt-Ct) represents net benefits in year t, and n is the number of years under consideration.

Based on the net benefit data in the fifth column of Table 6, the calculated value of i for which

PV=0 is 1.38 (rounded), implying an internal rate of return to NIST's investments in

radiopharmaceutical research of 138 percent.

 The internal rate of return estimate of 138 percent means that 1.38 (rounded) is the value of

the discount rate that equates the present value (1990 as the base year) of net benefits to zero, or stated

alternatively the present value of total benefits to the present value of costs.

Economists and policy makers generally use internal rate of return measures, for on-going or

completed public-sector research projects, to estimate what is referred to in the economics literature as

an approximation of the social rate of return. If 138 percent internal rate of return calculated above, is

greater than the hurdle rate (generally accepted rate of return) experienced by NIST, then the

radiopharmaceutical research program is surely worthwhile from a social perspective. This is especially

evident when one considers that this internal rate of return is a conservative estimate, as discussed in

Section V.

B. Adjusted Internal Rate of Return

It is not uncommon to misinterpret an internal rate of return measure as an annual yield similar

to that earned on, say, a bank deposit. One invests, say $100, and then earns interest on that $100 each
14

year plus interest on the interest. That is not the case in an R&D project, in general, or in the case of

NIST's radiopharmaceutical research, in particular.

Under an alternative set of assumptions, an adjusted internal rate of return (AIRR) can be

calculated from the data in Table 6 to provide a rate of return that is more analogous to an annual yield

(as earned on a bank deposit for example). It is important to re-emphasize that the AIRR is not the

same as the IRR and that the AIRR is not directly comparable to the economics literature on social

rates of return. Nevertheless, the AIRR remains an accepted evaluation metric.

If all net NIST costs are referenced to 1990 using a discount rate of 8.78 percent, the 1990

present value of NIST costs is $2,435,720. The discount rate of 8.78 percent is the sum of the Office

of Management and Budget’s [1992] recommended real rate of discount of 7.0 percent plus and

inflation factor of 1.78 percent.19 If all economic benefits (the sum of manufacturer benefits and patient

benefits) are referenced forward to the year 2001 using the same rate, the 2001 present value of

industry benefits is $596,138,530. The AIRR is the rate of return that equated a single NIST

investment of $2,435,720 in 1990 to a single benefit estimate of $596,138,530 in 2001.

Thus, the annual compounded rate of return that corresponds to such an investment in 1990

culminating in 2001 equals 65 percent (rounded) based on the value of x that satisfied the following

relationship:

(2) $2,435,720 (1+x)

11 = $596,138,530

C. Benefit-to-Cost Ratio

The third evaluation metric is a benefit-to-cost ratio (B/C). This ratio is, by definition the

present value of all benefits to the present value of all costs, where the point of reference for both

benefits and costs is the base time period, 1990. The relevant discount rate for the calculation of

present value is again 8.78 percent. For the data in Table 6, the present value of all benefits equals

$236,215,070 and the present value of all costs equals $2,435,720. Thus, the ratio of benefits-to-costs

is:

(3) B/C = 97-to-1

It should be noted that the calculated IRR of 138 percent from equation (1) implies a benefitto-cost ratio of unity. Rewriting equation (1) using summation notation:

(4) PV = [Σt=0 to n Bt/(1+i)

t

] - [Σt=0 to n Ct/(1+i)

t

]

 

19 The Office of Management and Budget [1992] recommends using a nominal discount rate equal to a 7 percent

real discount rate plus an inflation factor. The inflation rate of 1.78 percent is the Federal Reserve Board’s

published implicit price deflator for the past four quarters.
15

where i is the IRR that equated PV=0. When PV=0, then it follows that:

(5) [Σt=0 to n Bt/(1+i)

t

] = [Σt=0 to n Ct/(1+i)

t

]

or that the present value of benefits equals the present value of costs, or B/C=1. Thus, as is the case

here, when B/C > 1 it implies that the IRR is greater than the social rate of discount used in the benefitto-cost calculation.

V. Conclusions

NIST responded to the radiopharmaceutical industry’s needs for standards as outlined by the

Atomic Industrial Forum in 1973. Related research has been ongoing at NBS/NIST since 1974. This

research, in the opinion of manufacturers and in the opinion of medical experts today has been

invaluable. Quantitatively, the activities of the now Radioactivity Group are significantly benefiting

society as measured by accepted evaluation metrics.

Three standard evaluation metrics were quantified in this study: an internal rate of return of

138 percent, an adjusted internal rate of return of 65 percent, and a benefit-to-cost ratio of 97-to-1.

Each of these metrics was calculated under the most conservative set of assumptions possible. In every

instance when experts expressed an opinion in terms of a range of values, the most conservative end

point of the range was used. Also, there are certainly costs associated with both underdose and

overdose misadministrations that go beyond the direct cost of readministering a dosage (in the case of

an underdose). The benefits associated with these cost savings also have not been considered in this

analysis. Were they to be considered, the estimated values of all three of the metrics above would

increase. Still, when compared to the social cost of resources used to generate the economic benefits

described herein—an 8.78 percent return on resource costs—the radiopharmaceutical research at

NIST is without question valuable to society.
16

References

Collé, R., 1976. “AIF-NBS Radioactivity Measurements Assurance Program for the Radiopharmaceutical

Industry,” National Bureau of Standards Special Publication No. 456, pp. 71-76.

Golas, Daniel B., 1996. “NIST Radiopharmaceutical Standard Reference Materials and the NEI/NIST

Radiopharmaceutical Measurement Assurance Program” (mimeographed November 1996).

Golas, Daniel B., 1993. “USCEA/NIST Measurement Assurance Programs for the Radiopharmaceutical

and Nuclear Power Industries,” Proceedings of the Workshop on Measurement Quality Assurance

for Ionizing Radiation (March 1993).

Health Physics Society, 1996. “Centennial Calendar.”

Hutchinson, J. Robin, 1996. “Collaboration with and Support of Radiopharmaceudical Manufacturers

through the U.S. Council on Energy Awareness” (mimeographed).

Institute of Medicine, 1995. Committee on Biomedical Isotopes, Isotopes for Medicine and the Life

Sciences, edited by S.J. Adelstein and F.J. Manning, Washington, DC: National Academy Press.

Link, Albert N., 1996a. “Economic Impact Assessments: Guidelines for Conducting and Interpreting

Assessment Studies,” NIST Planning Report 96-1.

Link, Albert N., 1996b. Evaluating Public Sector Research and Development, New York: Praeger

Publishers.

Link, Albert N. and John T. Scott, 1997. “Evaluating Technology-Based Public Institutions: Lessons from

the National Institute of Standards and Technology,” in Policy Evaluation in Innovation and

Technology, edited by G. Papaconstantinou, Paris: OECD (in press).

Office of Management and Budget, 1992. “Memorandum for Heads of Executive Departments and

Establishments, Circular No. A-94,” Washington, DC, October 29, 1992.

Physics Laboratory Brochure, 1996. http://physics.nist.gov/overview/div846bro.html.

Seidel, C.W. and J.M.R. Hutchinson, 1976. “American National Standards Institute Quality Assurance

Program in Radioactivity Measurements,” NBS Special Publication No. 456, pp. 1-6.

Society of Nuclear Medicine, 1996a. Nuclear Medicine: 100 Years in the Making, 1896-1966, Reston,

VA: Society of Nuclear Medicine.

Society of Nuclear Medicine, 1996b. “Nuclear Medicine, X Rays, CT and MRI,”

http://www.snm.org/nmx-ray.html