The Feasibility of Economic Evaluation of

Diagnostic Procedures: The Case of CT

Scanning

April 1981

NTIS order #PB81-209777
CASE STUDY # 2

THE IMPLICATIONS OF

COST-EFFECTIVENESS

ANALYSIS OF

MEDICAL TECHNOlogy

APRIL 1981

BACKGROUND PAPER #2: CASE STUDIES OF

MEDICAL TECHNOLOGIES

CASE STUDY #2: THE FEASIBILITY OF ECONOMIC EVALUATIO N

OF DIAGNOSTIC PROCEDURES: THE CASE OF CT SCANNIN G

Judith L, Wagner, Ph. D.

Senior Research Asociate, The Urban Institute

Wshington, D.C.

I 1

OTA Background Papers are documents that contain information believed to be

useful to various parties. The informtion under-girds formal OTA assessments or is

a n outcome of interna exploratory planning and evaluation. The material is usually

not of immediate policy intereest such as is contained in an OTA Report or Technical

Memorandum, nor does it present options for Congress to consider.

1 J
Library of Congress Catalog Card Number 80-600161

For sale by the Superintendent of Documents,

U.S. Government Printing Office, Washington, D.C. 20402
Foreword

This case study is one of 17 studies comprising Background Paper #2 for OTA’s

assessment, The Implications of Cost-Effectiveness Analysis of Medical Technology.

That assessment analyzes the feasibility, implications, and value of using cost-effectiveness and cost-benefit analysis (CEA/CBA) in health care decisionmaking. The major, policy-oriented report of the assessment was published in August 1980. In addition

to Background Paper #2, there are four other background papers being published in

conjunction with the assessment: 1) a document which addresses methodological

issues and reviews the CEA/CBA literature, published in September 1980; 2) a case

study of the efficacy and cost-effectiveness of psychotherapy, published in October

1980; 3) a case study of four common diagnostic X-ray procedures, to be published in

summer 1981; and 4) a review of international experience in managing medical technology, published in October 1980. Another related report was published in

September of 1979: A Review of Selected Federal Vaccine and Immunization Policies.

The case studies in Background Paper #2; Case Studies of Medical Technologies

are being published individually. They were commissioned by OTA both to provide

information on the specific technologies and to gain lessons t-hat

the broader policy aspects of the use of CEA/CBA. Several of the

cally requested by the Senate Committee on Finance,

Drafts of each case study were reviewed by OTA staff; by

could be applied to

studies were specifimembers of the advisory panel to the overall assessment, chaired by Dr. John Hogness; by members of

the Health Program Advisory Committee, chaired by Dr. Frederick Robbins; and by

numerous other experts in clinical medicine, health policy, Government, and economics. We are grateful for their assistance. However, responsibility for the case studies remains with the authors.

Director
Advisory Panel on The Implications of

Cost= Effectiveness Analysis of Medical Technology

John R. Hogness, Panel Chairman

President, Association of Academic Health Centers

Stuart H. Altman Sheldon Leonard

Dean Manager

Florence Heller School Regulatory Affairs

Brandeis University General Electric Co.

James L. Bennington

Chairman

Department of Anatomic Pathology and

Clinical Laboratories

Children’s Hospital of San Francisco

John D. Chase

Associate Dean for Clinical Affairs

University of Washington School of Medicine

Joseph Fletcher

Visiting Scholar

Medical Ethics

School of Medicine

University of Virginia

Clark C. Havighurst

Professor of Law

School of Law

Duke University

Barbara J. McNeil

Department of Radiology

Peter Bent Brigham Hospital

Robert H. Moser

Executive Vice President

Americans College of Physicians

Frederick Mosteller

Chairman

Department of Biostatistics

Harvard University

Robert M. Sigmond

Advisor on Hospital Affairs

Blue Cross and Blue Shield Associations

Jane Sisk Willems

VA Scholar

Veterans Administration
OTA Staff for Background Paper #2

Joyce C. Lashof, Assistant Director, OTA

Health and Life Sciences Division

H. David Banta, Health Program Manager

Clyde J. Behney, Project Director

Kerry Britten Kemp, * Editor

Virginia Cwalina, Research Assistant

Shirley Ann Gayheart, Secretary

Nancy L. Kenney, Secretary

Martha Finney, * Assistant Editor

Other Contributing Staff

Bryan R. Luce Lawrence Miike Michael A. Riddiough

Leonard Saxe Chester Strobel’

OTA Publishing Staff

John C. Holmes, Publishing Officer

John Bergling* Kathie S. Boss Debra M. Datcher

Patricia A. Dyson* Mary Harvey* Joe Henson
Preface

This case study is one of 17 topics being issued that comprise Background Paper #2 to the

OTA project on the Implications of Cost-Effectiveness Analysis of Medical Technolology. * The

overall project was requested by the Senate

Committee on Labor and Human Resources. In

all, 19 case studies of technological applications

were commissioned as part of that project.

Three of the 19 were specifically requested by

the Senate Committee on Finance: psychotherapy, which was issued separately as Background Paper #3; diagnostic X-ray, which will

be issued as Background Paper #5; and respiratory therapies, which will be included as part of

this series. The other 16 case studies were selected by OTA staff.

In order to select those 16 case studies, OTA,

in consultation with the advisory panel to the

overall project, developed a set of selection

criteria. Those criteria were designed to ensure

that as a group the case studies would provide:

●

●

●

●

●

●

●

examples of types of technologies by function (preventive, diagnostic, therapeutic,

and rehabilitative);

examples of types of technologies by physical nature (drugs, devices, and procedures);

examples of technologies in different stages

of development and diffusion (new, emerging, and established);

examples from different areas of medicine

(such as general medical practice, pediatrics, radiology, and surgery);

examples addressing medical problems that

are important because of their high frequency or significant impacts (such as

cost );

examples of technologies with associated

high costs either because of high volume

(for low-cost technologies) or high individual costs;

examples that could provide informative

material relating to the broader policy and

methodological issues of cost-effectiveness

or cost-benefit analysis (CEA/CBA); and

. examples with sufficient evaluable literature.

On the basis of these criteria and recommendations by panel members and other experts,

OTA staff selected the other case studies. These

16 plus the respiratory therapy case study requested by the Finance Committee make up the

17 studies in this background paper.

All case studies were commissioned by OTA

and performed under contract by experts in academia. They are authored studies. OTA subjected each case study to an extensive review

process. Initial drafts of cases were reviewed by

OTA staff and by members of the advisory

panel to the project. Comments were provided

to authors, along with OTA’s suggestions for

revisions. Subsequent drafts were sent by OTA

to numerous experts for review and comment.

Each case was seen by at least 20, and some by

40 or more, outside reviewers. These reviewers

were from relevant Government agencies, professional societies, consumer and public interest

groups, medicaI practice, and academic medicine. Academicians such as economists and decision analysts also reviewed the cases. In all,

over 400 separate individuals or organizations

reviewed one or more case studies. Although all

these reviewers cannot be acknowledged individually, OTA is very grateful for their comments and advice. In addition, the authors of

the case studies themselves often sent drafts to

reviewers and incorporated their comments.

These case studies are authored works

commissioned by OTA. The authors are responsible for the conclusions of their specific case study. These cases are not statements of official OTA position. OTA does

not make recommendations or endorse particular technologies. During the various

stages of the review and revision process,

therefore, OTA encouraged the authors to

present balanced information and to recognize divergent points of view. In two cases,

OTA decided that in order to more fully

present divergent views on particular technologies a commentary should be added to

the case study. Thus, following the case
studies on gastrointestinal endoscopy and

on the Keyes technique for periodontal disease, commentaries from experts in the appropriate health care specialty have been

included, followed by responses from the

authors.

The case studies were selected and designed to

fulfill two functions. The first, and primary,

purpose was to provide OTA with specific information that could be used in formulating

general conclusions regarding the feasibility and

implications of applying CEA/CBA in health

care. By examining the 19 cases as a group and

looking for common problems or strengths in

the techniques of CEA/CBA, OTA was able to

better analyze the potential contribution that

these techniques might make to the management

of medical technologies and health care costs

and quality. The second function of the cases

was to provide useful information on the specific technologies covered. However, this was

not the major intent of the cases, and they

should not be regarded as complete and definitive studies of the individual technologies, In

many instances the case studies do represent excellent reviews of the literature pertaining to the

specific technologies and as such can stand on

their own as a useful contribution to the field. In

general, though, the design and the funding

levels of these case studies was such that they

should be read primarily in the context of the

overall OTA project on CEA/CBA in health

care.

Some of the case studies are formal CEAS or

CBAS; most are not. Some are primarily concerned with analysis of costs; others are more

concerned with analysis of efficacy or effectiveness. Some, such as the study on end-stage

renal disease, examine the role that formal

analysis of costs and benefits can play in policy

formulation. Others, such as the one on breast

cancer surgery, illustrate how influences other

than costs can determine the patterns of use of a

technology. In other words, each looks at evaluation of the costs and the benefits of medical

technologies from a slightly different perspective. The reader is encouraged to read this study

in the context of the overall assessment’s objectives in order to gain a feeling for the potential

role that CEA/CBA can or cannot play in health

care and to better understand the difficulties and

complexities involved in applying CEA/CBA to

specific medical technologies.

The 17 case studies comprising Background

Paper #2 short titles and their authors are:

Artificial Heart: Deborah P. Lubeck and John P.

Bunker

Automated Multichannel Chemistry Analyzers:

Milton C. Weinstein and Laurie A. Pearlman

Bone Marrow Transplants: Stuart O. Schweitzer and C. C. Scalzi

Breast Cancer Surgery: Karen Schachter and

Duncan Neuhauser

Cardiac Radionuclide Imaging: William B.

Stason and Eric Fortess

Cervical Cancer Screening: Bryan R. Luce

Cimetidine and Peptic Ulcer Disease: Harvey V.

Fineberg and Laurie A. Pearlman

Colon Cancer Screening: David M. Eddy

CT Scanning: Judith L. Wagner

Elective Hysterectomy: Carol Korenbrot, Ann

B, Flood, Michael Higgins, Noralou Roos,

and John P. Bunker

End-Stage Renal Disease: Richard A. Rettig

Gastrointestinal Endoscopy: Jonathan A. Showstack and Steven A. Schroeder

Neonatal Intensive Care: Peter Budetti, Peggy

McManus, Nancy Barrand, and Lu Ann

Heinen

Nurse Practitioners: Lauren LeRoy and Sharon

Solkowitz

Orthopedic Joint Prosthetic Implants: Judith D.

Bentkover and Philip G. Drew

Periodontal Disease Interventions: Richard M.

Scheffler and Sheldon Rovin

Selected Respiratory Therapies: Richard M.

Scheffler and Morgan Delaney

These studies will be available for sale by the

Superintendent of Documents, U.S. Government Printing Office, Washington, D. C. 20402.

Call OTA’s Publishing Office (224-8996) for

availability and ordering information.
Case Study #2

The Feasibility of Economic

Evaluation of Diagnostic Procedures:

The Case of CT Scanning

Judith L. Wagner, Ph. D.

Senior Research Associate

The Urban Institute

Washington, D.C.
Contents

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

LIST OF FIGURES

Figure Ale. Page

l. A Model for Economic Evaluation of Diagnostic Procedures . . . . . . . . . . . . . . . 6

2. Disease Detection Rates for Two Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3. Case-Finding Costs for 100 Patients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Case Study #2:

The Feasibility of Economic

Evaluation of Diagnostic Procedures:

The Case of CT Scanning

Judith L. Wagner, Ph. D.

Senior Research Associate

The Urban Institute

Washington, D.C.

INTRODUCTION

In the past, the evaluation of medical procedures—when it has occurred at all—has focused

on the assessment of the procedures’ effectiveness in altering the status of patients’ health or

improving the accuracy of diagnosis. Economic

criteria, such as the immediate or induced costs

of a procedure, have seldom been included in

the evaluation.

Recently, however, as the Nation’s ability to

pay for new advances in medical procedures has

been increasingly taxed, interest in economic

evaluation has been rising, and the application

of economic criteria has gained acceptance. One

manifestation of the increasing legitimacy of

economic evaluation of medical technologies is

the passage of the Health Services Research,

Health Statistics, and Health Care Technology

Act of 1978 (Public Law 95-623), establishing

a National Center for Health Care Technology

which is mandated to consider, among other

things, the cost effectiveness of medical technologies.

In spite of this new interest, the actual application of economic evaluation to medical procedures remains limited. Except for preventive

services such as immunization and asymptomatic screening (11,31,32,33,34,40), little effort has

been devoted to performing this type of evaluation, Although some recent literature represents

a start in this direction (42), the territory of

economic evaluation of medical procedures remains largely uncharted.

Nowhere do questions of the feasibility and

usefulness of economic evaluation arise more

than in the area of evaluating diagnostic procedures. Diagnostic testing has been widely characterized as subject to uncontrolled and explosive growth (16), and the existence of tradeoffs

between the information obtained and the cost

of such testing is often noted (22). But just how

such tradeoffs should be measured is an issue

that has not been pursued at length.

Computed tomographic (CT) scanning was

introduced in 1973. Initially limited to the head,

CT scanning can now be used to detect diseases

in other parts of the body. The use of this diagnostic technology has initiated a controversy of

unprecedented proportions. This controversy

has been cast in terms of the tradeoff between

the benefits and costs of CT scanning. Consequently, a great deal of literature extending

beyond the boundaries of traditional clinical

evaluation has emerged. Numerous articles and

reports have been published that describe the

historical pattern of diffusion of CT scanners in

hospitals and private practices throughout the

United States (5,10,35), the operating costs of

CT scanning units (13,14), patterns of utilization of CT scanners in particular institutions

(2,25,29), and most important here, the cost effectiveness of CT scanning (15,17,23,24,41,46).

Many of these studies have been reported in

clinical journals, demonstrating the intense interest of the medical community not only in the

.3
clinical uses of CT scanning, but also in the impact of this new technology on the practice of

medicine. Interestingly, almost all of the studies

of the economics of CT scanning have been authored by physicians or scientists without major

collaboration with economists.

Judging by the quantity of published research, one would surmise that the state of the

art of economic evaluation of diagnostic procedures—CT scanning, in particular—is well advanced. In fact, the opposite is true. The studies

of the cost effectiveness or economic benefit of

CT scanning reveal major conceptual and methodological weaknesses that not only mitigate

these particular studies’ value, but also cast

doubt on the overall feasibility of conducting

economic evaluations of diagnostic procedures.

The purpose of this case study is to explore

the feasibility of economic evaluation of diagnostic procedures. Several questions are addressed. To what extent can cost-benefit analysis (CBA) and its derivative, cost-effectiveness

analysis (CEA)—the cornerstones of economic

evaluation methodology—be applied to diagnostic procedures? What, if any, methodological and- conceptual obstacles to

analyses are there? How useful

such analyses?

CBA and CEA are methods

eating scarce resources among

performing such

are the results of

to assist in alloalternative uses,

These methods were developed primarily to

evaluate large public-sector investments such as

highways, dams, and airports. When applied to

diagnostic procedures, they are intended to provide information on two related questions: 1)

Under what circumstances should the procedure

be performed? and 2) How much investment in

capacity to perform the procedure is justified?

The answer to the second question rests on thorough study of the first, for only by knowing

when a procedure should be performed can one

assess how much investment in capacity is

justified.

The fundamental argument of this case study

is that the conceptual, methodological, and data

problems inherent in answering the first question (i.e., when should a given diagnostic procedure be used?) often create such inaccurate

and unreliable results that CEA/CBA is generally not worthwhile. In only a few situations (to

be enumerated later) is such economic evaluation likely to be particularly illuminating and

therefore useful to medical decisionmakers.

Such a pessimistic prognosis for the application of CBA and CEA needs to be explained and

supported with evidence. In this case study,

therefore, we begin with an idealized model of

economic evaluation of diagnostic procedures

and then use this model to explore the implications of problems that occur in attempts to apply it. Some typical second best approaches, designed to circumvent these problems, are then

described, with particular emphasis on the use

of these approaches in the evaluation of CT

scanning.

The economic evaluations of CT scanning include three general analytic approaches: 1) good

faith but limited attempts to apply the principles

of economic evaluation, 2) estimates of the impact of CT scanning technology on health care

expenditures, and 3) analyses of the amount and

location of CT scanning capacity required to

meet a predetermined level of demand. Each of

these approaches is assessed with respect to its

limitations as valid economic evaluation and the

particular

questions

usefulness

methodological weaknesses that raise

as to the ultimate feasibility and

of economic evaluation.

CBA AND CEA: THE THEORETICAL IDEAL

The purpose of CBA and CEA is to help decisionmakers determine the best allocation of resources among possible alternative uses. CBA is

used to measure the difference between the value of all benefits resulting from an activity, both

at present and in the future, and the costs. If this

difference, or net social benefit, is positive, the

activity would be said to be worth its costs, and
the decision should be made to allocate scarce

resources (labor, capital, etc. ) to producing the

benefits.

Straightforward as this decisionmaking tool

is, the benefits accruing from an activity, particularly those undertaken in the public sector,

are often difficult even to identify, much less to

value. Because CBA requires enumeration and

valuation of all significant benefits, analysts

have derived the seemingly simpler technique of

CEA. Here, a single measure of effectiveness is

designated, and the ratio of cost to effectiveness

guides resource allocation decisions among

competing alternatives.

Applying these principles to medical procedures, CBA would enumerate and place a value

on all benefits (both positive and negative) derived from performing a procedure on patients

with a specified set of conditions and would

compare those benefits to the cost of performing

the procedure. The resulting net social benefit

would indicate whether the procedure should be

performed under the specified conditions. In

CEA, a measure of procedure effectiveness

would be designated, and the ratio of that single

measure to cost would be the critical item for

resource allocation. Lives saved, life-years

saved, quality-adjusted life-years saved, disability-days saved, and age-adjusted disability days

saved are measures of effectiveness often chosen

in studies of health care programs (43).

Although CEA eliminates the problem of

placing a monetary value on benefits, its scope

is limited. While the scope of CBA is the whole

society, CEA’S is limited to the comparison of

strategies with similar impacts. Since diagnostic

procedures may have widely varying effects—

some procedures may affect mortality while

others may affect only the quality of life—CEA

can only be used to compare a narrow set of

procedures with similar purposes.

The application of CBA and CEA to diagnostic procedures can be described with reference to

figure 1. A set of patients with specific presenting signs and symptoms faces two or more alternative diagnostic pathways, each representing a

specific combination of diagnostic tests. These

alternative pathways may represent different

diagnostic tests or even different testing sequences. For example, pathway A might represent the administration of a radionuclide (RN)

brain scan followed by a CT head scan if the RN

scan is positive or equivocal, while pathway B

might represent a CT scan only. Each of these

two diagnostic pathways carries with it certain

resource costs and has specific diagnostic results. Four possible diagnostic results are shown

for each pathway: true positive, true negative,

false positive, and false negative. Each of these

outcomes implies a course of therapy (with negative findings implying no therapy), each involving its own resource costs. Finally, the therapies result in health outcomes measured by

relevant indicators. In CEA, a single indicator

or an index of indicators is chosen, and the impact of the pathway on this indicator is compared to the total resource costs (both therapeutic and diagnostic) associated with the pathway.

The diagnostic pathway with the lowest ratio of

cost to effectiveness is preferred. In CBA, all

relevant outcomes, both good and bad, would

be measured and their value assessed.

Notice that the purpose of economic evaluation is to decide under what conditions, if any, a

diagnostic procedure should be used. Typically,

a procedure will be of undisputed and obvious

value for some groups of patients. The important question for resource allocation is not

whether the diagnostic procedure or the equipment producing it is justified or whether the

procedure raises or lowers health care costs, but

how the procedures should be used in the practice of medicine. Only by knowing the costs and

effects of treating each group of patients with

the procedure is it possible to know whether

there is too much or too little capacity to perform the procedure in any community.

As seemingly obvious as this point is, it has

not been recognized by the authors of many

studies purporting to be analyses of the cost effectiveness of CT scanning. Too many of these

studies, whose titles include the term “cost effectiveness, ” are simply historical descriptions of

the aggregate impact of CT scanning on health
Figure 1 .—A Model for Economic Evaluation of Diagnostic Procedures

I

I Dx(A) +

I

❑

Dx(A)

Patients

with

specific

signs,

symptoms, or

other

demographic

characteristics

Dx(B)

I

I

I False - A A

care costs in a hospital or community, These proved by increasing, decreasing, or rearrangstudies are plagued with measurement prob- ing CT scanning capacity in the United States. 1-

lems, but even without those problems, the

I Similarly, estimating the impact of a new procedure on medi- studies would offer no guidance on whether the care or public program expenditures, while useful in Federal budgexisting allocation of resources could be im- eting, is not CEA.

PROBLEMS IN APPLYING THE IDEAL MODEL

The ideal model of economic evaluation is however, arise in or are exacerbated by the atstraightforward enough, but in attempts to ap- tempt to apply the model to diagnostic proceply it to diagnostic procedures, serious prob- dures. Problems in the following areas merit a

lems arise. Some of the problems are common closer look:

to all economic evaluations. For example, the

●

question of how one should compare costs and

effects accruing at different times has not yet

been fully resolved. z Several of the problems, ●

identifying homogeneous patient groups

for analysis,

specifying appropriate diagnostic pathways,

measuring diagnostic accuracy,
Q measuring diagnostic and therapeutic costs,

and

● specifying outcomes of the diagnostic/ therapeutic process.

Each of these areas is discussed below.

Identifying Homogeneous

Patient Groups

Application of the ideal model of economic

evaluation requires dividing the universe of

patients into reasonably homogeneous groups

based on their presenting conditions and personal characteristics. The patients in each

group, with a common set of signs and symptoms and perhaps other important characteristics, such as age or sex, are treated uniformly

with respect to the diagnostic pathways applied.

The outcome of a CEA can be extremely sensitive to the criteria (signs and symptoms) used

to identify a patient group. These criteria can be

so general that they lump together patients with

very different problems, or they can be so detailed that they essentially create a unique group

for each individual patient. If, for exampIe, an

analysis of CT head scanning is based on a universe of patients with all types of neurological

complaints, the CEA would average together

the costs and outcomes associated with very different kinds of patients. Patients with recent

strokes would be analyzed together with patients complaining of persistent headaches. At

this extreme, the usefulness of economic analysis in discriminating between the appropriate

and inappropriate uses of CT scanning is questionable. Conversely, if the criteria are so detailed that few patients fall into each group, the

analysis would need to be repeated for each of

the many groups of patients and would be prohibitively expensive. Moreover, such detailed

criteria are impractical for use in diagnostic

decisionmaking because they are likely to include subjective signs that are difficult to incorporate into a diagnostic protocol.

The problem of patient grouping is frequently

noted in connection with clinicaI trials (38). As

the number of groups of patients increases, the

sample size required to show statistically significant effects also increases, possibly exhausting

the supply of patients available for entry into a

clinicaI trial. At the same time, aggregating may

obscure significant benefits available to particular subgroups. But the problem is even more

acute in economic evaluation, which must estimate costs associated with diagnosing and treating patients. Cost data are rarely available on a

disaggregated basis, and allocation of aggregate

costs to individual patient groups involves a loss

of accuracy in cost measurement that can be

quite serious.

Specifying Appropriate

Diagnostic Pathways

The outcome of economic evaluation varies

with the number of diagnostic pathways studied. Omission of significant alternatives can

seriously compromise the results. If a new diagnostic test is developed for a previously undiagnosable disease, then the pathways in contention are easy to specify: the new test versus no

test. But when a new diagnostic procedure adds

information to that available from an existing

arsenal of procedures, then each diagnostic

pathway comprises a procedural sequence in

which the performance of later procedures is

conditioned on the achievement of certain results from earlier procedures. Moreover, two sequences that differ onIy in their rules for continuing the testing process must be considered

separate diagnostic pathways, since each sequence will have yields and costs that are different from those of the other sequence. Hence, the

number of pathways to be considered may be

quite large, and the best pathways may inadvertently be excluded from analysis.

The importance of comparing alternative testing sequences places economic evaluation

squarely at odds with principles for evaluating

the accuracy of diagnostic procedures. Some of

the best clinical studies of diagnostic procedures

have measured the diagnostic accuracy of a

given procedure independent of the outcome of

the tests that precede or follow it. Indeed, a cardinal principle in measuring diagnostic accuracy

of interpretive tests is that the reader or analyst

should not be aware of the outcomes of previous

tests (1). Yet, if knowledge of such outcomes

would improve the accuracy or timeliness of di-
agnosis in a real clinical setting, and hence the

potential outcomes of therapy, then that effect

should be considered in the economic evaluation. Conversely, if knowledge of previous results tends to bias the findings in a particular

direction, and hence leads to more misdiagnosis

and inappropriate and costly treatment, that

should also be considered in the economic evaluation. Unfortunately, the best efficacy studies

specifically eschew the measurement of diagnostic accuracy in such biased but realistic settings.

It is not surprising, then, that estimates of the

relative frequency of outcomes of different diagnostic pathways are often lacking.

Any aspect of diagnosis or treatment that

changes the parameters of the diagnostic pathway—either diagnostic accuracy, therapeutic

effectiveness, or resource costs—requires definition of a new diagostic pathway. Moreover, a

particular diagnostic pathway assumes a particular mix of inpatient and outpatient treatment.

If two treatment protocols are identical except

for the setting of care, then the protocols must

be considered as separate pathways, since their

costs may differ widely.

Technological change, which may improve

the accuracy of a diagnostic test or the effectiveness with which the diagnosed disease can

be treated, also requires the specification of new

pathways. One accepted way of dealing with

uncertainty about or changes in such parameters without constructing additional pathways

is use of “sensitivity analysis. ” Sensitivity

analysis systematically assesses the effect of

parameter changes on the solution to the economic analysis. The question is posed as follows: What percentage change in the diagnostic

accuracy of test X would be needed to induce a

change in the analytic findings? The approach is

powerful when only one or two parameters are

subject to change or uncertainty, but it breaks

down under conditions of general uncertainty

about the appropriate value of many parameters. Unfortunately, most economic evaluations of diagnostic procedures face such conditions,

It should now be clear that the need to specify

and compare a large number of alternative pathways to one another taxes both available data

and analytical resources.

Specifying Diagnostic Accuracy

The model pictured in figure 1 is constructed

as if each alternative diagnostic pathway yields

only two diagnostic findings: positive and negative. Often, a positive finding is associated with

several diseases or different stages of a disease.

Hence, estimates of the total sensitivity (i.e.,

false-positive rates) and specificity (i.e., true

negative rates)3 of a diagnostic pathway are generally inadequate for economic evaluation, Each

type of positive finding must be linked to the

particular therapeutic pathway it evokes, so

that the cost of that pathway can be estimated.

Thus, for any patient group with specific presenting signs and symptoms, accurate economic

evaluation requires data not only on the total

sensitivity of a diagnostic pathway, but also on

the rates of detection of the full spectrum of disease states that the diagnostic pathway might

identify. Such detailed estimates are seldom

available.

Measuring Diagnostic and

Therapeutic Costs

Economic evaluation requires measurement

of the diagnostic and therapeutic costs incurred

when patients embark on a particular pathway.

A CBA should measure the cost of all present

and future health care stemming from the health

problem that precipitated entry into the pathway in the first place. The cost of future health

care made necessary by a false-negative finding,

for example, should be included. In CEA, the

criterion for inclusion of health care costs

depends on the chosen indicator of effectiveness. The use of disability days as the effectiveness indicator would obviate the need for

measurement of health care costs associated

with treament of disability itself. Similarly, a

cost-effectiveness study using the 5-year survival rate as the outcome measure of interest

should count all costs up to the day of outcome
measurement. The cost of a false negative

would be relevant only if it induced the delivery

of additional medical care prior to the point at

which effectiveness is measured. Thus, with

CEA, there is a natural stopping point to cost

measurement that is dictated by the time horizon implicit in the effectiveness measure.

Most of the cost-effectiveness studies of CT

scanning have treated the measurement of diagnostic and therapeutic costs cavalierly, ignoring

the difficult conceptual and measurement problems and taking ad hoc approaches without

acknowledging their limitations. The authors of

these studies either do not appreciate the difficulties inherent in cost measurement, or they

recognize and are overwhelmed by the problems, and therefore ignore them. Nevertheless,

the problems are great and may defy solution.

Cost is a measure of value—in this case, the

value of productive resources (labor, equipment, supplies, etc. ) that must be forgone to

other uses when a particular diagnostic/ therapeutic pathway is chosen for the patient group

under study. The appropriate economic concept

is that of incremental or avoidable costs, those

incurred as a direct result of committing a given

increment of patients to a diagnostic therapeutic pathway or that can be avoided if the same

patients do not take the pathway. Avoidable

costs are treated as “variable, ” while unavoidable costs are assumed to be “fixed .“

The definition of what is avoidable or variable depends on the time perspective of the decision and the size of the increment. Generally,

the longer the time perspective and the larger

the increment, the more costs are avoidable.

Suppose, for example, that the decision is

whether to perform CT scanning on patients

with suspected stroke in a hospital with a scanner currently used at .50-percent capacity for

other types of patients. In the very short run,

the only truly avoidable costs are the materials,

supplies, and extra wear and tear on equipment

caused by serving the stroke patients. Most

labor costs may be invariant with serving the

new patients, because technicians must be paid

for full workshifts whether they are busy or not.

Consequently, the appropriate measure of cost

would exclude these fixed operating costs. Also,

because the equipment exists to serve other
assume that the service will be provided using

the most efficient technology currently available

to produce the expected diagnostic and therapeutic results. Therefore, the appropriate measure of capacity cost is not the original cost of the

equipment and plant, but the current cost of replacing it.

Depending on the rates of technological

change and general inflation, replacement cost

may be quite different from the original capital

cost of the equipment in any institution. But

whereas the original cost of existing equipment

is readily available from accounting records,

estimation of current costs of replacing capacity

is fraught with difficulty. Product improvements in CT scanning, for example, have increased equipment costs and improved the flexibility of CT scanning and the quality of resulting images. If these quality improvements

are unnecessary to produce the level of diagnostic accuracy specified for the pathway, the

measured costs of CT scanning should not be

based on these new models. Whole-body scanners, for example, are increasingly used in

studies of the head. Since the purchase price and

operating costs of head scanners are less than

those of body scanners, it would be incorrect to

allocate to head scanning patients a portion of

the extra costs associated with a body scanner.

But how will one estimate the cost of replacing

head scanners when they are no longer manufactured? The measurement of original capital

cost, while inappropriate, is at least precise.

Whichever concept of capacity cost is used—

original capital cost or current replacement

cost—it must be allocated between the patient

group under study and all other patient groups.

A common rule of thumb is to allocate these

costs in proportion to the share of total volume

accounted for by the patient group under study.

Underlying this rule is the implicit assumption

that the costs of any excess capacity should be

borne equally by each patient group using the

service. In fact, this assumption may bias the

analysis against certain patient groups. Headache patients, for example, may be able to wait

weeks for a CT scan without negative effects on

outcome, but may not have to do so because excess CT scanning capacity has been made available to serve trauma patients on an emergency

basis. CT scanning costs allocated to the headache patients should rightly be calculated as if

the equipment were used to full capacity, while

the trauma patient group should bear the entire

cost of the excess capacity. Such differential cost

allocation has not been attempted in the CT

scanning papers reviewed by this author.

Several economic studies of CT scanning

have used hospital procedure charges to estimate the cost associated with the use of CT

scanning. This approach is unacceptable, because procedure charges bear no relation to the

incremental costs of treating particular patient

groups. Even if each charge were set to equate

the revenues from a procedure with its associated costs, which is seldom the case, the unit

charge so constructed would be unlikely to reflect optimal utilization of capacity or appropriate allocation of indirect costs and overhead. It

should be noted, however, that in most American hospitals charges for specific procedures

are often completely unrelated to their costs.

Profits from one procedure typically subsidize

losses from another. Indeed, the hallmark of a

good hospital administrator is that charges have

been manipulated to maximize revenues. Consequently, the use of procedure charges as a proxy

for incremental cost seriously compromises an

analysis. The extensive reliance on procedure

charges in economic studies of CT scanning is

curious in light of the relatively detailed information available on the operating costs of

CT scanning units (13,14). Although the obstacles are formidable, improved estimates of the

cost of some procedures are within reach.

Another common practice in economic studies is to use physicians’ fees as estimates of the

value of physicians’ inputs into the diagnostic

and therapeutic processes. Although such fees

represent actual payments, they are not accurate reflections of the value of physician inputs.

Third-party payer policies have created anomalies in fee structures, so that some diagnostic

procedures offer higher returns on the physician’s investment of time and skill than others.

In particular, fees for new procedures are often

set higher than those for established procedures.

A better approach to valuing physicians’ serv-
ices would be to use carefully constructed relative value scales based on the time and skill requirements of individual procedures. Construction of good relative value scales is expensive,

however, and is unlikely to be undertaken as

part of the evaluation of medical procedures.

Some recent tentative evidence suggests that the

California Relative Value Scale may be reasonably well correlated with the level of difficulty

of surgical procedures i(19), but more investigation of this question is needed before it

would be possible to recommend reliance on

such a system.

To summarize, the obstacles to accurate

measurement of the resource costs associated

with a diagnostic/therapeutic pathway are

great, but substantial improvement in cost estimates would be possible if researchers would attempt to approximate long-run incremental

cost. At present, most cost analyses are grossly

inaccurate, so much so, in fact, that wrong inferences have been drawn about the usefulness

of CT scanning.

Specifying Outcomes of the

Diagnostic/Therapeutic Process

way. CEA assumes that a single measure or index can capture the essential nature of the outcomes. The former type of analysis challenges

our present capacity to objectively quantify all

types of effects, both good and bad, while the

latter runs the risk of oversimplifying the decisionmaking problem.

The problem of specifying and measuring

outcomes is common to all evaluation and is

therefore beyond the scope of this case study.

One type of outcome, however, is unique to

economic evaluation of diagnostic procedures:

the benefits derived from negative findings.

Negative findings are certainly valued by patients, for such findings can have dramatic impacts on their peace of mind. The reassurance

value of negative findings is generally ignored in

CEA, because satisfactory methods for estimating changes both in patient satisfaction and in

the value that patients place on those changes

are lacking at present (42). Consequently, CEA

is likely to be biased against diagnostic/therapeutic pathways offering high specificity (i. e.,

high true-negative rates) relative to those pathways with high sensitivity (i. e., high true-positive rates).

CBA requires the specification of all significant kinds of outcomes of each diagnostic pathECONOMIC EVALUATIONS OF CT SCANNING

Only two studies of CT scanning with which

the author is familiar are good-faith efforts to

implement the ideal model of economic evaluation in a comprehensive way. One such study,

by Knaus and Wagner (24), is successful; a second study, by Baker and Way (4), represents a

failure to understand and carry out the principles of economic evaluation.

Knaus and Wagner (24) analyzed the effectiveness and costs of two alternative diagnostic

pathways for patients presenting with sudden

severe headaches but no focal signs; such patients are generally viewed with skepticism as

candidates for CT scanning. The analysts in this

study compared the outcomes of two diagnostic

pathways: 1) performing a CT scan on all such

patients referred; and 2) performing a CT scan

only on patients presenting additional neurological signs (i. e., depriving the patient of a C T

scan unless and until his or her condition deteriorates). Only one possible positive finding was

considered: a ruptured intracranial aneurysm

resulting in subarachnoid hemorrhage. Because

early diagnosis of this disease significantly improves the prognosis of surgical intervention,

the authors concluded that the cost of each life

saved by early detection is about $511,000, or

$23,000 per life-year saved.

Knaus and Wagner observed that the analysis

depended on the attainment of a diagnostic yield

at least as high as that obtained at a large

teaching hospital: about one aneurysm per 125
toms, and other factors that might improve the

cost-effectiveness ratio for headache patients.

It is instructive to ask why Knaus’ study met

so many criteria for good economic evaluation.

A critical determinant appears to be the availability of empirical estimates of the outcome of

surgery, medical therapy, and no intervention

in patients in various symptomatic stages of

cerebral aneurysms. Without such probability

estimates, the study would have foundered. The

availability of data from clinical trials made

possible the decision analytic approach for an

admittedly narrow set of diagnostic findings for

patients with headaches.

The study by Baker and Way (4) of CT scanning in the abdomen is a manifest demonstration of what can go wrong when clinicians attempt to apply the principles of economic evaluation. The authors randomly selected the case

histories of approximately 200 hospitalized patients who had received body scans in 1976 or

1977. Each patient was classified according to

the organ system studied (e. g., biliary, liver,

etc. ). Baker and Way then constructed an arbitrary scale of effectiveness, where a value of 1

was assigned to a patient for whom the CT scan

had a lifesaving effect, a value of 18 was assigned to a patient if the CT scan actually precipitated a death, and a value of 10 indicated no

net value. The costs of CT scans and other related diagnostic tests were presumably estimated from the procedure charges, although the

authors did not specify their methods. A ratio of

cost to effectiveness was constructed for each

organ system, with the effectiveness scale value

as the denominator. The authors concluded that

“the differences between the categories [organ

systems] are insignificant, and the average result

suggests that, in general, the value of the scans

to medical care in relation to the cost of scans

was low, ”

Apart from the obvious question of the reliability of subjective assessments of effectiveness

and the cost measurement techniques, two criticisms of Baker and Way’s study are relevant

here. First, it is utterly meaningless to compare

the cost-effectiveness ratio of CT scanning

across patient groups. The appropriate comparison is between CT scanning and other diagnos-
of CT scanning on the cost of diagnosis is a

logical starting point but is generally an insufficient approach to economic evaluation.

The fundamental weakness of the studies remaining to be discussed, therefore, is not their

failure to consider therapies and outcomes.

Rather, it is their choice of diagnostic pathways.

Each one of these studies compares actual patterns of diagnosis with and without CT scanning. Most are pre-CT/post-CT comparisons of

diagnostic costs in an institution or country.

Given the system of medical practice in force today, however, there is no uniformity from

physician to physician in the use of diagnostic

procedures on similar patients. Since one physician may order a battery of tests simultaneously, while another may move cautiously from

one to another, each diagnostic pathway comprises many approaches to diagnosis. Thus, to

show that the introduction of CT scanning has

historically raised or reduced diagnostic costs is

to show that and nothing more. That finding

holds no implications for resource allocation,

because the pathways are too amorphous to determine how CT scanning should fit into the

diagnostic process.

Homogeneous patient groups, however, are

likely to undergo diagnostic strategies that are

reasonably uniform. Therefore, empirical studies of the impact of CT scanning on diagnostic

cost are more likely to be useful as resource

allocation tools when these studies are concentrated on a narrow set of presenting signs and

symptoms.

Three studies of specific patient groups with

relatively unambiguous and homogeneous signs

and symptoms were reported by Larson and colleagues at the University of Washington. They

analyzed the impact of CT scanning on the utilization and cost of diagnostic services for patients with presenting conditions suggestive of

brain tumors (26), cerebrovascular disease (27),

and hydrocephalus (30). Admission forms were

abstracted for all inpatients admitted to the university-affiliated hospital with signs and symptoms suggesting one of the three diseases during

the year prior to and 2 years after the introduction of a CT head scanner. Patterns of service
delivery and medical care in the pre-CT and

post-CT patient groups were compared, including the utilization of and charges for neurodiagnostic procedures; the length of hospital stay;

the time required to reach final diagnosis; the

level of detail of the diagnosis; and the type of

therapy employed, CT scanning had a major

impact on the configuration of diagnostic studies used for each of the three types of patients,

but the relative benefits resulting from these

changes varied among the three.

In the study of patients with suspected brain

tumors (26), CT scanning significantly reduced

the use of RN brain scanning (from 85 to 38 percent of patients), angiography (from 65 to 50

percent of patients), and pneumoencephalography (PEG) (from 15 to 4 percent of patients).

The length of hospitalization, speed of diagnostic workup, and timing and type of therapy for

these patients, though, did not change significantly. The overall detectability of brain tumors

increased markedly after CT scanning became

available. Total per-patient hospital charges for

neurodiagnostic procedures did not change significantly. The authors therefore concluded that

CT scanning as it was integrated into the neurodiagnostic process at the study hospital was cost

effective, because it permitted reductions in the

use of invasive and somewhat risky procedures

but did not raise hospital charges or otherwise

affect prognosis,

In the case of patients with suspected cerebrovascular disease (27), the introduction of CT

scanning significantly reduced the number of

lumbar punctures and RN brain scans, but the

total per-capita hospital charge for neurodiagnostic procedures increased about 33 percent.

The length of hospitalization and speed of diagnostic workup did not change, and although the

level of detail of the discharge diagnosis was

greatly increased, such detail was unnecessary

to plan therapy. In the judgment of the authors,

CT scan results were not essential to establishing a diagnosis in these patients but played a

confirming role. In patients with suspected cerebrovascular disease, therefore, CT scanning was

found to be of questionable value.

The introduction of CT scanning in the diagnostic workup of children with suspected hydrocephalus (30) led to major reductions in RN

brain scans and PEGs. CT scanning also reduced the total charges for neurodiagnostic procedures. Speed of diagnosis, length of hospitalization, and kind of therapy did not change.

Thus, CT scanning was found to be particularly

appropriate for such patients.

All three Larson studies depended on the use

of procedure charges as a proxy for the economic costs involved in performing the neurodiagnostic procedures. The resulting distortion of

estimated cost impacts is instructive. In the

study of suspected hydrocephalus (30), total

estimated diagnostic charges dropped precipitously with the advent of CT scanning, largely

because of the high charge for a PEG examination ($492 at the study hospital). The high

charge at least partly reflects that procedure’s

relative rarity and the existence of excess capacity. The long-run incremental cost of a PEG examination is probably much lower. Were such

examinations not painful and risky, and

therefore to be avoided wherever possible, a

substantial change in the estimated cost of this

procedure could have reversed the findings

regarding the benefit of CT scanning in patients

with suspected hydrocephalus. In fact, a PEG

procedure charge of $2OO would have resulted in

no difference between pre-CT and post-CT total

per-capita charge for neurodiagnostic procedures, and the analysis would have rested on the

implications of substituting CT for PEG examinations for patient morbidity and risk.

The investigation did not question the appropriateness of inpatient versus outpatient workups. Some neurodiagnostic studies can probably

be performed on an outpatient basis. By accepting the existing settings of care as the basis for

comparison, the studies may have overlooked

particularly efficient alternatives.

The three Larson studies do show, however,

that with good hospital data systems and effort

and persistence on the part of the investigators,

it is possible to identify patients with reasonably

homogeneous presenting signs and symptoms.

The ability to obtain medical records categorized by admission problem rather than discharge diagnosis is critical for estimating the

outcomes of different diagnostic pathways, and
although the Larson studies do not specify pathways with sufficient clarity, their patient identification methodology would support such

analyses.

A more recent study by the same authors investigated the impact of outpatient CT scans on

headache patients (28). Three cohorts of pa -

tients were selected for study: one cohort containing patients seen before installation of a CT

scanner, a second cohort containing patients

seen shortly after installation, and a third

cohort containing patients seen 1 year after installation of the CT scanner, The patients in

each of these cohorts were selected from a review of electroencephalogram (EEG) records,

where headache was the reason for the EEG.

Thus, the patient sample excluded those who

may have had CT scans without EEGs. The cost

of diagnostic tests was computed using procedure charges; the diagnostic testing charges per

patient increased 16 percent after the introduction of CT scanning. The neuroradiologic diagnostic yield of patients with normal neurological workups was low, and, in the case of CT,

was negligible. This finding led the authors to

suggest that abnormal findings on neurological

examination should be a screening criterion for

performing CT scans and other neuroradiological procedures for patients with headaches.

The fact that diagnostic charges increased

after the introduction of CT scanning says very

little about the appropriate allocation of resources; the more powerful evidence presented

by the authors is the apparent absence of real

clinical benefit, even with respect to diagnostic

capability, deriving from CT scanning—and,

indeed, from most neurodiagnostic procedures.

If radiological and other technology-bound procedures are to be performed on patients with no

focal findings, then this study has not adequately demonstrated that CT scanning cannot lower

the cost of diagnosis if it is appropriately substituted for other tests. A clear cost-reducing option, however, appears to ensue from stopping

the diagnostic workup of the patient when the

clinical examination reveals no abnormalities.

In light of the importance of such a conclusion,

it is unfortunate that the sample of patients in

this study was chosen by excluding those who

may have had CT scans without EEGs. More

needs to be done to refine the study of diagnostic procedures for patients with headaches.

Ambrose and his colleagues (2) in Britain

studied the impact of CT scanning on the diagnosis of head injuries, but they did not attempt

to attach cost estimates to the observed changes

in utilization of diagnostic procedures and exploratory surgery. The introduction of CT scanning at the study hospital dramatically reduced

the use of arteriography, PEG, and exploratory

craniotomy without changing the mortality

rate; however, once CT examinations were

available, they were ordered so frequently that

net costs may have increased. Nevertheless, the

realized effect is not the relevant question for resource allocation. CT scanning can undoubtedly reduce the cost of diagnosis for some patients

by substantially reducing the high cost, morbidity, and mortality associated with the diagnostic

procedures it replaces. The real question

though, is how to achieve the best results. Like

previous studies, the Ambrose study failed to

explore how CT scanning might optimally be

integrated into diagnostic and management protocols.

In a recent study of CT scanning in acute head

trauma at a U.S. teaching hospital, Zimmerman, et al. (47) noted a progressive decrease in

utilization of skull X-rays following the introduction of CT scanning, but they also observed

that “a certain proportion of the present skull

radiographic examinations are done because of

the referring physician’s adherence to traditional ways of evaluating trauma patients. ” A

critical question that economic analysis might

address is exactly when, if at all, skull X-rays

should be employed in the evaluation of patients

with head trauma.

Bahr and Hodges (3) investigated the impact

of CT scanning on the total cost of hospital care

(measured by billed charges) for neurological

and neurosurgical inpatients at a university

hospital. The total average hospital bill for these

patients decreased about 10 percent, after adjusting for inflation, but the decrease was not

statistically significant. The study population

was disaggregated into three diagnostic groups:
1) patients with extracerebral collections, 2) patients with proved intracranial tumors, and 3)

patients with vascular disease. The total inflation-adjusted hospital bill for patients in the first

two categories decreased by 23 and 35 percent

respectively, while that of the third group increased by 30 percent.

It is difficult to interpret such findings because

of the possibility that the patient mix changed

over the period, especially if the availability of

CT scanning enabled testing of some as outpatients. In addition, the study population is ambiguous, consisting of patients with all types of

neurological complaints. The disaggregated cost

comparisons are based on final diagnosis, not

on the patients’ presenting signs and symptoms.

Consequently, the cost or savings by diagnosis

are irrelevant to the use of CT scanning under

specified presenting conditions. The difficulty of

interpreting cost impacts based on procedure

charges has already been noted.

In one of the earliest economic impact studies,

Wortzman, Holgate, and Morgan (45) reviewed

the records of over 200 inpatients who were

given CT head scans at a Canadian hospital. In

each case, the authors judged whether an angiogram or PEG had been prevented or prompted

by the CT scan. The hospital costs (measured by

charges) saved or induced by use of the CT scan

were calculated on the assumption that the hospital stay for an angiogram is 1 day and that for

a PEG examination, 3 days. Net savings resulting from the availability of CT scanning were

estimated at $300 per patient. In a second part

of the study, the authors reviewed all outpatient

CT scans at the hospital and, with the help of a

survey of referring physicians, estimated that 58

percent of the outpatients would have been

hospitalized had the CT scan been unavailable.

Assuming that these patients would have stayed

the average length of stay in neurology (14

days), the authors estimated that CT scanning

saved $1,490 per outpatient scan.

The vulnerability of the savings estimate to

changes in assumptions is obvious and needs no

further comment. Nor does the use of charges to

reflect cost. The method for determining the

degree of substitution of CT for other procedures is judgmental and therefore subject to

investigator and respondent bias. Most important, however, the aggregated nature of the

patient group under study mitigates the value of

the findings. It is certain that savings, however

measured, could be further increased by more

detailed analysis of presenting signs and

symptoms.

In a recent reappraisal of their early study,

Wortzman and Holgate (45) revealed discrepancies between their estimates of potential savings

and realized effects in subsequent years. Hospital stays and admissions did not decline as

predicted in the study. Although the number of

angiograms and PEGs did decline, the cost of

these procedures, as measured by charges, rose,

thus wiping out much of the estimated savings.

The procedure charge increases were necessitated by decreases in utilization without concomitant capacity reduction. But the authors

wrongly assumed that estimated cost savings

should be altered to reflect these changes. The

principle of long-run incremental cost requires

the assumption that capacity will ultimately adjust to reduced demand.

A study by Thomson (41) at Frenchay Hospital in Great Britain traces changes in the use of

particular procedures and services in the 2 years

following the installation of a CT scanner.

There, as in most other institutions, the number

of arteriograms, RN brain scans, and PEGs declined after the introduction of CT scanning; the

length of stay of neurosurgical inpatients declined as well, and admissions were avoided.

The cost of operating a CT scanner at the study

hospital was meticulously calculated using a

modified concept of capacity replacement cost

and assuming operation at full capacity. The

economic cost of other neurodiagnostic procedures made unnecessary by CT scanning were

not measured so carefully, but were assigned

relatively low values in order to bias the study

against CT scanning. Hospital stay savings were

estimated at the average daily cost of a National

Health Service hospital bed. CT generated net

savings for the hospital, and, by inference, for

the National Health Service, although the savings were minimal,

Since Thompson made no attempt to follow a

specific patient group but merely reported on in-
stitutionwide changes in utilization of services,

it is impossible to know the kinds of patients for

whom CT scanning is cost effective. Other authors have documented such institutionwide

changes (26). This kind of information, even

when linked to good cost estimates, ignores the

central problem for medical decisionmaking and

health facilities planning: To what uses should

CT scanning capability be applied?

Two studies have attempted to estimate the

impact of CT scanning on the national cost of

diagnosis in the United States. Willems, et al.

(44) estimated net changes in total U.S. expenditures for diagnosis due to the introduction of CT

scanning. The performance of a CT scan generated a technical charge, professional fee, and

associated hospital room and board charges, all

of which totaled an amount between $295 million and $428 million in 1976. Savings resulting

from reduction in PEGs, RN brain scans, and

arteriograms were deducted from this total,

leaving a net estimate of $181 million to $390

million in increased expenditures due to CT

scanning in 1976.

Like analysts in most of the other studies

previously described, WilIems used procedure

charges to estimate the cost of most procedures.

Although charges are an inadequate surrogate

for resource costs, they do reflect real transfers

from payers to providers of health care; in that

sense, the total expenditure burden imposed by

CT scanning is of interest. It is unknown, however, whether the expenditures were used to

subsidize the provision of other services that

hospitals provide at a loss or, perhaps, whether

the expenditures were used to inflate physician

or other professional incomes.

A study by researchers at Arthur D. Little,

Inc. (17) represents the only effort to date to

estimate the national impact of both head and

body scanning on the cost of diagnosis. The national impact on the utilization of competing

diagnostic procedures and exploratory surgery

in 1977 and 1980 was estimated by a variety of

sources, including consultations with experts in

the field. The technical cost of providing each

type of procedure was estimated assuming

equipment at full utilization and current costs of

replacement of equipment. (Except for exploratory surgery, charges were not used to estimate

procedure costs. ) To arrive at a unit cost for

each of the various diagnostic procedures, the

national average hospital

charge was applied to the

hospitalization required by

cedure.

It was predicted in the A.

room and board

additional days of

each type of proD. Little study that

the availability of CT head scanning would raise

diagnostic costs by about $29 million in 1977

and $31 million in 1980. These low estimates of

net impact contrast with the results of the

Willems study but can be explained by different

assumptions about the effect of CT scanning on

exploratory surgery and skull X-rays.

The predicted impact of CT body scanning is

subject to greater speculation since, at the time

of the study, body scanning had only begun to

find its way into the diagnostic process. The

authors predicted that in 1980, as a result of the

use of CT scanning, the estimated cost of diagnosing abdominal and mediastinal disorders

will have increased by $152 million. This estimate is based on the assumption that the availability of CT body scanning will reduce the number of exploratory surgeries by 50 percent and

that no other diagnostic procedures will be affected by CT scanning.

Although the ultimate significance of these

estimates for decisionmaking is most certainly

low, the A. D. Little study represents the best effort to date at estimating the resources costs

associated with performing diagnostic procedures. Capacity costs were based on current

equipment replacement prices and full capacity.

However, indirect costs (administrative expenses in departments outside of the CT scanning unit) were arbitrarily assigned a value of 50

percent of direct costs, a level that probably

bears no relation to the actual incremental costs

associated with the existence of the CT scanning

capacity. The technical cost of surgery and

routine costs of hospital care were based on

charges rather than costs. With the high cost of

hospital construction today, hospital room and

board charges reflecting original capital costs

probably understate replacement costs, In spite

of these shortcomings, the method for estimat-
ing procedure costs is a model for economic

evaluation of CT scanning.

Studies of the Cost of Case Finding

A commonly advocated approach to economic evaluation is the calculation of the cost of case

finding, which is variously defined as “the cost

per positive finding, “ “the cost per correct finding, ” or “the cost per true positive” (6). The cost

of case finding is a concept borrowed from the

economic evaluation of screening procedures. It

is particularly useful in determining which of

two or more mutually exclusive screening procedures is more cost effective in detecting a particular disease in an asymptomatic population.

A procedure (or sequence of procedures) that

costs the least per true positive finding is preferred to all others.

Unfortunately, the concept does not travel

well. When applied to analysis of CT scanning,

which is used on symptomatic patients in conjunction with other tests, the cost of case finding

is meaningless. The results of studies using the

concept illustrate its limitations.

Carrera, et al. (9) calculated the cost per abnormal CT scan for three types of cases:

1) headaches without neurological findings,

2) headaches with neurological findings, and

3) suspected temporal lobe epilepsy. Because of

differences in the diagnostic yield of CT scanning among these three types of cases, the cost

per abnormal CT scan examination was $4,000;

$1,300; and $2,000 respectively. 7 In a separate

study, Knaus and Davis (23) calculated the casefinding cost of CT scanning for 25 leading indications for CT head scanning. The case-finding

cost at the study hospital ranged from $411 for

patients in coma to $3,500 for patients with

headache. Larson, et al. (28) estimated that the

cost of case finding for headache patients

without abnormal neurological findings was

$11,901. The authors of all three papers questioned the cost effectiveness of CT scanning in

patients with headaches without neurological

—

7 A CT procedure charge of $240 was used as a surrogate for

cost .

findings. Any conclusion about cost effectiveness is premature, however, for the cost of case

finding can only be compared to the benefits of

finding abnormalities in the patient group.

Evens and Jest (12) have used the concept of

case-finding cost to compare CT head scanning

and RN brain scanning on patients with neurological complaints. Using estimates of sensivitity and specificity of CT and RN scanning, the

authors calculated the cost per correct finding:

CT scanning cost $141 per correct finding; RN

scanning cost $73 per correct finding. Although,

on the surface, this finding would imply that RN

scanning is the procedure of choice, Evens and

Jest observed that CT is a more sensitive test.

Hence, in a large number of cases, the combination of the two tests would be ordered, with a

joint cost per correct finding of $214. If an initial

negative RN scan generates a confirmatory CT

examination in at least 60 percent of neurological cases, then an initial CT scan costs less per

correct finding than an initial RN examination.

Notice that Evens and Jest used a variant of

case-finding cost. Whereas Carrera (9) and

Knaus and Davis (23) calculated the cost per abnormal or positive finding, Evens and Jest calculated the cost per correct finding. A third approach, the cost per true positive finding, has

also been used. g The relative merit of these alternative measures of case-finding cost depends on

the errors in diagnosis and implications for therapy of each alternative. In fact, the most appropriate measure can be selected properly only

if the costs and benefits of pursuing diagnostic

pathways to their ultimate outcomes are

known.

This point can be illustrated algebraically. Let

TP, FP, TN, and FN denote the proportion of

true positives, false positives, true negatives,

and false negatives expected from a given diagnostic pathway in a specified population of patients. Let C be the total cost of performing

‘See, e.g., B. J. McNeil and S J. Adelstein, “Measures of Clinical Efficacy. The Value of Case Finding in Hypertensive Renovascular Disease, ” 1975 (32).
diagnostic procedures on these patients. The

three case-finding cost options are:

Cost per true positive = c

TP (100)

Cost per abnormal finding = c

(Tp + Fp) (1OO)

Cost per correct finding = c

(TP + TN) (100)

Suppose that two diagnostic procedures, A and

B, have the disease detection rates shown in

figure 2 when performed on a patient group

with a 0.50 disease prevalence rate. Assume also

Figure 2.—Disease Detection Rates for Two Tests

+

Tes t

result s

—

Diagnosti c

test

A

B

that both procedures cost the same to administer, $10 per patient. Figure 3 gives the casefinding costs of each procedure for each 100 patients as measured by the three case-finding options. In figure 3, diagnostic procedure B would

be preferred if the option “cost per correct finding” were used, and diagnostic procedure A

would be preferred under the options “cost per

true positive” and “cost per abnormal finding. ”

Suppose, however, that the implications of a

false negative are major (e.g., a fatal but completely treatable disease goes undetected), while

those of a false positive are minor (e. g., inexpensive additional tests are required). Clearly,

diagnostic procedure A would be preferred, and

use of the cost per correct finding would be unacceptable. Similarly, if the implications of a

false positive are major (e.g., application of inappropriate, expensive, and risky therapy),

while those of a false negative are also great,

then the use of cost per correct finding is superior. Thus, to accurately determine which

measure of case-finding cost is preferred, it is

necessary to know something about the subsequent benefits and costs resulting from a particular diagnostic procedure. The more that is

known, the more accurate can be the weights

put on the different kinds of errors. Thus, casefinding costs do not bypass the need for information concerning the full implications of

embarking on a diagnostic pathway.

Figure 3.—Case-Finding Costs for 100 Patients

Cost per Cost per Cost per

true positiv e correct finding abnormal finding

(C/TP) (C/TP + T N ) (C/TP + F? )
least 500 angiograms and 200 PEGs could be

eliminated each year, the CT scanner would

save money.

The cost analysis rightly assumed that all

radiographic equipment is operated at capacity

both before and after the introduction of CT;

this avoids ascribing to CT scanning savings

that would otherwise be obtainable from consolidating radiographic services in larger hospitals, In fact, the whole question of CT scanner

location appears to be inextricably linked to the

optimal distribution of neuroradiologic capacity

as a whole, Costs could be reduced by regionalizing such capacity, but these savings would

have to be compared to the added cost of transportation to regional facilities. For emergency

patients, the cost savings would also have to be

weighed against reduced access to care. Thus,

the Swedish study, though excellent in its costing methodology, is still limited as a planning

approach, since it analyzes the effect of introducing CT scanning to an already existing system in which the present allocation of radiological resources may very well be inappropriate.

Bartlett, et al. (7) reported on a comparison of

alternative options for the location of CT scanning capacity in a particular region of England.

The study assumed that the demand for CT

scans would be 3.5 per thousand residents per

year. Alternative configurations for meeting

(and exceeding) that demand were then compared. The critical question was whether CT

scanning capacity should be installed in all

hospitals with neurosurgery, neurology, and accident centers. The not so startling conclusion

was that more scanners would cost more money

and the added cost of installing a CT scanner in

every accident center (far exceeding the required

scanner-to-population ratio) must be compared

to the lifesaving potential of the scanner in head

trauma. We are thus led full circle back to the

basic question of the conditions for which CT

scanning is economically justified.

An American study by Greenwald, et al. (18)

is an application of a mathematical optimization

technique to the question of scanner capacity

and location. The authors formulated the problem thus: How does one choose the number and

location of CT scanners such that the sum of CT
operating and patient travel costs is minimized?

The solution must meet two constraints: 1) that

the region’s demand for CT scans is met, and 2)

that each scanner is assigned a patient load not

exceeding its capacity. Other considerations,

such as the desirability of placing CT scanning

capability in centers of radiological excellence,

were not included in the model, but easily could

be if the potential locations of CT scanners were

limited to those sites meeting certain criteria.

The authors showed that when patient travel

costs are taken into account, more scanners can

actually reduce the total cost of meeting a given

level of demand. As demand for CT services increases or decreases, however, the optimal number of facilities also changes. Hence, the model

begs the question of the optimal number of scanners by assuming that the answer to the prior

question—under what conditions is CT scanning worth its costs — has a 1 read y been answered.

CONCLUSIONS

Taken together, the literature on the economics of CT scanning reveals a plethora of

studies that either ask the wrong questions, or,

in asking the right ones, must make heroic and

unsupportable assumptions. Few studies have

addressed the fundamental question of economic evalution: For what kinds of patients is CT

scanning worth its costs?

The energy devoted to documenting historical

cost savings resulting from CT scanning is particularly misplaced. This literature offers no real

insight into the question facing health care decisionmakers today. Studies of the cost of case

finding, which appear on the surface to be more

useful in guiding resource allocation decisions,

offer such a partial view of costs and effects of

any diagnostic pathway that their results are

more obfuscating than enlightening. Finally,

those methods designed to assist directly in the

big resource allocation decision—the placement

of CT scanning capacity—must ignore the unanswered question of conditions of cost-effective use.

The mathematical programing approach to

facility location has a rich history,’” especially

with health care facilities. It is worth noting that

if CT scanning is considered a necessary diagnostic tool for head trauma and other emergency situations, then the formulation suggested by

Greenwald, et al. (18) may not be appropriate.

Instead, a preferred formulation might be to

minimize the number of scanners required to insure that all points in the region lie within some

maximum time or distance criterion.11 The resulting configuration of CT scanning facilities

would very likely be quite different from that

obtained in the Greenwald, et al, study.

Although the methodological and conceptual

obstacles to conducting useful economic evaluations are great, the literature shows that some

can be overcome. Unfortunately, the obstacles

have been dealt with in pieces—each study has

offered a solution to a particular problem, but

no study is completely satisfactory in all areas.

In particular, the proper estimation of resource

costs is an area that appears to be a good candidate for major improvements with relatively littIe effort, although some conceptual problems

will continue to exist. Certainly the use of

charges as a surrogate for cost can be eliminated. The identification of patients by specific

signs and symptoms also appears to be feasible,

although the cost of research based on specified

patient groups will depend on the existence of

good hospital data collection systems. The

problem of identifying patients by signs and

symptoms will be intensified with the use of CT

body scanning because of the ambiguity of signs

and symptoms related to diseases of the abdomen.
When the data needed to perform valid economic evaluations are considered, it is not surprising that so few appropriately constructed

studies exist. The cost of analyzing the signs and

symptoms of the many different kinds of patients who are reasonable candidates for CT

scanning would be great in itself; to imagine

conducting such analysis for all types of patients

and diagnostic procedures is unrealistic in the

extreme. At best, a few controversial indications for expensive diagnostic procedures might

be the subjects of economic evaluation. Research to date on CT scanning suggests that patients with headaches and suspected cerebrovascular disease would be excellent candidates for

additional analysis with more appropriate

methods of economic evaluation.

If one cannot expect a comprehensive analysis

of the appropriate conditions of use of a diagnostic procedure, how can those responsible for

decisions about the level and location of diagnostic capacity make rational choices? The task

is impossible, as most health planning agency

personnel have suspected all along. Instead of

attempting to find analytic approaches to support what is essentially a political decision,

health planning agencies might do better to set

fairly arbitrary regional capacity ceilings and

use economic evaluation to explore the cost and

access implications of alternative locational

choices.

Even better, the incentives might be improved

for those who are most knowledgeable about

the diagnostic alternatives: the providers themselves. If we had greater confidence that their incentives were consistent with the goals of public

policy, the limitations of formal evaluation

would be far less troublesome.

1.

2.

3.

4.

5.

6.

7.

8.

9.

Abrams, H. L., and McNeil, B, J., “Medical Implications of Computed Tomography (’CAT

Scanning’ ),” N. Eng. J, Med. 298:255, 1978.

Ambrose, J., et al., “EMI Scan in the Management of Head In juries,” Lancet 1:847, 1976.

Bahr, A. L., and Hodges, F. J., “Efficacy of

Computed Tomography of the Head in Changing Patient Care and Health Costs: A Retrospective Study, ” Am, ]. Roent. 131:45, 1978.

Baker, C., and Way, L. W., “Clinical Utility of

CAT Body Scans, ” Am. ). Surg. 136:37, July

1978.

Banta, D., “Number and Distribution of CT

Scanners, 1978, ” report to the Bureau of Health

Planning and to State and local health planning

agencies (Hyattsville, Md.: National Center for

Health Services Research, August 1978).

Banta, D., and McNeil, B. J., “The Costs of

Medical Diagnosis: The Case of the CT Scanner, ” Health Care Mgt. Rev, 3:7, 1978.

Bartlett, J. R., et al,, “Evaluating Cost-Effectiveness of Diagnostic Equipment: The Brain Scanner Case, ” Br. Med. ]. 2:815, 1978.

Bunker, J. P., et al, (eds. ), Costs, Risks, a n d

Benefits of Surgery (New York: Oxford University Press, 1977).

Carrera, G. F., et al., “Computerized Tomography of the Brain in Patients With Headache and

10.

11.

12.

13.

14.

15.

16.

17.

With Temporal Lobe

Cost Effectiveness, ” /,

1978.

Epilepsy: Findings and

Comp. Tomog, 2:108,

Creditor, M. C., and Garrett, J. B., “The Information Base for Diffusion of Technology:

Computed Tomography Scanning, ” N. Eng. J,

Med. 297:49, 1977.

Dales, L. G., et al., “Evaluation of a Periodic

Multiphasic Health Checkup, ” Meth. Inform.

Med. 13:140, 1974.

Evens, R, G., and Jest, R. G., “The Clinical Efficacy and Cost Analysis of Cranial Computed

Tomography and the Radionuclide Brain Scan, ”

Sem. Nuc. Med. 7:129, 1977,

“Economic Analysis of Body Computed

Tomography Units—Including Data on Utilization, ” Radio, 127:151, 1978.

“Economic Analysis of Computed Tomography Units, “ Am. ], Roent. 127:191, 1976.

Evens, R. G,, et al., “The Utilization, Reliability, and Cost Effectiveness of Computer Cranial

Tomography in Evaluating Pseudotumor Cerebri, ” Am, J. Radio. 129:263, 1977.

Freeborn, D. K,, et al., “Determinants of Medical Care Utilization: Physicians’ Use of Laboratory Services, ” Am. J. Public Health 62:846,

1972.

Gempel, P. A,, and Harris, G. H. “Comparative
18.

19.

20.

21.

22.

23.

24.

25.

26.

27.

28.

29.

30.

31.

32.

33.

Cost Analysis: Computed Tomography vs. Alternative Diagnostic Procedures, 1977-1980”

(Cambridge, Mass.: Arthur D. Little, Inc.,

1977).

Greenwald, H. P., et al., “Transportation or CT

Scanners: A Theory and Method of Health Resources Allocation, ” Health Serv. Res. 14:207,

1979.

Hsiao, W. C., and Stason, W, B,, “Toward Developing a Relative Value Scale for Medical and

Surgical Services, ” Health Care Fin. Rev. 1;23,

1979.

Institute of Medicine, Computed Tomographic

Scanning: A Policy Statement (Washington,

D. C.: National Academy of Sciences, 1977).

Jonsson, E., and Marke, L. A., “CAT Scanners:

The Swedish Experience, ” Health Care Mgt.

Rev, 2;37, 1977.

Kassirir, J. P., and Pauker, S. G., “Should Diagnostic Testing Be Regulated?” N. Eng. ]. Med.

299:947, 1978.

Knaus, W. A., and Davis, D. O., “Utilization

and Cost-Effectiveness of Cranial Computed Tomography at a University Hospital, ” ]. Comp.

Assist. Tomog. 2:209, 1978.

Knaus, W. A., and Wagner, D., “CT Scanning

Patients With Headache: A Cost-Benefit Calculation, ” Am. ]. Neuro. (in press).

Knaus, W. A., et al., “Impact of New Technology: The CT Scanner, ” Med. Care 15:533, July

1977.

Larson, E. B., and Omenn, G., “The Impact of

Computed Tomography on the Care of Patients

With Suspected Brain Tumor, ” Med. Care

15:543, 1977.

“Computed Tomography in Patients

With C’erebrovascular Disease: Impact of a New

Technology on Patient Care, ” Anl. ]. Roent.

131:35, 1978.

“Diagnostic Evaluation of Headache:

Impact of Computerized Tomography and CostEffectiveness, “ ]. A.M.A. 243:359, 1980.

“Effect of Computed Tomography on

Utilization of Cerebral Angiography, ” Am. ].

Roent, 129:1, 1977.

“Impact of Computed Tomography on

the Care of Patients With Suspected Hydrocephalus, ” Am. ]. Roent. 131 :41, 1978.

McNeil, B. J., and Adelstein, S. J., “Determining

the Value of Diagnostic and Screening Tests, ” ].

Nucl. Med. 17:439, 1976.

, “Measures of Clinical Efficacy: The Value of Case Finding in Hypertensive Renovascular Disease, ” N. Eng. ]. Med. 293:221, 1975.

McNeil, B. J., et al., “Measures of Clinical Ef -

34.

35.

36.

37.

38.

39,

40.

41.

42.

43.

44.

ficacy: Cost-Effectiveness Calculations in the

Diagnosis and Treatment of Hypertensive Renovascular Disease, ” N. Eng. ]. Med. 293:216,

1975.

Neuhauser, D., and Lewicki, A. J., “What Do

We Gain From the Sixth Stool Guaiac?” N. Eng.

j. Med. 293:266, 1975.

Office of Technology Assessment, U.S. Congress, Policy Implications of the Computed ToMography (CT) Scanner, GPO stock No.

052-003 -00565-4 (Washington, D. C.: U.S. Government Printing Office, 1978).

The Implications of Cost-Effectiveness) less

Analysis of Medical Technolology, GPO stock

No. 052-003 -00765-7 (Washington, D. C.: U.S.

Government Printing Office, August 1980).

Orkand Corp., Planning Approach, Criteria,

and Standards for Specialized Trauma Services:

Optimal, Intermediate, and Minimal Levels of

Care (Silver Spring, Md.: Orkand Corp., Sept.

30, 1976). (Also available as publication No.

HRP-0015437 from Department of Commerce,

National Technical Information Service, Springfield, Va. )

Pete, R., et al., “Design and Analysis of Randomized Clinical Trials Requiring Prolonged

Observation of Each Patient, ” Br. ]. Cancer

34:585, 1976.

ReVelle, C., et al., “Facility Location: A Review

of Context-Free and EMS Model s,” Health Serv.

Res. 12:129, 1977.

Schoenbaum, S. C., et al., “Benefit Cost Analysis of Rubella Vaccination Policy, ” N. Eng. ].

Med. 24:306, 1976.

Thomson, J. L. G., “Cost-Effectiveness of an

EMI Brain Scanner: A Review of a Two-Year Experience, ” Health Trends 9:16, 1977.

Weinstein, M. C., “Economic Evaluation of

Medical Procedures and Technologies, ” in Medical Technology, J. Wagner (ed. ), Urban Institute

Conference, NCHSR Research Proceedings Series, DHEW publication No. (PHS) 79-3254

(Washington, D. C.: National Center for Health

Services Research, Office of Health Research,

Statistics, and Technology, PHS, DHEW, 1979).

Weinstein, M. C., and Stason, W. B., “Foundations of Cost-Effectiveness Analysis for Health

and Medical Practices, ” N. Eng. ]. Med.

296:716, 1977.

Willems, J., et al., “The Computed Tomography

(CT) Scanner, ” in Medical Technology: The

Culprit Behind Health Care Costs? proceedings

of the 1977 Sun Valley Forum on National

Health, S. H. Altman and R. Blendon (eds. ),
DHEW publication No. (PHS) 79-3216 (Wash- mography: An Evaluation of Cost-Effectiveington, D. C.: DHEW, 1979). ness, ” Radio. 117:75, 1975.

45. Wortzman, G., and Holgate, R. C., “Reapprais- 47. Zimmerman, R. A., et al., “Cranial Computed

al of the Cost Effectiveness of Computed To- Tomography in Diagnosis and Management of

mography in a Government-Sponsored Health Acute Head Trauma, ” Am. ], Roernt. 131:27,

Care System, “ Radio. 130:257, 1979. 1978.

46. Wortzman, G,, et al., “Cranial Computed To-