Health Care Technology

Assessment in VA

Primer

Management Decision and Research Center

Health Services Research and Development Service

Office of Research and Development

Department of Veterans Affairs

In collaboration with

Association for Health Services Research
Health Care Technology Assessment in VA / by Clifford Goodman...et al.

Boston, Mass. : Management Decision and Research Center;

Washington, D.C. : Health Services Research and

Development Service, 1996.

p. cm.

Includes bibliographical references. Primer.

DNLM: 1. Hospitals, Veterans—organization & administration—

United States. 2. Technology Assessment, Biomedical—organization &

administration. 3. United States. Dept. of Veterans Affairs. Health 

Services Research and Development Service.

UH463 G653h 1996

VA’s Health Services Research and Development Service provides expertise in health services research, a field that examines the effects of organization, financing and management on a wide range of problems in health

care delivery — quality of care, access, cost and patient outcomes. Its programs span the continuum of health care research and delivery, from basic

research to the dissemination of research results, and ultimately to the

application of these findings to clinical, managerial and policy decisions.
Health Care Technology

Assessment in VA

Clifford Goodman, Ph.D.

Gordon Snider, M.D.

Karen Flynn, D.D.S.

Purpose of Primer Series: to help policy makers, managers, clinicians and health services researchers work together to

improve the quality and cost-effectiveness of health care for veterans. The primer series is part of a larger set of dissemination initiatives developed by the Department of Veterans Affairs’ Management

Decision and Research Center in collaboration with the Association

for Health Services Research.

Purpose of Health Care Technology Assessment

in VA: to introduce the purposes, scope and general approach of

technology assessment in health care. The primer describes the role

of technology assessment as a management tool in health care delivery, regulation and payment, and provides a basic framework for

understanding technology assessment programs. The primer is

intended to strengthen the roles of health care decision makers in

technology assessment and to help them become more informed

users of technology assessments. Rather than an exhaustive text or

a how-to manual, this is a general guide to the field. A list of more

in-depth reference material is provided in the bibliography.

Suggested Audience: health care professionals involved in

technology-related decision-making at the local and national levels.

This includes network and hospital managers involved in administration, clinical care, financial and strategic planning, and technology

procurement. Other users include health care information specialists, educators and students.

Suggested Uses: guide for developing assessment requests

or evaluating previous assessments, resource for strategic planning, technology assessment training programs in health care networks and hospitals, orientation for members of technology

assessment committees, and curriculum for medical schools,

schools of business administration and public health, continuing

medical education courses, and other medical and health professional training programs.

May 1996

Boston, Massachusetts

The primer describes the

role of technology assessment as a management

tool in health care 

d e l i v e r y, regulation and

payment, and provides 

a basic framework 

for understanding 

technology assessment

p r o g r a m s .

Table of Contents

i

Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .v

Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

What is health care technology?. . . . . . . . . . . . . . . . . . . . . . . . . . . .1

What is health care technology assessment?. . . . . . . . . . . . . . . . .1

What is the purpose of health care technology 

assessment and why is it important? . . . . . . . . . . . . . . . . . . . . . . .3

When are health care technology assessments requested?. . . . .4

What role do other organizations play in 

technology assessment?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4

What is the role of ethics in technology assessment?. . . . . . . . . .5

How is technology assessment conducted?. . . . . . . . . . . . . . . . . . .6

Step 1. Identify and rank assessment topics. . . . . . . . . . . . . . . . . . . 6

Step 2. Specify assessment problem. . . . . . . . . . . . . . . . . . . . . . . . . .7

Step 3. Determine locus of assessment. . . . . . . . . . . . . . . . . . . . . . . .8

Step 4. Retrieve available evidence. . . . . . . . . . . . . . . . . . . . . . . . . . .8

Step 5. Collect primary data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9

Step 6. Interpret evidence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Step 7. Synthesize and consolidate evidence. . . . . . . . . . . . . . . . . . . 9

Step 8. Formulate findings and recommendations. . . . . . . . . . . . .10

Step 9. Disseminate the findings and recommendations. . . . . . . . 11

Step 10. Monitor impact of assessment reports. . . . . . . . . . . . . . . . .11

Two case examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Concluding remarks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Appendix I: Bibliography. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Appendix II: Computer databases: sources of 

information for health care technology assessment. . . . . . . . . 19

Appendix III: Contacts for more information. . . . . . . . . . . . . . 21

Contributors

Clifford Goodman, Ph.D. is a senior manager at The Lewin Group,

a health care policy and management consulting firm in Fairfax, Virginia. He is a member of the governing boards of the International

Society of Technology Assessment in Health Care, and of the Division

of Health Care Technology Assessment of the International Federation

of Medical and Biological Engineering.

Gordon L. Snider, M.D. is Chief, Medical Service, Boston VA Medical

Center and Maurice B. Strauss Professor of Medicine and Vice-Chairman, Department of Medicine, Boston University School of Medicine.

His academic interests include research in chronic obstructive pulmonary disease, applied medical ethics, medical history and the appropriate use of high technology medicine in practice.

Karen Flynn, D.D.S. is the Manager of VA’s Technology Assessment

Program, a new effort to supply VA senior managers with technology

assessment information, coordinate existing VA technology assessment

activities, assess selected technologies of national importance to VA,

and promote evidence-based medicine within VA. The program is part

of the Management Decision and Research Center, VA Health Services

Research and Development Service.

Thomas L. Garthwaite, M.D. is VA’s Deputy Under Secretary for

Health. Prior to his appointment, he served as Chief of Staff of the

Clement J. Zablocki VA Medical Center in Milwaukee, where he directed all medical center professional activities, integrating the Medical

College of Wisconsin’s Educational and Research Program with VA’s

primary mission of quality patient care. While in Milwaukee, he also

served as Associate Professor of Medicine and Associate Dean at the

Medical College of Wisconsin.

Daniel Deykin, M.D. is Director, Health Services Research and

Development Service, in the Veterans Health Administration’s Office

of Research and Development. He also serves on the faculty at Boston

University, where he is Professor of Medicine and Public Health. His

research interests include the scientific basis for the clinical practice of

medicine. 

Acknowledgements. Special thanks go to Brian Mittman, Ph.D.,

Sepulveda VA Medical Center (Sepulveda, CA), for his role in helping

to identify technology assessment issues particularly relevant to policy

makers, managers and clinicians. Appreciation also is extended to colleagues throughout VA for their thoughtful comments in reviewing an

earlier draft of the primer: Dean Billik, Charleston VA Medical Center; Robert Burt, M.D., Richard L. Roudebush VA Medical Center

(Indianapolis); Wendy Carter, Department of Veterans Affairs; C.

Glenn Cobbs, M.D., Birmingham VA Medical Center; Roger Davis,

M.S., R.Ph., Omaha VA Medical Center; Gary M. DeGasta, White

River Junction VA Medical Center (White River Junction, VT); R.E.

Ecklund, M.D., Omaha VA Medical Center; Peter Fabri, M.D., James

H. Haley Veterans Hospital (Tampa); Wallace Geck, R.Ph., M.S., James

H. Haley Veterans Hospital; Nathan L. Geraths, William S. Middleton

Memorial Veterans Hospital (Madison, WI); David Hindson, M.D.,

Boise VA Medical Center; Willard Johnson, M.D., Boston VA Medical

iii
iv

Center; Robert F. Leveen, M.D., Omaha VA Medical Center; J.J.

Matoole, M.D., Omaha VA Medical Center; Lydia B. Mavridis,

Department of Veterans Affairs; James Nocks, M.D., Veterans Integrated Service Network (Baltimore, MD); John E. Ogden, M.S., Veterans Health Administration; Alan Robbins, M.D., Boston VA

Medical Center; Charles W. Smith, M.D., Seattle VA Medical Center;

Carlton Thygeson, P.D., Lincoln VA Medical Center; Eugene J. Towbin, M.D., Ph.D., Little Rock VA Medical Center; Jim Tuschschmidt,

M.D., North Chicago VA Medical Center; and Steven Wallner, M.D.,

Denver VA Medical Center.
Preface

v

Health care cost and inequity of access to care have become dominant themes of public debate in the United States. There is widespread concern that we spend far more on health care than is

justified by the overall benefit we achieve. While various approaches

to controlling costs and increasing access have been attempted

(including political processes, prepaid or capitated insurance coverage, limited access to specialists, practice guidelines, and diagnosisrelated groups for hospital reimbursement), none of these

approaches has been highly successful.

Health care technology has a paradoxical role in our culture and

health care system. It is frequently cited as a significant contributor

to unacceptable increases in health care costs. On the other hand,

technological innovation is seen as a strength of the United States’

health care system and its research and development infrastructure.

Both health care providers and consumers tend to want access to any

technology that may prove beneficial.

Clinicians, patients and insurers have different motives and expectations regarding whether to adopt and pay for new technologies.

Ideally, these decisions should be knowledge-based. However, in the

age of information overload even the most informed clinician can

find it virtually impossible to keep abreast of all the research results

that are relevant to his or her practice. Patients and policy makers

may have difficulty interpreting research results. Clinicians,

patients and policy makers must deal with the dilemma when multiple studies of the same technology have conflicting results, making

the information from these studies difficult to synthesize and interpret.

The Department of Veterans Affairs (VA) is committed to maintaining access to appropriately-used health care technologies and to

improving quality of care and outcomes for the veterans it serves.

VA actively supports technology assessment processes and the use of

technology assessment information to make evidence-based decisions. The Veterans Integrated Service Networks (VISNs) are

responsible for achieving the greatest possible health care value with

the resources that have been allocated. To fulfill this responsibility,

VISN managers will need technology assessment information on an

ongoing basis.

Health Care Technology Assessment in VA offers an overview of

the processes and uses of health care technology assessment. Readers will find the primer a useful entry point to wider reading and

thinking about incorporating evidence into health care delivery decisions. VA decision makers should employ this approach, whenever

feasible, in developing assessment requests, evaluating previous

assessments and incorporating technology assessment information

into the management of health care systems.

Thomas L. Garthwaite, M.D.

Deputy Under Secretary for Health

Department of Veterans Affairs

The Veterans Integrated

Service Networks (VISNs)

are responsible for

achieving the greatest

possible health care

value with the resources

that have been allocated.

To fulfill this responsibili t y, VISN managers will

need technology assessment information on an 

ongoing basis.

Introduction

vii

The Department of Veterans Affairs (VA) cannot stand apart from

critical self-examination and, indeed, the Secretary, Under Secretary,

Directors of Veterans Integrated Service Networks and hospital managers — are examining all aspects of health care delivery with a

sense of urgency. As part of this examination and evaluation process,

the Health Services Research and Development Service, through the

Management Decision and Research Center (MDRC), is actively

engaged in technology assessment and in information dissemination.

This primer is part of MDRC’s contribution to a broader effort

throughout VA to understand how to use technology assessment

results for better resource management.

This primer’s comprehensive definitions of health care technology

and health care technology assessment may prompt questions of how

technology assessment is different from other kinds of research.

Health care technology assessment is an emerging interdisciplinary

field that draws on many other types of inquiry in health care.

Rather than attempting to define precisely technology assessment as

a single kind of research, it may be more productive to look at it as a

process involving several types of actions. These include identifying

technologies needing assessment, collecting and analyzing data about

technologies, synthesizing existing information and the results of

data collection and disseminating findings and recommendations.

Technology assessment may also be seen as an analytic framework

that encourages critical thinking about everything we do in health

care, constantly reinforcing the need to base decisions and actions on

evidence to the greatest extent possible. The reader is encouraged to

look at technology assessment information as an everyday tool to

support evidence-based health care decision-making.

The primer provides a description of the process of technology

assessment. Three appendices offer practical suggestions on where

to go for more information on technology assessment.

I commend the authors for producing this primer and hope that

network and hospital administrators, physicians, house staff, nurses

and many others — inside and outside VA — find it valuable.

Daniel Deykin, M.D.

Director

Health Services Research and Development Service

Office of Research and Development

Veterans Health Administration

Technology assessment

may also be seen as an

analytic framework

that encourages critical

thinking about everything we do in health

c a r e .

Health care technology is the practical application of knowledge to

the prevention, diagnosis and treatment of disease. Although many

health care providers have classified only equipment, devices and

other hardware as technology, the discipline of health care technology assessment has embraced a broader definition that also encompasses drugs, clinical procedures and managerial systems. The

broad definition recognizes the multiple and complex interactions

between health care technologies and the wide range of human activities and social organizations that adopt and incorporate technologies. 

Health care technologies can be grouped into the following broad

categories:

Devices, equipment and supplies. Examples include cardiac

pacemakers, computed tomography (CT) and positron emission

tomography (PET) scanners, surgical gloves, diagnostic test kits.

Drugs. Examples include aspirin, beta-blockers, penicillin, vaccines, blood products, colony stimulating factors, monoclonal antibodies.

Medical and surgical procedures. Examples include psychotherapy, coronary angiography, gall bladder removal.

Systems of care. Examples include special care units (e.g. coronary care, medical, surgical and neurological intensive care), rehabilitation programs, screening programs.

Support systems. Examples include paper and electronic patient

record systems, telemedicine systems, blood banks, clinical laboratories.

Organizational and managerial systems. Examples include

prospective payment using diagnosis-related groups, alternative

health care delivery configurations such as vertically integrated

health care systems and clinical pathways.

Health care technology assessment (HCTA) is the systematic evaluation of the properties and effects of health care technology. It may

involve the investigation of one or more of the following attributes of

technologies:

Performance characteristics. Performance characteristics

include sensitivity and specificity of diagnostic tests, and conformity

with specifications for design, manufacturing, reliability, ease of use

and maintenance.

Clinical safety. Safety is a judgment of the acceptability of risk

(a measure of the chances of an adverse health outcome and its

severity) associated with using a technology in a particular situation. 

Efficacy. Efficacy refers to the benefit of using a technology to

address a particular problem under ideal conditions (e.g., within the

protocol of a carefully managed randomized controlled trial, involving patients meeting narrowly defined criteria, or conducted at a

“center of excellence”).

What is health care technology?

1

What is health care technology assessment?

Health care technology is

the practical application

of knowledge to the 

prevention, diagnosis

and treatment of disease.

Health care technology

assessment (HCTA) 

is the systematic 

evaluation of the 

properties and effects of

health care technology.
n Effectiveness. Effectiveness refers to the benefit of using a technology for a particular problem under general or routine conditions

(e.g., by a physician in a community hospital treating a variety of

patient types).

n Economic impacts. Health care technologies can have a wide

range of microeconomic and macroeconomic impacts. Microeconomic concerns may include costs, charges or payment levels associated

with individual technologies. Cost effectiveness, cost utility and cost

benefit analyses compare resource requirements and benefits of

technologies for particular applications. Macroeconomic impacts of

health care technologies include the impact of new technologies on

national health care costs, the effect of technologies on resource allocation among different health programs or among health and nonhealth sectors, and the effects of new technologies on outpatient

versus inpatient care. Other macroeconomic issues that pertain to

health care technologies include the effects of regulatory policies,

health care reforms and new policies on technological innovation,

technological competitiveness, technology transfer and employment.

n Social, legal, ethical and political impacts. Many technologies raise social, legal, ethical and political concerns. For example,

genetic testing, fertility treatments, organ transplants and life-support systems for the critically ill challenge legal standards and societal norms. Ethical questions continue to prompt improvements in

informed consent procedures for patients involved in clinical trials.

Allocating scarce resources to technologies that may be expensive,

inequitably used or non-curative raises broad social concerns.

Other key concepts in health care technology assessment:

n Appropriate and medically necessary. The terms “appropriate” and “medically necessary” often are used to encompass one or

more of the attributes discussed above for particular circumstances

to which a technology is applied. For example, the appropriateness

of a diagnostic test may depend on its safety, effectiveness and cost

for a given set of patient indications, clinical settings, resource constraints or availability of other technological options.

n Evidence-based medicine. In the U.S. and elsewhere, technology assessment organizations consider most or all of the attributes

listed above by synthesizing the available research findings (the “evidence”) and other relevant information into technology assessment

reports. Increasingly, this involves grading and weighing the available evidence according to its underlying methodological rigor. Evidence-based medicine refers to the practice of medicine that is

guided by or based on scientific fact. HCTA is a process in which

available evidence is used to formulate findings for guiding technology-related decisions, some of which are for clinical/medical purposes.

The terms – evidence-based medicine and HCTA–are related, but not

synonymous.

n Systematic review and meta-analysis. “Systematic review”

and “meta-analysis” are methods used in the synthesis component of

technology assessment. Systematic reviews are the result of using

scientific methods for identifying, assembling and synthesizing the

medical literature. When systematic reviews apply statistical methods for combining the findings of multiple research studies, they are

called meta-analyses. (See page 10.)

2
The purpose of HCTA is to assist health care policy makers, managers and clinicians at local, network and national levels in making

informed decisions. 

Health care technology assessment information may be particularly useful in supporting decisions when: a technology has high unit

or aggregate cost; explicit trade-off decisions must be made in allocating resources among technologies; a technology is highly complex, precedent-setting or involves significant uncertainty; a

proposed provision of a treatment, diagnostic test or medical equipment is innovative or controversial; or an established technology is

associated with significant variations in utilization or outcomes.

An assessment is usually carried out to meet a clearly defined policy, managerial or clinical goal as illustrated below:

n An organization responsible for financing or providing health care

(e.g., VA) might request an assessment of the relative merits of medical versus surgical treatment of a particular disease.

n A network director might request an assessment of two leading

inter-hospital medical information systems to help decide which

would be most appropriate to purchase.

A VA hospital director who approves patient referrals to other

facilities might request a number of assessments to determine what

therapies are appropriate for patients with specific medical profiles.

The assessment information may also help the director decide

whether to acquire a certain technology for the hospital, rather than

referring patients to other treatment centers.

A hospital pharmacy and therapeutics committee might request

an assessment of two different drug formulations and dosage schedules (e.g., a less expensive several-times daily formulation versus a

much more expensive twice-daily formulation) to inform the development of its hospital guidelines.

A primary care physician might request an assessment of a test

(e.g., spirometry– measurement of lung function) to determine

whether the test should be introduced into routine ambulatory care

practice.

An HMO might request an assessment of simultaneous pancreas

and kidney transplantation to support the design and interpretation

of covered benefits, case management and contracting arrangements

with “centers of excellence.”

Health care decision makers have a responsibility to ensure the

availability of the most effective technologies. They are also responsible for limiting the inappropriate use of technologies, especially in

an era of increasingly limited resources.

3

What is the purpose of health care technology 

assessment and why is it important?

An assessment is usually

carried out to meet a

clearly defined policy,

managerial or clinical

g o a l .
Assessments can be requested and conducted at any stage in a

technology’s life cycle (i.e., stage of development and diffusion).

Stages include:

n conceptual: in the earliest stages of development;

n experimental or investigational: undergoing initial testing

and evaluation;

n pre-established: adoption of an innovation by certain individuals

and institutions;

n established: considered to be a standard approach and diffused

into general use; and

n outmoded: superseded by another technology or demonstrated to

be ineffective or harmful.

These stages often are not clearly delineated, and technologies do

not necessarily mature through them in a linear fashion. A technology may be established for certain applications and may be investigational for others. A technology once considered obsolete may return

to established use for a better defined or entirely different purpose.

Technologies often undergo multiple incremental innovations after

their acceptance into practice.

Since technology is constantly evolving, HCTA must be viewed as

an iterative process. It may be necessary to revisit a technology

when competing technologies are developed, the technology itself

evolves or new information is introduced. Reassessment may

require additional data collection and ongoing assessments may be

enhanced with techniques that aggregate results of research.

Anumber of organizations in the public and private sectors use

HCTA to inform decision-making.

Public sector

The Food and Drug Administration (FDA) — concerned primarily

with safety and efficacy — makes decisions about whether or not to

permit the commercial use of drugs, devices and certain other technologies.

The Department of Veterans Affairs — one of the few federal

agencies actively engaged in the delivery of health care — develops

disease management programs and makes technology acquisition

and management decisions. It also supports a technology assessment program to coordinate assessment activities, conduct selected

assessments, and supply senior managers with technology assessment information.

The Center for Health Care Technology within the Agency for

Health Care Policy and Research (AHCPR) supplies the Health Care

Financing Administration (HCFA) with information that is used to

develop Medicare coverage and reimbursement policies.

4

When are health care technology assessments requested?

What role do other organizations play in

technology assessment?

Assessments can be

requested and conducted at any stage in a 

technology’s life cycle.
The National Institutes of Health (NIH) conduct consensus development conferences to define the optimal use of technologies indicated by currently available data.

n The U.S. Preventive Services Task Force in the Department of

Health and Human Services develops preventive services recommendations based explicitly on evidence-based approaches.

AHCPR supports the development of patient outcomes research

teams (PORTs) and clinical practice guidelines, both of which involve

technology assessment.

Private sector

n Private payers (such as Blue Cross and Blue Shield plans, HMOs,

and commercial insurers) support technology assessment programs

that use consensus methodologies to support coverage decisions.

n The University HealthSystem Consortium reviews specific technologies and coordinates small primary data collection studies

among its member institutions to support purchasing decisions,

guide clinical protocols and select drugs for formularies.

n A number of private organizations (such as ECRI) offer technology

assessment services (e.g., to assist hospitals, payers and others in

decision-making).

n Hospital biomedical engineering departments and pharmacy and

therapeutics committees use technology assessment for technology

acquisition, management and related activities.

Appendix III lists contact information for many of these organizations.

The discipline of medical ethics has developed in response to the

effects of technology on human interactions and the human condition. Since technology assessment is used to make judgments about

what ought to be done with health care technologies, there is significant overlap between it and medical ethics. Conducting a technology

assessment requires careful attention to ethical questions, such as:

n Should all assessments be driven by cost concerns?

n Are the individuals involved in the selection of topics, the conduct

of the assessment, and the use of its results free of conflicts of interest?

n Are judgments of value implicit in the statement of the assessment problem or the choice of methodology?

n Are informed consent, patient confidentiality and related means

for protecting patient welfare in clinical investigations properly

implemented?

n Do assessments provide means (e.g., in data collection, synthesis

and reporting) to determine how technologies challenge prevailing

legal standards and societal norms?

n Are the assessment’s recommendations ethically justified?

5

What is the role of ethics in technology assessment?

Conducting a technology

assessment requires

careful attention to 

ethical questions.
The ten steps listed below provide a basic framework for conducting a technology assessment. This framework also may be used to

guide the development of an assessment request or the evaluation of

an assessment. Not all assessments involve each of these steps or

conduct them in the same sequence.

Two case examples — one based on VA experience and another

from a private sector HMO — illustrate some of the ten steps. In the

VA example, the assessment was performed by a VA physician with

technical assistance from a VA technology assessment program. In

the private sector example, the assessment was performed by an

HMO with assistance from a private sector technology assessment

consultant, using evidence from the literature and the HMO’s internal data sources.

TEN STEPS OF HEALTH CARE 

TECHNOLOGY ASSESSMENT

1. Identify and rank assessment topics

2. Specify assessment problem

3. Determine locus of assessment

4. Retrieve available evidence

5. Collect primary data (as appropriate)

6. Interpret evidence

7. Synthesize and consolidate evidence

8. Formulate findings and recommendations

9. Disseminate findings and recommendations

10. Monitor impact of assessment reports

Step 1. Identify and rank assessment topics

Processes for soliciting candidate assessment topics and ranking

assessment priorities range from being informal to highly formal.

Identifying potential topics. To a large extent, assessment topics

are determined, or at least bounded, by the mission or purpose of an

organization. For instance, VA’s mission includes providing the

highest quality health care to eligible veterans. VA policy makers

are particularly interested in technologies that address the health

care needs of the veteran population.

Ranking topics. Some assessment programs have explicit procedures for setting priorities. Others set priorities in ad hoc or informal ways. Given limited resources for assessment and increased

accountability of assessment programs to their respective stakeholders, it is important for assessment programs to clearly articulate how

assessment topics are ranked and selected.

6

How is technology assessment conducted?

The ten steps. . . provide

a basic framework for

conducting a technology

assessment. This framework also may be used

to guide the development

of an assessment request

or the evaluation of an

a s s e s s m e n t .
The following are examples of criteria — listed in no particular

order — that might be used to set assessment priorities:

n high burden of morbidity or mortality;

n large number of patients affected;

n high unit or aggregate cost of a technology or health problem;

n substantial variations in practice;

n high potential to improve health outcomes or reduce health risks;

n availability of sufficient research findings to perform the 

assessment;

n scientific, professional or public controversy;

n need to make regulatory decision;

n need to make payment decision;

n available findings not widely disseminated or used by 

practitioners.

Step 2. Specify assessment problem

One of the most important aspects of an assessment is to specify

clearly the question(s) to be addressed; this will affect all subsequent

aspects of the assessment. Assessment problem statements should

recognize the relation of the new technology to existing technology.

For example, although a new diagnostic technology may yield superior information it may not affect treatment or improve health outcomes. The team charged with conducting the assessment should

have an explicit understanding of the purpose of the assessment and

the intended users of the assessment.

The following two examples illustrate the importance of how

assessment problems are constructed; the first is flawed, the second

is workable.

n Should our medical center set up a referral mechanism so

that patients have access to positron emission tomography

(PET) imaging? The question as stated does not specify the types

of patients for whom the technology would be used, the alternatives

to the technology, or the kinds of information that the policy maker

would find useful. It would be difficult to design and implement an

assessment to answer this question.

n When patients are being diagnosed and staged for cancer,

does PET imaging provide superior diagnostic information

over more readily available diagnostic technologies such as

CT, MRI or SPECT? This clear and precise assessment question

could be productively addressed.

The intended users or target audiences of the assessment report

should affect its content, presentation and dissemination strategy.

Health care professionals, patients, politicians, researchers, hospital

managers, company executives and others have different interests

and levels of expertise. Needs regarding the scientific or technical

level of reports, the presentation of evidence and findings, and the

format (e.g., length and appearance) of reports vary by target audience.

7

. . . assessment topics 

are determined, or at

least bounded, by the

mission or purpose of an

o r g a n i z a t i o n .

One of the most important aspects of an 

assessment is to specify

clearly the question(s) 

to be addressed; this will

affect all subsequent

aspects of the 

a s s e s s m e n t .
Step 3. Determine locus of assessment

The nature of an assessment problem will affect the determination

of the most appropriate organization or group to conduct the assessment.

A comprehensive assessment addressing multiple attributes of a

technology can be very resource-intensive. It can require considerable training and experience in the methods of evidence-based medicine. Hospitals, smaller insurers and HMOs often obtain technology

assessment reports from organizations that specialize in technology

assessment. Larger insurers, HMOs, or health care systems may

have internal programs for technology assessment. Such programs

usually dedicate a core staff to information retrieval and synthesis,

supplemented by clinical and methodologic expertise as needed. In

some cases, a health care organization may decide to commission one

component of an assessment, such as a systematic review, and perform the other steps in-house.

Factors that influence a HCTA “make or buy” decision include:

n Is an existing assessment available? If an existing assessment is

available, does it address the specific issues of concern to the organization? How recently was it conducted? Is the methodology used

sufficiently credible?

n If an existing assessment needs to be updated or is not available,

do people in the organization have the time and expertise to perform

the required data collection and analyses? If a synthesis of existing

information is needed, does the organization have database searching capabilities, staff to retrieve full text articles, and staff trained in

the conduct of systematic reviews? If new data are needed, does the

organization have the requisite resources and expertise?

n What methodology will be used? If a consensus of clinical experts

is the preferred methodology, does that consensus need to incorporate and reflect the opinions of the organization’s own clinicians?

Will local clinicians accept the results and report recommendations if

they do not participate in the assessment?

Step 4. Retrieve available evidence

One of the great challenges in HCTA is to assemble all of the evidence relevant to a particular technology before conducting a qualitative or quantitative synthesis. Although some sources are devoted

exclusively to health care topics, others cover the sciences more

broadly. Multiple sources should be searched to increase the likelihood of retrieving all relevant reports. Useful sources for relevant

evidence include: 

n computer databases of published literature;

n computer databases of clinical and administrative data;

n printed indexes and directories;

government reports and monographs;

n reference lists in available studies, reviews and meta-analyses;

n special inventories of reports;

n health newsletters and newspapers;

n company reports; and

n colleagues and other investigators.

8

One of the great 

challenges in HCTA is to

assemble all of the 

evidence relevant to a

particular technology

before conducting a

qualitative or 

quantitative synthesis.
Information specialists at health science libraries can be extremely

valuable in assisting with these literature searches. Increasingly,

most of the sources are accessible via the Internet. Further information on specific technology assessment information resources is provided in Appendix II.

Step 5. Collect primary data

Compiling evidence for an assessment may entail collecting new primary data after determining that existing evidence will not adequately address the assessment question(s).

Methods for generating new data on the effects of health care

technology include: 

n large randomized controlled trial (RCT);

n small RCT;

n nonrandomized trial with contemporaneous controls;

n nonrandomized trial with historical controls;

n cohort study;

n case-control study;

n cross-sectional study;

n surveillance (e.g., using databases, registers, or surveys);

n series of consecutive cases; and

n single case report.

These methods are listed in rough order of most to least scientifically rigorous for internal validity, (i.e., for accurately representing

the causal relationship between an intervention and an outcome).

There are other variations of these methodologic designs and some

investigators use different terminology for certain methods. The

demand for studies of higher methodological rigor (e.g., RCTs) is

increasing among health care technology regulators, payers,

providers and other decision makers.

Step 6. Interpret evidence

Evidence interpretation involves classifying the studies, grading the

evidence and determining which studies will be included in the synthesis. Assessors should use a systematic approach to critically

appraise the quality of the available studies. Interpreting evidence

requires knowledge of investigative methods and statistics.

Step 7. Synthesize and consolidate evidence

For many topics in technology assessment, a definitive study that

indicates one technology is better than another does not exist. Even

where definitive studies do exist, findings from a number of studies

often must be combined, synthesized or considered in broader social

and economic contexts in order to respond to the particular assessment questions.

Methods used to combine or synthesize findings from different

studies include:

n Non-quantitative literature reviews. Non-quantitative literature reviews are summaries of existing literature made by one or

more authors who select and weigh findings available from the literature. The procedures used to select studies and cognitively weigh

them can be quite subjective and poorly documented. Given

increased attention to the relative methodological rigor of available

9

The demand for studies

of higher methodological rigor. . . is increasing among health care

technology regulators,

payers, providers and

other decision makers.
evidence and improvements in quantitative approaches, non-quantitative literature reviews are regarded less often as standard

approaches for consolidating evidence.

n Meta-analysis. Meta-analysis refers to a group of statistical

techniques for combining results of multiple studies to obtain a

quantitative estimate of the overall effect of a particular technology

on a defined outcome. Meta-analysis may produce a stronger conclusion than would be yielded by any single study. Meta-analysis typically is used for topics that have no definitive studies or topics for

which studies disagree. The appropriate application of meta-analytic

technologies can be controversial. Although the statistical techniques for combining RCTs have been well defined, such techniques

may not account for certain methodological biases when used with

the weaker study designs. In all cases, careful attention to the rationale and techniques for pooling study results is essential.

n Decision analysis. Decision analysis identifies alternative choices and their associated potential outcomes in a series of sequenced

decisions. Decision models often are shown in the form of decision

trees with decision points, branching steps, outcomes and values

associated with those outcomes. Decision models can be used to: 

predict the distribution of outcomes for patient populations 

and associated costs of care; 

help develop clinical practice guidelines for specific health 

problems; and 

predict the likelihood of outcomes of alternative clinical 

strategies.

n Group judgment or consensus development. Group judgement or consensus development has multiple forms and purposes

(e.g., to make regulatory, payment and technology acquisition decisions; to formulate practice guidelines; and to define the state-of-theart of practice). Consensus development refers to discrete group

judgment processes that contribute to an assessment, or it can refer

to particular consensus-development programs, such as that of the

National Institutes of Health (NIH). Ranging from very informal to

more structured processes, consensus development is more or less

subject to the biases that may arise in group interaction. Consensus

processes do not necessarily involve systematic retrieval or critical

analysis of the available evidence. However, virtually all HCTA

efforts involve group judgment at some juncture, particularly in formulating findings and recommendations.

Step 8. Formulate findings and recommendations

Although the terms “findings” and “recommendations” are sometimes used interchangeably, they have different meanings. Findings

are the results or conclusions of an assessment; recommendations

are the suggestions, advice, or counsel that follow from the findings.

Recommendations can be made in various forms, such as options,

practice guidelines or directives.

Assessments should link explicitly the quality of the available evidence to the strength of their findings and recommendations. Doing

so facilitates an understanding of the rationale behind the assessment findings and recommendations. It also provides a more substantive basis on which to challenge the assessment as appropriate.

Further, it helps assessment programs and decision makers determine if a reassessment is needed as relevant new evidence becomes

available.

10

Recommendations can

be made in various

forms, such as options,

practice guidelines or

directives. 
Step 9. Disseminate the findings and recommendations

Dissemination strategies depend upon the mission or purpose of the

organization sponsoring the assessment. Dissemination should be

planned at the outset of an assessment along with other assessment

activities and should include a clear description of the target audience as well as appropriate mechanisms to reach them. The costs,

time and other resources needed for dissemination should be budgeted accordingly. Dissemination plans do not have to be rigid. The

nature of the findings and recommendations themselves may alter

the choice of target groups and the types of messages to be delivered.

Dissemination should be designed to influence the behavior of relevant decision makers.

Step 10. Monitor impact of assessment reports

The impact of HCTAs, from RCT reports to expert consensus statements, are variable and inconsistently evaluated. Plans for monitoring the impact of an assessment report should be considered in the

assessment design.

Some HCTA reports are translated directly into policies with clear

and quantifiable impacts, while the findings of some definitive RCTs

and authoritative, well-documented assessment reports go unheeded

and are not readily adopted into general practice.

As is the case for the technologies that are the subjects of HCTA, disseminated HCTA findings and recommendations can have intended and

unintended outcomes. Some of the effects of a HCTA report include:

n acquisition or adoption of a new technology;

n reduction or discontinuation in the use of a technology;

n change in clinician behavior;

n change in patient behavior;

n change in the use rate of a technology;

n change in the organization or delivery of care;

n reallocation of national or regional health care resources;

n change in regulatory policy;

n modification of marketing plan for a technology; and

n change in third-party payment policy.

Two case examples illustrate practical applications of technology

assessment. As stated earlier, not all assessments involve the

sequence of ten steps outlined in this primer. Each example demonstrates some of the ten steps; the reader is reminded that technology

assessment in an iterative process.

In the first case, a VA pulmonologist undertook an assessment of

lung volume reduction surgery for emphysema to assist in promoting

responsible, evidence-based practice and to enhance quality of care

for VA patients. The assessment consisted of a systematic review of

the literature, which ultimately resulted in a proposal to a national

professional organization to sponsor new data collection efforts.

In the second case, a large HMO used existing assessment information to project utilization and outcomes associated with treatment

11

Two case examples

Assessments should link

explicitly the quality of

the available evidence to

the strength of their

findings and 

r e c o m m e n d a t i o n s .
for high cholesterol for its enrolled population. It then conducted its

own evidence-based synthesis of the literature to modify the treatment recommendations to achieve better outcomes at a lower cost.

Case example:

Technology Assessment of Reduction Pneumoplasty Surgery

for Emphysema

Topic identification

A VA Medical Service Chief (a pulmonologist) had been watching

the development of a new surgical procedure, reduction pneumoplasty for emphysema, since the first reports were presented at a national meeting in April, 1994. Patients with end-stage chronic

obstructive pulmonary disease (COPD), most of whom have emphysema, make up a large share of the practice of pulmonologists. Previously, the only other procedure known to increase lung function in

emphysema was lung transplantation. The physician was concerned

that the new procedure would diffuse rapidly (including within VA)

prior to adequate evaluation of its long term outcomes, institutional

support, rehabilitation requirements and optimal patient selection

criteria.

Assessment problem(s) and locus of assessment

The physician wanted to conduct a systematic review that would

then be used to help set a national technology assessment agenda

(through a national professional organization) for the procedure. He

wanted to explore the potential usefulness of a multi-center registry.

The physician requested assistance from VA’s Management Decision

and Research Center Technology Assessment Program.

Available evidence

VA’s Technology Assessment Program provided the physician:

readings in technology assessment specific to the identified assessment problem, including materials on assessing surgical procedures,

critical evaluation of the literature, construction of evidence tables,

and the use of registries in technology assessment;

n names of contacts at existing registries who could offer advice on

the practical aspects of setting up a large-scale, multi-center registry;

n assistance in searching databases and generating selection criteria

for published articles to be included in the review; and

n editorial advice on writing a technology assessment report for submission to a professional journal.

Interpretation, synthesis and consolidation of evidence

The physician produced an evidence-based report summarizing relevant studies.

n A surgical team, based at a large lung transplant center, had published its results for the open-chest version of the procedure in a relatively small series of patients (in two peer-reviewed journal articles

and four abstracts); there was no early or late mortality in these

patients, and there were significant improvements in lung function

and quality of life.

n A second team using the open-chest procedure had presented its

results in one abstract, reporting no mortality and significant lung

function improvement in a very small series of patients.

n Several surgical teams had published results with a related procedure (thorascopic laser ablation of pulmonary bullae and diffuse

emphysema) in a small series of patients; while modest improvement

12

The physician was concerned that the new 

procedure would diffuse

rapidly (including within VA) in the absence of

adequate evaluation of

its long term outcomes.
in lung function was reported, the post-operative death rates were as

high as 21 percent.

The assessment concluded that short term outcomes of the openchest procedure in highly selected patients at a treatment center

with lung transplantation support and expertise were encouraging.

However, no data documenting the procedure’s outcomes in other

settings were available. In addition, technical details of the procedure and its long term survival and functional benefits remained to

be defined, as did the results of the thorascopic laser procedure.

Findings and recommendations

There was sufficient improvement in many patients that a randomized clinical trial — for which some patients would receive the treatment and others would not — raised ethical considerations. A

multi-center registry, for which specifics were defined, was proposed.

The registry raised no ethical issues, would permit rapid, prospective

data collection and evidence-based assessment of many questions.

However, because of limited control of bias, a registry would have

much less power than a randomized clinical trial.

Epilogue: Some months later, in the course of a feasibility study for a

clinical trial of reduction pneumoplasty, a planning meeting was convened with a biostatistician, clinical trial specialists, epidemiologists,

thoracic surgeons and pulmonologists. At that meeting, a design for

a randomized clinical trial which resolved ethical concerns was

developed.

Dissemination

The assessment findings and registry proposal were presented at

national meetings and in the peer-reviewed literature. VA’s Technology Assessment Program distributed a summary of the assessment to VA network and Medical Center Directors, alerting them to

the need for additional information before making the procedure

widely available.

For a citation of this case study, refer to Snider (1996) in Appendix I.

Case example:

Technology Assessment of Treatment for High Cholesterol

Levels

Topic identification

In pursuit of its mission to maximize the health of the population it

serves — subject to resource limitations— a large HMO was preparing to implement high blood cholesterol treatment recommendations

issued by the Second Adult Treatment Panel of the National Cholesterol Education Program (NCEP II, 1993).

Assessment problem

The NCEP II treatment guidelines used a simplified risk-assessment

protocol. The panel had hoped this simplicity would increase the

probability of adoption of their recommendations. The HMO

hypothesized that some efficiency may have been sacrificed to

achieve NCEP II’s simplicity and set out to determine the utility of a

more efficient strategy that would categorize patients by relevant

risk categories when determining who should get the treatment.

Locus of the assessment

The HMO hired a technology assessment consultant to work with its

staff.

13

The HMO. . . set out to

determine the utility of

a more efficient strategy

that would categorize

patients by risk when

determining who should

get the treatment.
Available evidence and primary data collection

Expected outcomes were assessed for the HMO’s population under

three scenarios: current practice (status quo); NCEP II recommendations; and the HMO’s more risk-sensitive treatment protocol.

The assessment team used the best available evidence from the literature on the epidemiology of different risk factors and the effectiveness of drug treatment. The HMO’s information system

supplied data on population characteristics and the costs of treatment, which were then used to model the impact of the three alternative protocols.

Synthesis of evidence and findings

n If the HMO were to maintain current practice, it could expect to

see 40,600 sudden deaths or myocardial infarctions (MIs) in its

enrolled population during the next five years, generating treatment

costs of $887 million.

n If the HMO were to adopt the NCEP II treatment recommendations, 35,800 sudden deaths and MIs would be expected (i.e., 4,800

less than with current practice) during the five year period, generating treatment costs of $933 million (i.e., $46 million more than current practice).

n If the HMO were to use the more risk-sensitive protocol, a larger

number of sudden deaths and MIs could be prevented. It could

accomplish this by treating less than half the number of people who

would have been treated under NCEP II recommendations. The

HMO could expect 35,500 sudden deaths and MIs (i.e., 5,100 less

than current practices and 300 less than NCEP II’s protocol) during

the five-year period. Treatment costs would be significantly lower

with the HMO’s risk sensitive protocol ($74 million for cholesterollowering drugs plus $775 million for CHD treatment for a total of

$850 million); The HMO’s protocol would result in savings of $37

million over five years compared to current practice, and savings of

$9 million over the NCEP II recommendations.

Recommendations

The assessment team recommended implementation of the more

risk-sensitive treatment protocols.

For a citation of this case study, refer to Eddy (1994) in Appendix I.

14
Concluding remarks

15

The primer introduces fundamental concepts of the dynamic field

of health care technology assessment. Health care technology assessment methods are evolving and applications for assessment results

are increasingly diverse. Broader participation of people from a variety of disciplines and roles in health care is enriching the field.

Heightened demand for technology assessment arising from private

and public organizations’ quest for value in health care, including

VA, is pushing the field to evolve keener processes and assessment

reports tailored for particular user groups.

Rather than make decisions for policy makers, network and hospital managers or clinicians, technology assessment informs decisions

by providing a systematic, thorough evaluation of available information — as permitted by available resources.

. . . technology assessment informs decisions

by providing a 

systematic, thorough

evaluation of available

information. . . 

Appendix I: Bibliography

17

For more in-depth information about technology assessment and

its role in health care delivery, the following publications may be

helpful:

n Banta, H.D., and Luce, B.R. Health Care Technology and its

Assessment: An International Perspective. New York: Oxford University Press, 1993.

n Department of Veterans Affairs, Health Services Research and

Development Service (in conjunction with Association for Health

Services Research). “Technology Assessment and the Cochrane 

Collaboration,” Forum 1996; (suppl): 1-8, February.

n Eddy, D.M. A Manual for Assessing Health Practices and Designing Practice Policies: The Explicit Approach. Philadelphia: 

American College of Physicians, 1992.

n Eddy, D.M. “Rationing Resources While Improving Quality: How

to Get More for Less.” Journal of the American Medical Association,

1994; 272 (10):817-24.

n American College of Physicians and British Medical Journal.

Evidence-Based Medicine, 1995 1(1) (new medical journal).

n Eysenck, H.J. “Meta-analysis and Its Problems.” British Medical

Journal, 1994; 309 (6957): 789-92.

n Frazier, H.S. and Mosteller, F., eds. Medicine Worth Paying For:

Assessing Medical Innovations. Cambridge: Harvard University

Press, 1995.

n Guyatt, G.H., Sackett, D.L., and Cook, D.J. “Users’ Guides to the

Medical Literature. II. How to Use an Article about Therapy or Prevention.” Journal of the American Medical Association, 1993; 

270 (21):2598-601.

n Heitman, E., and Sulmasy, D. “Ethical Considerations in Technology Assessment,” (paper written for Department of Veterans Affairs,

Health Services Research and Development Service, and Association

for Health Services Research), 1995, unpublished.

n Hasselblad, V., and McCrory, D.C. “Meta-analytic Tools for Medical Decision Making: A Practical Guide.” Medical Decision Making,

1995;15 (1):181-96.

n Institute of Medicine. Assessing Medical Technologies. Washington, DC: National Academy Press, 1985.
18

n Laird, N.M., and Mosteller, F. “Some Statistical Methods for Combining Experimental Results.” International Journal of Technology

Assessment in Health Care, 1990; 6 (1):5-30.

n Snider, G.L. “Health Care Technology Assessment of Surgical 

Procedures: The Case of Reduction Pneumoplasty for Emphysema.”

American Journal of Respiratory and Critical Care Medicine, 1996; 

153 (4 Part 1):1208-13.

n Tunis, S.R. and Gelband, H. “Health Care Technology in the

United States.” Health Policy, 1994; 30 (1-3):335-96.

n U.S. Congress, Office of Technology Assessment. Identifying

Health Technologies That Work: Searching for Evidence. OTA-H608. Washington, DC: U.S. Government Printing Office, 1994.
19

Appendix II: Computer databases:

sources of information for health

care technology assessment 

There are hundreds of publicly available computer databases in

health care and biomedicine of various types. Bibliographic databases have indexed citations for journal articles and other publications.

Factual databases provide information in the form of guidelines for

diagnosis and treatment, patient indications and contraindications,

and other authoritative information. Referral databases provide

information about organizations, services and other information

sources.

Examples of large bibliographic databases covering biomedicine

are MEDLINE, which is one of more than 40 MEDLARS (Medical

Literature and Retrieval System) databases provided by the National

Library of Medicine (NLM), and EMBASE (Excerpta Medica) produced by Elsevier. In addition, there are many specialized databases

in such areas as AIDS, bioethics, cancer treatment, and pharmaceutical research and development. Many databases are available from

vendors that repackage or reformat existing databases.

Two new NLM databases of particular relevance to HCTA are

HealthSTAR and HSTAT. HealthSTAR is a bibliographic database

for health services research, technology assessment, planning and

administration; it includes citations to journal articles, technical and

government reports, books and book chapters, meeting abstracts and

newspaper articles. HSTAT (Health Services/Technology Assessment Text) includes the full text of practice guidelines, consensus

development reports and certain other HCTA reports sponsored by

U.S. government agencies.

Examples of clinical trial registries in particular clinical areas are

the AIDSTRIALS database and the PDQ database on cancer treatment, both of which are available on MEDLARS. NIH and the

Department of Veterans Affairs maintain cross-topic registries of

controlled clinical trials sponsored by those respective agencies. The

Cochrane Collaboration is an international organization that prepares, maintains and disseminates a database of systematic reviews

of RCTs of treatments in more than 20 clinical areas.

See chart on the next page for more extensive listing of information sources.
20

descriptions of substances used in AIDS-related trials

references to recent AIDS literature

description of AIDS-related clinical trials

references to bioethics literature

references to journal literature in cancer

references to books, monographs

directory of organizations

ongoing health services research projects

references to health services research, technology assessment, and planning and administration literature

full text of AHCPR clinical practice guidelines and technology assessment reports, NIH consensus development conference and technology assessment reports, U.S. Preventive

Services Task Force Guide to Clinical and Preventive Services, and other federally sponsored guidelines

references to biomedical journals

descriptions of cancer treatment, supportive care, screening, prevention, clinical trials

references to biomedical journals

references to scientific journals

tables of contents of scientific journals

data sets and full text of public health reports

AIDSDRUGS:

AIDSLINE:

AIDSTRIALS:

BIOETHICS:

CANCERLIT:

CATLINE:

DIRLINE:

HSRProj:

HealthSTAR:

HSTAT:

MEDLINE:

PDQ:

EMBASE:

SCISEARCH:

Current Contents:

CDC WONDER:

NLM DATABASES

OTHER DATABASES

*HealthSTAR was formed by the 1996 merger of the Health and HSTAR databases
Appendix III: Contacts for more 

information

For more information about technology assessment services and resources, the following contacts inside

and outside VA may be helpful:

Contacts within VA

n Biomedical engineering departments in VA Medical Centers can provide information on vendors, costs,

maintenance requirements and technical specifications for equipment. VA Headquarters also provides biomedical engineering support for technology assessment and equipment acquisition activities.

Contact: Steven Wexler, M.S., Chief, Biomedical Engineering (138-C2), Veterans Health Administration, 

810 Vermont Avenue, N.W., Washington, DC 20420. 

Commercial phone: (202) 273-5881 

E-mail: wexler@101cmoob.med.dvacm.gov 

n Pharmacy and therapeutics committees at VA Medical Centers evaluate drugs for inclusion in formularies. VA Headquarters Clinical Pharmacy is one of VA’s contacts (along with the MDRC Technology

Assessment Program, below) for the University HealthSystem Consortium Technology Assessment and

Drug Monograph Programs.

Contact: Andy Muniz, M.S., Chief of Field Operations, Pharmacy Service (111-H), 

Veterans Health Administration, 810 Vermont Avenue, N.W., Washington, DC 20420.

Commercial phone: (202) 565-5302

E-mail: muniz.andrew@forum.va.gov

n VA’s Technology Assessment Program in the Management Decision and Research Center, supplies technology assessment information (updated existing assessments from other national and international technology assessment agencies, systematic reviews on request to Network directors, support for Medical

Center and Network assessment activities, and contacts with other VA technology assessment offices), and

conducts major assessments for the VA system (e.g. effectiveness of positron emission tomography as a

diagnostic procedure). The Program supports and works closely with the San Antonio VAMC Cochrane

Center (see below).

Contact: Karen Flynn, D.D.S., Program Manager, VA Medical Center (152-M), 

150 South Huntington Avenue, Boston, MA 02130.

Commercial phone: (617) 278-4469 

E-mail: flynn.karen@forum.va.gov

n The San Antonio VAMC Cochrane Center, a component of the international Cochrane Collaboration,

supports the Collaborative Review Groups that produce Cochrane systematic reviews and trains interested

VA investigators in the Collaboration methods. The Center maintains databases of existing reviews, review

groups and of randomized clinical trials in some specified areas.

Contact: Gilbert Ramirez, Ph.D., Co-Director, San Antonio Cochrane Center (11C6), 

Audie L. Murphy Memorial Veterans Hospital, 

7400 Merton Minter Boulevard, San Antonio, TX 78284.

Commercial phone: (210) 617-5190. 

E-mail: gramirez@merece.uthscsa.edu

n The Hybrid Open Systems Technologies (HOST) program evaluates commercial information on technologies and software for integration with the Decentralized Hospital Computer Program.

Contact: Kathleen Meyers, VA HOST Program Office Coordinator (192-8), 

VA Medical Center, Martinsburg, WV 25401. 

Commercial phone: (304) 264-4485. 

E-mail: meyers.kathleen@forum.va.gov

21
n The National Center for Cost Containment (NCCC) generates models for efficient resource allocation for

use by decision makers throughout VA. NCCC focuses on comparative data analyses (e.g. erythropoietin

utilization and costs at a number of VA dialysis sites), protocols and guidelines (e.g. Home Oxygen Program

Guidance), and an information clearinghouse for policies, practices, and methods (National Laboratory

Resource Guide).

Contact: Christian L. Houterman, M.S., M.B.A., Technology Assessment Coordinator, 

National Center for Cost Containment, Veterans Health Administration, 

5000 West National Avenue, Milwaukee, WI 53295.

Commercial phone: (414) 384-2000, extension 2356

E-mail: houterman.christian@forum.va.gov

Contacts outside VA

n A wide range of existing technology assessments from national and international agencies are available

in the public domain. These include assessments from: AHCPR/Center for Health Care Technology;

National Institutes of Health/Office of Medical Applications of Research (consensus development conferences); U.S. Preventive Services Task Force; American College of Physicians; American Medical Association’s DATTA program; and agencies in Canada, United Kingdom, France, Spain, Sweden, Australia, and

Netherlands. 

VA’s Management Decision and Research Center (MDRC) Technology Assessment Program maintains a

library of these assessments for use by VA decision makers.

Contact: Karen Flynn, D.D.S., Program Manager, VA Medical Center (152-M), 

150 South Huntington Avenue, Boston, MA 02130.

Commercial phone: (617) 278-4469 

E-mail: flynn.karen@forum.va.gov

A number of private sector and nonprofit agencies make technology assessment reports and services available for purchase:

n The Blue Cross and Blue Shield Association’s Technology Evaluation Center (TEC) evaluates technologies to determine their eligibility for coverage by BC and BS plans. TEC reports are published in cooperation with Kaiser Foundation Health Plan and Southern California Permanente Medical Group, and are

available for purchase.

Contact: Ellen Pearson, TEC Marketing Director, Blue Cross and Blue Shield Association, 

676 North St. Clair Street, Chicago, IL 60611.

Phone: (512) 440-6080

n ECRI is a nonprofit health services research agency and a Collaborating Center of the World Health

Organization that operates a technology assessment service and maintains a technology assessment bibliographic database. A variety of health care organizations commission its evidence-based technology assessment reports.

Contact: ECRI, 5200 Butler Pike, Plymouth Meeting, PA 19462.

Phone: (610) 825-6000

n Hayes Incorporated provides several kinds of technology assessment information to subscribers. These

include: the results of literature database searches, information on the regulatory status of technologies

and qualitative syntheses of existing information.

Contact: Pat Dehan Williamson, Account Executive, Hayes Incorporated, 

P.O. Box 190, Worcester, PA 19490. 

Phone: (215) 584-8354

22
Reader comments

Name:___________________________________________________ 

Title:___________________________________________________

Address:____________________________________________________

___________________________________________________ 

___________________________________________________

___________________________________________________

Telephone:_________________________Fax: ______________________

Comments:_______________________________________________________________________________________

_________________________________________________________________________________________________

_________________________________________________________________________________________________

_________________________________________________________________________________________________

_________________________________________________________________________________________________

_________________________________________________________________________________________________

_________________________________________________________________________________________________

_________________________________________________________________________________________________

Date:___________________________________________________

MDRC, AHSR and the authors are interested in your comments on Health Care Technology Assessment in

VA, the primer series in general and any suggestions for future primer topics. Please copy and complete

the form below and mail or fax to: 

Management Decision and Research Center (152-M)

VA Medical Center

150 So. Huntington Avenue

Boston, MA 02130

Fax (617) 278-4438 

Health Care Technology Assessment is available in 

print and electronic formats. Additional copies may

be obtained from the sources listed below.

Printed copies–contact:

Sue Harris

Special Projects Office (152)

VA Medical Center

Perry Point, MD 21902

FTS (700) 956-5442

Com (410) 642-1092

Fax (410) 642-1095

Email: harris.sue_e@forum.va.gov

Printed copies via fax–contact:

VA Fax-On-Demand System

(304) 264-3987

Follow the voice menu system instructions to order

document #3030. Requestors are advised that fax

transmission can take up to 30 minutes.

Electronic copies (PDF format) can be downloaded

from one of the VA electronic bulletin board systems

listed below. The primer is available in the HSR&D

file library. For additional information or assistance,

contact the system administrator at (304) 264-4486 

or (304) 263-0811 ext. 4037.

VA Online (all users)

(800) US1-VETS (871-8387) or (301) 427-3000

Modem Settings: 14,400-N-8-1; terminal type=ANSI

VA World (VA employees)

FTS 700-940-4393 or (304) 264-4493

IDCU Destination: VAWorld

Modem Settings: 14,400-N-8-1; terminal type=ANSI