Abstract

As an approach to linking patient outcomes research and

nursing practice, the paper begins with a framework defined by three aims of science: building knowledge, making decisions and carrying out applications—and goes on

to identify related, strategic questions about nursing research.

The National Center for Nursing Research (NCNR) is

capable of bringing its resources and position to bear on

those questions, which include: how are data about nursing practice related to the intellectual agenda of the initiative? How does nursing research contribute to evaluating

decision-making based on the findings of the initiative?

When would nursing research contribute to policies derived from the initiative; near term (practice and payment

policies) and/or long term (health care system reform)?

NCNR’s answers would establish a strategic framework

about nursing practice and research in patient outcomes.

The framework could guide science policy and bring decisions about design and methods to life. Those decisions are best made in light of both science and expected

application, particularly when the applications include

clinical decision-making, policies guiding practice, policies about health care financing, and policies affecting

health care reform.

NCNR’s strategic framework, and resulting science

policy, would foster a portfolio of nursing research that

meets the standards for patient outcomes research and falls

within the authorization of NCNR.

Implications for some pivotal design and methods issues

are discussed, followed by conclusions:

1. Improving the link to nursing practice requires specifying nursing data and inputs in relation to the samples,

measures and analytic models used in outcomes research

and creating simple ways of obtaining the pertinent data.

2. All studies of patient outcomes should address the limits of scope, rigor and generalizability. The field would

benefit from a formal, but pragmatic, evaluation of the

assets and liabilities of the knowledge base for clinical

decisions and policy analysis.

3. Outcomes research must include some controlled experimental studies. Those studies should be

multidisciplinary, of sterling quality, and focused on enduring policy questions.

As we discuss the topic for this session, design and methods and the link to nursing practice, it is important to remember the specific purpose of our discussion. We are

here to try to contribute to the NCNR’s deliberations over

their science policy regarding patient outcomes research.

The policy will reflect NCNR’s definition of “science” in

this field. If it affects how NCNR distributes its resources,

 Knowledge, Decisions And The Timing Of Policy

Development: Creating A Strategic Framework

For Science Policy

 Susan B. Meister, Ph.D., R.N., F.A.A.N.
the policy will play a role in shaping knowledge about

nursing practice and outcomes. By affecting the knowledge base for health policies, the science policy will eventually affect access to nursing care. In short, this is a

science policy with substantial potential.

Our aim is to analyze the technical issues in a manner that

helps to inform thinking about science policy. Here, I

outline a strategic framework designed to support that

analysis. It relates technical issues to the overarching

purpose of conducting research in order to improve health

services.

The framework is incomplete until NCNR’s decisions are

incorporated. Once completed, the framework should

establish connections between science policy and three

goals of science: to design a plan for building knowledge;

to create the ability to make decisions; to prepare for the

expected/desired applications. (See Figure 1) For patient

outcomes research, those three general goals include many

specific objectives. For example, research in the field

addresses an agenda for building knowledge about what

works and how to use it in health care. Studies also evaluate the quality of the knowledge base for decisions. Findings are, or will be, used to support policies guiding practice and payment now, and system reform later.

Related questions for science policy about nursing research include: how can data about nursing practice be

linked to studies of what works and how to use it; how

can the evaluation of the knowledge base as a premise for

decisions be extended; and, how can the timing of nursing research be estimated so that studies can be articulated with the right policy activities? (See Figure 2)

Clearly, I am posing more questions than answers. Even

so, each of the three goals is examined more closely in

the following sections. The discussion focuses on the

implications for linking data about nursing practice to

patient outcomes research. The paper outlines a strategic

framework defined by the aims of building knowledge,

making decisions and carrying out applications. This

framework could be considered as part of the context for

science policy.

I. National Center For Nursing Research

Major research efforts in patient outcomes are well underway, and will soon produce new premises for policies

guiding both practice and payment systems. However,

as the outcomes and effectiveness initiative gathers steam,

it does so essentially without establishing positions for

data about nursing practice. Notable as it is, the

groundswell of related activities within organized nursing and by individual nurse scientists has not established

those positions—in part because several strategic questions have not been answered. Yet.

The National Center for Nursing Research (NCNR) is

capable of bringing its resources and position to bear on

those questions, which include:

- how are data about nursing practice related to the intellectual agenda of the initiative?

- how does nursing research contribute to evaluating decision-making based on the findings of the initiative?

- when would nursing research contribute to policies derived from the initiative; near term (practice and payment

policies) and/or long term (health care system reform policies)?

A. Strategic Framework

NCNR’s answers, essentially a strategic framework about

nursing practice and research in patient outcomes, would

guide science policy and bring decisions about design and

methods to life. Those decisions are best made in light of

both science and expected application, particularly when
the applications include clinical decision-making, policies guiding practice, policies about health care financing, and policies affecting health care reform.

NCNR’s strategic framework, and resulting science

policy, would foster a portfolio of nursing research that

meets the standards for patient outcomes research and falls

within the authorization of NCNR. Under the Health

Care Research Extension Act of 1985, P.L. 99-158, NCNR

was authorized at NIH for the “conduct and support of,

and dissemination of information respecting, basic and

clinical nursing research, training and other programs in

patient care research”.

 Ii. Agenda And Links

The first component of the framework focuses on knowledge; in this case, the patient outcomes research agenda

of building knowledge about what works and how to use

it—as well as the strategic question of what data about

nursing practice would contribute.

 A. Patient Outcomes Agenda

The roots of the patient outcomes research agenda are in

quality of care and technology assessment. In the late

1980s, the Administrator of the Health Care Financing

Administration (HCFA) discussed four areas of work that

could support an initiative to improve effectiveness of

care: database development, research on outcomes and

clinical effectiveness, dissemination and assimilation of

findings, and development of practice guidelines that are

practical, based in science, explicit and revisable (Roper,

et al, 1988; Roper, 1990).

The research agenda is enormous. For example, Berwick

(1989a) outlined four components: efficacy (finding out

what works), appropriateness (using what works), execution of care (doing well what works), and purposes of

care (identify the values underlying actions). Even the

most pared-down empirical question becomes extraordinarily complex when it is placed in the domain of “usual”

health care delivery.

Brook & McGlynn’s (1990) definitions of “quality” illustrate such complexity. They identify two components

of quality; the technical process of care and the art of

care. The technical process refers to the amount, type

and manner of resource utilization. It is measured by determining whether the service was appropriate. That is,

was the right procedure done for the right reasons? It is

also measured in terms of the outcomes; was the procedure done competently, and what were the outcomes for

the patient? The art of care is measured in terms of patient satisfaction and compliance.

Elements of the patient outcomes agenda seem to fall

within the purview of NCNR, and could plausibly draw

on each of the branches (Nursing Systems, Health Promotion/Disease Prevention, Acute and Chronic Illness).

Improving the links to data about nursing practice involves

a number of actions, many of which have been discussed

during this conference. Two categories of those actions

are especially pertinent to this session:

- specify the nursing inputs for patient outcomes

- reconcile how we define and measure those inputs to

the design and methods of patient outcomes research

 B. Improving Knowledge About What Works

 1. Specify Nursing Inputs

Although a large number of nursing inputs have the potential to contribute to outcomes research, I will focus on

only two today: nursing data about the risks patients carry

at admission; and, nursing services which are provided

during hospitalization and affect the probability of receiving treatment and/or having various outcomes.
The first set of inputs, nursing data about risks, is indirect

in the sense of providing additional data about patients.

These are not data about nursing itself, but rather, unique

data that nurses collect routinely. The second set of inputs, nursing services delivered during hospitalization, are

direct in the sense of defining additional elements of clinical strategies in the context of the elements already under

study.

These two sets of nursing inputs are significant because

they could simultaneously improve the links to data about

nursing practice and make unique contributions to the

patient outcomes knowledge base. If the data were available, they would allow researchers to:

- use nursing data to enhance casemix measures, and

- use nursing data to refine definitions of clinical strategies, treatments, and outcomes.

2. Reconcile

A. Sampling

Sampling by outcome or clinical condition, sampling from

“at risk” populations, and using power analyses to determine sample sizes for low incidence variables are important to outcomes research (Donabedian, 1988A, 1986,

1982; Siu et al, 1991; Luft & Hunt, 1986). Related research on quality assurance and dissemination aims to

safeguard and improve the quality of care by changing

providers’ behavior (Donabedian, 1991). The research

focuses on increasing the systems’s capacity to implement an effective and comprehensive quality assurance

program, and thus, the sample might draw upon providers, institutions, payors and agencies (Lohr & Schroeder,

1991). Nurse researchers have certainly developed a foundation of experience in using the smapling techniques of

patient outcomes research and research to guide dissemination (Lang & Merek, 1990; Moritz, 1991).

However, nurse researchers have less of a foundation in

studying patient groups as aggregates and assessing their

status in relation to outcome variables. Linking outcomes

research to nursing practice may require efforts to increase

familiarity with the methods such as caseload analysis.

There are many models of caseload analysis. For example, using the basic dimensions of characteristics and

policy associations, Meister (1989) illustrated an approach

intended for research on families. The method includes

assessment of four characteristics (demographic, health,

economic, social) and four policy associations (family

economics, health care systems, education and transition

to work, services to young children and their families).

This method, or any designed to array knowledge from

practice so that it can be used in policy analyses, could

generate a concise, practice-based set of parameters for

describing groups of hospital patients. The parameters then

contribute to developing sampling frames linking outcomes research and nursing practice.

B. Measurement

Measurement is more of a problem—for outcomes research as well as for creating links to nursing practice.

Because the objectives of outcomes research include identifying the right interventions and how to use them to

obtain the best outcome, the studies must go beyond the

relationships between treatment and outcome to address

variables such as provider competence, access (market

level and beneficiary level), and acceptability of care

(Brook & Kosecoff, 1988; Palmer, 1991).

Outcomes research must demonstrate the link between

the processes of care and outcomes, and assess technology in ways that will explain the process of care, but the
link is obscured by the multiplicity of processes and providers embedded in care (Horn & Swain, 1988; Lohr,

1988). Research methods to untangle those effects are

advancing rapidly, but remain limited by both databases

and measures (Lohr, 1988; McNeil, 1990; Tarlov, et al,

1989; Ware, 1990). HCFA’s administrative databases,

often used in patient outcomes research, illustrate both

the advances and the limits.

The administrative databases offer an unbiased, but partial, view of whole populations of patients and providers

in the real world. They can be linked to mortality data,

used to derive rates and variations in use of technology,

and used to estimate costs of hospitalization (McNeil,

1990)

However, limits in data about how treatments were selected, how outcomes vary by provider, comorbidity and

severity of illness make it difficult for researchers to define an inception cohort (Jenks, 1990; McNeil, 1990).

Further, the risk adjustment variables are very limited—

even the best account for less than 30% of the variance in

mortality (Jenks, 1990). Therefore, the administrative

databases are least useful for studying diagnostic entities,

and have definite limitations in studies of therapies (Jenks,

1990; McNeil, 1990). Often, because research designs

require risk adjustment and administrative databases don’t

fully address it, the outcomes included in those databases

are viewed as relatively crude screening measures with

uncertain sensitivity and specificity (Jenks, 1990, Jenks

et al, 1988; McNeil, 1990; Schroeder, 1987).

Although most studies of patient outcomes make use of

the HCFA administrative databases, they often use supplementary data as well (Agency for Health Care Policy and

Research, [AHCPR] 1991). Similarly, researchers continue to examine ways of expanding the range of outcomes beyond morbidity, mortality and disability (Brook

& McGlynn, 1991; Greenfield, 1989; McCormick, 1991;

Ware, 1990).

In general, nurse researchers have been conducting work

related to the measures used in patient outcomes for 15

years. In 1978, Horn and Swain developed criterion measures of nursing care, followed by Hegyvary’s measures

of the quality of care, Strickland and Waltz work on nursing performance and standards, and Lang and Merek’s

(1990) discussion of frameworks to classify outcomes, to

name a few. During those same 15 years, nursing has

also conducted extensive work on patient classification

and nursing resource consumption, as well as the conduct and utilization of research in nursing practice. This

body of work is pertinent to patient outcomes, although

the basis for articulation has not yet been fully specified

(Lang & Clinton, 1983, 1984).

C. Analysis: Casemix

Patient outcome analyses usually aggregate data at the

level of institutions (ie hospitals) or areas. The design

often uses a measure of severity to reduce the confounding of degee of sickness with outcomes. For example,

studies often adjust for “casemix”, that is, the severity of

illness of the group of patients admitted to each hospital.

Methods for producing casemix indices have been studied extensively, and applied to various systems for hospital payment. Nursing has applied the principle, but not

usually the same methods, to patient classification studies, by examining the relationships between acuity of illness and use of nursing resources.

Outcomes research uses casemix variables to quantify the

effects on the probabilities of receiving the treatment or

having the outcomes and thus, obtains a clearer picture

of the probable role of the treatment in the outcome (Siu,
et al, 1991). Using nursing data in conjunction with

casemix indices is an effort to improve the power of those

indices. Casemix indices are already designed to move

from the concept of a group with health outcomes to a

more sensible, and powerful, view of a sample with health

outcomes corrected for severity.

For example, a simple view of a patient group receiving/

not receiving a treatment and then having outcomes (Figure 3) is greatly enhanced by including the concept of

comorbidity and its role in expected outcomes (Figure 4).

Nurses routinely collect data, especially at the time of

hospital admission, which could improve the empirical

power of casemix indices. The research questions center

on which data offer enough of an empirical boost to be

worth retrieving, while the application questions center

on how to best capture those data. The point is to find

nursing data capable of contributing to the complex work

of addressing the confounding of severity of illness with

the probabilities of receiving treatment and having particular outcomes.

D. Analysis: Clinical Strategies

The purpose for including nursing inputs in the analysis

is to have a more complete view of what actually happens, and works well, in treating a particular condition.

Once nursing inputs are specified, their direct and indirect effects must be demonstrated in relation to the variables already used in patient outcomes research, such as

treatment/procedure (occurs/does not) and outcome (mortality, morbidity, disability).

Data about nursing practice could increase the ability to

explain variance in the patient outcomes. However, the

data must be reconciled to the level of aggregation and

measures used in outcomes research. Scales should be

concise, and indices must separate patients into distinct

groups. Accuracy, sensitivity and specificity of measures

should be quantified.

Using nursing data to refine definitions of clinical strategies, treatments and outcomes is an effort to define the

nursing inputs into clinical strategies and to quantify their

relative power in producing variations in treatments and

outcomes.

For example, patients experience a number of events between admission and discharge. Some events, such as

surgeries, are already factored into patient outcomes research. Some events, particularly temporary problems

such as postoperative fever, have not yet been included.

If those events are related to the probability of treatment

and/or outcomes, then it may be necessary to include them

in the analysis. Figure 5 continues the example of the

groups with corrected mortality. Here, both subgroups

are exposed to the risk of having a temporary diagnosis.

Of those who have it, the ones who also receive a nursing

input then go on to receive only half a dose of another

treatment.

In this hypothetical example, analysis could determine

the relationship between the dose of treatment and mortality. If the relationship is strong enough, the interim

mechanisms—inlcuding the nursing input—merit closer

analysis. Even if the results demonstrate that the nursing

input is a significant mechanism, it may not be necessary

to add that input to the database. If, as shown in this

example, the input always occurs in the presence of a half

dose of the treatment, then the treatment dosage may serve

as a proxy measure for the nursing input.

Improving the links to data from and about nursing practice would allow a more complete view of hospital care.

Procedures, morbidity and mortality would retain key

roles, but they could also be related to other key elements
of care, such as risks at admission, recovery from admitting diagnoses, discharge planning and perhaps even wellbeing after discharge. There are nursing inputs into each

of the elements (Figure 6). Improving the link between

the outcomes agenda and data about nursing practice requires specifying the nursing inputs, and reconciling them

to the sampling methods, measures and level of analysis

used in patient outcomes research.

The effects of nursing inputs may be largest for outcomes

that occur during the course of hospitalization or after

discharge, in contrast to the medical diagnosis at admission or discharge (Lang & Merek, 1990). On the other

hand, patient outcomes research often measures admission diagnosis, the occurrence of a treatment or procedure, the discharge diagnosis, and mortality at discharge.

However, research focused on both direct and indirect

effects of nursing inputs could link the two empirical

tracks.

This places a particularly heavy empirical demand on

hypotheses involving outcomes occurring after hospital

discharge. Despite the sensibility of defining effectiveness in terms of the long term outcomes, most databases

cannot fully support that framework—and few policy

applications currently cross the boundary between hospital and home.

In summary, the existing body of work by nursing is significant for the agenda of patient outcomes research but

does not yet answer the pivotal questions: what are the

nursing inputs to patient outcomes? What really goes

on? What works? How are nursing inputs related to the

established variables in patient outcomes research? Sensitivity and specificity of patient outcomes measures have

been identified as empirical problems for the field (Cleary,

1990; Institute of Medicine Core Committee, 1990, Jenks,

1990). The combination of nursing admission data about

severity and data about the effects (direct and indirect) of

nursing inputs have the potential to reduce those problems. The potential can be realized if nursing admission

data and nursing inputs are used as explanatory variables—

they contribute less to understanding patient outcomes if

they become a parallel field of study.

III. Evaluation And Extension

The field of patient outcomes research is concerned with

creating the knowledge base for decisions, especially decisions about clinical practice and policies. Collecting

outcome data is one of the simplest steps; the real challenge lies in establishing decision-making processes capable of using those data (Ware, 1990). Therefore, the

caliber of the knowledge base as a premise for

decisionmaking becomes a central issue.

A. Knowledge Base And Clinical Decisions

Applications to practice include guidelines and monitoring. In both instances, the purpose is to improve the general level of standard practice (Lohr & Schroeder, 1990;

Roper, 1988). In fact, Ellwood (1988) envisioned a national system for outcomes management as a technology

based on guidelines and patient outcomes information

routinely collected and used by all providers. Brook &

McGlynn (1991) sketched a system of toll-free telephone

numbers to be used by beneficiaries to call HCFA and

obtain the relative risks of procedures their providers have

suggested.

Applications to clinical decision-making will include a

range of activities. The aim is to inform providers and

improve their ability to consider research findings when

making individual treatment decisions. This aim invokes

a number of issues related to applying knowledge from

research aggregates to individuals. The issues are familiar to researchers, but may not be so to providers. Thus,
one of the essential steps in fostering application of the

outcomes knowledge base is to help clinicians avoid errors in reasoning and logic (Mulley, 1990).

The scope of the knowledge base, and explicit analysis of

it, can affect the dissemination of findings. Ambiguity

in objectives and uncertainty about applications are significant sources of resistance, as are concerns that the rational basis for decision-making may be faulty (Berwick,

1989b; Epstein, 1990).

Clinical decisions would benefit from a knowledge base

that included analyses of the strategies used by providers

and patients, particularly if all elements of the strategies

were examined. Whether or not the presence/absence of

a procedure in the hospital discharge dataset can serve as

a proxy for complete strategies is a research question. It

is also important to find some way of assessing how well

a successful clinical strategy is specified. Are all of the

elements identified and tested? Data from nursing practice could offer a database about the elements of care,

that is, the components of strategies for combining the

processes, providers and recipients of care.

B. Knowledge Base And Policy Analyses

Exceptional as it is, outcomes research will produce

knowledge with specific limits in terms of scope and

generalizability. Decisions based on that knowledge will

be better informed if they are made with a clear view of

those limits and the implications for policy analysis.

Whenever policy decisions are made at the interface of

facts and values, those decisions are likely to be better if

the societal aspects of them are given serious consideration early on (Nightingale & Meister, 1987).

Policies based on outcomes research could address the

economic and institutional barriers to improving outcomes. They can reinforce or change delivery patterns,

such as the balance of authority for managing health care;

they might also affect the incentives for investing in quality improvements (Giegel & Jones, 1990). In short, policies emerging from outcomes rseearch can reach all levels of the care system and profoundly affect factors such

as efficiency of clinical care, accountability, allocation of

resources, internal policies, functional differentiation of

institutions, pricing, payment and competition

(Donabedian, 1989).

However, despite the scope of impact, outcomes research

is an emerging field and some fundamental questions are

as yet unresolved. For example, there is disagreement

over the fundamental question of whether or not consumers are capable of reporting or rating the quality of their

health care (Davies & Ware, 1988). There is no consensus regarding whether or not the constraints of resources

should be considered in the analysis of the quality of health

care (Lohr, 1991; Palmer, 1991). Experts differ in their

assessments of the magnitude and direction of effects of

outcomes research on health care costs (Aaron, 1990;

Epstein, 1990; Fleming, 1991; Lohr & Schroeder, 1990).

A formal, but pragmatic, evaluation could relate the assets and liabilities of the knowledge base for clinical decisions to policy development. If viewed as a research

issue by NCNR, evaluation studies could fit within the

Nursing Systems branch. The strategic questions for nursing include whether the evaluation can, and should, be

extended.

A number of models have been developed to assess the

logical quality of a knowledge base. For example, Rossi

and Freeman’s (1989) Impact Model can be used to develop the evaluation criteria. The model is based on causal,

intervention and action hypotheses. Those hypotheses

could also be used to identify relationships that should be

specified in the conceptual framework, design and measures of patient outcomes research. The quality of the

definitions of those causal, intervention and action rela-
tionships could then be evaluated. The evaluation should

go further than to identify shortcomings, i.e. places where

nursing practice was not included. It should produce a

scholarly definition of the nursing inputs, as well as empirical evidence of the utility to be gained by including

them.

A knowledge base may also be evaluated in terms of how

well it supports its intended applications. In this case, the

aims include enlightening policies to improve the general

quality of care. Therefore, the knowledge base could be

evaluated in terms of how well it incorporates societal

aspects of health policy. Nightingale and Meister (1987)

illustrated a model for this kind of evaluation, using prenatal screening as the example and well-being, autonomy,

and equity as the societal aspects. They evaluated the

quality of knowledge produced by cost-effectiveness

analysis, decision analysis and technology assessment in

terms of the caliber of the scientific findings and what

those findings contributed to understanding the societal

aspects of policies in the field.

Similar work could be done for patient outcomes. The

scientific elements of the knowledge base could be evaluated in terms of meta-analysis, decision analysis/models,

outcomes assessment, cost of care, dissemination and

evaluation of dissemination, and use of administrative

databases. In fact, AHCPR has six inter-PORT

workgroups focused on these elements.

IV. Policy And Timing

Patient outcomes research has direct applicability to policies about practice and payment systems. Those policy

effects are likely to occur in the near term, and research

efforts would have to employ designs that could be

launched and completed in a timely manner.

Work in this field can also contribute to health care reform. Since the development of system-level reforms is

likely to take longer, research efforts could employ designs such as large randomized trials and still be completed in time.

If the links to data about nursing practice are improved,

they will enable interactive and synergistic research and

demonstration projects. Timing becomes important, and

the strategic question for nursing becomes one of estimating when nursing research will be completed, as a

way of identifying which policies can be affected.

A. Near-Term Policies

A substantial portion of the urgency surrounding patient

outcomes research is related to the need for information

to guide policies affecting practice and payment. Policies based on patient outcomes research and affecting

practice and payment are likely to be incremental and

emerge in the next few years (Griner, 1990; McCormick,

1991; O’Connor, 1990). Research efforts aimed at affecting those policies will have to operate on the same,

relatively short, timeline.

The PORT projects funded by the Agency for Health Care

Policy and Research illustrate designs intended to affect

practice and policy. A PORT is designed to evaluate the

effectiveness and cost-effectiveness of alternative services

or procedures.

AHCPR identifies four components in the design for

PORT projects: 1) review and formal synthesis/

metanalysis of the literature to refine hypotheses, specify

the data, and finalize instruments, measures and procedures, 2) collect and analyze data to explain variations in

practice and outcomes; use secondary data and supplement with primary data, 3) disseminate findings and rec-
ommendations to providers and public, and 4) evaluate

effects of dissemination on reducing unwarranted variations in clinical practice (AHCPR, 1990).

As of September 1990, the PORT projects focused on

outcomes in back pain (surgical, nonsurgical treatment),

arthritis (total knee replacement, nonsurgical treatment),

acute myocardial infarction (variations in treatments),

cataracts (variations in surgical treatment), therapies for

benign prostatic hypertrophy and localized prostate cancer, chronic ischemic heart disease (diagnosis, treatment),

biliary disease (surgical, nonsurgical treatment), hip fracture and osteoarthritis (treatment patterns), diabetes (treatment patterns), pneumonia (treatment patterns), cesarean

section (diagnostic and practice patterns) (AHCPR, 1991).

 Most of these projects are focused on conditions and treatments affecting a substantial percentage of significant

populations, such as all adults or Medicare beneficiaries.

Most make use of administrative data, primarily Medicare claims data. Some add specialized survey data, information from private insurance, chart reviews, vital statistics and hospital data. Two projects also add prospective data; acute myocardial infarction project has an inception cohort and pneumonia project has a prospective

cohort. For the most part, the measures are selected to

allow analysis of variations using at least hospitals, if not

geographical areas, as the unit of analysis (see AHCPR,

1991).

Together, the PORT projects and the practice guidelines

will have a major influence on clinical decision-making

and policies guiding practice. Payors will be able to make

use of the results as well, in establishing standards for

services and payment—and thus, affecting access to care.

Data from and about nursing practice can improve the

quantification of the risks patients bring to treatment, the

elements in care the affect the probability of receiving

treatment or having outcomes. In sum, these data could

provide unique, incisive information about the nursing

inputs and the casemix. If timely, these findings could

make a significant contribution to policy analyses of practice guidleines and payment systems.

The science policy questions include: how would nursing data help to quantify the risks patients bring to the

treatment? How would nursing data help to quantify elements in the process of care that affect the probabilities

of treatments and/or outcomes? Would data from nursing practice would offer unique and incisive information

about inputs and casemix? If so, and if it could be marshalled within the topics and timeframes of the PORT

projects, and other patient outcome projects, it would contribute to the knowledge base for policies affecting practice and payment.

B. Long-term Policies

Patient outcomes research can identify what works. Using policy to leverage successful strategies will also require decisions about the organization and financing of

services.

Extensions to policy are difficult in patient outcomes research because it is necessary to be confident that changes

in health status are related to antecedant care (Donabedian,

1988b). Randomized clinical trials, while certainly the

“gold standard”, cannot be conducted for every test, procedure and therapy. If the scope of observational studies

is expanded to include large numbers of patients and providers, those studies can offer substantial information

(Greenfield, 1989). However, some questions require

controlled experiments—and, the timing of health care

reform will accomodate some of them.

Newhouse (1990) discusses the critical role of rigor in

longitudinal, controlled experiments to support policies

and system reforms. For example, the experiment must
meet the standards of randomized trials so that the findings can be used with confidence even if they differ from

observational studies.

Newhouse also points out the advantages and drawbacks

of controlled experiments. They are costly, although not

necessarily much more so than prospective observational

studies. They take time, because they are prospective.

Therefore, they require the funding agency to concentrate significant resources within a single project, and if

that project is large enough to produce major advances in

knowledge, to invest those resources for significant

amount of time. On the other hand, if the design and

execution are done well (with an interdisciplinary team)

and the project is focused on an enduring questions, then

the benefits in terms of the caliber of knowledge far outweigh the costs.

Blumenthal’s (1991) analysis of the timing and course of

health care reform establishes comprehensive reform as

being at least 10 years away. Reform is not likely until a

substantial portion of the middle class finds it impossible

to obtain health insurance and health care costs become a

palpable threat to economic well-being of the country.

Blumenthal estimates that a decade or so will pass while

the crisis builds, and meanwhile, health care costs will

increase to about 15% of the gross national product. In

that timeframe, research for policies related to health care

reform could make use of a wider range of designs. This

is an important opportunity, particularly in light of the

difficulties associated with designing health services research that goes beyond analytic description to policy

prescriptions (Ginzberg, 1991a, 1991b). At the same time,

with the adoption of FASB 106 scheduled in 1993, the

business community may well see a shorter timeframe

for some reforms.

In either case, outcomes research can affect the nature of

policies, particularly if it addresses the three components

of policy development: knowledge base, political will,

and social strategy (Richmond & Kotelchuck, 1983).

Obviously, research contributes to the knowledge base.

Linking the knowledge base to nursing practice can also

contribute unique information to the other two components. The nature of nursing practice lends unique expertise in how policies work or don’t work in the real world

(strategies), as well as in what people need and want from

health care, and for whom (political will).

If one of the aims of nursing research is to affect health

system reform, then there are several science policy issues to address. First, does nursing have a cadre of researchers with the experience to design and conduct a

large controlled experiment—or would it be wiser to work

on developing an interdisciplinary team that includes nurse

researchers? In their reviews of studies of the quality of

nursing care, Lang and Clinton (1983, 1984) found that

the field needed more rigor, more comparative analyses,

more study of the relationships between intervention and

outcome, and more study of the effect of structure on those

relationships. More recently, Fetter et al (1989) addressed

nursing’s relative unfamiliarity with the methodological

and feasiblity issues of randomized clinical trials. Science policy, for example, could foster the use of interdisciplinary teams as an expedient choice.

Second, which enduring questions involve patient outcomes, nursing data and nursing inputs—and what are

the policy applications of the answers? This complex

issue is especially difficult when gaps occur between perspectives and technical ability. For example, a conference of nursing experts recently failed to reach a consensus about whether the quality of nursing care is a national

issue (Aiken, 1989). On the other hand, they recommended more emphasis on multisite work as well as more
studies of the use of clinical and nursing intensity data as

the basis for payment (Hegyvary, 1989). Science policy

could bring empirical questions into focus and clarify their

articulations with methods issues.

The implications of research designed to affect health care

reform include the costs outlined by Newhouse, as well

as the benefits. The magnitude and duration of controlled

experiments can affect the whole portfolio of the funding

agency. If NCNR determined that this kind of design fit

within its science policy, the implications for its portfolio

could be significant.

V. Developmental Perspective

Some highlights of the development of outcomes research

are listed here, as part of the context for NCNR’s science

policy.

Nightingale’s severity-adjusted mortality tables for the

Queen of England locate the history of patient outcomes

and effectiveness research well over a century ago—followed by the work of Groves, an English surgeon who

used standardized mortality data in the early 1900s (Brook

& McGlynn, 1991). Codman, a surgeon at Massachusetts General Hospital, devoted his work to “the end result” and convincing hospitals to begin routine surveillance of every patient’s post-discharge course, and reporting to the public (Berwick, 1989b; Donabedian, 1989;

Mulley, 1989). Donabedian’s explication of the concept

of quality of care infused evaluation with renewed purpose in the late 1960s. At the same time, the era of expansion in health care was winding down, and the era of

cost containment was dawning (Relman, 1988). In the

late 1970s, the concept of “sentinel events” in public health

was defined in relation to unnecessary disease, disability,

and untimely death (Rutstein, et al, 1976). The revival of

interest in the field has made use of those negative indexes for the aggregate, while attempting to develop positive indexes as well.

Relman (1988) observed the beginnings of the effectiveness initiative and noted that it marked the dawning of

the age of “assessment and accountability”. This initiative has been framed as “simply, what works?” (Clinton,

1990). Since then, a substantial handful of PORT projects

and nearly 50 effectiveness studies have been launched.

Ambitious projects in measurement and design continue.

This is an international field. For example, the Royal

College of Nursing in England is operating a multifaceted standards of care project designed to improve quality. Major controlled experiments for policy, such as

Newhouse’s Health Insurance Experiment, stand in the

midst of these developments.

Looking ahead, features of the American health care system are likely to include: expansions in workplace-based

private insurance to cover most employed Americans and

their dependants, a strong role for fee-for-service practice and free choice of providers, a reliance on the States

to administer both private and public insurance, and some

discretion for the States in cost containment and establishing standards of care (Blumenthal, 1991). Pressure

for reform may rise sharply with the implementation of

FASB 106 in 1993, and the direct effects of retiree medical benefits on corporate earnings and net worth, as well

as stock values to shareholders.

Nurse researchers have already created and tested some

links between outcomes and nursing practice. The question is how to improve the link. Several strategic actions

have been discussed here, and the implications for individual researchers, academic nursing, clinical nursing, and

NCNR may be different.
VI. Conclusions

1. Improving the link to nursing practice requires specifying nursing data and inputs in relation to the samples,

measures and analytic models used in outcomes research

and creating simple ways of obtaining the pertinent data.

2. All studies of patient outcomes should address the limits of scope, rigor and generalizability. The field would

benefit from a formal, but pragmatic, evaluation of the

assets and liabilities of the knowledge base for clinical

decisions and policy analysis.

3. Outcomes research must include some controlled experimental studies. Those studies should be

multidisciplinary, of sterling quality, and focused on enduring policy questions.

This research will go on for decades and results will accrue slowly. The research will lead, in some cases, to

less care or more care, and probably in a large number of

cases, to different care that may be roughly as costly as

what we have now. Thus, rather than saving money, this

field first promises to improve quality. (Aaron, 1990) The

government’s role is to call the public’s attention to this

work, set priorities, commit unique resources (databases),

fund research, develop partnerships to carry out this work

(Roper, 1990). The challenge to scientists includes setting priorities to advise Congress (Rockefeller, 1990), as

well as sharpening the focus of the knowledge base and

expanding the basis for policy devlopment.

Linking patient outcomes research to nursing practice

could contribute to both aims by creating an incisive database about severity and casemix, and by establishing

the methods for relating specific nursing inputs to the field

of outcomes research. Eventually, those links would affect decisions about policies, which in turn, affect patient

outcomes and access to nursing care. Other important

effects might also occur—this paper focuses primarily on

those that would affect the knolwedge base for practice

and policy.

This paper described a strategic framework defined by

aims for knowledge, decisions and applications. Today,

the focus is on implications for NCNR science policy and

perspective is important. Outcomes research is not a silver bullet for our health care dilemmas. In fact, some

argue that the strength of outcomes reserch lies as much

in the pre-existing vacuum as in its science (Mulley, 1990).

Nursing has some experience with this field, although

probably not enough and too much of it in parallel. There

is some real risk of failure in entering this field, although

tolerating the risk of failure is necessary to success in science (Shapiro, 1990 ). Beginning with where we are,

sound science policy—designed to shape science and

guide decisions affecting access to nursing care—seems

vital. Berwick (1989a) points out that issues of leadership, financing and conduct of health services research

vary with the aggregate to be affected by the work. NCNR

has a leadership role of national significance, particularly

in relation to the community of nurse researchers.

References

Aaron, H. J. (1990) The need for reasonable expectations.

In K.A. Heithoff & Lohr, K.N. (Eds.), Effectiveness and

outcomes in health care. (pp. 215-217) Washington D.C.:

National Academy Press.

Agency for Health Care Policy and Research (1991)

Medical treatment effectiveness porgram (MEDTEP) active research projects as of September 30, 1990.

Rockville: Agency for Health Care Policy and Research,

Parklawn Building, Room 18-12, Rockville, Maryland.
Agency for Health Care Policy and Research (1990) Program note: Medical treatment effectiveness research.

Rockville: Agency for Health Care Policy and Research,

Parklawn Building, Room 18-12, Rockville, Maryland.

Aiken, L. (1989) Summary of Day 1. In Resources and

the delivery of patient care (Invitiational Conference). (pp.

27-28) National Center for Nursing Research, NIH Pub.

No. 89-3008.

Berwick, D. M. (1989a) Health services research and

quality of care: Assignments for the 1990s. Medical Care,

27, 763-771.

Berwick, D. M. (1989b) E.A. Codman and the rhetoric of

battle: A commentary. The Milbank Quarterly, 67, 262-

267.

Blumenthal, D. (1991) The timing and course of health

care reform. New England Journal of Medicine, 325, 198-

200.

Brook, R. H. & Kosecoff, J. B. (1988) Competition and

quality, Health Affairs, 7, 150-161.

Brook, R. H. & McGlynn, E. A. (1991) Maintaining quality of care. In E. Ginzberg (Ed.), Health services research:

Key to health policy. (pp. 284-314) Cambridge: Harvard

University Press.

Cleary, P. D. (1990) Using patient reports of outcomes to

assess effectiveness of medical care. In K.A. Heithoff &

Lohr, K.N. (Eds.), Effectiveness and outcomes in health

care. (pp. 152-159) Washington D.C.: National Academy Press.

Clinton, J. J. (1990) Research on the effectiveness of

medical treatment: new challenges and opportunities. In

K.A. Heithoff & Lohr, K.N. (Eds.), Effectiveness and

outcomes in health care. (pp. 21-26) Washington D.C.:

National Academy Press.

Davies, A. R. & Ware, J. E. (1988) Involving consumers

in quality of care assessment. Health Affairs, 7, 33-48.

Donabedian, A. (1966) Evaluating the quality of medical

care. Milbank Memorial Fund Quarterly, 44, 166-203.

Donabedian, A. (1982) The criteria and standards of quality. Ann Arbor: Health Administration Press.

Donabedian, A. (1988a) The quality of care—How can it

be assessed? Journal of the American Medical Association, 260, 1743-1748.

Donabedian, A. (1988b) Quality assessment and assurance: Unity of purpose, diversity of means. Inquiry, 25,

173-192.

Donabedian, A. (1989) The end result of health care:

Ernest Codman’s contribution to quality assessment and

beyond. The Milbank Quarterly, 67, 233-256.

Donabedian, A. (1991) Reflections on the effectiveness

of quality assurance. In R. H. Palmer, A. Donabedian, G.

J. Povar (Eds.) Striving for quality in health care: An inquiry into policy and practice. (pp. 59-128). Ann Arbor:

Health Administration Press.

Ellwood, P. (1988) Outcomes management: A technology of patient experience. New England Journal of Medicine, 318, 1549-1556.

Epstein, A. M. (1990) The outcomes movement: Will it

get us where we want to go? New England Journal of

Medicine, 323, 266-270.

Fetter, M. S., Feetham, S.L., D’Apolito, K., Chaze, B.A.,

Fink, A., Frink, B. B., Hougart, M. K., & Rushton, C. H.

(1989) Randomized clinical trials: Issues for researchers.

Nursing Research, 38, 117-120.

Fleming, S. T. (1991) The relationship between quality

and cost: Pure and simple? Inquiry, 28, 29-38.
Geigle, R. & Jones, S.B. (1990) Outcomes measurement:

A report from the front. Inquiry, 27, 7-13.

Ginzberg, E. (1991A) Health services research and health

policy. In E. Ginzberg (Ed.), Health services research:

Key to health policy. (pp. 1-19) Cambridge: Harvard

University Press.

Ginzberg, E. (1991B) The challenges ahead. In E.

Ginzberg (Ed.), Health services research: Key to health

policy. (pp. 315-332) Cambridge: Harvard University

Press.

Greenfield, S. (1989) The state of outcome research: Are

we on target? New England Journal of Medicine, 320,

1142-1143.

Griner, P. F. (1990) The clinical perspective. In K.A.

Heithoff & Lohr, K.N. (Eds.), Effectiveness and outcomes

in health care. (pp. 38-43) Washington D.C.: National

Academy Press.

Hegyvary, S.T. (1989) Panel III: Research strategies for

maintaining quality in the delivery of patient care. In

Nursing Resources and the delivery of patient care

(Invitiational Conference).(pp. 19-20) National Center

for Nursing Research, NIH Pub. No. 89-3008.

Horn, B. J. & Swain, M.P. (1978) Criterion measures of

nursing care quality: NCHSR Research Summary Series.

DHEW Pub. No. (PHS) 78-3187.

Institute of Medicine Core Committee (1990) Promise and

limiations of effectiveness and outcomes research. In K.A.

Heithoff & Lohr, K.N. (Eds.), Effectiveness and outcomes

in health care. (pp.8-20) Washington D.C.: National

Academy Press.

Jenks, S. F. (1990) Issues in the use of large data bases

for effectiveness research. In K.A. Heithoff & Lohr, K.N.

(Eds.), Effectiveness and outcomes in health care. (pp.

94-104) Washington D.C.: National Academy Press.

Jenks, S. F., Daley, J., Draper, D., Thomas, N., Lenhart,

G., & Walker, J. (1988) Interpreting hospital mortality

data: The role of clinical risk adjustment. Journal of the

American Medical Association, 260, 3611-3616.

Lang, N. M. & Clinton, J. F. (1983) Assessment and assurance of the quality of nursing care. (pp. 211-231) In

Evaluation and the health professions. Beverly Hills:

Sage.

Lang, N. M. & Clinton, J. C. (1984) Assessment of quality of nursing care. In H. H. Werley and J. J. Fitzpatrick

(Eds.) Annual review of nursing research. (pp. 135-163)

New York: Springer.

Lang, N. M. & Merek, K. D. (1990) The classification of

patient outcomes. Journal of Professional Nursing, 6, 158-

163.

Lohr, K. N. (1988) Outcome measurement: Concepts and

questions. Inquiry, 25, 37-50.

Lohr, K. N. (1991) Quality of health care: An introduction to critical definitions, concepts, principles and

practicalities. (pp. xi-xxi). In R. H. Palmer, A.

Donabedian, G. J. Povar (Eds.) Striving for quality in

health care: An inquiry into policy and practice. Ann

Arbor: Health Administration Press.

Lohr, K. N. & Schroeder, S.A. (1990) A strategy for quality assurance in Medicare. New England Journal of Medicine, 322, 707-712.

Luft, H. S. & Hunt, S. S. (1986) Evaluating individual

hospital quality through outcome statistics. Journal of

the American Medical Association. 255, 2780-2784.
McCormick, K. A. (1991) Future data needs for quality

of care monitoring: DRG considerations, reimbursement

and outcome measurement. Image, 22, 29-32.

McNeil, B.J. (1990) Claims data and effectiveness: Acute

myocardial infarction and other examples. In K.A.

Heithoff & Lohr, K.N. (Eds.), Effectiveness and outcomes

in health care. (pp. 65-70) Washington D.C.: National

Academy Press.

Meister, S. B. (1989) Family health policy: A perspective

on its development and trends for the future. In L. G.

Krentz (Ed.) Proceedings of the National Conference on

Family Nursing (pp. 65-76). Portland, Oregon: Oregon

Health Sciences University.

Moritz, P. (1991) Innovative nursing practice models and

patient outcomes. Nursing Outlook, 39, 111-114.

Mulley, A.G. (1989) E.A. Codman and the end result idea:

A commentary. The Milbank Quarterly, 67, 257-261.

Mulley, A. G. (1990) Applying effectiveness and outcomes research to clinical practice. In K.A. Heithoff &

Lohr, K.N. (Eds.), Effectiveness and outcomes in health

care. (pp. 179-189) Washington D.C.: National Academy Press.

Newhouse, J. P. (1991) Controlled experimentation as

research policy. In E. Ginzberg (Ed.), Health services

research: Key to health policy. (pp. 161-194) Cambridge:

Harvard University Press.

Nightingale, E. O. & Meister, S. B. (1987) Prenatal screening, policies and values. Boston: Harvard University.

O’Connor, K. S. (1990) Effectiveness and outcomes of

health care services: Implications for nursing. (Final Report on a conference funded by AHCPR) Kansas City:

American Nurses’ Association.

Palmer, R. H. (1991) Considerations in defining quality

of health care. In R. H. Palmer, A. Donabedian, G. J.

Povar (Eds.) Striving for quality in health care: An inquiry into policy and practice. (pp. 1-58). Ann Arbor:

Health Administration Press.

Reinhart, U. E. (1990) The social perspective. In K.A.

Heithoff & Lohr, K.N. (Eds.), Effectiveness and outcomes

in health care. (pp. 34-37) Washington D.C.: National

Academy Press.

Relman, A. S. (1988) Assessment and accountability: The

third revolution on medical care. New England Journal

of Medicine, 319, 1220-1222.

Richmond, J.B. & Kotelchuck, M. L. (1983) Political

influences: Rethinking national health policy. In McGuire

(ed.) Handbook on health professions education. San

Francisco: Jossey-Bass.

Rockefeller, J. D. (1990) The legislative perspective. In

K.A. Heithoff & Lohr, K.N. (Eds.), Effectiveness and

outcomes in health care. (pp. 44-48) Washington D.C.:

National Academy Press.

Roper, W. L. (1990) The effectiveness initiative: Retrospective and prospects. In K.A. Heithoff & Lohr, K.N.

(Eds.), Effectiveness and outcomes in health care. (pp.

31-33) Washington D.C.: National Academy Press.

Roper, W. L., Winkenwerder, W., Hackbarth, G.M., &

Krakauer, H. (1988) Effectiveness in health care: An initiative to evaluate and improve medical practice. New

England Journal of Medicine, 319, 1197-1202.

Rossi, P. H. & Freeman, H. E. (1989) Evaluation: A systematic approach. Beverly Hills: Sage.
Rutstein, D. D., Berenberg, W., Chalmers, T. C., Child,

C. G., Fishman, A. P. & Perrin, E. B. (1976) Measuring

the quality of medical care. New England Journal of

Medicine, 294, 582-588.

Schroeder, S. A. (1987) Outcome assessment 70 years

later: Are we ready? New England Journal of Medicine,

316, 160-162.

Shapiro, H. T. (1990) The willingness to risk failure. Science, 250, 609.

Siu, A. L., McGlynn, E. A., Morgenstern, H. & Brook,

R. H. (1991) A fair approach to comparing quality of care.

Health Affairs, 10, 62-75.

Sox, H. C. (1990) Effectiveness research and changing

physician practice patterns. In K.A. Heithoff & Lohr, K.N.

(Eds.), Effectiveness and outcomes in health care. (pp.

173-178) Washington D.C.: National Academy Press.

Strickland, O. L. & Waltz, C. F. (1988) Measurement of

nursing performance, practice, education and research

(Volume 2). New York: Springer.

Tarlov, A. R., Ware, J. E., Greenfield, S., Nelson, E. C.,

Perrin, E., & Zubkoff, M. (1989) The medical outcomes

study—An application of methods for monitoring the results of medical care. Journal of the American Medical

Society, 262, 925-930.

Ware, J. E. (1990) Measuring patient function and wellbeing: Some lessons from the Medical Outcomes Study.

In K.A. Heithoff & Lohr, K.N. (Eds.), Effectiveness and

outcomes in health care. (pp. 107-119) Washington D.C.:

National Academy Press.