This paper examines the state of knowledge

regarding the effects of nursing practice models on patient outcomes. Many hospitals, and to some extent other

care facilities, are experimenting with innovative models of nursing practice, and some of these models are

the subject of evaluation research. This paper will attempt to summarize what is known from studies in which

the practice model is the independent variable, an element of the “structure” of health care settings

(Donabedian, 1988), and patient outcomes are dependent variables.

The paper will present an overview of innovative nursing practice models; will summarize the types

of patient outcome measures employed in studies of these

models and present some examples of findings; will discuss some of the methodological issues that arise in this

type of research; and will suggest future research needs

in this area.

Since many models are still in the formative

stages and evaluations are ongoing, much of the evaluation research on practice models has not yet been reported. Nevertheless, the studies represent a significant

departure from earlier research on patient outcomes.

Previous research on patient outcomes rarely had included variables representing the organization of nursing services. Hospital studies frequently considered attributes of medical staff but neglected attributes of nursing staff. Figure 1 provides some information on four

studies that did consider aspects of nursing service organization as predictors of mortality among surgical

patients, intensive care patients, and Medicare patients.

In these studies, the measures of nursing organization

were somewhat crude and did not reflect practice model

issues.

Overview of Nursing Practice Models

For purposes of this paper, the term “nursing

practice model” refers to operational models for redesigning nursing practice for the provision of patient care

in organizational settings, primarily hospitals and

longterm care facilities. Though the models may be

implemented organization-wide, they typically redesign

nursing practice at the point of patient care delivery, that

is, at the nursing unit level. Thus the models are distinguished from organization-level management innovations (such as clinical advancement programs or innovative pay systems) that do not specifically address care

delivery.

Nursing practice models are innovative practice arrangements that differ from traditional models on

one or more of the following structural dimensions:

(1) The degree to which the practice of individual nurses is differentiated according to education level or performance competencies;

(2) The degree to which nursing practice at the

unit level is self-managed, rather than managed

by traditional supervisors;

Nursing Practice Models: Research on Patient Outcomes1

Carol S. Weisman, Ph.D.
(3) The degree to which case management is

employed; and

(4) The degree to which “teams” (either nursing or multidisciplinary) are employed.

Many practice models contain more than one of these

elements and also include elements of primary nursing.

A recent compendium of innovative nursing

practice models (Mayer, Madden, and Lawrenz, 1990)

includes the selected examples displayed in Figure 2.

These models, which have been evaluated to varying

degrees, represent a wide variety of structural approaches

to reorganizing nursing practice. Some have been motivated primarily by the need to address nurse staffing

shortages, whereas others have been motivated primarily by the need to contain costs. Cost savings might be

achieved through better coordinated care, through use

of non-RN providers, or through reductions in turnover

and replacement costs. Most of these models have been

developed for, and field tested on, a small number of

nursing units in one hospital, rather than hospital-wide

or in multiple hospitals.

In addition to the models already described in

the published literature, there are some ongoing evaluations of innovative models. Two research demonstration projects funded by the National Center for Nursing

Research (NCNR) and the Division of Nursing are underway in New York and Arizona. The University of

Rochester School of Nursing is implementing and evaluating an Enhanced Professional Practice Model for Nursing, designed to increase nurses’ control over practice

at the unit level and to provide professional compensation. The evaluation design includes five hospitals, experimental and control units, and a pretest-posttest design. Patient outcomes being studied include patients’

perceptions of the hospital experience, morbidity and

mortality, and unplanned hospital readmission up to 30

days post-discharge. The University of Arizona

College of Nursing is implementing and evaluating a

unit-based Differentiated Group Professional Practice

Model that includes three components: group governance

(including participative management, staff bylaws, peer

review, and professional salary structure); differentiated

care delivery (including differentiated RN practice, use

of nurse extenders, and primary case management); and

shared values (including a culture-building process that

values quality of care, intrapreneurship, and recognition

for excellence in practice). The evaluation design includes three hospitals, demonstration and comparison

units, and a 36-month followup. Quality of care outcomes include complications, medication errors, infections, and chart audits.

NCNR is also funding the evaluation of The

Johns Hopkins Professional Practice Model. That model

consists of a contract between a unit’s registered nurses

and the hospital in which the nurses agree to provide

24-hour patient care on the unit for one year in exchange

for unit self-management (including peer review, selfscheduling, and quality assurance), salaried compensation, and shared savings if the unit contains its costs.

The evaluation design is posttest only and includes 16

professional practice units and 8 comparison units

matched for gross clinical area and unit size. The Professional Practice Model units include units in neurology, psychiatry, pediatrics, general surgery and several

surgical specialties, two general operating rooms and

labor and delivery. Process variables and patient outcomes studied include in-hospital mortality, medication

errors, falls, length of stay, patient satisfaction with nursing care measured on the day of discharge, post-discharge perceived health status, unmet needs for care

during the first two weeks after discharge, post-discharge

unplanned health services utilization, and hospital readmission within 30 days. Post-discharge outcomes are

included because the model was expected to improve

discharge planning; the two-week post-discharge period

was selected for study based on previous research on
the effects of discharge planning on patients’ needs for

care (Steinwachs et al., 1989). Our data are being analyzed now.

Twenty projects have been funded by The Robert Wood Johnson Foundation and The Pew Charitable

Trusts in their program entitled “Strengthening Hospital Nursing: A Program to Improve Patient Care.” The

implementation phase of these projects began in the fall

of 1990, so definitive descriptions of the practice models being tested are not yet available. According to the

projects’ proposals, the models range from system-wide

interventions to unit-based practice models. Further,

fourteen of the twenty proposals indicated that patient

outcomes would be measured in order to assess the impact of the models.2

 These projects are likely to produce some interesting findings with regard to patient

outcomes by 1994.

Another project with plans to collect patient

outcome data is the New Jersey Nursing Incentive Reimbursement Awards (NIRA) Program (Knickman et

al., 1991). This project is evaluating nursing innovations in 23 New Jersey hospitals. The innovations include redesigned work environments, including case

management models; shared governance structures; computerized nursing process; and educational programs to

address nurse satisfaction. The pretest-posttest evaluation (with comparison units, as possible) will include

the Hinshaw and Atwood (1982) patient satisfaction instrument and three indicators of quality of care: nosocomial infections, medication errors, and falls. The final

report on this project is planned for June, 1992.

Other current projects include studies of practice models in specialty units, such as dedicated AIDS

units, special care units for chronically critically ill patients (Daly et al., 1991), pediatric critical care units

(Murphy, Walts, and Cavouras, 1989), and intensive care

units (Phillips et al., 1990).

Patient Outcome Measures and Illustrative Findings

The motivation for creating innovative nursing

practice models has included the need to attract and retain nurses in hospital practice and the need to contain

costs (Tonges, 1989). Many of the models were designed initially to increase nurses’ job satisfaction and

retention and to produce efficiencies in care delivery.

Only more recently have researchers turned to the question of the models’ effects on patient outcomes. Hypothesized effects on patient outcomes differ by study.

In cases where the primary focus is on achieving cost

savings, the hypothesis might be that the model maintains current quality of care (i.e. does not result in poorer

patient outcomes). In cases where the primary focus is

on retaining a more experienced nursing staff or on improving care coordination or case management, the hypothesis might be that the model will improve patient

outcomes.

The types of patient outcomes being measured

in these studies vary widely and reflect the general shift

in patient outcomes research from traditional measures

of morbidity and mortality to a broader set of measures

including patient-perceived health status, post-discharge

outcomes, and patient satisfaction (Tarlov et al., 1989;

Moritz, 1991). Often, “process” indicators of quality of

care — such as length of stay, number of medication

errors, or nursing chart documentation — also are included in these studies.

The variations in practice models and in patient

outcomes, as well as the preliminary nature of many of

the studies, makes it difficult to summarize findings.

Instead, illustrative examples of findings will be presented from six published studies. (See Figure 3.)

Dear et al. (1985) reported on results of a oneyear pretest-posttest evaluation of the Professional Practice Model unit at Johns Hopkins Hospital, compared

with a unit using traditional management. The only

outcome studied that related to quality of patient care
was the nursing audit; audit scores were generally higher

for the professional practice model unit than for the comparison unit, at both baseline and followup. As previously noted, the ongoing evaluation of this practice

model includes a number of patient outcomes.

Burnes-Bolton et al. (1990) reported findings

from a pretest-posttest evaluation conducted over 10

months for one medical-surgical unit using the Cost

Containment Model at Cedars-Sinai Medical Center. Patient care planning, nursing documentation, and patient

education improved over the 10-month period; patient

satisfaction increased, primarily because patients perceived that nurses spent more time with them and their

families.

Lamb and Huggins (1990) reported on some

pilot studies of the St. Mary’s Professional Nursing Network, a case management model; the data covered a

three-year period and referred to several “tracer” diagnoses. The authors reported that client satisfaction increased over time and that length of stay decreased in

two diagnostic categories (chronic obstructive pulmonary disease and total hip replacement).

Horvath (1990) reported some data from evaluations at Beth Israel Hospital in Boston, in which patients on primary nursing units had significantly lower

stress scores than patients on team nursing units.

Brett and Tonges (1990) reported a number of

findings from an 8-month pretest-posttest pilot evaluation of one surgical orthopedic unit using the ProACT

model at Robert Wood Johnson University Hospital.

Patient satisfaction and four indicators of the quality of

nursing care were measured; the general hypothesis was

that these outcomes would be maintained or improved

as a result of the model. Patient satisfaction was measured using Risser’s Patient Satisfaction with Nursing

Care Instrument (Risser, 1975) as adapted by Hinshaw

and Atwood (1982). Satisfaction scores remained stable

over the 8-month study period. Percentage achievement

of nursing process criteria and of nursing outcome criteria, measured using quality assurance indicators, improved over the study period. Neither the number of

incidents attributable to nursing care nor the number of

nosocomial infections attributable to nursing care

changed during the study period.

Daly et al. (1991) compared patients randomized to either a special care unit for chronically critically ill patients or to traditional ICUs at University

Hospitals of Cleveland. Qualitative data suggest an increase in patient and family satisfaction for the special

unit patients. The project plans to collect data on length

of stay, complications (respiratory and infections), mortality, readmission rate, and patient and family satisfaction with care.

In sum, there is some evidence of positive effects of some models on nursing process and on patient

satisfaction. To date, there is little evidence of effects

on post-discharge outcomes.

Methodological Issues

This review of studies of the effects of nursing

practice models on patient outcomes suggests a number

of methodological issues for evaluation research in this

area. These issues pertain to the selection of appropriate patient outcomes for study; defining appropriate patient samples for study; selecting controls for patient severity or care needs; and the timing of studies using patient outcome measures. Each issue, with an example

from the Johns Hopkins evaluation, will be discussed.

Selecting Patient Outcomes for Study

Donabedian (1988) states, “As a general rule,

it is best to include in any system of assessment, elements of structure, process, and outcome.” Most of the

studies cited are following this rule in that they are including measures of nursing care process as well as

measures of patient health status and satisfaction. (How-
ever, some studies neglect measures of structure by failing to include comparison units that are not employing

the practice model or by neglecting measures of the degree to which the model has been implemented or the

length of time the unit has been using the model.)

Interestingly, though, the studies do not always

provide a rationale for the specific patient outcomes selected or for the instruments used to measure these outcomes. For example, although patient satisfaction is a

frequently measured outcome, few studies provide a

theoretical argument as to why a specific practice model

would be expected to affect patient satisfaction. Further,

although the Hinshaw and Atwood (1982) measure of

patient satisfaction is the most frequently used, it is not

clear that their scale taps the dimensions of nursing care

that would be affected by practice models and that could

be perceived by patients assessing the quality of their

care. A concern is that some studies may be “fishing”

for patient effects and may be reducing their chances of

finding effects by using instruments that may not be sensitive to the dimensions of patient care affected by practice models.

In the Johns Hopkins evaluation, we decided

that no existing patient satisfaction scales measured the

patient’s perception of nursing teamwork or coordination of care, two dimensions thought to be improved by

the Professional Practice Model. Focus groups of nurses

convened early in our study planning alerted us to these

issues. We therefore developed and pretested some items

assessing patients’ perceptions of how well the nurses

work together to coordinate their care. Also, since the

nurses reported their perception that the Professional

Practice Model improves discharge planning, our study

includes several outcomes that should reflect the impact

of discharge planning on the patient’s post-hospitalization status.

Studies need to select patient outcomes carefully so as to reflect the dimensions of care affected by

the practice model being evaluated. In some cases, instrument development may be necessary to assess outcomes of interest.

Selecting Patients for Study

Defining appropriate patient samples for study

is another important methodological issue. Many reports do not provide power computations or specify how

the patient samples were selected or how representative

the samples are of patients treated within the practice

models being evaluated. Since most studies are collecting data directly from patients (e.g. patient satisfaction

measures), issues of response rates and response biases

also need to be addressed. Most studies appear to collect data on a case series of patients in each of the nursing units studied. Some studies employ “tracer” conditions and examine outcomes only for patients with these

conditions (Donabedian, 1988). (When tracer conditions are used, outcomes are disease-specific.)

In the Johns Hopkins study, we could not use

tracer conditions due to the distribution of diagnoses in

our Professional Practice and traditional units. Instead

we conducted day-of-discharge and two-week post-discharge interviews with a case-series of approximately

40 consenting patients from each of the units. By obtaining case-mix records on all patients discharged from

the units during the study period, we are able to report

response rates by unit and to assess differences between

responding and non-responding patients on

sociodemographics, hospital LOS, principal diagnosis

category, etc.

Controls for Patient Condition

In comparing patient outcomes for practice

model units and comparison units, the comparability of

patients needs to be addressed. Comparability might be

addressed through randomization of patients to both
types of units, through matching of units or patients (e.g.

on discharge diagnoses), or through statistical controls

for patient diagnosis, severity of illness, or needs for

care at the point of entry to the study unit. Randomization of patients is rare in the literature.

In the Johns Hopkins study, we required a means

of controlling for the severity level of patients with a

wide range of diagnoses across clinical departments. In

addition, we required a measure of severity at the point

of entry to the nursing unit, prior to substantial resource

consumption on that unit, and that could be assessed,

using retrospective chart review, by a rater other than a

nurse providing care to that patient.

Our decision was to use APACHE II (Knaus et

al., 1985) to measure adult patient severity of illness in

the first 24 hours on the nursing unit, and the Pediatric

Risk of Mortality (PRISM) instrument (Pollack et al,

1988) to measure pediatric patient severity. The two

instruments are similar in that they are both physiologybased and can be scored objectively through retrospective review of the medical record. Both instruments were

developed to predict risk of mortality in patients in intensive care, but APACHE II has been used in non-ICU

settings as well (Larvin and McMahon, 1989; Daley et

al., 1988). We reviewed the charts of patients discharged

from each of our study units and grouped the scores on

these two instruments (by risk of mortality) to develop

categories of low, medium, and high severity. Using

these groupings, we were able to show that the matched

Professional Practice and traditional units are comparable with regard to patient severity. In addition, the

patient’s severity score can be used as a covariate in

analyses of patient outcomes by type of practice model.

Timing of Posttest Measures

The timing of outcome measures is another key

methodological issue. Most of the reported evaluations

of practice models have assessed effects over a short

period of time (e.g. eight months to one year following

full implementation of the models). The length of time

that a unit has been using a practice model, however,

could affect patient outcomes. Two competing hypotheses are relevant here. The “honeymoon” hypothesis

would predict that patient outcomes, if they are to be

improved by a given model, will show greater improvement early in the model’s history, rather than later, because the nurses will be more motivated during the honeymoon phase of the new model. The “settling in” hypothesis would predict that patient outcomes, if they are

to be improved by a given model, will show greater improvement later in the model’s history, after the model

has settled in, been fine tuned, and the nurses have

learned their new roles. (Conventional wisdom suggests

that it takes one year for a model to settle in.) These

issues can be addressed by measuring “time on the practice model” and examining patient outcomes in repeated

followup surveys.

In the Johns Hopkins evaluation, Professional

Practice Model units were classified as early or late

adopters of the model: early adopters had been using

the model for at least three years at the time of our study;

late adopters had been using the model for less than three

years. Our analysis will investigate whether length of

time on the practice model affects patient outcomes.

Future Research Needs

Based on this discussion, several recommendations for future research on the effects of nursing practice models on patient outcomes are made:

1. Researchers should make explicit the theoretical links

between practice model components and the patient outcomes studied. Many nursing practice models were designed primarily to affect nurses’ work environment and

retention; the possible impacts on patient outcomes need

to be conceptualized and measured.
2. Patient samples need to be representative of all patients served on the nursing units studied or of specified

subgroups of patients served (e.g. “tracer” diagnoses)

for whom critical pathways or specific outcomes can be

defined. Studies need to select an appropriate sampling

frame and address the issue of representativeness.

3. To ascertain practice model effects, patients should

either be randomized to model and non-model units,

matched by diagnosis or DRG, or classified according

to severity level at the point of entry to the unit.

4. Studies of patient outcomes need to address the “honeymoon” and “settling in” hypotheses in order to account for short-term versus longer-term impacts of practice models.

Endnotes

1. The assistance of Sandra Cassard in the preparation

of this paper is gratefully acknowledged.

2. This information was provided by Sue Taft, Associate Professor of Nursing, Kent State University, in her

role as evaluator of the Robert Wood Johnson/Pew

Charitable Trusts’ “Strengthening Hospital Nursing Program.”

References

Brett, Judy L. Luckenbill, and Mary Crabtree Tonges.

“Restructured Patient Care Delivery: Evaluation of the

ProACT Model.” Nursing Economics 8:36-44, 1990.

Burnes-Bolton, Linda, Mary Ann Daviver, Margaret M.

Voxburgh, Kathy Harrigan, Lori Urbanec, and Roxane

B. Spitzer-Lehmann. “A Cost Containment Model of

Primary Nursing at Cedars-Sinai Medical Center.” Chapter 8 in Mayer, Madden, and Lawrenz, 1990.

Daley, Jennifer, Stephen Jencks, David Draper, Gregory

Lenhart, Neal Thomas, and Janice Walker. “Predicting

Hospital-Associated Mortality for Medicare Patients: A

Method for Patients with Stroke, Pneumonia, Acute

Myocardial Infarction, and Congestive Heart Failure.”

Journal of the American Medical Association 260:3617-

3624, 1988.

Daly, Barbara J., Ellen B. Rudy, Kathryn Sabo Thompson. “Development of a Special Care Unit for Chronically Critically Ill Patients.” Heart and Lung 20:45-51,

1991.

Dear, Margaret R., Carol S. Weisman, and Sharon

O’Keefe. “Evaluation of a Contract Model for Professional Nursing Practice.” Health Care Management Review 10:65-77, 1985.

Donabedian, Avedis. “The Quality of Care: How Can

It Be Assessed?” Journal of the American Medical Association 260:1743-1748, 1988.

Flood, Ann Barry, and W. Richard Scott. Hospital Structure and Performance. Baltimore, Maryland: The Johns

Hopkins Press, 1987.

Hartz, Arthur J., Henry Krakauer, Evelyn M. Kuhn, et

al. “Hospital Characteristics and Mortality Rates.” The

New England Journal of Medicine 321:1720-1725, 1989.

Hinshaw, Ada Sue, and Jan R. Atwood. “A Patient Satisfaction Instrument: Precision by Replication.” Nursing Research 31:170-175, 1982.

Horvath, Kathy J. “Professional Nursing Practice

Model.” Chapter 13 in Mayer, Madden, and Lawrenz,

1990.
Knaus, William A., Elizabeth A. Draper, Douglas P.

Wagner, and Jack E. Zimmerman. “APACHE II: A Severity of Disease Classification System.” Critical Care

Medicine 13:818-829, 1985.

 . “An Evaluation of Outcome from Intensive

Care in Major Medical Centers.” Annals of Internal

Medicine 104:410-418, 1986.

Knickman, James, Christine Kovner, Gerry

Hendrickson, David Whittier, and Heidi Graf. “An

Evaluation of the New Jersey Nursing Incentive Reimbursement Awards Program: An Interim Report.” Health

Research Program of New York University, January

1991.

Lamb, Gerri S., and Delma Huggins. “The Professional

Nursing Network.” Chapter 10 in Mayer, Madden, and

Lawrenz, 1990.

Larvin, Michael, and Michael J. McMahon. “APACHEII Score for Assessment and Monitoring of Acute Pancreatitis.” The Lancet July 22:201-205, 1989.

Mayer, Gloria Gilbert, Mary Jane Madden, and Eunice

Lawrenz (editors). Patient Care Delivery Models.

Rockville, MD: Aspen, 1990.

Moritz, Patricia. “Innovative Nursing Practice Models

and Patient Outcomes.” Nursing Outlook 39:111-114,

1991.

Murphy, Cynthia Adams, Lynn Walts, and Carol Ann

Cavouras. “The PRN Plan: Professional Reimbursement

for Nurses.” Nursing Management 20:64Q-64X, 1989.

Phillips, Russell S., Veronica F. Rempusheski, Ann

Louise Puopolo, Mary Naccarato, and Lee Mallatratt.

“Decision Making in SUPPORT: The Role of the

Nurse.” Journal of Clinical Epidemiology 43, Supplement:555-585, 1990.

Pollack, Murray A., Urs E. Ruttimann, and Pamela R.

Getson. “Pediatric Risk of Mortality (PRISM) Score.”

Critical Care Medicine 16:1110-1116, 1988.

Risser, Nancy L. “Development of an Instrument to

Measure Patient Satisfaction with Nurses and Nursing

Care in Primary Care Settings.” Nursing Research 24:45-

52, 1975.

Shortell, Stephen M., and Edward F.X. Hughes. “The

Effects of Regulation, Competition, and Ownership on

Mortality Rates Among Hospital Inpatients.” The New

England Journal of Medicine 318:1100-1107, 1988.

Steinwachs, Donald M., Joyce Mamon, Maureen Fahey,

Lee Bone, et al. “Impact of Discharge Planning on Patient Outcome.” Final Report to Agency for Health Care

Policy and Research, D.H.H.S. Grant No. 5R18 HS

0541002, 1989.

Tarlov, Alvin R., John E. Ware, Sheldon Greenfield,

Eugene C. Nelson, Edward Perrin, and Michael Zubkoff.

“The Medical Outcomes Study: An Application of Methods for Monitoring the Results of Medical Care.” Journal of the American Medical Association 262:925-930,

1989.

Tonges, Mary Crabtree. “Redesigning Hospital Nursing Practice: The Professionally Advanced Care Team

(ProAct) Model, Part 1.” Journal of Nursing Administration 19:31-38.