Abstract

The use of patient outcomes to describe and measure nursing practice is not a new phenomenon. The use of

patient outcomes has been documented since the early efforts of Florence Nightingale. Throughout the following

years numerous studies using outcome measures were conducted in hospitals, nursing homes, and community settings.

These outcomes are described most frequently in the nursing literature in terms of physiological status, psychosocial

status, safety, functional ratings, behavioral measures, knowledge, symptom control, home functioning, family strain,

quality of life, goal attainment, patient satisfaction, caring

achievement, nursing diagnosis resolution, and utilization.

The relationship of outcomes to process and structure variables requires a naming and categorization of these

variables. Consensus about the naming of the variables is

needed among nurses in practice, in research, and in organized nursing. The clinical testing of these variables is the

basis for a clinical research outcome agenda. The seating of

these variables into large data sets such as Medicare claims

data and uniform clinical data sets will enhance the ability

of researchers to identify patterns of effective nursing care.

The challenge for the research agenda is to build upon the

outcome work that has already been done within nursing

and to move the past and present nursing research into an

interdisciplinary effort where the impact of multiple interventions and providers can be studied.

Outcomes That Reflect Clinical Practice

The use of patient outcomes in the measurement of

effectiveness in health care is not a new phenomenon in nursing. Florence Nightingale was one of the earliest, if not the

earliest, proponents of outcome evaluation, demonstrated by

her use of mortality and morbidity statistics to portray to the

public the terrible standards of care experienced by the military personnel in the Crimean War (Nightingale, 1858).

There is little question that care delivered by the three million nurses in the United States makes a difference in health

outcomes. Nurses are a major component of the health care

delivery team and system. Yet data on what these nurses do

(interventions and treatments), for what type of patient/client populations, problems/conditions or diagnoses, and with

what results or outcomes are virtually unavailable. The fact

that nurses care for thousands of patients and yet data describing patient conditions, interventions, and outcomes, and

the variations that are relevant to nursing are not available is

a major challenge for our research agenda.

This paper is written from a disciplinary view because the authors feel a disciplinary view is necessary in

order to enter into interdisciplinary work. The purpose of

this paper is to present an overview of the past and present

use of outcome measures in evaluation of the effectiveness

of nursing practice. In addition, recommendations for a future research agenda is discussed.

Clinical Conditions And Interventions:

Their Relationship To Outcomes

Outcomes That Reflect Clinical Practice

Norma M. Lang, Ph.D, RN, FAAN

Karen Dorman Marek, MSN, RN Ph.D Candidate
Definition of Outcome

The construct of outcome is highly complex. An

outcome is most commonly defined most commonly as an

end result of a treatment or intervention. This definition has

several components. First, is the determination of the outcome measure to represent the end result. For example, is

the outcome measured by mortality, functional status, self

care ability, satisfaction, financial savings, or some combination of measures? A second component of the definition

is the determination of when the end point occurs. The outcome of some prevention interventions may not be recognizable for several years, whereas some treatments may produce immediate results. In addition, some interventions may

require multiple measurement times to determine effectiveness of a treatment. Thirdly, there is the treatment or intervention. Is the treatment a type of care delivery, such as

case management, or a specific intervention like a bowel

management program?

Finally, there is the identification of the problem,

diagnosis, or population for which the treatment is done and

to which the outcomes are related. The way outcomes are

linked influences what aspects of care will be examined for

effectiveness. Outcomes are most frequently linked to medical diagnoses (Fitzgerald, Moore, & Dittus, 1988; Decker,

Stevens, Vancini, & Wedeking, 1979; Jones, Jones, & Katz,

1989; Wolf, Lesic, & Leak, 1986; Tarlov et al, 1989) and

medical treatments (Wennberg, 1984; Anderson, 1977; Barry

et al, 1988). However, other examples of outcome linkages

include nursing diagnoses (Kerr, Rudy, & Daly, 1991; Harris, 1991), nursing interventions (Heater, Becker, & Olson,

1988; Hjelm-Karlsson, 1989; Hyman et al, 1989; Hodnett &

Osborn, 1989), site of care (Hartz et al, 1989; Rooks et al,

1989; Knause et al, 1989; Chassin et al, 1989), and type or

organization of care delivered (Greenfield et al, 1975;

Hinshaw, Scofield, & Atwood, 1981; Liem, Chernoff, &

Carter, 1986). To what extent should the fifteen years of

work on nursing diagnosis be further developed, tested, and

included in the studies linking problems, interventions, and

outcomes (North American Nursing Diagnosis Association,

1987; Fitzpatrick et al, 1989)?

Health care is a multidimensional phenomena. It is

provided in multiple sites, by multiple providers, with multiple environmental and organizational variables influencing the effectiveness. However, health care outcomes are

frequently viewed related to one dimension such as site of

care, usually the acute care facility. The challenge is to expand the concept of care beyond one episode to a continuum

of care so that longitudinal outcomes can be identified, as

well as the relationship between multiple sites and providers

of care examined.

Historical Overview of the Use of

Outcome Measures in Nursing

Aydelotte (1962) used patient welfare as an outcome of nursing care. In her study, clinical measures of

outcomes, which were easily retrievable from the patient’s

record, included the number of days in bed, fever days, postoperative days, and doses of narcotics, analgesics, and/or

sedatives. Outcomes measured were patient’s mental attitude, physical independence, special aspects of physical independence, mobility, skin condition, patient’s opinion of

care given, and physician’s evaluation of patient’s condition and progress.

In the 1970’s, outcome measures were emphasized

as a measurement of the quality of care. The American

Nurses Association (ANA) developed 17 sets of outcome

criteria for use in the review of nursing care within the Professional Standards Review Organizations (PSRO’s) (American Nurses Association, 1976). Examples of outcome measures included knowledge of condition, compliance with

treatment, pain level, mobility, and physical status.
Also during this time, the Joint Commission of

Accreditation of Hospitals (JCAH) emphasized the use of

outcome criteria with the implementation of the PEP (Performance Evaluation Procedure) primer methodology (Joint

Commission of Accreditation of Hospitals, 1975). Many

sets of outcome criteria were developed during the 1970’s

and may still exist. Unfortunately, most of this work cannot

be retrieved from published literature.

Examples of published literature include Hover and

Zimmer (1978) who offered a classification of five categories of outcomes derived from an examination of 35 previously developed sets of outcome criteria. The classification

categories are: (1) knowledge of illness and its treatments,

(2) skills, (3) knowledge of medications, (4) adaptive behaviors, and (5) health or physiologic status.

Horn and Swain (1978) developed an outcome classification consisting of a total of 539 measurement items

categorized into eight universal and ten health deviation

self-care demand dimensions based upon the work of Orem.

Each dimension also was classified into four domains which

were: (1) requirements met (physiological), (2) knowledge,

(3) skills and performance abilities, and (4) motivation. The

work of Horn and Swain has provided a comprehensive instrument for measuring patients’ health status using a number of dimensions under the influence of nursing care.

Daubert’s early work on home care classification

was based on rehabilitation potential (Daubert, 1979). Outcomes were classified into the categories of recovery,

self-care, rehabilitation, maintenance, and terminal care.

Patient outcome was viewed as the actual functioning of a

patient at the time of discharge. This system has been used

in several studies which have examined financial and clinical outcomes of home health care (Harris, Santoferraro, &

Silva, 1985; Harris, Peters, & Yuan, 1987; Harris, 1991).

The Visiting Nurse Association (VNA) of Omaha

has conducted a well known research project in which a patient classification system for community health nursing was

developed (Martin et al, 1986). One aspect of this study

was the problem rating scale for outcomes in which outcomes were classified in categories of knowledge, behavior,

or status. A five point rating scale was used to further specify

the outcome level at each category. The VNA of Omaha

currently is conducting its fourth testing, funded by the National Center for Nursing Research, of the Omaha System in

which reliability, validity, utility, and generalization potential of the Omaha Classification System are being intensely

examined (Martin & Scheets, in press). The Omaha System

also has been used by the Nursing Center of the University

of Wisconsin-Milwaukee School of Nursing to describe the

nursing diagnoses, interventions, and outcomes of the

community-based underserved population (Lundeen,

Coenen, & Marek, 1991). In addition, this system is being

considered for use by other nurse managed centers and parish nurse centers with the anticipation of obtaining comparable data across numerous community-based nursing care

practice settings.

Lang and Clinton (1984) completed a critical review of empirical work from 1974 to 1982 designed to assess the quality of nursing care. A total of 164 research

studies were reported categorized into structure, process,

outcome, structure-process, structure-outcome,

process-outcome, and structure-process-outcome. The need

for data banks of descriptions or norms of nursing practice

was identified, as well as, the inclusion of elements of nursing care in minimum data sets for health care information

systems.

The Nursing Minimum Data Set (NMDS) was developed to establish uniform standards for the collection of

minimum essential nursing data (Werley & Lang, 1988). The

NMDS includes three broad categories of elements which
are (a) nursing care, (b) patient or client demographics, and

(c) service elements. The nursing care elements consist of

(a) nursing diagnosis, (b) nursing intervention, (c) nursing

outcome, and (d) intensity of nursing care. In the NMDS,

nursing outcome is defined as an aspect of patient or client

health status that is influenced by nursing intervention and

recorded at specific times for an episode or encounter of

care. The nursing outcome is identified as the resolution

status of the nursing diagnosis as resolved, not resolved, or

referred for continuing care.

The ANA and specialty nursing organizations have

developed multiple sets of standards for nursing practice.

One example is the monograph entitled Outcome Standards

for Nursing Care of the Critically Ill by the American Association of Critical-Care Nurses (1990). Other examples include home health nursing, gerontological nursing, hospice

nursing, and psychiatric and mental health nursing practice.

The standards have been designed using the format of structure, process, and outcome criteria. The criteria are both

nurse and patient focused. These standards represent significant work in outcome identification, outcomes which are

sensitive to nursing and all but ignored in current outcome

indicator development.

Types of Outcome Indicators

A wide variety of outcome indicators have been

used in nursing quality assurance and research studies. In

the following section, an overview of the various types of

outcome indicators used in nursing is presented (Marek,

1989; Lang & Marek, 1990).

Physiological status is a frequently occurring categories in the literature related to outcomes and includes

measures of processes which maintain life (Hegyvary &

Haussmann, 1976; Horn & Swain, 1978; Hover & Zimmer,

1978; Howe, 1980; Blom, 1985; Votava, Cleveland, &

Hiltunen, 1985; Martin et al, 1986; Sorgen, 1986; Lalonde,

1986; Heater, Becker, & Olson, 1988; Rinke, 1988; Stevens

& Pavlides, 1989). Frequent measures in this area include

changes in parameters such as blood pressure, pulse, temperature, lung sounds, and blood glucose levels. In addition, skin integrity, weight, and the healing of a wound are

often indicators in this area. This category also has been

referred to as health status, although measures usually are

limited to physiologic measures. Physiologic measures are

often more easily quantified than other outcome measures

and are often readily available in the client record.

Psychosocial outcome indicators refer to the patterns of behavior, communications, and relationships both

intrapersonally and interpersonally of the client (Vincent &

Price, 1977; Choi, 1983; Padilla & Grant, 1985; Martin et

al, 1986; Sorgen, 1986; Toth, 1988; Dolbee & Creson, 1988;

Heater, Becker, & Olson, 1988; Hjelm-Karlsson, 1989;

Olson, Heater, & Becker, 1990; Rinke, 1988). Examples of

measures in this domain include mentation, emotion, attitude, mood, and affect. Concepts such as coping, social contact, and social functioning also have been incorporated in

this category.

Functional measures relate to activities of daily living, mobility, and communication are (Linn, Gurel, & Linn,

1977; Horn & Swain, 1978; Daubert, 1979; Posavac & Carey,

1982; Liem, Chernoff, & Carter, 1986; Lalonde, 1986;

Christensen, 1987; Morris & Buckwalter, 1988; Wilson,

1988; Buchanan, 1988; Rinke, 1988; Myles, 1989). Another category which could be merged with functional measures is self care. In this category the locus of functioning is

often expanded beyond the client to include the client’s family and/or caregivers.

In the behavior domain activities, skills, and actions presented by the client are included (Marston, 1970;

Horn & Swain, 1978; Hover & Zimmer, 1978; Given, Given,

& Simoni, 1979; Sullivan & Armignacco, 1979; Howe, 1980;

Sorgen, 1986; Heater, Becker, & Olson, 1988; Martin et al,
1986; Hilbert, 1988; Rinke, 1988). Areas referred to in this

domain include application of knowledge and skills, problem solving, compliance, motivation, and therapeutic competence.

Knowledge is the next type of outcome indicator

identified (Hover & Zimmer, 1978; Gallant & McLane, 1979;

Given, Given, & Simoni, 1979; Hageman & Ventura, 1981;

Hill, 1986; Sorgen, 1986: Martin et al, 1986; Lalonde, 1986;

Heslop & Bagnall, 1988; Rinke, 1988). It refers to the cognitive level of understanding of the client. Specific areas of

knowledge in the literature include knowledge of nursing

problems, diet, medications, and treatments. Knowledge as

an outcome measure has received greater attention recently

due to the increase in the amount and acuity of the care clients are expected to provide to themselves.

Another category that is closely related to physiological measures is symptom control (Sorgen, 1986; Davis

et al, 1986; LaLonde, 1986; Davis, 1988; Hyman et al, 1989).

The most frequently occurring symptoms utilized as outcome

measures include pain and comfort. Examples of other symptom measures include fatigue, nausea, constipation, incontinence, and diarrhea.

Quality of life is a broad category that focuses on

outcome measures beyond biological functioning (Padilla

& Grant, 1985; Sorgen, 1986; Caldwell, 1988). Quality of

life measures include dimensions such as life satisfaction,

well-being, symptom control, standard of living, and functional capacity. This category is reliant on the patient perception of subjective experiences in life and requires the

collection of concepts that are more qualitative than traditional clinical endpoints (Ware, 1991).

The next category is home functioning which is a

crucial area, especially in home health care (Votava, Cleveland, & Hiltunen, 1985; Sorgen, 1986; Dolbee & Creason,

1988; Hazlett, 1989; Myles, 1989). This category refers to

the functioning of the client in the home environment. Areas in this domain include family living patterns, home environment, support, and role function.

Family strain is another measure that has been examined in nursing studies (Votava, Cleveland, & Hiltunen,

1985; LaLonde, 1986; Sorgen, 1986; Foxall & Watson, 1988;

Caldwell, 1988; Allen, Becker, & Swank, 1991). The functioning of the family caregiver often is a critical component

of the patient’s recovery at home as well as in the acute care

facility.

Goal attainment as a measure of quality is another

category of outcome measures (Daubert, 1979; Inzer &

Aspinall, 1981; Sorgen, 1986; King, 1988). In this type of

measure, expected outcomes or goals are determined at the

beginning of care and the attainment of the goals is measured at designated intervals or at discharge of the client from

care. This type of outcome measurement is more sensitive

to individual client differences and outcome achievement.

Utilization of service is often used as an outcome

measure. This refers to use of health care resources and can

include items such as length of hospital stay, number of clinic

visits, and rehospitalization (Anderson, 1977; Sullivan &

Armignacco, 1979; Brooten et al, 1986; Marchette &

Holloman, 1986; Finkler, Brooten, & Brown, 1988; Kane et

al, 1989; Miller, 1989; Stevens & Pavlides, 1989). These

measures are more easily accessible in health care facilities.

Lang et al (1990) used an outcome framework of overuse

and underuse of services by older people in a review of 4758

papers on the quality of health care for older people. Resource use can be a negative or positive outcome depending

on its context. An example of a positive outcome related to

high resource use would be if a client was referred and utilizing services which were available to assist the client to

independence. However, frequent use of resources could be

a negative outcome, for example, if a client has made unnecessary trips to the emergency room for problems which
could have been addressed by a home care visit. Utilization

measures also are converted to financial terms such as cost

of an episode of care or treatment (Fagin & Jacobsen, 1985;

Ventura et al, 1985; U. S. Congress 1986; Brooten et al, 1986;

Wolf, Lesic, & Leak, 1986; Neidlinger, Scroggins, &

Kennedy, 1987; Dolbee & Creason, 1988; Gamroth, 1988;

Hogan & Roher, 1989; Mitchel et al, 1989).

Safety is another outcome indicator mentioned in

the nursing literature (Votava, Cleveland, & Hiltunen, 1985;

Buddi, 1987). Many nursing interventions are targeted at

providing the outcome of safety such as fall prevention. In

addition, the use of devices such as Hickman catheters and

mechanical ventilation has made safety a more critical outcome, especially in the home environment.

Resolution of nursing diagnosis or nursing problems is another outcome category (Werley & Lang, 1988;

Felman & Richard, 1988; Keating, 1988; Kerr, Rudy, & Daly,

1991). This category overlaps with the majority of other

categories. Since nursing diagnoses are often more temporary than medical diagnoses, the resolution of nursing diagnoses would be more sensitive to changes in client health

status. If all relevant nursing diagnoses were identified during an episode of client care, the resolution of nursing diagnoses at designated intervals, such as discharge, would yield

valuable data regarding the nursing needs of clients, as well

as, the effectiveness of nursing interventions.

Patient satisfaction has become more popular in

recent years as a critical component in the measurement of

the quality of care (Greenfield, et al, 1975; Graham, 1978;

Hill, 1986; Hinshaw, Scofield, & Atwood, 1981; Ventura et

al, 1982; Koerner, Cohen, & Armstrong, 1985; U.S. Congress, 1986; LaMonica et al, 1986; Erickson, 1988). This is

a difficult dimension to measure due to a general patient

bias to answer favorably. In addition, tools which can discriminate actual changes in patient satisfaction are difficult

to develop.

There has been much discussion in the nursing literature related to caring as an outcome versus a process

(Shiber & Larson, 1991; Lang & Krejci, 1991; Valentine,

1991). Outcomes associated with caring are multidimensional, however, the patient’s subjective perception related

to care provided is a key component (Paulen & Rapp, 1981:

Coffey & Reagan, 1984; Laffrey, 1990). Considerable debate can be found in nursing regarding the need for a balance of quantitative and qualitative methodologies (Benner

& Wrubel, 1989; Westra, 1991). Over the years nursing has

been warned that quantification may destroy the essence of

nursing (Lewis, 1976). Further development of caring outcomes is one way that nursing can balance the reductionistic

focus of the current health care system.

Influence of Contemporary Projects

Health care is entering into a new era in which the

use of health care data and information has never been as

critical. Advances in computer technology have enabled the

collection of complex data bases and use of sophisticated

analytic techniques.

The data collected in health care information systems is used to direct health care policy and affects the standard of care delivered. This information is used to establish

reimbursement, determine access to health care, define services, and monitor the quality of care. In addition, it is the

patient and service characteristics collected in health care

data bases that define what information must be collected

by health care providers.

Although large national data bases exist, the major

focus has been on patient demographics, medical diagnoses,

medical procedures, and health care costs. Data bases related to utilization and outcomes of health care are in the

formative stages. Large data sets are needed to perform sophisticated multivariate analysis. If nursing data are not
included in these large data sets the impact of nursing care

will remain largely unmeasured and invisible (Mallison,

1990).

The need for nursing data and a classification system for these data has never been more important. The ANA

has recommended that data include the NMDS elements with

high priority given to nursing diagnoses, interventions, outcomes, and nurse providers (O’Connor, 1990).

There currently are several national data bases that

merit close attention by the nursing community. The first is

the Uniform Clinical Data Set (UCDS) which is being developed by the Health Care Financing Administration

(HCFA) for use by the Peer Review Organizations (PROs)

(Lohr, 1990). Second is the Uniform Minimum Data Set

(MDS) which is designed to provide case mix information

for all nursing facilities in the nation (Health Care Financing

Administration, 1990). Third is the Uniform Needs Assessment Instrument which will provide data on the needs of

patients for post hospital extended care services, home health

services, and long-term care services of a health-related or

supportive nature (Lohr, 1990). Finally, there is the indicator development project of the Joint Commission on Accreditation of Healthcare Organizations (JCAHO). In this project

a national performance measure database will be established

for ongoing monitoring of organizational performance

(Nadzam, 1991).

The development of guidelines by the Agency for

Health Care Policy and Research (AHCPR) has had a major

impact on nursing and nursing on the AHCPR (1990). It is

critical that nurses be multidisciplinary partners in guideline

development panels that address specific conditions where

medical intervention is primary, as well as, all aspects of

preventative, acute, and long term care. At the same time,

the nursing profession should develop guidelines for areas

in which nursing intervention is primary, for which costs are

high, in which large patient populations are affected, and for

which there is an established data base. Bergstrom’s research

on the prevention of skin breakdown should be used as an

example or prototype (Bergstrom, 1991). Her own basic

clinical research provided direction for the diagnosis, prevention, and treatment of skin integrity (Sponsored by

NCNR). Her research findings will be integrated with findings of other research as part of the guideline development

process for the AHCPR Guidelines Development and Dissemination Project. The use of these guidelines as one determinate of what should be included in data bases is yet to

be developed. This development would lead to the use of

data elements identified in guideline development in describing conditions, treatments, and outcomes of large groups of

people. Other examples that provide models for nursing research are the Guidelines for Incontinence co-chaired by

Kathleen McCormick and Guidelines for Pain co-chaired by

Ada Jacox.

A Framework for Outcomes Research

In 1980, the ANA issued Nursing: A Social Policy

Statement in which four defining characteristics of nursing

are specified (American Nurses Association, 1980). These

characteristics include phenomena, theory application, nursing action, and evaluation of effects of action in relation to

phenomena. The evaluation of the outcomes of nursing action reflects the effectiveness of the nursing actions in resolving or improving the condition or diagnosis to which

actions were directed. Thus, the need to identify outcomes

was reaffirmed.

Since most outcomes are influenced by a wide variety of structures and interventions, any single discipline’s

approach would be narrow and limited. Therefore, there is

the need to not only to study nursing interventions separately,

but also to integrate these interventions with those of other

disciplines to measure multiple influences on outcomes. For

example, when studying interventions for the diagnosis of

fractured hip, open reduction and hip replacement interven-
tions are included. Complementarily, it is essential to include a matrix of such nursing problems and diagnoses as

immobility, confusion, sleep disturbance, pain, incontinence

and the associated nursing interventions. Further, it is essential to follow these patients from home to hospital to nursing home to home not only based upon the medical diagnoses but also using the referent nursing diagnoses. Ultimately, the effectiveness is measured by answering the questions: Can the person walk, talk, manage pain and continence, etc.?

Nursing must build on its rich research history. For

example, in 1975 Bloch proposed a comprehensive model

for quality assessment research that should be revisited as

we research the effectiveness of nursing care. In this model

she proposed extensive testing of process-outcome relationships in nursing practice (Bloch, 1975).

Nursing scholars must continue to perform “horizontal” research in which the relationship between a specific phenomena of nursing (nursing diagnoses), nursing

action (interventions), and the effects of actions (outcomes)

are identified. In addition, “vertical” research must continue

with the aim to develop and refine a classification of the

phenomena of nursing, including nursing diagnoses/problems, nursing interventions, and nursing sensitive outcomes

(North American Diagnosis Association, 1987; Werley &

Lang, 1988; American Nurses Association, 1989; Peters,

1989; Bulechek & McCloskey, 1990; Jacox, 1990; Grobe,

1991: Maas, 1991; Saba et al, 1991). Results or findings

from this basic “horizontal” and “vertical” research supported

by the National Institutes of Health (NIH) National Center

for Nursing Research must then be used in guideline development and dissemination, as well as, in the effectiveness

research and health services research supported by the

AHCPR and other agencies.

In summary, Nursing has a rich history of identifying and using relevant patient outcomes. The relationship

of outcomes to process and structure variables requires a

naming and categorization of these variables. Consensus

about the naming of the variables is needed among nurses in

practice, in research, and in organized nursing. The clinical

testing of these variables is the basis for a clinical research

outcome agenda. The seating of these variables into large

data sets such as Medicare claims data and uniform clinical

data sets will enhance the ability of researchers to identify

patterns of effective nursing care. The challenge for the research agenda is to build upon the outcome work that has

already been done within nursing and to move the past and

present nursing research into an interdisciplinary effort where

the impact of multiple interventions and providers can be

studied.

References

Agency for Health Care Policy and Research. (1990). Nursing advisory panel for guideline development: Summary.

Rockville, MD: U. S. Department of Health and Human Services Public Health Service

Allen, J. K., Becker, D. M., Swank, R. T. (1991). Impact of

spouse concordance of psychological adjustment on functional status after coronary bypass surgery. Journal of Nursing Quality Assurance, 5(2), 69-74.

American Association of Critical-Care Nurses (1990). Outcome standards for nursing care of the critically ill. Laguna

Niguel, CA: Author.

American Nurses’ Association (1976). Guidelines for Review at the Local Level. Kansas City, MO: Author.

American Nurses’ Association (1980). Nursing: A social

policy statement. Kansas City, MO: Author.
American Nurses’ Association (1989). Classification systems

for describing nursing practice: Working papers. Kansas City,

MO: Author.

Anderson, N. (1977). Audit of care processes and patient

outcome: One facet of QA. Nursing Administration Quarterly, 1(3), 117-128.

Aydelotte, M. K. (1962). The use of patient welfare as a

criterion measure. Nursing Research, 11, 10-14.

Barry, M. J., Mulley, A. G., Fowler, F. J., Wennberg, J. E.

(1988). Watchful waiting vs immediate transurethral resection for symptomatic prostatism: The importance of patients’

preference. Journal of the American Medical Association,

259, 3010-3017.

Bergstrom, N. (1991). (Personal communication, September 1991, University of Nebraska Medical Center, College

of Nursing).

Benner, P. & Wrubel, J. (1989). The primacy of caring.

Menlo Park, CA: Addison-Wesley.

Bloch, D. (1975). Evaluation of nursing care in terms of process and outcome: Issues in research and quality assurance.

Nursing Research, 24, 256-263.

Blom, M.F. (1985). Dramatic decrease in decubitus ulcers.

VA quality assurance program stimulated changes. Geriatric Nursing, 6(2), 84-87.

Brooten, D., Kumar, S., Brown, L. P., Butts, P., Finkler, S.

A., Bakewell-Sachs, S., Gibbons, A., &

Delivoria-Papadopoulos, M. (1986). A randomized clinical

trial of early hospital discharge and home follow-up of

very-low-birth-weight infants. New England Journal of

Medicine, 315, 934-939.

Buchanan, B. F. (1988). Functional assessment: Measurement with the Barthel Index and PULSES Profile. Home

Healthcare Nurse, 4(6), 11-17.

Bulechek, G. M., & McCloskey, J. C. (1990). Nursing intervention taxonomy development. In J. C. McCloskey & H.

K. Grace (Eds.), Current issues in nursing (pp. 23-28). St.

Louis, MO: C. V. Mosby.

Buddi, J. (1987). Quality assurance in a home health care

agency. In L. Rinke & A. Wilson (Eds.), Outcome measures

in home health care: Volume II Service (pp. 129-138). New

York: National League for Nursing.

Caldwell, S. M. (1988). Measuring family well-being: Conceptual model, reliability, validity, and use. In C. Waltz &

O. Strickland (Eds.), Measurement of nursing outcomes:

Volume one measuring client outcomes (pp. 396-422). New

York: Springer.

Chassin, M. R., Park, R. E., Lohr, K. N., Keesey, J., & Brook,

R. H. (1989). Differences among hospitals in Medicare patient mortality. Health Services Research, 24(1), 2-31.

Choi, T., Josten, L., & Christenson, M. L. (1983).

Health-specific family coping index for noninstitutional care.

American Journal of Public Health, 73, 1275-1277.

Christensen, M. L. (1987). Development and use of functional client outcome measures. In L. Rinke & A. Wilson

(Eds.), Outcome measures in home care: Volume II Service

(pp. 249-253). New York: National League for Nursing.

Coffey, J., & Reagan, J. T. (1984). Assessing patient-centered

nursing care. The Journal of Long-term Care Administration, 12(4), 25-30.

Daubert, E. A. (1979). Patient classification system and outcome criteria. Nursing Outlook, 27, 450-454.
Davis, A., Nagelhout, M., Hoban, M., & Barnard, B. (1986).

Bowel management: A quality assurance approach to upgrading progress. Journal of Gerontological Nursing, 12(5),

13-17.

Decker, F., Stevens, L., Vancini, M., & Wedeking, L. (1979).

Using patient outcomes to evaluate community health nursing. Nursing Outlook, 27, 278-282.

Dolbee, S. F., & Creason, N. S. (1988). Outcome criteria for

the patient using intravenous antibiotic therapy at home.

Home Healthcare Nurse, 6(4), 22-29.

Eriksen, L. (1988). Measuring patient satisfaction with nursing care: A magnitude estimation approach. In C. Waltz &

O. Strickland (Eds.), Measurement of nursing outcomes:

Volume one measuring client outcomes (pp. 523-534). New

York: Springer.

Fagin, C. M., & Jacobsen, B. S. (1985), Cost-effectiveness

analysis in nursing research. In Werley, H. H., & Fitzpatrick,

J. J. (Eds.) Annual Review of Nursing Research (pp.

215-238). New York: Springer Publishing Co.

Felman, J., & Richard, R. (1988). The measurement of nursing outcomes in home health care agencies. In C. Waltz &

O. Strickland (Eds.), Measurement of nursing outcomes:

Volume one measuring client outcomes (pp. 475-495). New

York: Springer.

Finkler, S. A., Brooten, D., & Brown, L. (1988). Utilization

of inpatient services under shortened lengths of stay: A neonatal care example. Inquiry, 25, 271-280.

Fitzgerald, J. R., Moore, P. S., & Dittus, R. S. (1988). The

care of elderly patients with hip fracture: Changes since

implementation of the prospective payment system. New

England Journal of Medicine, 319, 1392-1397.

Fitzpatrick, J. J., Kerr, M., Saba, V., Hoskins, L., Hurley,

M., Mills, W., Rottkamp, B., & Warren, J. J. (1989) NANDA

Taxonomy I: Proposed ICD-CD 10 Version. Applied Nursing Research, 2(2), 90-91.

Foxall, M. J., & Watson, P. (1988). Evaluation of measures

assisting family responses to chronic illness. In C. Waltz &

O. Strickland (Eds.), Measurement of nursing outcomes:

Volume one measuring client outcomes (pp. 231-283). New

York: Springer.

Gallant, B. W. & McLane, A. M. (1979). Outcome criteria:

A process for validation at the unit level. Journal of Nursing

Administration, 9(1), 14-21.

Gamroth, L. (1988). Long-term care resource requirements

before and after the prospective payment system. Image,

20(1), 7-11.

Gillis, C. (1984). Reducing family stress during and after

coronary artery bypass graft surgery. Nursing Clinics of

North America, 19(1), 103-112. during early convalescence

Given, B., Given, C. W., & Simoni, L. E. (1979). Relationships of processes of care to patient outcomes. Nursing Research, 28, 85-93.

Graham, N. (1978). A quality of care assessment: Pediatricians and pediatric nurse practitioners. Image, 10(2), 41-48.

Greenfield, S., Anderson, H., Winickoff, R. N., Morgan, A.,

& Komaroff, A. L. (1975). Nurse-protocol management of

low back pain: Outcomes, patient satisfaction and efficiency

of primary care. The Western Journal of Medicine, 123,

350-359.

Grobe, S. (1991) (Personal communication, September 1991,

University of Texas-Austin).

Hageman, P. T., & Ventura, M. R. (1981). Utilizing patient

outcome criteria to measure the effects of a medication teaching regimen. Western Journal of Nursing Research, 3, 25-33.
Harris, M. D. (1991). Clinical and financial outcomes in

patient care in a home health care agency. Journal of Nursing Quality Assurance, 5(2), 41-49.

Harris, M., Peters, D., & Yuan, J. (1987). Relating quality

and cost in a home care agency. Quality Review Bulletin,

13, 93-99.

Harris, M. D., Santoferraro, C., & Silva, S. (1985). A patient

classification system in home health care. Nursing Economics, 3(5), 276-282.

Hartz, A. J., Krakauer, H., Kuhn, E. M., Young, M., Jacobsen,

S. J., Gay, G., Muenz, L., Katzoff, M., Bailey, R. C., & Rimm,

A. A. (1989). Hospital characteristics and mortality rates.

New England Journal of Medicine, 321(25), 1720-1725.

Hazlett, D. E. (1989). A study of pediatric home ventilator

management: Medical, psychological, and financial aspects.

Journal of Pediatric Nursing, 4(4), 284-294.

Health Care Financing Administration. (1990). Long term

care resident assessment instrument training manual (NTIS

# PB-91-131292) Springfield Virginia : U. S. Department of

Commerce.

Heater, B. S., Becker, A. M., & Olson, R. K. (1988). Nursing interventions and patient outcomes: A meta-analysis of

studies. Nursing Research, 37(5), 303-307.

Hegyvary, S. T., & Haussmann, R. K. D. (1976). The relationship of nursing process and patient outcomes. Journal of

Nursing Administration, 6(9), 18-21.

Heslop, A. P., & Bagnall, P. (1988). A study to evaluate the

intervention of a nurse visiting patients with disabling chest

disease in the community. Journal of Advance Nursing,

13(1), 71-77.

Hilbert, G. (1988). The measurement of compliance as a

nursing outcome. In C. Waltz & O. Strickland (Eds.), Measurement of nursing outcomes: Volume one measuring client outcomes (pp. 80-107). New York: Springer.

Hill, J. (1986). Patient evaluation of a rheumatology nursing

clinic. Nursing Times, 82(27), 42-43.

Hinshaw, A. S., Scofield, R. & Atwood, J. R. (1981). Staff,

patient and cost outcomes of all-registered nurse staffing.

Journal of Nursing Administration, 11(9), 30-36.

Hjelm-Karlsson, K. (1989). Effects of information to patients

undergoing intravenous pyelography: An intervention study.

Journal of Advanced Nursing, 14, 853-862.

Hodnett, E. D., & Osborn, R. W. (1989). Effects of continuous intrapartum professional support on childbirth outcomes.

Research in Nursing and Health, 12, 289-297.

Hogan, A. J., & Rohrer, J. E. (1989). The effects of nursing

and physician services: Some preliminary results. Social

Science Medicine, 29(4), 527-536.

Horn, B., & Swain, M. A. (1978). Criterion Measures of

Nursing Care Quality (DHEW Publication No. PHS 78-3187)

Hyattsville, Maryland: U.S. Department of Health, Education, and Welfare, Public Health Service, National Center

for Health Services Research.

Hover, J., & Zimmer, M. J. (1978). Nursing quality assurance: The Wisconsin System. Nursing Outlook, 26, 242-248.

Howe, M. J. (1980). Developing instruments for measurement of criteria: A clinical nursing practice perspective.

Nursing Research, 29, 100-103.

Hyman, R. B., Feldman, H. R., Harris, R. B., Levin, R. F., &

Malloy, G. B. (1989). The effects of relaxation training on

clinical symptoms: A meta-analysis. Nursing Research,

38(4), 216-220.

Inzer, F., & Aspinall, M. J. (1981). Evaluating patient outcomes. Nursing Outlook, 29, 178-181.
Jacox, A. K. (1990). A classification of nursing technology.

Nursing Outlook, 38(2), 81-85.

Joint Commission on Accreditation of Hospitals. (1975). PEP

workbook for nurses. Chicago: Author.

Jones, S. L., Jones, P. K., & Katz, J. (1989). A nursing intervention to increase compliance in otitis media patients. Applied Nursing Research, 2(2), 68-73.

Kane, R. L., Garrard, J., Skay, C. L., Radosevich, D. M.,

Buchanan, J. L., McDermott, S. M., Arnold, S. B., &

Kepferle, L. (1989). Effects of a geriatric nurse practitioner

on process and outcome of nursing home care. American

Journal of Public Health, 79(9), 1271-1277.

Keating, S. B. (1988). The measurement of client outcomes

in home health care agencies. In C. Waltz & O. Strickland

(Eds.) Measurement of nursing outcomes: Volume one measuring client outcomes (pp. 457-474). New York: Springer.

Kerr, M. E., Rudy, E. B., & Daly, B. J. (1991). Human response patterns to outcomes in the critically ill patient. Journal of Nursing Quality Assurance, 5(2), 32-40.

King, I. (1988). Measuring health goal attainment in patients.

In C. Waltz & O. Strickland (Eds.) Measurement of nursing

outcomes: Volume one measuring client outcomes (pp.

109-127). New York: Springer.

Knaus, W. A., Draper, E. A., Wagner, D. P., & Zimmerman,

J. E. (1986). An evaluation of outcome from intensive care

in major medical centers. Annals of Internal Medicine,

104(3), 410-418.

Koerner, B. L., Cohen, J. R., & Armstrong, D. M. (1985).

Collaborative practice and patient satisfaction: Impact and

selected outcomes. Evaluation and the Health Professions,

8(3), 299-321.

Laffrey, S. C. (1990). Elderly clients’ perceptions of public

health nursing care. Public Health Nursing, 7(2), 111-117.

La Monica, E. L., Oberst, M. T., Madea, A. R., & Wolf, R.

M. (1986). Development of a patient satisfaction scale. Research in Nursing and Health, 9, 43-50.

Lalonde, B. (1986). Quality assurance manual of the Home

Care Association of Washington. Edmonds, WA: The Home

Care Association of Washington.

Lang, N. M. & Clinton, J. F. (1984). Assessment of quality

of nursing care. In H. H. Werley & J. J. Fitzpatrick (Eds.)

Annual Review of Nursing Research Volume 2 (pp. 135-163)

New York: Springer.

Lang, N. M., Kraegel, J. M., Rantz, M. J., Krejci, J. W. (1990).

Quality of health care for older people in America. Kansas

City, MO: American Nurses’ Association.

Lang, N, M., & Marek, K. D. (1990). The classification of

patient outcomes. Journal of Professional Nursing, 6(3),

158-163.

Lang, N. M., Krejci, J. W. (1991). Standards and holism: A

reframing. Holistic Nursing Practice, 5(3), 14-21.

Lewis, E. (1976). Editorial: Quantifying the unquantifiable.

Nursing Outlook, 24, 147.

Liem, P. H., Chernoff, R., & Carter, W. J. (1986). Geriatric

Rehabilitation Unit: A 3-Year outcome evaluation. Journal

of Gerontology, 41(1), 44-50.

Linn, M. W., Gurel, L., & Linn, B. S. (1977). Patient outcome as a measure of quality of nursing home care. American Journal of Public Health, 67(4), 337-344.

Lohr, K. (1990). Medicare: A strategy for quality assurance. Washington, D.C.: National Academy Press.
Lundeen, S., & Coenen, A., & Marek, K. D. (1991) Nursing

clinical information: Describing practice and utilization in a

nursing center. (manuscript submitted for publication).

Maas, M. (1991) (Personal communication, September 1991,

University of Iowa, College of Nursing, Iowa City, Iowa).

Mallison, M. B. (1990). Editorial. Access to invisible expressways. American Journal of Nursing, 90, 7.

Marchette, L., & Holloman, F. (1986). Length of stay: Significant variables. Journal of Nursing Administration, 16(3),

12-20.

Marek, K. D. (1989). Outcome measurement in nursing.

Journal of Nursing Quality Assurance, 4(1), 1-9.

Marston, M. V. (1970). Compliance with medical regimens:

A review of the literature. Nursing Research, 19, 312-323.

Martin, K., Scheet, N., Crews, C., & Simmons, D. A. (1986).

Client management information system for community health

nursing agencies: An implementation manual. Rockville,

MD, Division of Nursing, US DHHS, PHS, HRSA. (NTIS

No. HRP-0907023).

Martin, K., & Scheet, N. J. (in press) The Omaha System:

Applications for community health nursing. Philadelphia:

W. B. Saunders.

Miller, J. (1989). Evaluating structure, process, and outcome

indicators in ambulatory care: The AMBUQUAL approach.

Journal of Nursing Quality Assurance, 4(1), 40-47.

Mitchell, P. H., Armstrong, S., Simpson, T. F., & Lentz, M.

(1989). American Association of Critical-Care Nurses Demonstration Project: Profile of excellence in critical care nursing. Heart & Lung, 18(3), 219-237.

Morris, J., et al. (1990). Development of resident assessment

system and data base for nursing home residents: Report on

the small scale trial of the minimum data set for resident

assessment and care screening (MDS). Research Triangle

Park, N.C.: Research Triangle Institute.

Morris, W. W., & Buckwalter, K. C. (1988). Functional assessment of the elderly: The Iowa self-assessment inventory. In C. Waltz & O. Strickland (Eds.) Measurement of

nursing outcomes: Volume one measuring client outcomes

(pp. 328-351). New York: Springer.

Myles, S. (1989). Monitoring patient outcomes in an oncology unit. Journal of Nursing Quality Assurance, 4(1), 35-39.

Nadzam, D. M. (1991). The Agenda for Change: Update on

indicator development and possible implications for the nursing profession. Journal of Nursing Quality Assurance, 5(2),

18-22.

Neidlinger, S. H., Scroggins, K., & Kennedy, L. M. (1987).

Cost evaluation of discharge planning for hospitalized elderly. Nursing Economics, 5(5), 225-230.

Nightingale, F. (1858). Notes on matters affecting the health,

efficiency, and hospital administration of the British Army.

London: Harrison & Sons.

North American Nursing Diagnosis Association (1987).

Taxonomy I with complete diagnoses. St. Louis, Missouri:

Author.

O’Connor, K. S. (1990). Effectiveness and outcomes of

health services: Implications for nursing. Kansas City, Missouri: American Nurses’ Association.

Olson, R. K., Heater, B. S., & Becker, A. M. (1990). A

meta-analysis of the effects of nursing interventions on children and parents. Maternal Child Nursing, 15(2), 104-108.
Padilla, G. V., & Grant, M. M. (1985). Quality of life as a

cancer nursing outcome variable. Advances in Nursing Science, 8(10), 45-58.

Paulen, A., & Rapp, C. (1981). Person-centered caring. Nursing Management, 12(9), 17-21.

Peters, D. (1989). An overview of current research relating

to long-term outcomes. Nursing & Health Care, 10(3),

133-136.

Posavac, E. J., & Carey, R. G. (1982). Using a level of function scale (LORS-II) to evaluate the success of inpatient rehabilitation programs. Rehabilitation Nursing, 7, 17-19.

Rinke, L. T. (1988). Outcome standards in home health: State

of the art. New York: National League for Nursing.

Rooks, J. P., Weatherby, N. L., Ernst, E. K. M., Stapleton,

S., Rosen, D., & Rosenfield, A. (1989). Outcomes of care in

birth centers: The national birth center study. The New England Journal of Medicine, 321, 1804-1811.

Saba, V., O’Hare, P. A., Zuckerman, A. E., Boondas, J.,

Levine, E., & Oatway, D. (1991). A nursing intervention

taxonomy for home health care. Nursing & Health Care,

12(6), 296-299.

Shiber, S., Larson, E. (1991). Evaluating the quality of caring: Structure, process, and outcome. Holistic Nursing Practice, 5(3), 57-66.

Sorgen, L. M. (1986). The development of a home care quality assurance program in Alberta. Home Health Care Services Quarterly, 7(2), 13-29.

Stevens, K. A., & Pavlides, C. (1989). Individualized prenatal nursing care of pregnant adolescents makes a difference.

Journal of Obstetric, Gynecologic, and Neonatal Nursing,

18(6), 521-522.

Sullivan, J. A. & Armignacco, F. (1979). Effectiveness of a

comprehensive health program for the well-elderly: By community health nurses. Nursing Research, 28, 70-75.

Tarlov, A. R., Ware, J. E., Greenfield, S., Nelson, E. C.,

Perrin, E., & Zubkoff, M. (1989). The medical outcomes

study: An application of methods for monitoring the results

of medical care. JAMA, 262(7), 925-930.

Toth, J. C. (1988). Measuring the stressful experience of

hospital discharge following acute myocardial infarction. In

C. Waltz & O. Strickland (Eds.) Measurement of nursing

outcomes: Volume one measuring client outcomes (pp. 3-23).

New York: Springer.

U. S. Congress (1986). Nurse practitioners, physician assistants, and certified nurse midwives: A policy analysis (Health

technology case study 37). OTA-HCS-37 Washington,, D.C.:

U.S. Government Printing Office.

Valentine, K. L. (1991). Comprehensive assessment of caring and its relationship to outcome measures. Journal of

Nursing Quality Assurance, 5(2), 59-68.

Ventura, M. R., Fox, R. N., Corley, M. C., & Mercurio, S.

M. (1982). A patient satisfaction measure as a criterion to

evaluate primary nursing. Nursing Research, 31(4), 226-230.

Ventura, M. R., Young, D. E., Feldman, M. J., Pastore, P.,

Pikula, S., & Yates, M. A. (1985). Cost savings as an indicator of successful nursing intervention. Nursing Research,

34(1), 50-53.

Vincent, P. & Price, J. R. (1977). Evaluation of a VNA mental health project. Nursing Research, 26, 361-367.

Votava, J. M., Cleveland, M. S., & Hiltunen, K. (1985). Home

care of the patient dependent on mechanical ventilation:

Home care policy development and goal setting using outcome criteria for quality assurance. Home Healthcare Nurse,

3(2), 18-25.
Ware, J. (1991, April) The use of health status and quality of

life measures in outcomes and effectiveness research. Paper

presented at the National Agenda Setting Conference on

Outcomes and Effectiveness Research, Alexandria, Virginia.

Wennberg, J., & Gittelsohn, A. (1982). Variations in medical care among small areas. Scientific American, 246,

120-134.

Wennberg, J. E. (1984). Dealing with medical practice variations: A proposal for action. Health Affairs, 3(2), 6-32.

Werley, H., & Lang, N. M. (Eds.) (1988). Identification of

the nursing minimum data set. New York: Springer Publishing.

Westra, B. (1991). Getting back to living life: Older adult’s

experiences at home after hospitalization, (Doctoral Dissertation, University of Wisconsin-Milwaukee, School of Nursing).

Wilson, A. (1988). Measurable patient outcomes: Putting

theory into practice. Home Healthcare Nurse, 6(6), 15-18.

Wolf, G. A., Lesic, L. K., & Leak, A. G. (1986). Primary

nursing: The impact on nursing costs within DRG’s. Journal

of Nursing Administration, 16(3), 9-11.