Evidence Report/Technology Assessment 

Number 137 

Cancer Care Quality Measures: Symptoms and 

End-of-Life Care 

Prepared for: 

Agency for Healthcare Research and Quality 

U.S. Department of Health and Human Services 

540 Gaither Road 

Rockville, MD 20850 

www.ahrq.gov 

Contract No. 290-02-003 

Prepared by:

Southern California Evidence-based Practice Center 

RAND, Santa Monica, CA 

SCEPC Directors 

Paul G. Shekelle, M.D., Ph.D. 

Sally C. Morton, Ph.D. 

Principal Investigators 

Karl Lorenz, M.D., M.S.H.S. 

Joanne Lynn, M.D. 

Reviewers

Sydney Dy, M.D. 

Ronda Hughes, Ph.D., M.H.S., R.N. 

Richard A. Mularski, M.D., M.S.H.S. 

Lisa R. Shugarman, Ph.D. 

Anne M. Wilkinson, Ph.D. 

Medical Editor 

Sydne J. Newberry, Ph.D. 

Programmer/Analyst 

Afshin Rastegar, M.S. 

Task Order Coordinator 

Cony Rolón, B.A. 

Staff Assistant 

Susan Chen, B.A. 

AHRQ Publication No. 06-E001 

May 2006
This report is based on research conducted by the Southern California Evidence-based Practice 

Center (EPC) under contract to the Agency for Healthcare Research and Quality (AHRQ), 

Rockville, MD (Contract No. 290-02-003). The findings and conclusions in this document are 

those of the author(s), who are responsible for its contents, and do not necessarily represent the 

views of AHRQ. No statement in this report should be construed as an official position of AHRQ 

or of the U.S. Department of Health and Human Services. 

The information in this report is intended to help clinicians, employers, policymakers, and others 

make informed decisions about the provision of health care services. This report is intended as a 

reference and not as a substitute for clinical judgment. 

This report may be used, in whole or in part, as the basis for development of clinical practice 

guidelines and other quality enhancement tools, or as a basis for reimbursement and coverage 

policies. AHRQ or U.S. Department of Health and Human Services endorsement of such 

derivative products may not be stated or implied. 
ii

This document is in the public domain and may be used and reprinted without permission except 

those copyrighted materials noted for which further reproduction is prohibited without the 

specific permission of copyright holders. 

Suggested Citation: 

Lorenz K, Lynn K, Dy S, et al. Cancer Care Quality Measures: Symptoms and End-of-Life 

Care. Evidence Report/Technology Assessment No. 137. (Prepared by the Southern California 

Evidence-based Practice Center under Contract No. 290-02-003.) AHRQ Publication No. 

06-E001. Rockville, MD: Agency for Healthcare Research and Quality. May 2006. 

None of the investigators has any affiliations or financial 

involvement that conflicts with the material presented in 

this report. 
iii

Preface 

 The Agency for Healthcare Research and Quality (AHRQ), through its Evidence-Based 

Practice Centers (EPCs), sponsors the development of evidence reports and technology 

assessments to assist public- and private-sector organizations in their efforts to improve the 

quality of health care in the United States. This report was requested and funded by the Centers 

for Disease Control and Prevention (CDC), the Centers for Medicare & Medicaid Services 

(CMS), and the National Cancer Institute (NCI). The reports and assessments provide 

organizations with comprehensive, science-based information on common, costly medical 

conditions and new health care technologies. The EPCs systematically review the relevant 

scientific literature on topics assigned to them by AHRQ and conduct additional analyses when 

appropriate prior to developing their reports and assessments. 

 To bring the broadest range of experts into the development of evidence reports and health 

technology assessments, AHRQ encourages the EPCs to form partnerships and enter into 

collaborations with other medical and research organizations. The EPCs work with these partner 

organizations to ensure that the evidence reports and technology assessments they produce will 

become building blocks for health care quality improvement projects throughout the Nation. The 

reports undergo peer review prior to their release. 

 AHRQ expects that the EPC evidence reports and technology assessments will inform 

individual health plans, providers, and purchasers as well as the health care system as a whole by 

providing important information to help improve health care quality. 

 We welcome comments on this evidence report. They may be sent by mail to the Task Order 

Officer named below at: Agency for Healthcare Research and Quality, 540 Gaither Road, 

Rockville, MD 20850, or by e-mail to epc@ahrq.gov. 

Carolyn M. Clancy, M.D. 

Director 

Agency for Healthcare Research and Quality 

Andrew C. von Eschenbach, M.D. 

Director, National Cancer Institute 

National Institutes of Health 

Acting Commissioner of Food and Drugs 

U.S. Food and Drug Administration 

Mark B. McClellan, M.D., Ph.D. 

Administrator 

Centers for Medicare & Medicaid Services 

Julie Louise Gerberding, M.D., Ph.D. 

Director 

Centers for Disease Control and Prevention 

Jean Slutsky, P.A., M.S.P.H. 

Director, Center for Outcomes and Evidence 

Agency for Healthcare Research and Quality 

Beth A. Collins Sharp, Ph.D., R.N. 

Acting Director, EPC Program 

Agency for Healthcare Research and Quality 

William Lawrence, M.D., M.S. 

EPC Program Task Order Officer 

Agency for Healthcare Research and Quality 
iv

Acknowledgments 

We acknowledge the important contributions of Rena Hasenfeld Garland, B.A., who conducted 

all Internet searches (for guidelines and of relevant professional associations). 
v

Structured Abstract 

Objectives. To systematically identify quality measures and the evidence for them⎯ to 

support quality assessment and improvement in the palliative care of patients with cancer in the 

areas of pain, dyspnea, depression, and advance care planning (ACP), and to identify important 

gaps in related research. 

Data Sources. MEDLINE®, CINAHL©, and PsycINFO© in English 1995-2005. We also 

conducted an extensive Internet search of professional organizations seeking guidelines and 

other grey literature (i.e., not published in peer-reviewed journals) using similar terms and 

attempted to contact all measure developers. 

Review Methods. We searched using terms for each domain for patients (adults and children) 

with a cancer diagnosis throughout the continuum of care (e.g., diagnosis to death). Pain and 

depression searches were limited to cancer, but we searched broadly for dyspnea and ACP, 

because the evidence base for dyspnea is more limited and experts advised that ACP measures 

would be generalizable to cancer. Measures were included if they expressed a normative 

relationship to quality and included a measurable numerator and denominator. Citations and 

articles were each reviewed/abstracted by two of six palliative care researcher/clinicians who 

described populations, testing, and attributes for each measure. 

Results. The literature search identified 5,187 titles, of which 4,650 were excluded at abstract 

review. Of 537 articles, only 25 contained measures: 21 on ACP, 4 on depression, 2 on dyspnea, 

and 12 on pain. Ten relevant measure sets were identified: ACOVE, QA Tools, Cancer Care 

Ontario, Cancer Care Nova Scotia, Dana-Farber, Georgia Cancer Coalition, University Health 

Consortium, NHPCO, VHA, and ASCO. We identified a total of 40 operationalized and 19 nonoperationalized measures. The most measures were available for pain (12) and ACP (21), 

compared with only 4 for depression and 2 for dyspnea. Few of the measures were published, 

and few had been specifically tested in a cancer population. 

Conclusions. A large number of measures are available for addressing palliative cancer care, 

but testing them in relevant populations is urgently needed. No measures or indicators were 

available to evaluate the quality of supportive pediatric cancer care. Basic research is urgently 

needed to address measurement in populations with impaired self-report. Funding field testing 

of highest quality measures should be an urgent patient and family-centered priority to meet the 

needs of patients with cancer. 

vii 

Contents 

Executive Summary.........................................................................................................................1 

Evidence Report.............................................................................................................................9 

Chapter 1. Introduction ..................................................................................................................11 

Supportive Care and the Cancer Continuum ...........................................................................11 

Summary..................................................................................................................................13 

Chapter 2. Methods.......................................................................................................................15 

Final Task Order Questions .....................................................................................................15 

Overview..................................................................................................................................15 

Technical Expert Panel and Approach to the Report...............................................................16 

Analytic Framework ................................................................................................................17 

Processes Relevant to Symptom Care and Advance Care Planning Quality Measurement....17 

Evidence Sources and Searches...............................................................................................19 

Literature Searches.............................................................................................................19 

Grey Literature - Internet Search .......................................................................................20 

Grey Literature - Measure Developers ..............................................................................22 

End-of-Life Care and Outcomes Literature Database........................................................22 

Title Screening, Article Review, and Selection of Individual Studies ....................................22 

Summarizing the Evidence for Key Questions........................................................................24 

Peer Review Process................................................................................................................25 

Chapter 3. Results .........................................................................................................................27 

Overview..................................................................................................................................27 

Literature Flow.........................................................................................................................30 

Pain ..........................................................................................................................................33 

Introduction........................................................................................................................33 

Measures and Indicators ....................................................................................................34 

Potential Indicators and Measurement Gaps......................................................................40 

Dyspnea....................................................................................................................................43 

Introduction........................................................................................................................43 

Measures and Indicators ....................................................................................................43 

Potential Indicators and Measurement Gaps......................................................................45 

Depression................................................................................................................................48 

Introduction........................................................................................................................48 

Measures and Indicators ....................................................................................................48 

Potential Indicators and Measurement Gaps......................................................................50 

Advance Care Planning............................................................................................................52 

Introduction........................................................................................................................52 

Measures and Indicators ....................................................................................................52 

Potential Indicators and Measurement Gaps......................................................................62 
viii

Measure That Crosses All Domains ..................................................................................63 

Summary..................................................................................................................................63 

Chapter 4. Discussion ...................................................................................................................65 

Future Research .......................................................................................................................65 

Field Testing in Relevant Populations ...............................................................................65 

The Need for Measures To Address Impaired Self-Report ...............................................66 

The Lack of Pediatric Measures ........................................................................................67 

Defining the “End-of-Life”................................................................................................67 

The Publication Gap for Quality Measurement.................................................................67 

Strengthening Quality Measurement of Depression in Cancer..........................................68 

Building an Evidence Base To Compare Population Subgroups.......................................68 

Conclusions..............................................................................................................................69 

References Cited in the Evidence Report ......................................................................................71

Figures 

Figure 1. Article flow.....................................................................................................................32 

Tables 

Table 1. Processes potentially relevant to symptom care and advance care planning...................19 

Table 2. Overview of identified measure sets................................................................................27 

Appendixes 

Appendix A: Technical Expert Panel Members and Peer Reviewers 

Appendix B: TEP Worksheet 

Appendix C: Literature Search Strategies 

Appendix D: RAND SCEPC Cancer Quality Measures Project - Measure Developer 

Contact Form 

Appendix E: Cancer Measure Review Article Screener 

Appendix F1: Quality Measures and Indicators for Supportive Cancer Care: Pain 

Appendix F2: Quality Measures and Indicators for Supportive Cancer Care: Dyspnea 

Appendix F3: Quality Measures and Indicators for Supportive Cancer Care: Depression 

Appendix F4: Quality Measures and Indicators for Supportive Cancer Care: 

Advanced Care Planning 

Appendix F5: Quality Measures for Cancer Palliation: Measures that Cross All Domains 

Appendix G: Peer Reviewer Comments 

Appendix H: Excluded Studies 

The Appendixes and Evidence Tables cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/eolcanqm/eolcanqm.pdf 
 1 

Executive Summary 

Overview 

Cancer is a common, costly problem that affects many Americans and their families. 

Approximately 9.6 million Americans who were alive in January 2000 were estimated to be 

living with a previous diagnosis of cancer, and approximately 1.4 million cases of cancer were 

diagnosed in 2004. Direct medical expenditures on cancer exceeded $180 billion in 2000. The 

most common malignancies include lung, breast, colorectal, and prostate, for which 2003 

estimates of 5-year survival rates varied from 15% (lung) to 99% (prostate). 

Cancer imposes its burdens from diagnosis to death, and even survivors live with substantial 

impairments; therefore, supportive care is an important consideration of all phases of 

management. Supportive care addresses the direct and treatment-related impacts of cancer 

including, pain, and symptom management; the psychosocial context, including spirituality; and 

caregiving. Supportive approaches focus on assisting patients and caregivers to maximize wellbeing and can ameliorate many of the debilitating consequences of living with a cancer 

diagnosis, regardless of cancer stage. As early diagnosis and more effective treatment extend the 

experience of living with cancer, seamless integration of supportive principles and approaches 

becomes even more imperative. 

In spite of the prevalence of cancer and its enormous costs, little effort has been made to 

systematize the assessment of the quality of palliative cancer care. AHRQ and a collaborative of 

Federal Partners, including the Centers for Disease Control and Prevention (CDC), the Centers 

for Medicare & Medicaid Services (CMS), and the National Cancer Institute (NCI), requested a 

systematic review of quality measures for supportive cancer care. This report, prepared by the 

Southern California Evidence-Based Practice Center, is intended to assist the National Quality 

Forum (NQF) by describing quality measures in the areas of pain, dyspnea, depression, and 

advance care planning (ACP) and to identify priorities for a quality measurement research 

agenda. 

Reporting the Evidence 

 

This report addressed the following questions: 

1. What quality-of-care measures are available and what evidence supports those 

measures to assess pain, depression, dyspnea, and advance care planning for 

patients with cancer, including: 

a. Patient assessment, 

b. Intervention and appropriate management, and 

c. Timeliness and effectiveness of intervention. 

2. What gaps in knowledge about quality measurement are evident from the currently 

available literature, including absence of measures or measures lacking evidence of 

their scientific soundness, whether for the population of cancer patients as a whole 

or for specific subpopulations?
 2 

Methodology 

In consultation with the AHRQ Task Order Officer (TOO) and the NQF, we created a 

Technical Expert Panel (TEP) of leading scientists and clinicians with expertise in oncology and 

palliative medicine and a broad knowledge of research and policy issues in the field of oncology 

and palliative care in the United States to guide the evidence report. Project staff worked closely 

with AHRQ, NQF, and the TEP to refine the research questions and focus on the relevant 

outcomes. Drs. Karl Lorenz and Joanne Lynn met with the NQF and members of their Steering 

Committee in December 2004 to discuss the task order question and plans for the systematic 

review. 

Originally, Question 1 specified focusing on “measures to assess pain, depression, distress, 

and advance care planning.” However, the Committee indicated that distress had been 

incorporated in order to address broad psychosocial and spiritual concerns. However, the group 

expressed concerns relative to the feasibility of conducting a systematic review of distress. They 

also agreed that the broader spiritual and psychosocial concerns could be taken into account in 

conducting the literature review of depression measures. Because dyspnea is an important 

symptom among cancer patients and a recent systematic review found promising evidence 

related to its treatment, the Committee recommended that RAND should add dyspnea as a topic. 

After consulting with the AHRQ Task Order Officer, the review team agreed to focus on pain, 

depression, dyspnea, and advance care planning. 

The first of a series of calls was held on February 9, 2005 with our TOO and the TEP. An 

important issue that arose for discussion during our initial conference call with the TOO and TEP 

in February 2005 was the extent to which searches should be restricted to cancer. Limiting the 

search logic to cancer provided an extremely limited number of references, particularly for ACP 

and dyspnea quality measures. Our TEP advised that measures that were applied to the care of 

symptoms and advance planning in other disease states would also be relevant to cancer; thus, 

for those two topics, we broadened the search to include conditions other than cancer. However, 

because our initial searches indicated that the literature on pain and depression was extensive, we 

restricted our review of these citations to cancer only. 

Literature Search and Review 

We searched MEDLINE®, CINAHL©, and PsycINFO© for English language literature 

(January 1995 - February 2005) using terms for pain, dyspnea, depression, and ACP combined 

with terms for cancer, quality assessment or improvement, and palliative care. Because our 

previous experience identified limitations in the indexing of quality measures in scientific 

databases, we used pre-specified terms to search the Internet for evidence-based standards or 

quality measures that had been developed by healthcare organizations or specialty societies. 

(February to March 2005). We directly contacted developers of all measures or measure sets we 

identified and sought additional information using a ‘contact form.’ 

Study Selection and Abstraction 

Six reviewers, with clinical backgrounds in internal medicine and nursing, and research 

careers in palliative care, organized into four teams by domain, each reviewed documents 
 3 

independently, resolving disagreement by consensus. We applied the following inclusion and 

exclusion criteria at both the title/abstract and article screening phases. Study reports or other 

documents were accepted only if they contained either indicators or measures. An indicator was 

defined as “a descriptive statement with a normative relationship to quality that includes a 

numerator and denominator and that is expressed as a measurable standard.” A measure is an 

indicator fully specified for measurement, including data elements, the data collection approach, 

data sources, analysis, and presentation. 

Documents were excluded if they: 

• Were non-Western (i.e., not United States, Canada, Australia, New Zealand, Great 

Britain, or European), 

• Were published in a non-English language, 

• Focused exclusively on a disease other than cancer UNLESS they were on the topic of 

pain or depression, or 

• Focused on cancer but considered only a topic other than ACP, depression, dyspnea, or 

pain, 

• Addressed only specific types of cancer and/or processes of care that were not 

generalizable across the most common cancer conditions (e.g., particular pain syndromes 

for tumor subtypes), 

• Guidelines were excluded if they did not contain any description of their methods or how 

they were developed. 

For the domains of pain and depression, if a measure or indicator was developed for 

populations (e.g., hospice) that generally include cancer patients, but the development or field 

testing for the indicator did not explicitly include cancer patients, we did not exclude the 

document or citation. 

Articles accepted at the screening stage were subjected to full abstraction using standardized 

forms. Some studies or documents described only indicator or measure development, whereas 

others described testing or use in an actual population. Detailed information on the attributes of 

these tools was abstracted. 

The Report 

In general, the review was organized around fundamental domains of symptom 

management—assessment, treatment, and follow-up. Assessment includes all activities related 

to evaluating the presence of a symptom. Treatment includes all activities subsequent to 

identification related to mitigating the symptom. Follow-up includes re-assessment and/or retreatment of the symptom. For consistency, ACP was also organized around these domains since 

they are reasonably analogous, although for ACP, the task of ‘assessment’ is to evaluate goals 
 4 

and preferences, ‘treatment’ is the application of advance care plans, and ‘follow-up’ refers to 

actual implementation. 

Peer Review 

We received comments from eight peer reviewers and eleven TEP members who reviewed 

our draft report. We compiled the comments and made appropriate changes to the report. 

Findings 

Literature Review 

The literature search identified 4,580 titles. We identified an additional 389 articles from the 

database we had assembled for a systematic review of research on End-of-Life Care and 

Outcomes. Our web search identified 105 guidelines. An additional 109 articles were suggested 

from by the project members. Four additional articles were suggested by our TEP upon review 

of the draft version of this evidence report. In total, the RAND reviewers examined 5,187 titles. 

Of the 5,187 possibly relevant titles, 4,599 were excluded at abstract review, leaving 588. 

Repeat review by one principal investigator (KL) excluded an additional 51 titles, leaving 537 

reports and guidelines. 

Of the 537 articles ordered, we retrieved 536 prior to the cut-off date (June 10, 2005). 

Screening resulted in the exclusion of 485 articles: 347 included no indicator; 113 had no domain 

of interest; 8 considered a disease other than cancer; 5 were guidelines with no methods; 6 

originated in a non-Western location; 2 presented duplicate data; 2 were duplicate articles; and 2 

were useful for background only. 

Of the remaining 51 articles, 25 that contained measures and/or indicators and described their 

methods were reviewed in detail. Potential indicators were identified by each team based on 

their working knowledge of research literature and prioritized based on each team’s opinion of 

the most important gaps in measurement. The remaining 26 articles contained only potential 

indicators. As one article can report on multiple topics and may include multiple measures 

and/or indicators/potential indicators the numbers in the domain boxes in Figure 1 represent the 

numbers of measures, indicators or potential indicators that were abstracted into the Evidence 

Tables and not the actual number of articles.

Pain - Measures and Indicators 

This category addressed the general conceptual areas of pain assessment (five measures, two 

indicators) with numeric or nonspecific scales; treatment (two measures, two indicators), 

including timely intervention; side effect/constipation prophylaxis; radiotherapy; and follow-up 

(four measures, one indicator) including timely relief, satisfaction, and degree of relief (Table 2). 

Assessment measures and indicators specifically focused on cancer addressed routine pain 

assessment using numeric or non-specific scales in inpatient, intensive care, outpatient, and 

unspecified settings. 

 
 5 

Dyspnea - Measures and Indicators 

Dyspnea measures and indicators addressed the general conceptual areas of routine 

assessment (one measure) with a numeric scale; treatment (three indicators), which included any 

treatment, terminal treatment, and addressing hypoxia; and follow-up (one measure), addressing 

timely relief. 

Depression - Measures and Indicators 

Depression measures and indicators addressed the general conceptual areas of assessment 

(one measure, one indicator), including psychosocial, spiritual assessment, treatment (two 

measures) including regular assessment or treatment. 

ACP - Measures and Indicators 

ACP measures and indicators addressed the general conceptual areas of assessment (1 measures, 

4 indicators), including surrogate identification, preference evaluation, patient/family 

participation, and timely planning; ACP application (2 measures), addressing documentation 

across venues and evaluation of specific life sustaining preferences; and ACP follow-up (10 

measures, 1 indicators), which addressed preference-treatment consistency, hospice admission, 

late life utilization of hospital or intensive care settings, emergency care, and chemotherapy. 

Recommendations for Future Research 

We identified priorities for future research in a number of important areas: 

Field Testing in Relevant Populations 

Many of the measures may be useful now for local quality assessment and improvement, but 

most measures lack even basic published evidence of reliability and validity. Before these 

measures are deployed at higher levels of the health care system, information about their ability 

to discriminate quality performance and responsiveness to change will be required. Furthermore, 

health care systems must consider the cost and feasibility of data collection and must be assured 

that measures are at low risk for perverse incentives and that they can provide timely feedback. 

The Need for Measures To Address Impaired Self-Report 

None of the indicators address how to evaluate symptoms among individuals with impaired 

self-report, which can be due to either temporary (e.g., delirium) or durable factors (e.g., 

dementia, brain metastasis). There is no doubt that delirium is common among cancer patients, 

although its incidence and prevalence are not well studied. Some recent reports noted a 

prevalence of 18-50% among patients with cancer in general, and observed rates are even higher 

in certain treatment settings such as the intensive care unit or the nursing home, where almost 

60% of the population is also living with dementia. 
 6 

The Lack of Pediatric Measures 

We noted the complete absence of pediatric measures or indicators. Cancer represents the 

second leading cause of death among children ages 5 to 15 in the United States, and serious 

deficiencies in the care of pediatric cancer cases have been noted. It is insufficient to propose 

that adult measures or indicators could simply be adopted for pediatric care, as the kinds of 

disease, challenges of symptom reporting and intervention, and basic approach to care differ so 

significantly between pediatric and adult oncology care. 

Defining the “End-of-Life” 

The lack of consensus on definitions of “end-of-life” presents what has been called the 

“denominator” problem. Many of the indicators define end-of-life retrospectively (e.g., last six 

months of life). This definition does not provide guidance on important clinical service, and 

retrospective analyses may introduce a number of biases, since the timing of death is often not 

predictable. Prospective indicators would account for patient preferences but will require 

methodological work to define which cancer patients (those with certain types that are usually 

fatal, those at advanced stage) should be in the denominator. 

The Publication Gap for Quality Measurement

An important finding of our report was that much of the evidence on the measures that we 

identified was unpublished. Additional information that might be available to inform the 

evidence base for these measures was likely collected by non-academic organizations and is 

probably unpublished. Much of the actual experience with measures and measurement has 

probably been reported in an anecdotal rather than an archival form. Those who conduct future 

systematic reviews should be aware of this important issue and make an explicit effort to capture 

this unpublished information. Methods for systematically compiling and analyzing this type of 

data might help to improve the evidence base for quality measurement. 

Strengthening Quality Measurement of Depression in Cancer 

Given the recognized prevalence of depression among cancer patients, surprisingly few 

indicators have been evaluated depression in cancer. Indicators developed for other settings may 

be appropriate but have not been tested in this population. For late-stage disease, more 

fundamental clinical research is needed, for example to assess the optimal approach to screening 

and the usefulness of short term treatment. 

Building an Evidence Base To Compare Population Subgroups 

We found insufficient evidence to comment on population differences. An important priority 

for future research is evaluating measures in important sub-populations (e.g., those that differ by 

race/ethnicity, language, or gender). It is also important to define subgroups of cancer by stage 

of illness and to understand the performance of indicators among major subgroups, including 
 7 

disease type and settings of care. Fewer indicators are available for outpatient than for inpatient 

care evaluation, even though the majority of patient care over time occurs in the outpatient 

setting. 
 
EVIDENCE REPORT 

11 

Chapter 1. Introduction 

 Cancer is a common and costly problem that affects many Americans, their families, and 

their caregivers, since many are affected by a cancer diagnosis. Approximately 9.6 million 

Americans who were alive in January 2000 were living with a diagnosis of cancer, and 

approximately 1.4 million new cases of cancer were diagnosed in 2004.1 The most recent 

estimate of lifetime risk suggests that males have an approximately one in two risk and women a 

one in three risk of developing cancer.1

 Direct medical expenditures on cancer exceeded $180 

billion in 2000. The costs of care are concentrated in the time just following diagnosis and the 

last few months before death, with about one-third of expenditures in the final year.2

 The most 

common malignancies include lung, breast, colorectal, and prostate, for which 2003 estimates of 

5-year survival rates varied from 15% (lung) to 99% (prostate).3

 

In spite of the prevalence of cancer and its enormous personal and societal costs, little effort 

has been made to systematize the assessment of the quality of palliative cancer care. In 2004, the 

Agency for Healthcare Research and Quality requested a systematic review of quality measures 

for supportive cancer care in the areas of pain, dyspnea, depression, and advance care planning 

(ACP). AHRQ and a collaborative of Federal Partners, including the Centers for Disease 

Control and Prevention (CDC), the Centers for Medicare & Medicaid Services (CMS), and the 

National Cancer Institute (NCI), sponsored the report in order to assist the National Quality 

Forum (NQF) by identifying and describing quality measures for supportive cancer care 

including the evidence that they are ready or nearly ready for incorporation into routine clinical 

practice. An additional purpose of this report was to identify gaps in measurement that could be 

the basis for a quality measurement research agenda, including areas in which no measures exist, 

or in which measures exist but for which additional evidence is needed. 

 

Supportive Care and the Cancer Continuum 

Cancer imposes its burdens on patients and caregivers from diagnosis to death, and even 

long-term survivors often live with substantial impairments. Supportive care addresses the direct 

and treatment-related impacts of cancer, pain and symptom management and the psychosocial 

context, including spirituality and the challenges of caregiving; it is important in all phases of 

cancer care. Symptoms may prompt (e.g., dyspnea) or be precipitated by (e.g., depression) the 

diagnosis of cancer, and treatment and progressive illness typically impose substantial 

symptomatic burdens. Some common cancers are diagnosed at a late stage: Only 16% of lung 

cancer cases are localized at the time of diagnosis.1

 Supportive approaches focus on assisting 

patients and caregivers to maximize well-being and can ameliorate many of the debilitating 

consequences of living with a cancer diagnosis, regardless of cancer stage.4-6 As early diagnosis 

and more effective treatment extend the experience of living with cancer, seamless integration of 

supportive principles and approaches becomes even more imperative. 

Pain is one of the most common symptoms in patients with cancer and is often inadequately 

treated when measured by patient or staff perceptions, particularly in vulnerable subgroups.7-9

Cancer pain can result from direct primary or metastatic tumor involvement, including bone and 

neurologic pain; from diagnostic interventions; and as a side effect of chemotherapy or radiation 

therapy.7

 One-third to one-half of patients in active treatment, and three quarters of patients with 

advanced cancer experience pain.10 Chronic pain also may be a component of cancer 
12 

survivorship, and many patients with cancer also have pain due to unrelated causes. Many 

studies have documented that pain is often inadequately treated, and the quality of treatment 

varies widely among centers.8 Racial and ethnic disparities in pain assessment and management 

have been described.11

Dyspnea is an unpleasant, distressing, and common experience among cancer patients: over 

half of cancer patients experience significant shortness of breath.12-18 In the final months and 

days of life with cancer, rates of dyspnea range from 60% to 90%, with the symptom especially 

prominent in those with lung cancer.13-16,19 For example, 87% of 673 patients with stage III or 

IV non-small cell lung cancer reported dyspnea.19 Numerous etiologies for dyspnea exist in 

cancer, including pulmonary or pleural metastasis, anemia, and muscular weakness. Despite the 

prevalence of dyspnea among cancer patients, the majority of dyspnea research has been 

conducted in patients with chronic pulmonary conditions.13,20,21 However, a promising evidence 

base supports assessment and interventions to improve the experience of cancer patients with 

dyspnea.21

Depression has received increasing attention as a cause of distress or suffering in patients 

living with cancer.10 Studies suggest that around one quarter of cancer patients experience 

depression during the course of their care.10 Depressive symptoms often accompany the 

diagnosis of cancer,22 and depression is strongly associated with other symptoms⎯ such as 

pain⎯ that may fluctuate in severity over the course of cancer treatment.10 Depressive 

symptoms accompany patient and caregiver experiences throughout all stages of cancer care, 

including wrestling with treatment disappointments, the complications of therapy and of cancer 

itself, and progressive illness. Evidence has demonstrated an important relationship between 

depression and spiritual or existential well-being, especially related to how patients and 

caregivers maintain hope along the trajectory of the cancer experience.23-26

Another important area of consideration regarding cancer care is ACP. Early on, the 

emphasis in ACP was on legal initiatives, “living wills,” and other similar documentation; 

however, over time, the concept has broadened to emphasize a process of effective 

communication and planning ahead to shape an appropriate course of care.27 Now, high quality 

practice includes ACP for a number of reasons: to allow patients to shape care to their 

preferences, to avoid imposition of unwanted treatment in urgent situations, to relieve patient and 

family anxiety, and to ensure availability of the specific practical arrangements that will allow 

the patient to have optimal care, on his or her own terms. Cancer treatment commonly requires 

potentially difficult interventions such as chemotherapy or surgery. The utilization of high 

intensity care such as hospitalization, particularly in an ICU, varies significantly among elderly 

cancer patients, without a clear relationship to patient-centered outcomes.28-30 Because of the 

emphasis on documentation of choices among a narrow set of treatment preferences, research on 

this topic emphasizes outcomes related to health care utilization generally or the actual use of 

particular services.21 However, informed consent and self-determination are cardinal ethical 

principles, and self determination is endorsed by both patients and caregivers as an important 

value in late life care.31 Furthermore, planning ahead allows deliberate forgoing of undesired 

care. 
13 

Summary 

Supportive care emphasizes treatment to minimize symptoms and enhance patient and 

caregiver well-being and is relevant across the trajectory of cancer care, from diagnosis and 

initial treatment to chronic progressive illness and the end-of-life. Palliative care is a term that is 

sometimes used interchangeably with the term supportive care, except that palliative care 

generally refers to a focus on care for patients in more-advanced stages of illness. Pain, dyspnea, 

depression, and ACP are common, high impact concerns to be considered as targets for quality 

assessment and improvement. All of these areas represent problems that affect many patients 

receiving care for common malignancies, although the importance of particular symptoms varies 

among major cancers and throughout the illness trajectory. Pain, dyspnea, and depression, and 

ACP represent areas in which substantial initial research has begun to define the scope of the 

challenges and effective interventions for patients with cancer. ACP is relevant to all phases of 

cancer, as potentially difficult treatments may be employed, and effective communication is 

always important to cancer patients and their caregivers. 

15 

Chapter 2. Methods 

Final Task Order Questions 

The Agency for Healthcare Research and Quality (AHRQ), Centers for Disease Control and 

Prevention (CDC), Centers for Medicare & Medicaid Services (CMS), and the National Cancer 

Institute (NCI) requested this systematic review. They also requested that the review be 

available in time for review by the National Quality Forum’s (NQF) Steering Committee in 

September 2005. The following key questions were addressed in the response to the Request for 

Task Order (RFTO): 

1. What quality-of-care measures are available and what evidence supports those 

measures to assess pain, dyspnea, depression, and advanced care planning for 

patients with cancer, including: 

a. Patient assessment, 

b. Intervention and appropriate management, and 

c. Timeliness and effectiveness of intervention. 

2. What gaps in knowledge about quality measurement are evident from the currently 

available literature, including absence of measures or measures lacking evidence of 

their scientific soundness whether for the population of cancer patients as a whole 

or for specific subpopulations? 

Overview 

In order to accomplish the tasks as directed, we assembled a team of clinical and 

methodological experts and staff who worked closely with the directors and staff of the Southern 

California Evidence Based Practice Center (SCEPC). Dr. Karl Lorenz led the day-to-day work 

of the review and writing teams with the close assistance and regular involvement of Drs. Joanne 

Lynn, Paul Shekelle, and Sally Morton. Our team included six literature reviewers (with Dr. 

Lorenz) with experience and expertise in palliative care and with diverse clinical training and 

experience, including an oncology nurse, one intensivist, and two general internist/palliative care 

physicians. The reviewers were trained and experienced in the critical analysis of scientific 

literature and had conducted a systematic review of palliative care in December 2004.21

The entire team met weekly to review and refine the methodology of the task order. 

Meetings and teleconferences of the SCEPC staff with the TEP helped specify issues central to 

this report within the framework of the key questions. The SCEPC conducted a comprehensive 

search of the medical literature to identify studies that addressed the key questions. Staff 

conducted an extensive search for grey (unpublished) literature, including a Web search; 

reviewed relevant articles; contacted quality-of-care measure developers; compiled tables of 

measures; appraised the methodological validity of studies and measures; and summarized the 

results. 
16 

Technical Expert Panel and Approach to the Report 

In consultation with the AHRQ Task Order Officer (TOO) and the NQF, we created a 

Technical Expert Panel (TEP) to guide the evidence report. We invited a multidisciplinary group 

of leading scientists and clinicians with expertise in oncology and palliative medicine and a 

broad knowledge of research and policy issues in the field of oncology and palliative care in the 

United States to participate. The list of potential technical experts and their curriculum vitae 

were submitted to the Task Order Officer for approval, and the final list of members is included 

in Appendix A*

. 

Project staff worked closely with AHRQ, NQF, and the TEP to refine the research questions 

and focus on the relevant outcomes in the topic areas. Drs. Karl Lorenz and Joanne Lynn met 

with the NQF and members of the NQF Steering in December 2004 to discuss the task order 

question and plans for the systematic review. At that initial meeting, the specific domains that 

were the focus of the task order and the task order questions were discussed at length and 

finalized. 

The first of a series of teleconferences was held on February 9, 2005 with our TOO and the 

TEP. In preparation for the call, we conducted a series of exploratory MESH-based literature 

searches for each of the domains. We also conducted an initial Web-based search for grey 

literature, including important guidelines and recommendations of major specialty societies. We 

solicited the TEP’s knowledge about the relative strengths and weaknesses of the evidence base 

for quality measures in the domains of interest and asked them to complete a worksheet listing 

existing measures and important sources, as well as priorities for improving quality measurement 

(Appendix B*

). 

An important issue that arose for discussion with our TEP was the extent to which our 

searches should be restricted to cancer. Although the purpose of the systematic review was to 

provide measures for improving palliative cancer care, restricting the search logic to cancer 

would severely limit the number of pertinent references, particularly for ACP and dyspnea 

quality measures. Furthermore, our TEP advised that measures of the care of symptoms and 

advance planning in other disease states would also be applicable to patients with cancer and 

should be considered in the systematic review, as they would likely be useful to the NQF 

Committee. 

During the call, we reviewed terms, logic, and results of our initial searches. We finalized 

our formal literature (Appendix C*

) and Web searches and finalized the list of organizations from 

which we would attempt to retrieve guidelines and other relevant information. Because our 

initial searches indicated that the literature on pain and depression was relatively extensive, we 

restricted our review of these citations to cancer only, because of resource limitations. However, 

for the topics of dyspnea and ACP, we followed our TEP’s advice and in consultation with our 

TOO, broadened the search to include conditions other than cancer. We limited our search to the 

past decade because our experience with this topic suggested that most of the relevant research 

would be relatively recent based on typical progress in medical knowledge.21 Although the task 

order specified limiting the review to US measures only, we decided to include all Englishlanguage sources in the systematic review and to highlight the origin of particular measures in 

our report. 

 

*

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/eolcanqm/eolcanqm.pdf 
17 

Analytic Framework 

According to the model of Donabedian, the evaluation of quality of care includes structure, 

process, and outcomes.32 Structural measures focus on the capacity of the health care delivery 

system and depend upon demonstrating a relationship between structure and the health outcomes 

of interest, which is often difficult to establish. Process-of-care measures evaluate whether 

providers deliver the appropriate service for the patient’s presenting condition. Outcome 

measures directly assess relevant effects of care on the patient (or possibly, in the case of 

palliative care, the caregiver). Focusing on either processes or outcomes alone has both 

advantages and disadvantages. 

Compared with outcomes, processes are more numerous (thus providing more measurement 

options), they are more directly attributable to the health care system rather than the patient, and 

the performance of some processes can be measured in routine health care records or data. 

When processes are selected for quality monitoring, they should exhibit a strong relationship to 

outcomes. Even when outcomes are satisfactory, inadequate processes of care can reflect 

problems with health care provision. 

In contrast, outcomes provide direct insight into relevant effects of care, but they can be 

difficult to measure, especially in frail or very sick patients, they may not be routinely measured 

in health care encounters, and they are affected by the individual patient’s health status and 

comorbidities. Thus, outcomes must typically be risk-adjusted, but adjusting for patient 

characteristics can be difficult. In addition, the low frequency of serious adverse events requires 

large sample sizes. The task order directed us to focus our attention on process measures, 

because they would be most relevant to NQF’s concerns. Thus, our review and discussion 

emphasize process measures; however, for the sake of completeness, our report also includes 

outcome measures. 

Processes Relevant to Symptom Care and Advance Care 

Planning Quality Measurement 

We organized the review in general around three domains of care that are fundamental to 

symptom management – assessment, treatment, and follow-up (Table 1). Assessment includes 

all activities related to measuring or evaluating the presence of a symptom in a patient. 

Treatment includes all activities (including distinct concurrent or stepwise activities such as 

medication and counseling) subsequent to identification that are related to mitigating the 

symptom. Follow-up includes re-assessment and/or re-treatment of the initial symptom. We 

organized our review of ACP around these same domains for clarity since they are reasonably 

analogous, although “assessment” becomes evaluation of goals and preferences, “treatment” is 

the application of advance care plans, and “follow-up” refers to actual implementation. 

Both symptom management and ACP fundamentally begin with the patient’s own symptom 

description. Thus, assessing self-report ability is a fundamental process to be considered for 

quality measurement and has important implications for denominator determinations in these 

domains. Patients who are impaired in this regard may need to “opt-out” of measures, or 

providers may need to seek alternative measures that are appropriate for them. Symptoms are 

“subjective evidence of disease,”33 and ACP requires eliciting the patient’s values and care 

preferences. Our review included both adults and children, and self-reports may be impossible 
18 

for very sick or very young individuals. Pediatric assessment must be developmentally 

appropriate.34 Adults may experience a variety of sensory or expressive impairments, most 

commonly dementia in the elderly, that interfere with self-expression. Surrogate reporting is 

essential for symptom assessment in young children and adults with impairments; and 

designation of a surrogate for decision-making and care planning are critical for ACP. 

Symptom management begins with addressing the presence of the symptom, as well as 

frequency, severity, and bother or functional impact.35,36 Specific historical, physical 

examination, and technical diagnostic tests are employed in searching for specific etiologies.37

Treatment must consider targeting specific etiologies as well as relieving symptoms alone. 

Pursuing specific symptom etiologies involves tradeoffs in the burden for patients, even when 

those etiologies are identified. Therefore, measuring the quality of symptom care may need to 

include patients’ right to “opt-out” among care preferences as well as considering the 

contraindications for certain supportive care diagnostic or treatment algorithms.38 Some 

treatments for dyspnea such as supplemental oxygen, may be relatively easy to arrange and 

tolerate, but others such as pulmonary stenting are more challenging. For that reason, the phase 

of illness and preferences may moderate decisions about appropriate diagnostic testing or the 

specific etiologic treatment.39 

Differences in the time course of symptom occurrence and treatment, as well as the side 

effect profile of therapies, offer considerations for quality measurement. Some symptoms 

fluctuate over brief intervals, and the reliability and validity of measurement vary with patients’ 

cognitive ability and the length of the retrospective lookback.40 Chronologic considerations also 

affect appropriate follow-up intervals. For example, pain and dyspnea may fluctuate over short 

periods; whereas, depression is typically a more durable symptom. However, depression also 

varies with other physical symptoms (e.g., pain), and transient mood states are common in 

cancer.7,41 The frequency with which these effects influence measurement varies with the 

treatment setting. Although follow-up of the symptom depends on both chronological variability 

in the symptom and expected timing of responses to treatment, the side effects of treatment may 

be experienced before the benefits. High quality care includes prophylaxis for some side effects 

(e.g., constipation with opioids); whereas, other effects (e.g., sedation with opioids) require 

timely monitoring and supportive care until they abate. 

Measures of ACP as a component of quality care may reflect a variety of processes or steps 

that are necessary in focusing on the most crucial medical decisions that are likely to confront the 

patient and the process of considering alternative care approaches. Measurement may need to 

account for the patient’s changing clinical status, because preferences about treatments fluctuate 

over time.42, 43 An important consideration for culturally sensitive care is whether or not care 

planning begins with an assessment of informational preferences and how others should 

participate in decisions. Elements of care planning include prediction of the clinical situation or 

prognostication, awareness of alternative treatment plans, eliciting preferences through clear 

communication, including key decisionmakers, and synthesizing preferences and alternatives. 

Advance care plans must be available to medical providers when patients need them and through 

important transitions in care. Ultimately, care plans should be reflected in the actual medical 

treatment a patient receives. 
19 

Table 1: Processes potentially relevant to symptom care and advance care planning 

Symptom Management Advance Care Planning 

Assessment

 Evaluation of self-report 

capacity; determination of 

severity, frequency, 

bother/functional impact; 

history, physical 

examination, and diagnostic 

testing; preferences for 

evaluation and treatment 

including desired level of 

relief; consideration of 

specific etiologies. 

Evaluation of self-report 

capacity; evaluation of 

information preferences and 

caregiver(s) involvement. 

Treatment (or “Application” of Advance Care Planning)

Specific etiologic 

treatment(s); symptomguided treatment; patient 

education including 

information about side 

effects; prophylaxis of side 

effects. 

Clear communication; 

prognostication of patient’s 

clinical situation and 

potential clinical scenarios; 

provision of information 

about alternative choices; 

elicitation of patient and/or 

surrogate values and 

preferences; appropriate 

documentation. 

Follow-up

Timely follow-up of 

symptom changes and side 

effect manifestations; 

treatment adjustment; 

appropriate care for 

complications of therapy. 

Availability of preferences 

to other providers, and 

across settings; 

implementation of 

preferences to actual care 

delivered; re-assessment 

with notable changes in 

clinical context. 

Evidence Sources and Searches 

Literature Searches 

Sources for our review included MEDLINE®, the Cumulative Index to Nursing & Allied 

Health Literature (CINAHL©) database, and PsycINFO.© The RAND Library staff performed 

the searches. Members of the project team worked closely with the TEP and the librarians to 

refine the search strategy. We limited the searches to articles published in the English language, 

appearing in journals between the years 1995 and 2005, involving human subjects, and excluding 
20 

individual case reports. We limited our search to the past decade because our prior experience 

demonstrated that most of the progress in the science of supportive or palliative care was limited 

to that period. 

The main search strategy included an extensive list of terms intended to identify all research 

publications associated with all of the following: 

• Each of the domains of interest (pain, dyspnea, depression, and advance care planning) 

• Quality assessment or improvement 

• Quality measurement 

As described above, searches for pain and depression were limited with terms related to 

cancer. Searches for dyspnea and ACP were not limited with cancer-specific terms. We 

searched both the adult and pediatric cancer literature. We also searched “related articles” (e.g., 

using similar articles linked by MEDLINE®) for citations that described using measures within 

the Assessing Care of the Vulnerable Elders (ACOVE) measure set that were likely to meet our 

criteria. All of these searches were conducted during February 2005. The search strategies can 

be found in Appendix C*

. 

Grey Literature - Internet Search 

Because of limitations in the indexing of quality measures in traditional scientific databases, we 

employed a number of additional search strategies. We sought to identify guidelines that would 

point to recommended evidence-based standards of practice or specific quality measures that had 

been developed by health care organizations or specialty societies. Most of these searches 

involved the Internet. First, we used the National Guideline Clearinghouse 

(www.guidelines.gov) and the National Quality Measures Clearinghouse (http://www. 

qualitymeasures.ahrq.gov/) to identify guidelines and indicators that fit our criteria. In addition, 

we accessed the websites of specific professional societies, disease societies, medical 

organizations, medical associations, and patient advocacy groups to search for any relevant 

guidelines or measures, as follows: 

• ASCO (American Society of Clinical Oncology) - www.asco.org 

• European Society for Medical Oncolgy - www.esmo.org 

• Canadian Cancer Society - http://www.cancer.ca/ccs/internet/frontdoor/ 

0,,3172___langId-en,00.html 

• Canadian Association of Psychosocial Oncology - http://www.capo.ca/eng/index.asp 

• ONS (Oncology Nursing Society) - www.ons.org/evidence 

 

*

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/eolcanqm/eolcanqm.pdf 
21 

• Canadian Association of Nurses in Oncology - http://www.cos.ca/cano/web/en/ 

index.html 

• American Cancer Society - http://www.cancer.org/docroot/home/index.asp 

• AAHPM (American Academy of Hospice and Palliative Medicine) - www.aahpm.org 

• NHPCO (National Hospice and Palliative Care Organization) - www.nhpco.org 

• EAPC (European Association of Palliative Care) - www.eapcnet.org 

• PCA (Palliative Care Australia) - www.pallcare.org.au/ 

• Canadian Palliative Care Association - http://www.chpca.net/ 

• Growthhouse - www.growthhouse.org 

• Center to Advance Palliative Care (CAPC) - www.capc.org 

• Center for Palliative Care Studies - www.medicaring.org 

• Americans for Better Care of the Dying - www.abcd-caring.org 

• Innovations in End of Life Care (online journal) - http://www2.edc.org/lastacts/ 

• IHI (Institute for Healthcare Improvement) - www.ihi.org 

• Promoting Excellence in End of Life Care - www.promotingexcellence.org 

• Supportive Care of the Dying - www.careofdying.org 

• MacColl Institute for Healthcare Innovation - http://www.improvingchroniccare.org/ 

• American Alliance of Cancer Pain Initiatives - http://www.aacpi.wisc.edu/ 

• American Pain Society - http://www.ampainsoc.org/ 

• Pain and Policy Studies Group - http://www.medsch.wisc.edu/painpolicy/ 

• American Thoracic Society - www.thoracic.org 

• The National Comprehensive Cancer Network - www.nccn.org. 

We also conducted Internet searches using GoogleTM and the following search terms: 

guideline, clinical practice guideline, indicator, measure, quality measure, performance measure, 
22 

cancer, metatstatic cancer, prostate cancer, breast cancer, lung cancer, colorectal cancer, 

depression, depressive symptoms, emotional depression, antidepressive agents, tricyclic 

antidepressants, selective serotonin re-uptake inhibitors, pain, morphine, opioids, nonsteroidal 

anti-inflammatory drugs, treatment of bone metastasis, radiation therapy, dyspnea, shortness of 

breath, anemia, erythropoeitin therapy, advance care planning, advance directives, goals of care, 

DNR, DNI, withholding or withdrawal of treatment. 

Finally, we searched the following published sources for relevant guidelines or indicators: 

Directory of Clinical Practice Guidelines, Guide to Clinical and Preventive Services, Health 

Plan Employer Data and Information Set (HEDIS©) 2005, The Medical Outcomes and 

Guidelines Sourcebook, Clinical Process and Outcomes Measurement Directory, and Behavioral 

Outcomes and Guidelines Sourcebook.

44-49 Whenever possible, we downloaded publications 

from the Internet; otherwise, we ordered them from the UCLA Biomedical Library or directly 

from the publishing organizations. All of these additional searches were conducted from January 

2005 through March 2005. 

Grey Literature - Measure Developers 

We sought supplemental information on measures that we identified by directly contacting 

developers of those measures or measure sets. In our communications, we specified which 

additional types of information might be helpful for this report and used a “contact form” as a 

guide to solicit this additional information (Appendix D*

). 

End-of-Life Care and Outcomes Literature Database 

We searched the reference database for the End-of-Life Care and Outcomes Project, a systematic 

review recently completed by this report’s authors, for articles coded as “cancer.”21 This search 

yielded 389 articles that met the criteria and were screened for relevance to this project. 

Title Screening, Article Review, and Selection 

of Individual Studies 

The literature review team described above selected studies for further review. The six 

researchers formed four teams, organized by each domain of interest. Each reviewer on each 

team independently reviewed the documents or studies, and the teams resolved disagreements by 

consensus. Dual review was used at all stages of the project. Any questions or needs for 

clarification that arose throughout the literature review were resolved by the principal 

investigators. 

We adopted the following inclusion and exclusion criteria that were applied at both the 

title/abstract and article screening phases. 

Published reports or Internet documents were accepted only if they contained either 

indicators or measures. An indicator was defined as a descriptive statement with a normative 

relationship to quality that includes a numerator and denominator and that is expressed as a 

 

*

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/eolcanqm/eolcanqm.pdf 
23 

measurable standard. As specified by the task order, we also defined a measure as an indicator 

that is fully specified for measurement, including data elements, the data collection approach, 

data sources, analysis, and presentation.2

 We generally defined any indicator that had been 

previously operationalized and used for quality assessment or improvement activities as a 

measure. A potential indicator was defined as a statement of a normative standard for which 

there exists substantial evidence of a relationship to quality of care. We also identified articles 

with potential indicators; however, potential indicators were not included or identified 

systematically. Potential indicators were identified by each team based on their working 

knowledge of the research literature and prioritized based on each team’s opinion of the most 

important gaps in existing measurement. Potential indicators are based on reasoned expert 

judgment and are intended only as suggestions for further expert discussion. We did not attempt 

to express them as fully operationalized measures. 

Study reports or other documents were excluded if they: 

• Were non-Western (i.e., not United States, Canada, Australia, New Zealand, Great 

Britain, or European), 

• Were published in a language other than English, 

• Focused exclusively on a disease other than cancer UNLESS they were on the topic of 

pain or depression, or 

• Focused on cancer but considered only a topic other than ACP, depression, dyspnea, or 

pain, 

• Addressed only specific types of cancer and/or processes of care that were not 

generalizable across the most common cancer conditions. (e.g., particular pain syndromes 

for tumor subtypes). 

We also excluded documents or citations that addressed only specific types of cancer and/or 

processes of care that were not generalizable across the most common cancer conditions (e.g., 

particular pain syndromes for tumor subtypes). For the domains of pain and depression, some 

measures or indicators were developed for populations (e.g., hospice) that generally include 

cancer patients, although the development or field testing of these indicators may not have 

explicitly included cancer. We did not exclude these documents or citations. Reviewers 

screened all titles identified through the formal literature review and all documents found 

through our Internet and other grey literature searches for pertinence to the key questions and 

therefore their relevance to this project. Project staff entered data from screening into an 

electronic database and tracked all studies through the process. Titles and abstracts were 

screened simultaneously at the first stage. If a title and abstract were accepted, then the full 

document was ordered and subjected to screening using a standardized screening form 

(Appendix E*

). Internet documents were also screened using a standardized screening form. 

 

*

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/eolcanqm/eolcanqm.pdf 
24 

Articles accepted at the screening stage were subjected to full abstraction using standardized 

abstraction forms. Some studies or documents described only indicator or measure development, 

whereas others described testing or use in an actual population. To the extent possible, at this 

stage, the following data were extracted for indicators and measures: 

• A general indicator description and/or statement 

• Numerator and denominator 

• The disease for which the indicator or measure was specified 

• Patient/population characteristics (age, race/ethnicity, and gender) 

• Setting of care for the indicator or measure application 

• Phase of illness to which the indicator was intended to apply 

• Data source 

• Data about psychometric performance – specifically, reliability and validity 

For guideline statements that were abstracted, we rated the quality of the guideline reports 

using statements following the Appraisal of Guidelines Research and Evaluation (AGREE) and 

Shaneyfelt criteria.50,51

Summarizing the Evidence for Key Questions 

We report the evidence in several forms. First, the evidence tables (Appendix F*) offer a 

detailed description of the individual indicators and measures that we identified, addressing each 

of the topic areas and attributes of indicators or measures that were specified by the task order 

and are important to understanding the selection of indicators for a national measure set.2,52

Narrative text summarizes each of the indicators and measures, including the specification of the 

indicator or measure, its use or application in actual practice, and evidence supporting its 

relationship to quality of care and its application. We provide statistical information (e.g., 

background rates, reliability, validity, etc.) useful to evaluating the indicator or measure when we 

found that information. In addition, we provide tables of potential indicators to highlight 

priorities for future research in measurement in each of the topic areas. We organized the 

indicators according to whether they most applied to the assessment, treatment, or follow-up of a 

symptom, and applied an analogous approach to the advance care planning indicators. Because 

the strength of evidence related to available indicators and measures varied across topic areas, 

the degree to which each topic emphasizes existing indicators or measures versus potential 

measures varies accordingly. 

 

*

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/eolcanqm/eolcanqm.pdf 
25 

Peer Review Process 

We identified potential peer reviewers through project staff and in consultation with our 

TOO. We attempted to include representation from organizations representing adult and 

pediatric oncology, and palliative medicine, as well as major stakeholders in health care quality 

measurement. Based on these inquiries, we contacted 20 individuals with wide expertise in the 

field representing 17 organizations and with deep knowledge of the literature, 14 of whom 

agreed to participate. The final list of peer reviewers and their affiliations can be found in 

Appendix A*

. We mailed a copy of the draft evidence report to the peer reviewers and TEP 

members, with a request for their comments. Comments were compiled and addressed in 

Appendix G*

. 

 

*

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/eolcanqm/eolcanqm.pdf 

 27 

Chapter 3. Results 

Overview 

This chapter reports the findings of our systematic review of the evidence supporting 

indicators and measures for the quality of care for pain, dyspnea, depression, and advance care 

planning in cancer care. The chapter first describes the literature review flow and then provides 

additional information about the specific indicators and measures that are outlined in our 

evidence tables (Appendix F*

). As described below, we identified 10 measure or indicator sets 

that were the source of most individual indicators and measures that we identified within the 

topic areas of pain, dyspnea, depression, and advance care planning. In general, these indicator 

sets were developed to comprehensively evaluate health (Quality Assurance (QA) Tools, 

Assessing the Care Of Vulnerable Elders (ACOVE)), overall cancer care (Georgia Cancer 

Coalition, Cancer Care Ontario), supportive cancer care, or palliative care (Dana-Farber Cancer 

Center, Cancer Care Nova Scotia, Veterans Health Administration (VHA), University Health 

Consortium, National Hospice and Palliative Care Organization). We successfully contacted 9 

of the 10 developers of the measure or indicator sets that are described (the exception being 

Georgia Cancer Coalition). In addition, we noted several widely recognized measurement sets 

that contain potentially relevant measures (e.g., Joint Commission of Accreditation of Healthcare 

Organizations (JCAHO), National Community for Quality Assurance (NCQA)), but we did not 

abstract these indicators as these measurement systems are already widely recognized. Table 2 

summarizes basic information about each measure set. The section that follows describes 

measures, indicators, and potential indicators (as defined in our Methods) for pain, dyspnea, 

depression, and advance care planning derived from these sets and other sources. 

Table 2. Overview of identified measure sets 

Indicators 

QA Tools RAND Health developed the QA Tools system as a comprehensive, clinically based 

system for assessing the overall quality of care for children and adults. Each QA Tools 

indicator is based on a focused review of the scientific literature and on the ratings of a 

panel of experts in the field (based on the “RAND appropriateness method”). The QA tools 

system includes 46 clinical areas and all four functions of medicine: screening, diagnosis, 

treatment, and follow-up. It also covers a variety of modes of providing care, including 

history, physical examination, laboratory study, medication, and other interventions and 

contacts. 

Relevant to supportive cancer care, QA Tools identified indicators for the following: 

1. Pain assessment 

ACOVE Assessing the Care Of Vulnerable Elders (called ACOVE) was intended to 

comprehensively address quality measurement for this population because vulnerable 

elders are at risk for serious declines in health and function from poor quality care and use 

health care resources disproportionately. ACOVE developed a method of identifying a 

 

*

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/eolcanqm/eolcanqm.pdf 
 28 

Table 2. Overview of identified measure sets (continued) 

Indicators 

Dana-Farber cont’d. community-based sample of vulnerable elders, selected clinical conditions for quality 

measurement, and developed an evidence-based set of 236 ACOVE-1 quality-of-care 

 process indicators to evaluate the care provided to vulnerable elders. The indicators are 

categorized into 22 clinical conditions of importance to vulnerable elders. Most of these 

quality indicators require only data from medical record review, though some require 

interviews or administrative data only. Relevant to supportive cancer care, ACOVE 

indicators addressed the following: 

1. Pain assessment and management (although these indicators excluded patients with 

cancer and therefore were ineligible for the current review) 

2. Depression 

3. Advance care planning (as part of dementia and end-of-life care) 

4. End-of-life care in general (including dyspnea) 

Dana-Farber The Dana-Farber Cancer Institute at Harvard University has pursued an effort to develop 

a set of administrative data-based indicators for palliative cancer care as part of its recent 

research agenda. 

 These measures focus on utilization as an outcome and address the following general 

aspects of supportive cancer care: 

1. Utilization-based intensity of service use (late chemotherapy, emergency, 

hospital/ICU use, site of death) 

2. Use of hospice 

CCNS Cancer Care Nova Scotia (CCNS) is a Canadian provincial organization established to 

improve the delivery of cancer services across Nova Scotia. CCNS is actively refining a 

quality measure set for palliative cancer care with the aim of determining which quality 

indicators are measurable from administrative datasets. The project has two phases: 

 Qualitative – The project systematically reviewed end-of-life quality indicators for 

cancer (generic indicators) and breast cancer (unique indicators). It also drew from a 

general literature review (project identified 90 potential indicators at this stage) and 

assessed the feasibility of measurement with administrative data (i.e., good 

communication, having advance directives – not measurable using local resources). 

In addition, the project conducted focus groups with patients living with metastatic 

cancer and bereaved caregivers, and expert panels (modified Delphi) with 

multidisciplinary providers. This process yielded a set of final indicators for testing. 

 Quantitative – The project has defined a cohort of all women whose cause of death 

was related to breast CA over five years (1998-2002f) in Nova Scotia and in Ontario, 

a true population-based cohort; linked all available datasets that describe this 

population where feasible; and programmed measurable indicators. The project plans 

to validate the final indicator set by a detailed review of 100 charts in both provinces, 

assessing reliability using a 2nd set of abstractions. 

 

 The project identified 19 palliative cancer indicators on these topics: 

1. Pain management 

2. Nausea and vomiting 

3. Advance directives 

4. Utilization-based intensity of service use (site of death, hospital/ICU care, 

emergency, late chemotherapy use, and others) 

5. Hospice and multidisciplinary service use 

6. Transitions and coordination among settings and providers of care 
 29 

Table 2. Overview of identified measure sets (continued) 

Indicators 

CCO Cancer Care Ontario (CCO) is a Canadian provincial organization whose primary mission 

includes the effort to drive system quality, availability, and innovation. CCO developed a 

set of quality measures for a range of uses organized around measures of access to care, 

outcomes, evidence-based practice, efficient practice, and the data gathering performance 

of the cancer care system. Within these areas, individual clinical indicators address 

aspects of prevention, early detection, treatment, and palliative care. Measures relevant to 

palliative cancer care include those that address the following: 

1. Pain management (satisfaction) 

2. Utilization-based intensity of service use (site of death, emergency, and home-based 

services) 

GCC The Georgia Cancer Coalition (GCC) was created in 2001 as a legacy of the tobacco 

industry’s Master Settlement Agreement with 50 states. GCC contracted with the Institute 

of Medicine (IOM) to develop a set of quality measures for evaluating and improving 

cancer care in Georgia. Efforts were guided by the principles and selection criteria of the 

National Quality Forum’s Strategic Framework Board in recommending a set of 52 cancer 

care measures. GCC focused its 2003 review on potential measures that could be used to 

evaluate care for the most common cancers in Georgia and the United States as a whole 

– breast, colorectal, prostate, and lung cancer. GCC used various sources for measures 

including accreditation organizations, Federal agencies, provider and professional groups, 

State cancer control programs, and other sources (RAND Health, FAACT, NQF). GCC 

appointed an expert panel using the RAND appropriateness method to winnow an initial 

pool of more than 80 candidate measures. The final set included four items addressing the 

following: 

1. Pain assessment and management 

2. Use of hospice care 

VHA In 2001, VHA, Inc initiated a rapid-cycle quality improvement program, the Transforming 

the Intensive Care Unit (TICU) initiative. Phase 1.0, which included 23 hospitals, focused 

on safety and 2 pain measures shown to be effective in pilot studies at Johns Hopkins. At 

the request of the participating ICUs, a key portion of phase 3.0, to be implemented in the 

Fall 2005, will be a palliative care bundle, which includes the pain measures and eight 

additional performance measures. 

 The measure set was developed using information from the Critical Care Peer Workgroup 

of Promoting Excellence in End-of-Life Care, the National Consensus Project for Quality 

Palliative Care, JCAHO standards, and other evidence. The measure set is currently 

undergoing small-scale pilot testing. It will be implemented and tested in the TICU ICUs. 

As with other TICU projects, outcomes will be defined and abstracted for evidence of 

impact from sources such as administrative data, chart review, and satisfaction surveys. 

The palliative care bundle measures are as follows: 

Day 1: 

o Identify decisionmaker 

o Address advance directive status 

o Address CPR status 

o Distribute information leaflet 

o Assess pain regularly 

o Manage pain optimally

Day 3: 

o Offer social work 

support 

o Offer spiritual support

Day 5: 

o Family meeting
 30 

Table 2. Overview of identified measure sets (continued) 

Indicators 

UHC The University Health Consortium (UHC), formed in 1984, is an alliance of academic 

health centers situated in the United States. The 2004 Palliative Care Benchmarking 

Project was conducted within the Clinical Process Improvement (CPI) division of the 

benchmarking program. The Benchmarking program provides members with the unique 

opportunity to evaluate and compare their clinical and operational practices with the finest 

university hospitals in the country. The UHC staff conducted an extensive literature review 

and assembled an expert panel of constituent members to choose final indicators based 

on the review and expert opinion. UHC collected data on a set of palliative care quality 

indicators from December 2003-February 2004. UHC employed its members’ Clinical Data 

Base (CDB) to select a retrospective cohort defined on the basis of adult status, several 

serious chronic conditions (HIV, cancer, pulmonary disease, congestive heart failure), and 

mortality risk (e.g., multiple recent admissions). The charts of these patients were 

subjected to review (40 patient records/each of 35 participating sites). A total of 1,596 

charts were included in the evaluation. Quality measures included the following: 

1. Pain assessment and treatment 

2. Dyspnea assessment and treatment 

3. Psychosocial care 

4. Care coordination/discharge planning 

NHPCO The National Hospice and Palliative Care Organization (NHPCO), founded in 1978, is the 

largest nonprofit membership organization that represents hospice and palliative care 

programs and professionals in the United States. NHPCO has developed standards for 

hospice care from its inception. In 1998, NHPCO began to develop the National Data Set 

(NDS) to establish national benchmarks for hospice practice. Development was led by an 

expert group that included hospice and State government representatives and 

researchers; this group decided to focus on outcome measures. The measurement set 

underwent two sets of pilot testing. Data submission is currently voluntary, but enough 

hospices have submitted data (78 hospices in 2002) that published benchmarks are 

available.53 

 Quality measures included the following: 54 

1. Comfortable dying 

2. Self-determined life closure 

3. Safe dying 

4. Effective grieving 

5. Family evaluation of hospice care 

Literature Flow 

The literature search identified 4,580 titles. We identified an additional 389 potentially 

relevant articles from the database we had assembled for the End-of-Life Care and Outcomes

review. Our web search identified 105 guidelines. An additional 109 articles were suggested 

from the personal libraries of the project members. Four articles were suggested by our TEP 

upon review of the draft version of this evidence report. In total, the RAND reviewers examined 

5,187 titles. 

Of the 5,187 titles identified as possibly relevant to our topics, 4,599 were excluded at 

abstract review, leaving 588 that might be useful to this project. Repeat review by one principal 

investigator (KL) excluded an additional 51 titles. Thus, the project sought to review 537 

articles and guidelines. 
 31 

Of the 537 articles ordered, we retrieved 536 prior to the cut-off date (June 10, 2005) 

Screening resulted in exclusion of 485 articles and guidelines: 347 had no indicator; 113 had no 

domain of interest; 8 covered a disease that was not cancer; 5 presented guidelines with no 

methods; 6 dealt with a non-Western location; 2 presented duplicate data; 2 represented duplicate 

articles; and 2 were useful for background only. (For a list of excluded studies, please refer to 

Appendix H*

). 

Of the remaining 51 articles, 25 contained measures and/or indicators that described their 

methods and were reviewed in detail. Potential indicators were identified by each team based on 

their working knowledge of research literature and prioritized based on each team’s opinion of 

the most important gaps in existing measurement. The remaining 26 articles contained only 

potential indicators. Because one publication can report on multiple topics and may include 

multiple measures and/or indicators/potential indicators, the numbers in Figure 1 add up to more 

than the total number of articles considered for detailed review. 

 

*

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/eolcanqm/eolcanqm.pdf 
 32 

Figure 1. Article flow 

Project Reviewers

n= 109

EOL Evidence Report 

n= 389

Total number of titles identified for title review

n=5,187

EOL Evidence Report

n= 33

Total number of titles considered potentially relevant and articles ordered

n= 588

Literature Searches

n= 4,580

Literature Searches

n= 337

485 Articles excluded

8 disease not cancer

5 guidelines with no methods 

6 non-Western location 

347 no indicator 

113 no domain of interest

2 duplicate data

2 duplicate article, accidentally ordered

2 useful for background only

Total number of articles reviewed

n= 537

51 excluded at abstract review

Total number of articles considered for detailed review

N = 25 containing Measures and/or Indicators *(3 of these also contained PI)

N = 26 containing Potential Indicators (PI)

1 article not received by cut off date

Depression

Measures = 4

Indicators = 1

Potential Indicators = 9

Dyspnea

Measures = 2

Indicators = 3

Potential Indicators = 6

Advance Care Planning

Measures = 21

Indicators = 5

Potential Indicators = 14

Pain

Measures = 12

Indicators = 5

Potential Indicators = 8

Web Guideline 

Search

n= 105

Web Guideline 

Search

n= 105

TEP

n= 4

TEP

n= 4

Project Reviewers

n= 109
 33 

Pain 

Introduction 

For the review of pain measures, we reviewed all relevant clinical guidelines, indicator and 

measure sets, potential indicators, systematic reviews, and articles related to quality 

improvement initiatives and clinical trials, including one review of quality improvement 

initiatives for cancer pain. Please refer to Evidence Table Appendix F1*

. Pain was among the 

topics most commonly addressed within existing measure sets (6/8 of the sets identified). We 

designated most of the items we identified within these measure sets as quality measures since 

all of them have been operationalized and used for at least quality assessment. The exception is 

a handful of indicators that are either under development (e.g., Cancer Care Nova Scotia) or for 

which we were unclear whether or not they had previously been used for actual measurement or 

we were unsuccessful in obtaining information about that experience (e.g., Georgia Cancer 

Coalition). 

The report of IOM’s Committee on Assessing Improvements in Cancer Care in Georgia 

(National Cancer Policy Board)55 (Chapter 6, Treating Cancer) has two indicators related to pain, 

one each for assessment and follow-up. 

The performance measurement framework for Cancer Care Ontario56 has a measure relevant 

to pain management, defined as patients’ self-reported pain and perception of pain management 

by providers; this measure was operationalized as an indicator as “satisfaction with pain 

management.”57 

ACOVE generally excluded cancer patients, so the ACOVE set of measures for chronic 

pain58 were not included in our review, although some of these measures may be appropriate for 

patients with cancer. However, ACOVE measures related to the end –of life did not exclude 

cancer; thus, that set includes one measure relevant to pain assessment.59 This measure has also 

been adapted for nursing homes.60 

The VHA Transforming the Intensive Care Unit (TICU) initiative includes two measures for 

pain assessment and follow-up that are currently being used in 30 ICUs and have evidence for 

efficacy in a cancer ICU.61,62 

The University Health Consortium (UHC) included two measures for assessment, one for 

treatment, and one for follow-up. QA Tools included three items relevant to pain, although only 

two were accepted by the expert panel. However, the indicator not accepted by the QA Tools 

panel has been proposed as part of Cancer Care Nova Scotia. 

The NHPCO set includes one outcome measure for pain.63 

A broader cancer quality measurement set, QOPI (The Quality Oncology Practice Initiative), 

includes one measure for assessment of pain at the end of life. We also identified a treatment 

indicator in a guideline for radiation therapy.64 

 

*

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/eolcanqm/eolcanqm.pdf 
 34 

Measures and Indicators 

Assessment. We identified five measures and two indicators relevant to cancer pain assessment. 

Measure - Regular Assessment of Pain. Quality Assurance Tools (QA Tools), proposed in 

2000,64 includes one measure relevant to pain assessment. The numerator of this measure is 

“patients with the presence or absence of pain noted every six months,” and the denominator is 

“all patients with cancer metastatic to the bone.” The measure was limited to patients with bone 

metastases because this group was felt to be well defined and to have the highest prevalence of 

cancer pain. The prevalence of every six months was chosen as a minimum requirement, based 

on older versions of clinical practice guidelines from the early 1990s. Although this measure 

was tested as part of the QA Tools evaluation in a VA- and community-based population,65,66 the 

numbers of patients eligible for this indicator was too small to provide any useful validity, 

reliability, or comparability information. This measure is intended for use in patients with any 

type of cancer metastatic to the bone. The measure is not in current use. 

Measure - Routine Inpatient Pain Assessment. The University Health Consortium (UHC) 

Palliative Care Benchmarking Project67 included two measures relevant to pain assessment. The 

numerator for the first measure is “persons with pain assessment within 48 hours of admission.” 

The denominator is “adults ≥18 years of age, with admission for CHF (DRG 127), Cancer (DRG 

82, 203, 172, 274, 346, 10) HIV (DRG 489) OR respiratory (DRG 475, 483) AND length of stay 

> 4 days AND 2 prior admissions for any cause in preceding 12 months.” This denominator was 

developed by an expert panel to represent a potential palliative care population in hospitals. A 

mean of 96% (median 98.5%, range 83-100%) of charts in the UHC cohort satisfied this 

indicator, including 98% of cancer patients, and little variability was seen among participating 

sites. No reliability or validity information is available; however, the benchmarking project 

found that using a combination of all of the UHC measures in a “palliative care bundle” was 

associated with lower patient length of stay. In addition, several elements of the bundle could be 

used to identify hospitals with “best practices” for palliative care. Scores from the bundle were 

instrumental in achieving advances in palliative care at several hospitals, including justifying at 

least one new palliative care program. The measure is intended for use as a tool for comparing 

performance in all hospitals. However, it is very broad, does not include an element of 

timeliness (within 48 hours of admission is late for pain assessment to occur), and is not in 

current use. 

The numerator for the second UHC pain assessment indicator (routine inpatient pain 

assessment with a numeric scale) is “persons assessed with a numeric pain scale.” The 

denominator is “adults ≥ 18 years of age, with admission for CHF (DRG 127), Cancer (DRG 82, 

203, 172, 274, 346, 10) HIV (DRG 489) OR respiratory (DRG 475, 483) AND length of stay > 4 

days AND 2 prior admissions for any cause in preceding 12 months in the hospital, AND 

reported pain within the first 48 hours of admission.” A mean of only 76% (median 85%, range 

13-100%) of the charts reviewed in the UHC cohort met this criterion, including 77% of those of 

cancer patients. Some variability was seen among sites; at ten low-performing sites, only 15 to 

70% of charts included this information. Information is the same as for the first UHC indicator; 

no additional reliability or validity information is available. 
 35 

Measure - Regular ICU Pain Assessment. The VHA TICU project, initially proposed in 2002,61

includes one pain assessment measure, which was modified for inclusion in the VHA TICU 

palliative care bundle in 2005 (Quality Indicator #5).68 The numerator is “the total number of 4-

hour intervals (within the first 24 hours following ICU admission) for which pain was assessed 

and documented, using an appropriate rating scale.” The denominator is “the total number of 4-

hour intervals (within the first 24 hours following ICU admission) for patients with an ICU 

length of stay ≥ 24 hours.” Documentation of pain assessment may include (1) a numeric value 

of 1 to 10, or (2) pain was assessed but the patient was unresponsive (e.g., heavily sedated, 

comatose). A four-hour interval is defined as a four-hour interval of patient care (patient-nursing 

interval). The four-hour intervals are set at 8 am; 12pm; 4 pm; 8 pm; 12 am; and 4 am. An 

appropriate rating scale is defined as either a visual analogue scale (VAS with numeric 

translation of 1-10) or another numerical rating scale of 1-10. Exclusion criteria include 

intervals for which a patient was not physically available in the ICU (e.g. patient expired; not yet 

admitted to the ICU; was discharged/transferred out of the ICU (for part of the day); went into 

the operating room or off the unit for a procedure); and intervals for which pain was assessed, 

but without the use of an appropriate rating scale as defined above. This measure is supported by 

the JCAHO Hospital Standards RI.1.2.9, PE.1.4 and the National Consensus Project for Quality 

Palliative Care Clinical Practice Guideline, 2.1.69 Evidence of its validity includes a rapid-cycle 

quality-improvement intervention in two ICUs, including a surgical cancer ICU, demonstrating 

that its use as part of a rapid-cycle quality improvement intervention, in which the percent of 

nursing intervals where pain scores were measured using a VAS scale increased from 42 to 94%, 

contributed to a decrease in pain scores ≥3 from 41% to 6%.62 In addition, a similar rating scale 

was used in an RCT of interrupted sedation in the ICU.70 No reliability information is available. 

This measure is intended for use in all ICU populations. As with other pain outcome 

measures, variability in the quality of pain assessment may limit its use for comparing 

performance. In the ICU population, as at the end of life or in nursing homes, many patients 

may have difficulty or may be unable to verbalize a numerical pain score; depending on local 

standards of assessment, this outcome variable may therefore apply to very different proportions 

of the population in different ICUs or settings. The measure also does not specify whether this 

score should represent minimum, maximum, or average pain. Also, the measure applies only to 

the first 24 hours in the ICU. This measure is currently being used as part of the VHA TICU 

initiative and is part of the VHA palliative care bundle, which will be initiated in September 

2005. 

Measure - Assessing Pain in Patients Close to Death. QOPI (the Quality Oncology Practice 

Initiative) included one end-of-life pain assessment measure in their practice-based system of 

quality assessment. The numerator of this measure is “medical records where there is an explicit 

practitioner’s notation quantifying their physical pain or lack thereof on their last visit to the 

office prior to death,” and the denominator is “recently deceased patients from oncology 

practices.”71 QOPI was developed and sponsored by individual oncologists and ACSO (the 

American Society of Clinical Oncology). QOPI selected and developed quality measures that 

were easily identifiable through outpatient medical record review. The measures also had to 

relate to important issues, as defined by (1) consensus among QOPI participants, (2) evidencebased standards, or (3) requirements from organizations such as JCAHO. This measure was 

evaluated through review of 10 medical records of patients who had died in 41 oncology 

practices. The initial round of assessment showed that 56% of medical records met this measure, 
 36 

69% met the measure in a second round 6 months later, and there was substantial variation 

among practices (range, 30-90%). No validity or reliability information is available. 

This measure is intended for use in self-assessment in outpatient oncology practices. The 

retrospective nature of the measure is a drawback, since death is often not predictable, and 

improvements might be due to changes in documentation that do not necessarily reflect 

improvement in practice. This measure will be part of a new voluntary quality improvement 

initiative for oncology practices sponsored by ASCO, NCI, and the National Health Council, to 

begin in 2005. End-of-life will be the first initiative and will include this measure as well as 

whether the pain was quantified by either a numerical scale or another descriptor. 

Indicator - Routine Pain Assessment in Expected Dying. The Assessing Care of the Vulnerable 

Elderly (ACOVE) end-of-life measure set59 includes one measure relevant to assessment for 

pain. Its numerator is “the number of vulnerable elders with documentation of pain or lack of 

pain during the last three days of life.” The denominator is “vulnerable elders who are conscious 

during the last three days of life and who died an expected death.” Although the eight measures 

from the ACOVE end-of-life set were evaluated for reliability and validity72 as part of the 

ACOVE project, this measure was not included in the evaluation. The denominator may be 

difficult to define, since “expected death” requires explicit documentation of that expectation in 

the medical record more than one day before death. This measure is intended for use in all 

vulnerable elders. An expert panel accepted a modified version of this indicator as part of a 

revised set for nursing homes.60 The nursing home indicator requires a denominator of all 

nursing home residents who died an expected death and were conscious at all during the last 

seven days of life. 

Indicator - Routine Assessment of Pain. The IOM’s Committee on Assessing Improvements in 

Cancer Care in Georgia (National Cancer Policy Board) proposed an indicator set in 2005 with 

two indicators related to pain, one of which targeted regular assessment.55 The numerator for 

this indicator is “the number of cancer patient encounters where the patient was assessed for 

pain,” and the denominator is “the number of cancer patient encounters.” Available information 

does not specify the appropriate frequency of cancer pain assessment (i.e., every encounter) or 

how pain should be measured. The indicator is intended for use for all cancer patients, across the 

continuum of care and all health care settings. This indicator is newly proposed and information 

on actual use is not available. 

Treatment. We identified three measures and one indicator relevant to cancer pain treatment. 

Measure - Responsive Pain Treatment. QA Tools includes one measure relevant to treatment of 

pain, proposed in 2000.64 The numerator is “patients offered a change in pain management 

within 24 hours of the pain complaint,” and the denominator is “all cancer patients whose pain is 

uncontrolled.” As with the other QA tools indicators, although this measure was tested as part of 

the QA Tools evaluation in VA and community-based populations,65,66 the number of eligible 

patients was too small in these studies to determine validity or reliability. This measure is 

intended for use in patients with any type of cancer across the continuum of care. The measure 

is not in current use.
 37 

Measure - Regular Prophylaxis of Opioid-Induced Constipation. The UHC Palliative Care 

Benchmarking Project also included one measure relevant to pain treatment that addresses safety 

or avoiding side effects that can be effectively prevented. The numerator is “patients with a 

bowel regimen initiated within 24 hours of an opioid, and in whom a bowel regimen is not 

contraindicated.” The denominator is “adults ≥ 18 years of age, with admission for CHF (DRG 

127), Cancer (DRG 82, 203, 172, 274, 346, 10) HIV (DRG 489) OR respiratory (DRG 475, 483) 

AND length of stay > 4 days AND 2 prior admissions for any cause in preceding 12 months in 

the hospital, AND treated with narcotics.” The treatment of constipation as a side effect of 

opioid administration was addressed in a recent systematic review.73 The review found that 25-

50% of cancer patients have constipation and that it is the most frequently occurring adverse 

effect of opioid use in advanced cancer; however, the review of 17 studies found none that 

directly compared patients treated with an opioid regimen with those not treated. A similar 

potential measure has also undergone some field testing in a population that included cancer 

patients as part of the JCAHO-AMA-NCQA pain management performance measure 

development project.74 A mean of only 59% (median 59%, range 20-93%) of the charts 

reviewed in the UHC cohort met this criterion, and the distribution of the outcome appears 

broadly normal. Eighty-three percent of cancer patients met the indicator. No validity or 

reliability information is available for this indicator; more details on the testing of the 

measurement set are described in the UHC pain assessment measures. This measure is intended 

for use across all types of cancer. The measure set is not in current use. 

Indicator - Minimizing Radiotherapy Burden. Cancer Care Ontario’s Program in EvidenceBased Care reported a clinical practice guideline in 2003, Radiotherapy Fractionation for the 

Palliation of Uncomplicated Painful Bone Metastases, Practice Guideline Report #13-2. This 

report includes an indicator relevant to the choice of radiotherapy regimen in patients with 

advanced cancer.75 The numerator is “patients receiving single dose therapy as a single 8Gy 

treatment, prescribed to the appropriate target volume.” The denominator is “all adult patients 

with single or multiple radiographically confirmed bone metastases of any histology 

corresponding to painful areas in previously nonirradiated areas without pathologic fractures or 

spinal cord/cauda equina compression.” It does not apply to the management of malignant 

primary bone tumor and is intended for patients receiving radiotherapy where the objective or 

intent is pain relief. The report, which was based on a systematic review, also includes a number 

of qualifying statements. The evidence was reviewed by a four-member group and then 

approved by a multidisciplinary group. External review was obtained through a mailed survey of 

Ontario practitioners. The indicator is based on 2 systematic reviews and 16 RCTs. A metaanalysis of eight RCTs, conducted as part of the guideline process, found no significant 

difference in response rate between single-fraction and fractionated radiotherapy. There were no 

differences in quality of life (for the few studies where this was assessed) or side effects. 

Observed re-irradiation rates were higher with single-fraction than with multiple-fraction 

therapy, but details were not described. A separate systematic review and meta-analysis76

derived the same conclusions about pain, but concluded that the pathologic fracture rate was 

significantly higher for single-fraction than for multiple-fraction treatment. The indicator is 

intended for use in a mixed cancer population, for palliative treatment. This indicator is relevant 

to the efficiency of care. The indicator is not known to be in current use. 
 38 

Indicator - Effective Treatment for Painful Bony Metastasis. Cancer Care Nova Scotia77 has 

proposed a set of quality indicators for palliative cancer care that includes one pain treatment 

indicator, adapted from a proposed QA tools indicator that was not accepted by the QA Tools 

expert panel and was not part of the final set.64 The numerator of this indicator is “patients who 

are offered one of the following within one week of the notation of pain: radiation therapy to the 

sites of pain, or radioactive strontium therapy.” The denominator of this indicator is “patients 

with cancer metastatic to the bone and pain uncontrolled by opioids.” Although there is 

excellent meta-analysis evidence of the efficacy of radiation therapy for bone pain,76 another 

meta-analysis found that the evidence supporting the efficacy of radioisotopes such as strontium 

was inconclusive,78 and we did not identify any evidence that supports the use of radiation 

therapy only for pain uncontrolled by opioids or that supports the timing of radiation therapy. 

Although this indicator was proposed, based on feedback from CCNS expert panels and 

available datasources, the indicator will not be field tested at the current time because 

“uncontrolled pain” could not be operationalized. The indicator is intended for use in patients 

with metastatic breast cancer and addresses both effectiveness and timeliness. The indicator may 

be limited by the lack of definition of “pain uncontrolled by opioids.” The indicator set is 

proposed and is not in current use. 

Follow-up. We identified four measures and one indicator relevant to the follow-up of cancer 

pain treatment. 

Measure - Timely Treatment of Pain in Hospice. The NHPCO indicator set for hospice,54,63,79

includes one outcome measure for pain. The numerator is “the proportion of patients between 48 

and 120 hours after admission who report ‘yes’ when asked, ‘Was your pain brought to an 

acceptable level within 48 hours of your admission to the hospice program?’” The denominator 

is “patients admitted to hospice who answer ‘yes’ when asked, ‘Are you uncomfortable because 

of pain?” The measure does not apply to patients in nursing facilities, children, non-Englishspeaking patients, or others who cannot self-report, because of concerns about measurement. It 

assumes that all patients are assessed for pain on admission to hospice, and detailed information 

on how to administer the measure is available.63 The NHPCO chose to focus only on outcome 

measures, and considered and rejected alternative data collection methods for pain outcomes, 

including chart audits and retrospective surveys. They also chose not to use proxy reports 

because of issues with reliability and validity.54 This concern, as well as the difficulty in 

assessing pain in the last days of life, led to the timing for the indicator at hospice admission. A 

non-numerical dichotomous rating was chosen because of concerns about the validity of 

numerical ratings. For construct validity, in pilot testing, more people said that they were 

“uncomfortable” because of their pain than said their pain was “unacceptable,” so the word 

“uncomfortable” was chosen as more sensitive for the denominator.54 Results from the use of 

this measure as part of the measurement set have also been published.79 For the 78 agencies that 

submitted data in 2002, the mean was 87% (median, 90%; 25th & 75th percentiles, 79% & 100%). 

These data suggest that there may be sufficient variation among hospices for this measure to be 

useful for comparing performance or quality improvement. No other reliability or validity 

information is available. A previous review of the palliative care literature found that general 

measures of satisfaction or subjective ratings of health care were subject to a strong ceiling 

effect, and items or measures that addressed specific processes tended to exhibit better 

variability.21 This measure is intended for use in hospice care. It is limited by the inability of 
 39 

many hospice patients to report their pain. In addition, as an outcome measure, it would require 

evaluation for whether there needs to be risk adjustment for variation among organizations. 

Population characteristics such as the percentage of patients who report pain on admission, 

patients who cannot self-report, and patients with a history of drug abuse may significantly 

influence results. It is currently being used by some hospice programs for internal quality 

assurance purposes and voluntary, confidential reporting to NHPCO. 

Measure - Effective Treatment of Pain in the ICU. This measure, initially evaluated in 2002,61

was modified for inclusion in the VHA TICU palliative care bundle in 2005 as an outcome 

measure for pain (Quality Indicator #6).68 The numerator is “the total number of 4-hour intervals 

(within the first 24 hours following ICU admission) for which the documented pain score was 

<3.” The denominator is “the total number of 4-hour intervals (within the first 24 hours 

following ICU admission) with numerical pain values of 1 to 10, for patients with an ICU length 

of stay ≥ 24 hours.” Exclusion criteria include intervals for which a patient was not physically 

available in the ICU (e.g. patient expired; was not yet admitted to the ICU; was 

discharged/transferred out of the ICU [for part of the day]; went into the operating room or off 

the unit for a procedure); intervals for which a pain score was not documented; intervals for 

which pain was assessed, but the patient was unresponsive (e.g., heavily sedated; comatose); and 

intervals for which pain was assessed, but without the use of an appropriate rating scale as 

defined above. This measure is based upon a rapid-cycle quality-improvement intervention in 

two ICUs, including a surgical cancer ICU, demonstrating its responsiveness to changes in care 

processes (including the other VHA measure listed under assessment as well as whether house 

staff documented the pain score). It is also based on the JCAHO Hospital Standards RI.1.2.9., 

PE.1.4; and the National Consensus Project for Quality Palliative Care Clinical Practice 

Guideline, 2.1. No additional reliability or validity information is available. 

This measure is intended for use in all ICU populations. As with other pain outcome 

measures, variability in the quality of pain assessment may limit its use for comparing 

performance. In the ICU population, as at the end of life or in nursing homes, many patients find 

it difficult or may be unable to verbalize a numerical pain score; depending on local standards of 

assessment, this measure may therefore be applicable to very different proportions of the 

population in different ICUs. The measure also does not specify whether this figure should be a 

minimum, maximum, or average pain score. It is currently being used as part of the VHA TICU 

initiative and is part of the VHA palliative care bundle, which will be initiated in September 

2005. 

Measure - Satisfaction With Pain Treatment. This measure is part of the indicator set proposed 

by Cancer Care Ontario in 2005.57 The numerator of this measure is reported in three categories: 

“Patients where the response to the question “Do you think staff did everything they could to 

control your pain or discomfort?” was (1) yes, completely; (2) yes, somewhat; or (3) no.” The 

denominator is “outpatient cancer patients reporting mild to severe pain.” The source of this 

indicator is patient surveys. Information on reliability/validity as an indicator is not available. In 

a survey of more than 5,000 cancer outpatients in 2004, 70% of patients answered “yes, 

completely,” 25% answered “yes, somewhat,” and only 5% answered “no.” Variability by 

center was not reported, and all four provinces where the survey was done apparently had similar 

results.57 The low number of “no” answers is consistent with previous studies of satisfaction for 

pain, which have found that overall satisfaction reports may not reliably correlate with actual 
 40 

pain scores and may be difficult to use because of ceiling effects.21 The indicator is intended for 

use in ambulatory/outpatient care. It may be affected by recall bias (used for care in the previous 

six months), low response rate among palliative care patients, and a requirement for a minimum 

denominator of 300 patients per participating center. It is currently being used in Ontario and 

three other Canadian provinces. 

Measure - Timely Treatment of Inpatient Pain. The UHC Palliative Care Benchmarking 

Project,67 introduced in 2004, includes one outcome measure for pain. The numerator is 

“persons with pain relief or reduction within 48 hours of admission to ≤ 3 on a 0-10 scale.” The 

denominator is “adults ≥18 years of age, with admission for CHF (DRG 127), Cancer (DRG 82, 

203, 172, 274, 346, 10) HIV (DRG 489) OR respiratory (DRG 475, 483) AND length of stay > 4 

days, 2 prior admissions for any cause in preceding 12 months in the hospital, AND reported 

pain within the 1st 48 hours of admission.” A mean of 76% (median 78%, range 46-92%) of the 

UHC cohort met this indicator. Only 70% of cancer patients met the indicator. There was 

moderate variability in this measure; although the range of performance was somewhat 

restricted, the distribution of results appeared to be relatively normal. As with the other UHC 

measures, no individual reliability or validity information is available. It is not known to be in 

current use. 

Indicator - Effective Treatment of Pain. The IOM Committee on Assessing Improvements in 

Cancer Care in Georgia’s (National Cancer Policy Board) 2005 indicator set proposed an 

indicator relevant to the follow-up of pain treatment.55 The numerator is “the number of cancer 

patients who report being in more than minor pain.” The denominator is “the number of cancer 

patients who are not comatose.” This indicator is based on the 2004 cancer pain guideline from 

the National Comprehensive Cancer Network (NCCN),80 the 1996 cancer pain guideline from 

the World Health Organization,81 the cancer pain position statement from the Oncology Nursing 

Society (ONS),82 and the American Pain Society Quality Improvement Guidelines for the 

treatment of acute pain and cancer pain.83 Evidence is based on the AHRQ evidence report on 

treatment of cancer pain7 and NIH Consensus State of the Science Statement on symptom 

management in cancer.84 

The indicator is intended for use for all cancer patients, across the continuum of care and all 

health care settings. A validated pain scale that defines “minor pain” must be used in each health 

care setting, and the threshold should be reported along with the indicator. Poor medical record 

documentation may lead to low prevalence estimates. Potential benchmark sources are baseline 

patient surveys and medical records. Evidence suggests that members of ethnic minority groups, 

women, and elderly patients may be less likely to be adequately treated for pain (Chapter 6, p. 

23).55 This indicator is newly proposed and information on use is not available. 

Potential Indicators and Measurement Gaps 

We identified a large number of potential indicators in the substantial guideline, quality 

improvement, and research literature that we reviewed. Many of these guideline statements and 

potential indicators address specific management issues, such as the treatment of neuropathic 

pain, or more details on the indicators we have identified, such as more-comprehensive 

assessment for pain or pain outcomes.74 We highlight here eight potential indicators with 

relatively strong supporting evidence and/or expert consensus, highlighting critical issues not 
 41 

addressed by the indicators and measures listed above, that may have potential for demonstrating 

variability in quality and improving patient outcomes, and applicable to cancer patients across 

the continuum and to diverse populations. 

Assessment 

Physician Documentation of Pain Assessment. The numerator is “patients where a pain 

assessment is documented in the physician note,” and the denominator is “all patients with 

cancer.” This potential indicator was used as a part of the pain continuous quality improvement 

study that led to the VHA pain measures61 and is supported by evidence that pain assessment by 

nurses or aides often does not lead to documentation of pain by the physician. Improvements in 

pain assessment by non-physician staff may not lead to improved outcomes because the 

physician may not attend to the report of pain.85 Another consideration with pain assessment 

indicators is the extent to which they might address aspects of assessment including specific 

symptom attributes (e.g., timing, duration, and functional impairment) and specific etiologies. 

Disparities in Pain Assessment. The numerator is “non-white patients with cancer who are 

assessed for pain,” and the denominator is “all patients with cancer who are assessed for pain.” 

Ensuring that pain assessment is equitable across ethnic groups is important because large 

studies have demonstrated that members of minority groups with cancer have higher rates of 

inadequate pain assessment and treatment than others.86,87 A recent nonsystematic review of this 

topic found that this disparity persists for both assessment and treatment, across multiple settings 

and types of pain.88 Whenever possible, assessment of disparity by ethnicity should also include 

assessment of other contributing factors, such as socioeconomic status and access to care, since 

these are often the primary cause of perceived disparities by race. 

Assessment in Patients Who Have Difficulty Verbalizing Their Pain. The numerator of this 

potential indicator is “patients who have difficulty verbalizing their pain but who are assessed for 

pain,“ and the denominator is “cancer patients who have difficulty verbalizing their pain.” This 

potential indicator is important because many of the indicators and measures we identified 

exclude patients who are unable to verbalize their pain. Many cancer patients at high risk for 

pain would therefore be excluded from many of these measures, including infants and small 

children, many seriously ill or perioperative cancer patients, cancer patients with severe 

dementia, and many patients in the last days of life, where a decreased level of consciousness is 

very common. In addition, patients with some degree of cognitive impairment or sedation may 

also be excluded from the denominator even though many of these patients are able to report 

their pain.40 Variable exclusion of these patients limits the usefulness of pain assessment 

measures for comparisons between facilities or over time. Recent systematic reviews have 

summarized the variety of tools available for assessing pain in older adults with cognitive 

impairment89 and in infants.90 In addition, some evidence exists that cognitively impaired adults 

receive less pain medication than those who are cognitively intact and that untreated pain may 

lead to higher rates of adverse outcomes, such as delirium.91

Assessment for Depression in Patients With Pain. The numerator is “patients assessed for 

depression,” and the denominator is “all cancer patients with pain.” This potential indicator is 

supported by evidence that cancer patients with pain are more likely to be depressed than cancer 
 42 

patients without pain.92 In addition, there is substantial descriptive evidence that the symptoms 

of depression and pain often occur together. 10

Treatment 

Educating Patients About Pain Management. The numerator is “patients who are educated about 

pain management,” and the denominator is “cancer patients with pain.” This guidance is 

included in many cancer pain guidelines, some of which also include education of caregivers. A 

related potential outcome measure, that patients should be adequately informed and 

knowledgeable about pain management, is included in the 2005 American Pain Society 

Recommendations for Improving the Quality of Acute and Cancer Pain Management,74 based on 

an expert review of the measurement tools used in 20 quality improvement studies in pain 

management.93 A systematic review of educational interventions for patients and family 

caregivers found eight studies, two of which were randomized, controlled trials; both of these 

and all of the nonrandomized studies where pain management education was measured found a 

significant impact of education on pain.94

Use of Opioids for Severe, Persistent Pain. The numerator of this potential indicator is “patients 

who are treated with opioids,” and the denominator is “cancer patients with severe, persistent 

pain.” This potential indicator is supported by a number of clinical practice guidelines for pain, 

including the World Health Organization81 and NCCN.80 One of the pilot measures that was 

field-tested as part of the JCAHO-AMA-NCQA initiative also relates to this issue,74 but it needs 

further development before use as a measure. A randomized trial in advanced cancer patients 

comparing the use of strong opioids to the WHO stepped-care approach where patients are 

started on non-opioids, then given weak opioid medications, found that patients treated with 

opioids had better pain outcomes.95

Use of Long-Acting Pain Medications. The numerator is “all adults who are prescribed longacting opioids,” and the denominator is “all adults with cancer who have persistent pain and are 

prescribed opioids.” This expert guidance is in numerous pain guidelines, including the Agency 

for Health Care Policy and Research cancer pain guideline96 and the American Geriatrics 

Society cancer pain guideline for older adults.97 A systematic review of the cancer pain literature 

found only two relevant trials, which showed no difference between long-acting and around-theclock short-acting formulations.7

 However, the expert consultants for the Cancer Symptom 

Management State-of-the-Science Statement84 also included this recommendation because 

improved adherence with long-acting drugs in clinical practice may improve outcomes. 

Procedure-Related Pain in Children. The numerator is “all children who receive appropriate 

preparation and/or anesthesia” and the denominator is “all children with cancer receiving painful 

procedures.” We chose to highlight this potential indicator because it is emphasized in 

guidelines for cancer pain in children but is typically not mentioned in guidelines for adults. The 

World Health Organization34 describes this area as important because the pain from procedures 

is often worse than that from the cancer and because inadequate pain control may lead to anxiety, 

reduced adherence, and poorer relationships with health care providers. 

In addition, many pharmacological and nonpharmacological approaches, including the 

presence of parents whenever possible, local anesthetics, and opioids when needed, can reduce 
 43 

procedural pain. An assessment tool for postoperative pain management in children has been 

developed and psychometrically tested in children as young as eight years old.98 Many 

randomized, controlled trials have demonstrated that use of pharmacologic measures such as 

topical anesthetics not only reduces pain, but reduces procedure time and improves procedure 

success rates.99 Such an indicator might also be reasonable to consider for adults. 

Dyspnea 

Introduction 

We failed to identify any quality measures that were developed specifically for dyspnea in 

cancer. Please refer to the Evidence Table in Appendix F2*. Two measures were developed and 

tested as part of the UHC Project on palliative care for dyspnea assessment and dyspnea 

reduction/relief within 48 hours of hospital admission.67 The UHC indicators were used in an 

adult population with cancers that included lung, gastrointestinal, breast, male reproductive, and 

nervous system neoplasms – approximately 25% of the UHC cohort had cancer. 

We identified two indicators for dyspnea applicable to the cancer population derived from 

the Assessing Care of Vulnerable Elders (ACOVE) project59 and one indicator developed for 

COPD from the RAND QA Tools project that is applicable to patients with cancer and 

hypoxia.65

Measures and Indicators 

Assessment. We identified one measure and zero indicators relevant to dyspnea assessment. 

Measure - Regular Assessment of Dyspnea. The University Health System Consortium (UHC) 

Palliative Care Benchmarking Project used a performance measure across an alliance of 

academic health centers in the United States in 35 participating hospitals for dyspnea assessment 

within 48 hours of admission.67 The numerator is “persons with dyspnea assessment within 48 

hours of admission.” The specific criteria for how dyspnea assessment was operationalized were 

not described in available reports (other than the use of a 10-point scale). The denominator is 

“adults 18 years of age and older, with admission for congestive heart failure (DRG 127), cancer 

(DRG 82, 203, 172, 274, 346, 10), HIV (DRG 489), or respiratory illness (DRG 475, 483) and a 

length of stay more than 4 days and 2 prior hospital admissions for any cause in the preceding 12 

months.” This denominator was developed by an expert panel to represent a potential palliative 

care population in hospitals. No reliability or validity information is available; however, the 

benchmarking project found that a combination of use of all of the UHC measures in a 

“palliative care bundle” was associated with shorter patient length of stay. In addition, several 

elements of the bundle could be used to identify hospitals with “best practices” for palliative 

care. Scores from the bundle were instrumental in achieving advances in palliative care at 

several hospitals, including justifying at least one new palliative care program. 

 

*

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/eolcanqm/eolcanqm.pdf 
 44 

The dyspnea assessment measure is intended for use as a tool for comparing performance in 

all hospitals. The data for this measure come from billing, administrative, and clinical databases 

combined in the UHC data set. The UHC report indicates that a mean of 91% (median 95%, 

range 53-100%) of patients was assessed for dyspnea within 48 hours with the average rate for 

cancer being 89.9%. The measure criterion appears broad and does not include an element of 

timeliness (within 48 hours of admission may be too long for dyspnea assessment to occur). 

Further details about operationalization and details on psychometric testing are needed. The 

measure is not known to be in use. 

Treatment. We identified zero measures and three indicators relevant to dyspnea treatment. 

Indicator - Effective Dyspnea Treatment in Expected Dying When Forgoing Ventilator. ACOVE

describes a set of quality indicators for adults age 65 years and older in 2001.59 The numerator is 

“whether or not the patient received or had an order for an opioid, benzodiazepine, or barbiturate 

infusion to reduce dyspnea and the chart documented whether the patient has dyspnea.” The 

denominator is “all non-comatose vulnerable elders who are not expected to survive or those for 

whom a mechanical ventilator is withdrawn or intubation is withheld.” The indicator is designed 

to be evaluated by retrospective chart review and is most applicable to hospitalized patients; 

however, the indicator could be applied to any setting where ventilator withdrawal or 

withholding could occur. The withdrawal and withholding for mechanical ventilation indicator 

is supported by documentation of variation and inadequate management of dyspnea,100,101 bestpractice recommendations,102,103 and statements from professional societies.59,104,105 

Indicator - Regular Treatment and Follow-up of Dyspnea in Expected Dying. A second ACOVE 

indicator addresses the follow-up of dyspnea treatment at the end of life.59 The numerator is 

“whether or not the medical record documents how dyspnea was treated and followed up.” The 

denominator is “all vulnerable elders who had dyspnea in the last seven days of life and died 

expected deaths.” The indicator is designed to be evaluated by retrospective chart review and is 

applicable to any setting of care. This indicator was based on the premise that symptoms such as 

dyspnea should be a focus of end-of-life care and consensus calling for improvement in the care 

and management of dyspnea. In supporting documentation, the developers stated that dyspnea 

can be effectively treated with oxygen and pharmacologic agents.59 Like the withholding and 

withdrawal of ventilation indicator, this indicator was not evaluable in the community-based 

study of ACOVE measures.72 

Indicator - Treatment of Dyspnea Caused by Hypoxia. QA Tools did not include indicators for 

dyspnea within its multiple cancer sets.64-66,106 However, within the category of chronic 

obstructive pulmonary disease (COPD), one indicator for hypoxia is applicable to cancer patients 

with dyspnea caused by hypoxia.65,66,107 The numerator is “whether home oxygen therapy was 

used.” The denominator is “all patients with baseline room air oxygen saturation < 88% at rest.” 

The original indicator in the QA Tools project for patients with COPD read as follows: “COPD 

patients should receive home oxygen if their baseline room air oxygen saturation is <88% at rest 

(not during an exacerbation).”65 This indicator was based on multiple studies demonstrating 

benefit to both survival in hypoxic patients and subjective improvement of dyspnea symptoms 

through oxygen therapy.107 More recent data supporting use of oxygen for symptomatic 

improvement in hypoxic patients exists, including a recent review by a working group of the 
 45 

scientific committee of the Association of Palliative Medicine.108 This review included primarily 

patients with COPD and demonstrated more consistent symptom improvement with oxygen for 

those with hypoxia. 

The indicator is designed to be evaluated by retrospective chart review. This indicator was 

used in the COPD population for outpatient care during routine clinical interactions that did not 

occur during an exacerbation, however it may be pertinent to other settings such as physician 

offices, home health care, and home hospice. The indicator was operationalized by looking at 

the lowest oxygen saturation at least two weeks before the close of the study and scored as 

meeting the criterion if an order, prescription, or discharge order for home oxygen was identified 

within two weeks of a saturation value less than 88% or a note that the patient was already on 

home oxygen that was not discontinued. Reliability was based on a 4% duplicate abstraction 

from the medical record and demonstrated a kappa of 0.83 for agreement on the condition 

(COPD), a kappa of 0.76 for agreement on eligibility for the indicator, and a kappa of 0.80 for 

scoring. To become a measure for cancer care, the numerator and denominator of this indicator 

would need to be explicitly operationalized and tested in cancer populations. 

Follow-up. We found one measure and zero indicators related to follow-up for dyspnea. 

Measure - Timely Treatment of Inpatient Dyspnea. The UHC Palliative Care Benchmarking 

Project used a second performance measure regarding dyspnea relief or reduction of dyspnea.67 

The numerator is “persons with dyspnea relief or reduction within 48 hours of admission.” 

Significant reduction was operationalized as a 3-or-more-point reduction in dyspnea on a 10-

point scale. The denominator is “adults 18 years of age and older, with admission for congestive 

heart failure (DRG 127), cancer (DRG 82, 203, 172, 274, 346, 10), HIV (DRG 489), or 

respiratory illness (DRG 475, 483) and a length of stay more than 4 days and 2 prior admissions 

for any cause in the preceding 12 months and reported dyspnea within the first 48 hours of 

admission.” No reliability or validity information is available. Information is the same as for the 

other UHC dyspnea indicator. The UHC report indicates that 62% of patients assessed had 

dyspnea within 48 hours after admission and that of these, 77% overall and 77% of cancer 

patients had relief or reduction of these symptoms. There was substantial variation in results 

with a range of 37.5% to 96.6% criteria met across the 35 hospitals participating in the 

benchmarking project. Further details about operationalization and details on psychometric 

testing are lacking. The indicator set is not known to be in use. 

Potential Indicators and Measurement Gaps 

Assessment. We identified a large number of potential indicators in the substantial guideline, 

quality improvement, and research literature that we reviewed. We highlight here six potential 

indicators with relatively strong supporting evidence and/or expert consensus, highlighting 

critical issues not addressed by the indicators and measures listed above. These indicators have 

potential for demonstrating variability in quality and improving patient outcomes, and are 

applicable to cancer patients across the continuum and to diverse populations. 

Although guidelines and evidence support assessment of dyspnea,15,18,21,109 future research is 

needed to understand the most appropriate approach and instruments for assessment (i.e., 

provider observation, single-item symptom measures, or multi-item self-report effects of 
 46 

dyspnea) in particular settings and diseasesi

 as well as operating characteristics as quality 

indicators.15,16,109 Subsequent research is indicated for indicators in dyspnea assessment. In the 

interim, it would be reasonable to develop and use a general indicator that at least some form of 

dyspnea symptom assessment was performed for all cancer patients. 

Assessment for Treatable Causes of Dyspnea. The numerator is “all patients who have been 

assessed for treatable causes of dyspnea.” The denominator is “all cancer patients with 

dyspnea.” We identified four reports on the assessment of underlying causes of dyspnea.14,109-111 

The American College of Chest Physicians (ACCP) and American Thoracic Society (ATS) have 

recommended specific assessment for potentially correctable causes of dyspnea in this 

setting.109,110 A prospective observational study demonstrated potentially correctable causes for 

dyspnea such as hypoxia (40%), anemia (20%), and bronchospasm (52%).14 Another report 

supports the use of spirometry to detect any airway obstruction in patients with dyspnea.111 A 

reasonable potential indicator would be to use medical record review to document that 

correctable conditions were sought in evaluation of dyspnea in cancer patients. Some 

consideration of time windows for recent evaluation would be important in operationalizing this 

as a standard. 

Treatment. We identified eight potential indicators related to the treatment of dyspnea, from 

seven reports.110,108,109,112-115 As with the general research gap in assessment of dyspnea, the 

operationalization of how much and when to assess for response to therapy requires future 

research and refinement of quality indicators. 

Treatment for Malignant Pleural Effusions. The numerator is “patients who have had 

thoracentesis and appropriate subsequent therapy (pleurodesis or chemotherapy) as indicated.” 

The denominator is “cancer patients with dyspnea and malignant pleural effusions.” Evidencebased guidance from the ACCP Lung Cancer Guidelines, published by the American College of 

Chest Physicians includes (1) Patients with malignant pleural effusions that cause dyspnea 

initially should be drained by thoracentesis; (2) Patients with non-small cell lung cancer and 

better performance status and recurrent malignant pleural effusions, and whose lungs re-expand 

with initial thoracentesis or thoracoscopy, should be followed up by pleurodesis;110 and (3) 

Patients with small cell lung cancer receive systemic chemotherapy when malignant effusions 

are present to relieve the symptoms of dyspnea.110 Potential indicators for these specific 

conditions could be developed and tested. This guidance is clinically applicable to managing 

dyspnea in other cancers that cause malignant pleural effusions (breast, gastrointestinal, prostate, 

melanoma, others). 

Bronchoscopic Therapy. The numerator is “patients who are offered, or who have, 

bronchoscopic therapy.” The denominator is “cancer patients with central airways obstruction.” 

 

i

 Measures may include the Lung Cancer Symptom Scale, Oxygen Cost Diagram, Baseline Dyspnea Index, 

Transition Dyspnea Index, University of California Shortness of Breath Questionnaire, Borg or Modified Borg 

Dyspnea Scale, Visual Analogue Scale, Chronic Respiratory Disease Questionnaire, Saint George Respiratory 

Questionnaire, Pulmonary Functional Status Scale, Breathing Problems Questionnaire, Modified Medical Research 

Council [Dyspnea Scale], American Thoracic Shortness of Breath Scale, and many other candidates including single 

item symptom scales, severity of symptom tools, generic health-related quality of life instruments, dyspnea related 

quality of life instruments, generic palliative care and satisfaction measures, symptom specific treatment response 

measures, and quality of dying measures.15, 16, 109
 47 

The ACCP110 also present evidence based on a review of the literature that bronchoscopic 

therapy can reduce dyspnea in the presence of central airway obstruction, recommending that 

patients with central airway obstruction should have bronchoscopy and that in patients with 

central airway obstruction, rapid relief of dyspnea can be accomplished by removal of 

intraluminal tumor and/or by inserting a stent. A potential indicator may be that if a patient has 

cancer and central airway obstruction, then bronchoscopy and bronchoscopic therapy should be 

considered. This potential indicator is most applicable to lung cancer. 

Patient Education and Non-Pharmacologic Interventions. The numerator is “patients who have 

received education about their dyspnea.” The denominator is “all cancer patients with dyspnea.” 

Kvale and colleagues110 recommended that all lung cancer patients with dyspnea receive nonpharmacologic, non-interventional treatments including patient education to reduce the dyspnea 

symptoms. Patient education can encompass breathing control education, relaxation techniques, 

and psychosocial support. The evidence for this potential indicator was poor based on their 

review of the literature, and these interventions were felt to have poor supporting evidence but 

moderate net benefit due to the lack of side effects. This indicator could be broadly formulated 

based on clinical practice to all forms of malignancy with dyspnea. 

Palliative Radiation and Chemotherapy.113,114 The numerator is “patients who are offered or 

have received appropriate palliative chemotherapy or palliative radiation therapy.” The 

denominator is “all patients with particular malignancies.” The Lung Cancer Guidelines 

published by the American College of Chest Physicians114 present evidence that chemotherapy 

can reduce dyspnea symptoms among patients with Stage IV non-small cell lung cancer. 

Socinski et al., identified seven studies that explored the palliation of symptoms by 

chemotherapy; four of the seven reduced dyspnea symptoms in over half of the patients 

enrolled.114 Palliative chemotherapy should be considered for patients who are deemed suitable 

for treatment, including other malignancies. 

The American Society of Clinical Oncology published a guideline for the treatment of 

unresectable non-small cell lung cancer113 in 2003 in which they cite research that radiotherapy 

can provide relief from dyspnea symptoms. However, the evidence cited in this report included 

only nonrandomized comparisons to conventional therapy. This indicator may likely be 

operationalized only in retrospective medical record review with fulfillment of criteria as 

consideration for therapy.

General Approaches. The numerator is “patients who have had oxygen, bronchodilators, 

corticosteroids, opioids, or antibiotics attempted. The denominator is “patients with cancer and 

dyspnea (and possibly co-morbid conditions such as COPD).”110 The evidence for the 

effectiveness of particular interventions is modest, but the net benefit may be moderate.110 The 

ATS endorsed oxygen therapy, opioids, anxiolytics, and bronchodilator trials to diminish 

dyspnea symptoms among terminally ill patients with lung disease.109 Several systematic 

reviews support the efficacy of opioids and oxygen in patients with advanced illness including 

cancer.21,112,116 A reasonable criterion may be to consider whether therapy was considered, 

offered, or tried. The operationalization of this indicator would require further specification 

based on underlying disease, the physiologic mechanism of dyspnea, and particular therapeutic 

interventions (for example, specifying concomitant obstructive lung disease when evaluating use 

of bronchodilators). 
 48 

Depression 

Introduction 

For the condition of depression, we searched for measures and indicators restricted to the 

condition of cancer. We identified one measure and three indicators that were relevant. Please 

refer to the Evidence Table in Appendix F3*

. 

The UHC Palliative Benchmarking Project has evaluated one indicator related to formal 

psychosocial assessment in mixed hospitalized populations that include cancer patients.67 

ACOVE’s end-of-life indicators addressed spiritual assessment prior to death, which we 

considered broadly applicable.59 ACOVE also included measures and indicators for depression, 

as did QA Tools.65,72 Some of these may be regarded as relevant, although we included only two 

additional ACOVE indicators, because one explicitly addressed depression in the elderly in the 

context of a new cancer diagnosis, and the other addressed somatic symptoms in vulnerable 

elders that overlap with presenting symptoms of cancer. QA Tools relied on AHCPR depression 

guidelines that emphasized cancer as an associated risk factor, although none of its indicators 

address this situation directly. 

Measures and Indicators 

Assessment. We identified one measure and one indicator relevant to cancer and depression 

assessment. 

Measure - Regular Assessment for Psychosocial Well-Being. The UHC Palliative Care 

Benchmarking Project67 included one measure relevant to depression assessment. The numerator 

is “persons with a formal psychosocial assessment (usually an assessment by a social worker) up 

to one year prior to admission during a previous admission OR within four days of the index 

admission.” The denominator is “adults ≥18 years of age, with admission for CHF (DRG 127), 

Cancer (DRG 82, 203, 172, 274, 346, 10) HIV (DRG 489) OR respiratory (DRG 475, 483) AND 

length of stay > 4 days AND 2 prior admissions for any cause in preceding 12 months.” This 

denominator was developed by an expert panel to represent a potential palliative care population 

in hospitals. Only a mean of 25% (median 17%, range 0-95%) of the UHC sample met the 

criteria for this indicator. This indicator identified four high performing facilities where more 

than 60% of patients had been assessed in the prior year. No reliability or validity information is 

available; however, the benchmarking project found that a combination of use of all of the UHC 

measures in a “palliative care bundle” was associated with lower patient length of stay. In 

addition, several elements of the bundle could be used to identify hospitals with “best practices” 

for palliative care. Scores from the bundle were instrumental in achieving advances in palliative 

care at several hospitals, including justifying at least one new palliative care program. Finally, 

results for the “bundle” varied significantly among the 35 hospitals participating in the project. 

It is intended for use as a tool for comparing performance in all hospitals. The measure is not 

known to be in current use. 

 

*

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/eolcanqm/eolcanqm.pdf 
 49 

Indicator - Regular Spiritual Assessment in Expected Dying. ACOVE identified one indicator 

relevant to depression assessment. The numerator is “vulnerable elders with a documentation of 

a spiritual assessment in the medical record.” The denominator is “vulnerable elders who were 

conscious during the last three days of life and who died an expected death.” Although eight 

items from the ACOVE end-of-life set were evaluated for reliability and validity72 as part of the 

ACOVE project, this indicator was not included in the evaluation. The denominator may be 

difficult to define, since “expected death” requires explicit medical record documentation more 

than one day before death that the patient is expected to die. This measure is intended for use in 

all vulnerable elders. An expert panel accepted a modified version of this indicator as part of a 

revised set for nursing homes.60 The nursing home indicator requires a denominator of “nursing 

home residents who died expected deaths and were conscious at all during the last seven days of 

life.” The numerator for the adapted measure requires that pain, spirituality, and emotional 

distress all have been addressed. Neither indicator is in current use. 

Treatment. We identified two measures relevant to depression treatment among cancer patients. 

Measure - Regular Assessment or Treatment of Depression in Newly Diagnosed Cancer. 

ACOVE identified two indicators relevant to both depression assessment and treatment.117,60 For 

the first indicator, the numerator is “patients asked about or treated for depression, or referred to 

a mental health professional within two months of the diagnosis of a condition.” The 

denominator includes “all persons diagnosed with stroke, myocardial infarction, dementia, 

malignancy (excluding skin cancer), chronic pain, alcohol or substance abuse or dependence, 

anxiety disorder, or personality disorder.” These conditions were accepted for the indicator on 

the basis of their association with incident or prevalent depression. Although 13 indicators from 

the ACOVE depression set were evaluated for reliability and validity in a community 

population,72 this indicator was not included, although it was operationalized for cancer patients. 

No information on reliability and validity is available for this indicator. The indicator is not in 

use. 

A second ACOVE measure (routine assessment or treatment of depression in symptomatic 

patients) relevant to depression assessment and treatment addressed both affective and somatic 

symptoms among vulnerable elders.117,60 The numerator is “patients asked about or treated for 

depression, or referred to a mental health professional within two weeks of presentation.” The 

denominator is “vulnerable elders with new onset of one of the following symptoms-sad mood, 

feeling down, insomnia or difficulties with sleep, apathy or loss of interest in pleasurable 

activities, reports of memory loss, unexplained weight loss of more than 5% of body weight in 

the past month or 10% over one year, or unexplained fatigue or low energy.” This indicator was 

included in the 13 indicators from the ACOVE depression set, although ACOVE field evaluation 

excluded patients undergoing active cancer treatment. Only 26 of the 34 quality indicators 

triggered were passed.72 No information on reliability or validity is available. This indicator is 

not in use. 

Follow-up. We identified zero measures and zero indicators relevant to the follow-up of 

depression treatment in cancer patients. 
 50 

Potential Indicators and Measurement Gaps 

Within the literature that we identified, only the ACOVE set provided measures and 

indicators for depression among a population that did not include cancer patients, per se or was 

not limited to cancer patients; thus we considered these as potential indicators. Quite a number 

of other measurement sets address depression care in general. ACOVE provides only a general 

example of that class of non-cancer depression indicators that might be considered. However, 

many of these indicators may not be appropriate for hospice or similar settings, since some of 

them address issues such as follow-up within months of depression diagnosis, and life 

expectancy in such settings may be very short. Nevertheless, within ACOVE, we identified the 

following additional potential indicators for cancer care. 

Assessment. We identified three potential indicators relating to initial depression assessment. 

Appropriate Diagnosis. Two ACOVE indicators are based on the assumption that depression 

diagnosis should document the presence or absence of major affective symptoms and suicidality. 

Both of these indicators use as a denominator population “all vulnerable elders with a new 

diagnosis of depression.” The first addresses the need to distinguish between major depression 

and other affective syndromes and uses as a numerator “patients where the medical record 

documents at least three of nine DSM-IV target symptoms within the first month of diagnosis.” 

The second indicator addresses the need to evaluate suicidality and uses as a numerator “patients 

where the medical record on day of diagnosis documents the presence or absence of suicidal 

ideation and psychosis.” These indicators are based on the findings that only patients with major 

depression are known to respond to medication117 and that evidence of psychosis and suicidality 

should be considered in the approach to treatment.117 The documentation of depressive 

symptoms may differ somewhat for cancer patients, especially late stage patients where fatigue 

and other manifestations of illness overlap with the “vegetative symptoms.” 

Evaluating Suicidality. One ACOVE indicator addresses the need to evaluate and treat 

suicidality appropriately when it is present in a patient with newly diagnosed depression. The 

numerator for this indicator is “patients where the medical record documents that the person has 

no immediate plans for suicide or that the patient was referred for evaluation for psychiatric 

hospitalization.” Expert opinion supports this approach even though ethical considerations make 

studies of interventions difficult. Suicide is a disproportionate concern among the elderly in 

general, and cancer patients in particular.117

Treatment. We identified 2 potential indicators that were related to initial treatment. 

Use of the Most Appropriate Anti-Depressants. Two ACOVE indicators address the appropriate 

use of medication in patients newly treated for depression. The denominator for both of these 

indicators is “vulnerable elders being treated for depression with antidepressants.” The 

numerator for the first potential indicator is “patients where the following medications are not 

used as first or second-line therapy: tertiary amine tricyclics, monoamine oxidase inhibitors 

(unless atypical depression is present), benzodiazepines, or stimulants (except 

methylphenidate).” This indicator is based on the fact that for the most part, antidepressants 
 51 

have equal efficacy, although side-effect profiles differ and are generally worse for older 

antidepressants highlighted by the indicator.117 

Dose Titration. The denominator for this indicator is “vulnerable elders being treated for 

depression with antidepressants.” The numerator for the second potential indicator is “patients 

with antidepressants started at appropriate doses, and with an appropriate titration schedule to a 

therapeutic dose, therapeutic blood level, or remission of symptoms by 12 weeks.” Many studies 

demonstrate that older patients do not receive appropriate follow-up including therapeutic doses 

of antidepressant medications.117 

Follow-up. We identified four potential indicators that were related to follow-up of initial 

treatment. 

Addressing Non-Responders to Initial Treatment. The numerator for this potential indicator is 

“patients with antidepressant treatment, psychotherapy, or electroconvulsive therapy (ECT) 

offered within two weeks after depression diagnosis unless there is documentation within that 

period that the patient has improved, or unless the patient has substance abuse or dependence, in 

which case treatment may wait until eight weeks after the patient is in a drug or alcohol free 

state.” The denominator is “vulnerable elders with a new diagnosis of depression.” This 

indicator addresses the fact that psychotherapy may be the initial treatment attempted in 

depression, and that although the time-course of response is unclear, other treatments should be 

started within weeks of diagnosis if it is ineffective.117

Addressing Partial Responders to Treatment. Two ACOVE indicators addressed the appropriate 

care for patients who were only partial responders to initial pharmacotherapy at 6 and 12 weeks. 

The denominators for those potential indicators are “vulnerable elders who have only had a 

partial response to therapy (at each time period).” For the first indicator that addresses week-6 

nonresponders, the numerator is “patients with one of the following treatment options initiated 

by the eighth week of care: medication dose should be optimized or the patient should be 

referred to a psychiatrist (if initial treatment was medication); or medication should be initiated 

or referral to a psychiatrist should be offered (if initial treatment was psychotherapy alone).” 

The second indicator addresses partial responses at 12 weeks. The numerator for this indicator is 

“patients with one of the following options instituted by the 16th week of treatment: switch to a 

different medication class or add a second medication to the first (if initial treatment includes 

medication); add psychotherapy (if the initial treatment was medication); try medication (if 

initial treatment was psychotherapy without medication); consider ECT; or refer to a 

psychiatrist.” 

Maintaining Effective Anti-Depressant Care. This potential indicator addresses the need for 

maintenance therapy. The denominator is “vulnerable elders who have responded successfully 

to antidepressant medication treatment.” The numerator is “patients continued on the drug at the 

same dose for at least six months, and making at least one clinician contact (office visit or phone) 

during that period.” This potential indicator acknowledges uncertainty in the exact amount of 

time that therapy should be continued in responders but upholds a lower limit of six months.117
 52 

Advance Care Planning 

Introduction 

The review of literature in the area of advance care planning was not limited only to cancer. 

We identified a total of 30 indicators concerning the identification and documentation in the 

patient’s medical record of advance directives, designation of a proxy decisionmaker, 

preferences for care in cognitively impaired elderly, referral to hospice, and utilization-based 

indicators of high intensity care. Please refer to the Evidence Table in Appendix F4*. We 

included utilization indicators because much of the literature on advance care planning has 

focused on avoiding (e.g., resuscitation) or facilitating (e.g., hospice referral) particular treatment 

or utilization outcomes. However, we acknowledge that such indicators are limited as patientcentered metrics and may inadequately account for patient preferences. Furthermore, measures 

of particular types of utilization (e.g., hospitalization) may also reflect structural factors or the 

adequacy of alternative services. For that reason, other investigators might have made a 

reasonable decision to exclude such outcomes from consideration, although we included them 

for completeness. The indicators tended to overlap significantly and, where there was overlap, 

the primary differences were in care setting (e.g., ICU vs. nursing home, nursing home vs. 

hospital)68 or populations (e.g., dementia vs. incapacitated patients).118 In addition, the literature 

on advance directive/advance care planning overlaps with the literature on good end-oflife/palliative care standards. Where appropriate, these overlapping indicators will be discussed 

together. Some items that did not clearly address “assessment,” “application,” or “follow-up” 

but crossed all domains are addressed separately. Finally, indicators may identify potential 

standards for hospital, system,67 or regional55,119,120 performance. 

Measures and Indicators 

Assessment. We identified eight measures and four indicators relevant to advance care planning 

assessment.

Measure - Regular Identification of a Surrogate in Outpatient Setting. ACOVE’s end-of-life 

indicators included one measure addressing the identification of a surrogate decisionmaker 

among outpatients.59,72 The numerator is “the number of vulnerable elderly patients with 

medical record documentation of their surrogate decisionmaker’s name and contact information, 

or of a discussion with the patient of who would be surrogate, or documentation of a search for a 

surrogate, or an indication that there is no identified surrogate.” The denominator is “all 

vulnerable elderly outpatients.” No clinical trials or observational studies have examined this 

issue, yet documentation is an important tool for clinician access to an incapacitated patient’s 

decisionmaker,21,121 and many consensus statements promote designation of a surrogate 

decisionmaker (e.g., AGS Ethics Committee, 1995). One prospective study of physicians 

discussing advance directives with patients revealed that the physicians recognized the correct 

surrogate decisionmaker 89% of the time after such a discussion.122 Only 4% of 370 eligible 

 

*

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/eolcanqm/eolcanqm.pdf 
 53 

charts in the 420-person ACOVE community-based sample passed this indicator.72 No 

reliability or validity information is available for the individual measure. The measure is not in 

use. 

Measure - Regular Identification of a Surrogate Among Hospital Admissions With Impaired 

Cognitionii. One ACOVE measure addresses surrogacy among patients admitted to the 

hospital.59,72 The numerator is “the number of vulnerable elders (among those with the 

denominator conditions) who have chart documentation within 48 hours of either an advance 

directive indicating the patient’s surrogate decisionmaker, a discussion about who would be a 

surrogate decision maker or a discussion about a search for a surrogate, or an indication that 

there is no identified surrogate.” The denominator is “all vulnerable elders with dementia, coma, 

or altered mental status admitted to a hospital who survive 48 hours.” This indicator has 

implications for cancer patients who may have temporary (e.g., delirium) or durable cognitive 

impairment. Research has shown that the majority of patients admitted to the hospital have 

neither named a surrogate decision maker nor expressed their treatment preferences in a written 

document.123 However, observational data suggest that physician understanding of patient 

resuscitation preference (a component of advance directives) is associated with a better match 

between those preferences and resuscitation attempts and is associated with less end-of-life 

resource use.124-126 Only 25% of 20 eligible charts in ACOVE’s 420-person community-based 

sample passed this indicator.72 No reliability or validity data are available on this individual 

item. The measure is not in use. 

Measure - Regular Assessment of Preferences Among Inpatients With Dementia. One ACOVE 

measure addresses documentation of preferences among inpatients with severe dementia.59,72 

The numerator is “the number of severely demented elders with documentation in their medical 

record that the patient’s prior preferences for care have been considered or that these preferences 

could not be elicited or are unknown.” The denominator is “all vulnerable elderly patients with 

severe dementia admitted to the hospital who survive 48 hours.” This indicator has implications 

for cancer patients who may develop cognitive impairment associated with their cancer (e.g., 

delirium). Aggressive medical interventions and inadequate pain management are common in 

hospitalized dementia patients.127 This measure could be evaluated among only 2 of 420 cases in 

ACOVE’s community-based cohort.72 No reliability or validity information is available for the 

specific measure. The measure is not in use. 

Measure - Regular Assessment of Preferences in an ICU. This ACOVE measure addresses 

documentation of preferences in an ICU.59,72 The numerator for this indicator is “the number of 

vulnerable elders admitted directly to the intensive care unit from an outpatient or ER setting and 

surviving 48 hours with documentation in the medical record that the patient’s preferences for 

care have been considered or that these preferences could not be elicited or are unknown within 

48 hours of admission.” The denominator for this indicator is “all vulnerable elderly patients 

admitted directly to the ICU from the outpatient setting or emergency department and who 

 

ii Dementia and altered mental status were defined as: diagnosis of dementia OR MD or RN note on Day 1 or 2 of 

cognitive impairment (confused, disoriented, Ox2,Ox1, unable to follow commands) OR neurologic change 

(somnolent, lethargic, poorly arousable, semicomatose, stuporous, obtunded, comatose) OR inattentive, restless, 

agitated. 
 54 

survive 48 hours.” Only 6 of 420 patients in ACOVE’s community sample were eligible for this 

measure, and 17% passed.72 No reliability or validity data are available for this specific measure. 

The measure is not in use. 

Measure - Regular Assessment of Preferences in Hospice. NHPCO54 measure addresses 

common decisions about activities of daily living, life closure, and care received by patients and 

family members facing a terminal illness. The numerator for this indicator is “the number of 

hospice patients who received care (either hospitalization or resuscitation) consistent with their 

expressed preferences as assessed at the time of admission.” The denominator is “all patients 

admitted to hospice.” Experts identified a number of global categories of decisionmaking, such 

as determination of the degree of symptom management desired, setting goals in relation to 

sentinel events, expressing or not expressing religious and spiritual needs, determination of one’s 

wishes for the site of death, hospitalization, and CPR.63 The intent of this indicator is to assist 

hospice patients in identifying care preferences and then meeting these needs by conducting a 

comprehensive assessment, including physical, functional, emotional, and spiritual and 

preferences, and allowing full autonomy to the patient and family to make decisions about how 

the remainder of the patient’s life is to be spent upon admission. Hospices report these data 

voluntarily to NHPCO. In 2002, 120 agencies reported this measure, and the mean and median 

values were 97%. No reliability or validity data are available. This measure is in current use as 

a voluntary reporting standard. 

Measure - Regular Patient Participation in Decisions To Limit Treatment. This ACOVE 

measure focuses on ensuring participation in decisions for the withdrawal or withholding of lifesustaining treatments (LST), which is consistent with ethical and professional guidelines.59,72 

The numerator is “the number of vulnerable elderly patients with documentation in the medical 

record of participation in the decision or why the patient did not participate in a decision 

regarding LST.” The denominator is “all vulnerable elderly patients in the hospital or nursing 

home with written orders to withdraw or withhold a LST.” A small study comparing chart 

documentation to patient recall about life-sustaining treatment decisions showed that medical 

record documentation reflected patient understanding of these decisions.128 Ten of the 420 cases 

in ACOVE’s community-based sample met criteria for the measure, and 70% passed.72 No 

reliability or validity information is available on this specific measure. The measure is not in 

use. 

Measure - Regular Family Meetings Among Hospitalized Patients. The numerator for this UHC 

measure is “the number of adults admitted to the hospital with documentation in the medical 

record that the health care team conducted a patient/family meeting within one week of 

admission that included discussion of the patient’s treatment preferences and/or a plan for 

discharge disposition.”67 The denominator is adults ≥18 years of age, with admission for CHF 

(DRG 127), cancer (DRG 82, 203, 172, 274, 346, 10) HIV (DRG 489) OR respiratory (DRG 

475, 483) AND length of stay > 4 days AND 2 prior admissions for any cause in preceding 12 

months.” This indicator is similar to the ACOVE measures59,72 that specify documentation of 

patient participation in decisionmaking (implying communication of some sort between the 

patient and provider team) and the VHA indicators68 that specify conducting an interdisciplinary 

team meeting with the family within 120 hours of ICU admission. A mean of 39% (median 

41%, range 0-93%) of the UHC cohort met this indicator. Only 55% of cancer patients met the 
 55 

indicator. The distribution of results appeared to be broadly normal. As with the other UHC 

measures, no individual reliability or validity information is available. It is not known to be in 

current use. 

Measure - Timely and Effective Discharge Planning. The numerator for this UHC measure is the 

number of adults admitted to the hospital with documentation in the medical record of a plan for 

discharge disposition within four days of admission.”67 The denominator is “adults ≥18 years of 

age, with admission for CHF (DRG 127), cancer (DRG 82, 203, 172, 274, 346, 10) HIV (DRG 

489) OR respiratory (DRG 475, 483) AND length of stay > 4 days AND 2 prior admissions for 

any cause in preceding 12 months.” This measure implies that all patients who enter the hospital 

and stay at least one week should receive a plan for discharge disposition within four days of 

admission. Planning for discharge is usually not explicitly included in discussions of advance 

directives, although many critically ill patients frequently move in and out of the hospital and 

across a variety of health care settings as they progress on a downward trajectory toward death. 

This explicit recognition of the importance of planning for care after leaving the hospital is a 

contribution to the advance directives discussion. A mean of 53% (median 53%, range 18-94%) 

of the UHC cohort met this indicator. Only 62% of cancer patients met the indicator. The 

distribution of results appeared to be broadly normal. As with the other UHC measures, no 

individual reliability or validity information is available. It is not known to be in current use. 

Indicator - Regular Identification of a Surrogate in the ICU. This VHA indicator focuses on the 

identification of a medical decisionmaker (family member or other appropriate surrogate) and is 

similar to the ACOVE indicator68,72,129 identifying a surrogate/proxy for community-dwelling or 

hospitalized vulnerable elders. Other than setting, the major difference with the ACOVE 

indicators is the timeframe identified for performance of the indicator (24 vs. 48 hours): The 

denominator is “the total number of ICU patients with a stay of > 24 hours.” The numerator is 

“the number of patients who have documentation on the status of identification of a health care 

proxy (or other appropriate surrogate decisionmaker) within 24 hours.” Support for this 

indicator is derived from JCAHO Hospital Standard RI.1.2.3, literature focusing on the 

withdrawal of life-support in the ICU,130 and practice guidelines.100 This measure is currently 

being used as part of the VHA TICU initiative and is part of the VHA palliative care bundle, 

which will be initiated in September 2005. 

Indicator - Regular Assessment of Advance Directives for ICU Patients. This VHA indicator 

focuses on the specification and documentation of the patient’s advance directive if admitted to 

the ICU within 24 hours of admission.68 The numerator for this indicator is “the number of 

patients admitted to the ICU with documentation of the patient’s advance directive (written or 

oral instructions from the patient specifying the type of medical treatment that is desired if the 

patient becomes incapacitated, including a living will, durable power of attorney (note Statespecific status) or any document that State law recognizes as an “advance directive” entered in 

the medical record within 24 hours following admission.” The denominator is “the total number 

of patients with an ICU length of stay greater than 24 hours.” Support for this indicator is 

derived from JCAHO Hospital Standard RI.1.2.5; the National Consensus Project for Quality 

Palliative Care Clinical Practice Guideline 1.2, 8.169,131 and literature on decision making and 

medical outcomes.132,133 This measure is currently being used as part of the VHA TICU 
 56 

initiative and is part of the VHA palliative care bundle, which will be initiated in September 

2005. 

Indicator - Regular Assessment of Specific Resuscitation Preferences in the ICU. This VHA 

indicator focuses on the percent of patients with documentation in the medical record of 

resuscitation status within 24 hours following admission to the ICU.68 The numerator for this 

indicator is “the number of patients admitted to the ICU with documentation of the patient’s 

advance directive (written or oral instructions from the patient specifying the type of medical 

treatment that is desired if the patient becomes incapacitated, including a living will, durable 

power of attorney (note State-specific status) or any document that State law recognizes as an 

‘advance directive’) entered in the medical record within 24 hours following admission.” The 

denominator is “the total number of patients with an ICU length of stay greater than 24 hours.” 

Support for this indicator is derived from the literature on decisionmaking and medical 

outcomes.132 This measure is currently being used as part of the VHA TICU initiative and is part 

of the VHA palliative care bundle, which will be initiated in September 2005. 

Indicator - Regular Clinician-Patient-Family Communication in the ICU. This VHA indicator 

addresses inadequate communication among patients, family members, and health care providers 

in ICUs.68,134 The numerator for this indicator is “the number of patients admitted to the ICU 

who survive more than 5 days (120 hours) with documentation in the medical record that an 

interdisciplinary team meeting (with at least the attending physician (primary care or ICU 

attending) and a nurse) and the patient and/or family was conducted within 72 hours of ICU 

admission and involved a discussion addressing each of the following topics: the patient’s 

condition (diagnosis and prognosis), goals of treatment, the patient’s and family’s needs and 

preferences, and the patient’s and family’s understanding of the patient’s condition and goals of 

treatment at the conclusion of the meeting.” The denominator is “total number of patients with 

an ICU length of stay more than 5 days (120 hours).” Support for this indicator is derived from 

JCAHO Standards RI.1.2.2.-RI.1.2.8, RI.1.3.6; NCP_CPG 1.3.135,136 A joint meeting with the 

surrogate and family members and the health care team, labeled a “family conference,” should be 

used to answer any questions family members might have regarding the condition of the patients 

and to explore goals of care. The literature identifies various elements of good communication 

as well as barriers to advance directive discussions.137-141 This measure is currently being used 

as part of the VHA TICU initiative and is part of the VHA palliative care bundle, which will be 

initiated in September 2005. 

Application. We identified two measures and zero indicators addressing the application of 

existing care plans. 

Measure - Documentation of Care Preferences Across Venues. Several ACOVE measures 

address advance directive continuity.59, 72 The goal of advance care planning is to ensure that 

patient’s preferences for care are identified and followed. The denominator is “all vulnerable 

elderly with an advance directive in the outpatient, inpatient, or nursing home medical record or 

who report the existence of an advance directive in an interview, and are receiving care in 

another venue.” The numerator is “those cases with documentation in their medical record of the 

existing advance directive at the second venue or documentation acknowledging the existing 

advance directive, it contents, and the reason that it is not included in the medical record.” Five 
 57 

observational studies have documented that even when advance directives are completed, 

patients’ physicians often do not know that they exist,126,142,143 and the directives are not always 

documented in the patients’ charts144 or transferred from outpatient to inpatient settings.145 This 

measure passed in 25% of the 8 cases among 420 community-dwelling elderly in the ACOVE 

study.72 No reliability or validity information is available on the specific measure. The measure 

is not in use. 

Measure - Documentation of Specific Life Sustaining Preferences. This ACOVE measure 

addresses the need to document decisions a vulnerable elder makes concerning future health 

states.59, 72 The numerator is “patients with one of the following documented in the medical 

record: a discussion of life-sustaining treatment preferences, an advance directive, or that the 

patient discussed this topic with the physician or does not wish to discuss this topic.” The 

denominator is “all vulnerable elderly patients who indicate (during an interview) that they 

would rather die than live permanently comatose, ventilated, or tube fed.” Observational studies 

show that large proportions of patients would prefer to die than live permanently comatose, 

mechanically ventilated, or tube fed and that physicians and surrogate decision makers often do 

not know patients’ preferences about life-sustaining treatments.59 Consensus statements promote 

the documentation of these preferences (e.g., The American Geriatric Society Ethics Committee, 

1995). Of the 238 of 420 eligible patients in ACOVE’s community-based sample, only 12% 

passed the measure.72 No reliability or validity information is available on this specific measure. 

The measure is not in use. 

Follow-up. We identified 14 separate measures (covering 10 general categories) and 7 

indicators (covering 8 general categories) addressing follow-up. Many of these administrative 

measures and indicators are closely related. For example, Cancer Care Ontario119 and CCNS146

are both using indicators and measures very similar to those developed by investigators at the 

Dana-Farber Cancer Institute.146 For that reason, these similar measures and indicators are 

discussed together whenever possible. 

Measure - Consistency of Preferences With Use of Ventilator Support. This ACOVE measure 

focuses on consistency between use of ventilator support and documented preferences.59, 72 The 

numerator is “the number of records with documentation of the goals of care and the patient’s 

decision for mechanical ventilation or why this information is unavailable.” The denominator is 

“all hospitalized vulnerable elders requiring mechanical ventilation (except short-term and post 

operative mechanical ventilation).” This indicator is intended for use with all vulnerable elderly 

patients requiring mechanical ventilation, including cancer patients. Practices concerning 

withdrawal of treatment for patients receiving mechanical ventilation vary widely.123,135,143,147,148 

Practices regarding withholding of specific treatments among surveyed physicians also varied 

greatly.100 The SUPPORT study found that a primary predictor of whether a ventilator was 

withdrawn was the existence of communication about care preferences.149 Only 2 of 420 persons 

in ACOVE’s community-based study met criteria for the measure. No reliability or validity 

information is available on this specific measure. The measure is not in use. 

Measure - Late Life Hospital Use.77,150 Several measures or indicators address late-life hospital 

use. One measure was developed by Dana-Farber Cancer Institute146 investigators, and a closely 

related measure is in use by Cancer Care Ontario.119 A similar indicator is in the process of 
 58 

being operationalized by CCNS.146 The numerator for the Dana-Farber measure is “the 

proportion of cancer decedents with more than one hospitalization or ER visit in the last month 

of life. The denominator is “all cancer decedents.” Support for its validity comes from literature 

reviews; focus groups of patients, caregivers, and providers; expert panel review; and 

quantitative analysis of the indicator in Medicare claims data. This measure was evaluated in a 

Medicare-SEER database, and using the highest performing decile of this measure implied that 

in high performing health care service areas (HCSAs), fewer than 4% of cancer decedents would 

have more than one hospitalization or emergency room visit in the last month of life. When a 

sample of 150 charts at Dana-Farber were abstracted as the gold standard against which the 

sensitivity and specificity of the measure would be determined, this measure was determined to 

have a sensitivity of 0.96, specificity of 1.00, and accuracy of 0.097. As a measure of variability, 

investigators determined the 5th/95th percentile ratio (2.38) (range: 1.85-3.16). The Cancer Care 

Ontario measure is “hospital use in days by stage of cancer.” The populations for the “stage of 

cancer” consist of those in initial care (within six months of diagnosis), continuing care, and 

terminal care (within six months of death). A separate category, investigative care, is defined as 

patients with a benign outcome. Provincial resource use in 2002 indicated that as a proportion of 

total cost, the terminal phase of care used 29 to 42% of resources across Ontario. The initial 

phase of care used 31 to 38% of total resources.120 A closely related indicator, hospital days near 

the end of life, is being operationalized by CCNS. 

Measure - Late Life ICU Use.77,146,150 Several measures or indicators address late-life ICU use. 

One measure was developed by Dana-Farber Cancer Institute. A similar indicator is currently 

being operationalized by CCNS. The numerator for the Dana-Farber measure is “the proportion 

of cancer decedents with admission to the ICU in the last month of life.” The denominator is “all 

cancer decedents.” Support for its validity comes from literature reviews; focus groups of 

patients, caregivers, and providers; expert panel review; and quantitative analysis of the indicator 

in Medicare claims data. This measure was evaluated in a Medicare-SEER database and, using 

the highest performing decile of this measure, the data implied that in high performing HCSAs 

fewer than 4% of cancer decedents would have more than one ICU admission in the last month 

of life. When a sample of 150 charts at Dana-Farber were abstracted as the gold standard against 

which the sensitivity and specificity of the measure was determine, this measure was determined 

to have a sensitivity of 0.87, specificity of 0.97, and accuracy of 0.095. As a measure of 

variability, investigators determined the 5th/95th percentile ratio (3.28) (range: 2.38-4.67). A 

closely related indicator, ICU days near the end of life is being operationalized by CCNS. 

Neither the measure nor the indicator are currently in use. 

Measure - Late Life Rate of Emergency Care. This measure was developed by Dana-Farber 

Cancer Institute77,146,150 investigators, and a closely related measure is in use by Cancer Care 

Ontario.119 A similar indicator is currently being operationalized by CCNS.146 The numerator 

for the Dana-Farber measure is “the proportion of cancer decedents with more than one 

emergency room visit in the last month of life.” The denominator is “all cancer decedents.” 

Support for its validity comes from literature reviews; focus groups of patients, caregivers, and 

providers; expert panel review; and quantitative analysis of the indicator in Medicare claims 

data. This measure was evaluated in a Medicare-SEER database, and, using the highest 

performing decile of this measure, the data suggested that in high performing HCSAs, fewer than 

4% of cancer decedents would have more than one emergency room visit in the last month of 
 59 

life. When a sample of 150 charts at Dana-Farber were abstracted as the gold standard against 

which the sensitivity and specificity of the measure was determined, this measure was 

determined to have a sensitivity of 0.82, specificity of 0.96, and accuracy of 0.89. As a measure 

of variability, investigators determined the 5th/95th percentile ratio (2.78) (range: 2.04-3.88). The 

Cancer Care Ontario measure is “the rate of cancer patients utilizing an emergency room in the 

last two weeks of life.” The denominator is “all cancer patients dying in a hospital.” Provincial 

rates of this measure varied between about 20% and 25% in 2002. A closely related indicator, 

the frequency of emergency visits, is being operationalized by CCNS. 

Measure - New Chemotherapy Regimen in Last 30 Days of Life.77,150 This measure was 

developed at the Dana-Farber Cancer Institute146 and a closely related indicator is being 

operationalized in ongoing work by CCNS.77 The numerator is “the number of dying cancer 

patients started on a new chemotherapy regimen in the last 30 days of life.” The denominator is 

“all deceased cancer patients.” A high proportion (more than 2%) of cancer patients receiving a 

new chemotherapy regimen in the last 30 days of life indicates poor quality care.120 Support 

comes from literature reviews; focus groups of patients, caregivers, and providers; expert panel 

review; and quantitative analysis of the indicator in Medicare claims data. This measure was 

normed in a Medicare-SEER database and, using the highest performing decile of this measure, 

the data implied that in high performing HCSAs, fewer than 2% of patient would initiate new 

chemotherapy regimens in the last month of life. A sample of 150 charts at Dana-Farber were 

abstracted as the gold standard against which the sensitivity and specificity of the measure was 

determined. This measure was determined to have a sensitivity of 0.83, specificity of 0.94, and 

accuracy of 0.85. As a measure of variability, investigators determined the 5th/95th percentile 

ratio (3.19) (range: 2.03-5.41). CCNS is operationalizing a closely related indicator – the interval 

between new chemotherapy and death. This measure is not in use. 

Measure - Chemotherapy in Last 14 Days of Life.77,150 This measure is derived from the DanaFarber Cancer Institute146 measure set and a closely related indicator is being operationalized in 

ongoing work by CCNS.77,120 The numerator is “the number of dying cancer patients receiving 

chemotherapy in the last 14 days of life.” The denominator is “all deceased cancer patients.” A 

high proportion (more than 10 %) of cancer patients receiving chemotherapy in the last 14 days 

of life indicates poor quality care. Support for this indicator comes from literature reviews; focus 

groups of patients, caregivers, and providers; expert panel review; and quantitative analysis of 

the indicator in Medicare claims data. This indicator was normed in a Medicare-SEER database 

and, using the highest performing decile of this measure, the data suggest that in high performing 

HCSAs, fewer than 10% of patients would receive chemotherapy in the last 14 days of life. 

When a sample of 150 charts at Dana-Farber were abstracted as the gold standard against which 

the sensitivity and specificity of the measure was determined, this measure was determined to 

have a sensitivity of 0.92, specificity of 0.94, and accuracy of 0.92. As a measure of variability, 

investigators determined the 5th/95th percentile ratio (2.24) (range: 1.74-2.97). CCNS is 

evaluating a closely related indicator, using a “short” interval between the last chemotherapy and 

death. This measure is not in use. 

Measure - Admission to Hospice.150 This measure was developed by investigators at the DanaFarber Cancer Institute146 and is currently being operationalized as an indicator by CCNS77,120

and Cancer Care Ontario.77 A closely related measure was also selected for use by the Georgia 
 60 

Cancer Coalition.55 The numerator is “the number of cancer patients not admitted to hospice.” 

The denominator is “all deceased cancer patients.” Numerous observational studies as well as 

several recent methodologically rigorous systematic reviews qualitatively support the 

effectiveness of hospice and palliative care in addressing the full range of quality-of-care 

domains.4,21 Support for the validity of this item comes from literature reviews; focus groups of 

patients, caregivers, and providers; expert panel review; and quantitative analysis of the indicator 

in Medicare claims data. This measure was normed in a Medicare-SEER database and, using the 

highest performing decile of this measure the data suggested that in high performing HCSAs, 

fewer than 45% of patients would die without being admitted to hospice. When a sample of 150 

charts at Dana-Farber were abstracted as the gold standard against which the sensitivity and 

specificity of the measure was determined, this measure was determined to have a sensitivity of 

0.24, specificity of 0.96, and accuracy of 0.88. As a measure of variability, investigators 

determined the 5th/95th percentile ratio (5.00) (range: 3.76-6.89). Several closely related 

indicators are being evaluated and operationalized by CCNS, including enrollment in hospice, 

access to palliative care or palliative physician assessment, and periodic palliative care. The 

Georgia Cancer Coalition has operationalized this item as “the rate of cancer deaths in hospice.” 

Using a SEER-Medicare dataset, the proposed numerator is “the number of adults with cancer 

discharged due to death” and the denominator is “the number of adults with cancer (ICD-10 

codes C00-C97, ICD-9 140-208).”55 No reliability or validity information is available other than 

from the Dana-Farber experience. This item is not known to be in current use as a measure. 

Measure - Late Referral to Hospice.77,150 This measure was developed by investigators at the 

Dana-Farber Cancer146 and is very similar to a measure developed by CCNS146 and the Georgia 

Cancer Coalition.55 The numerator is “the number of dying cancer patients referred to hospice or 

palliative care less than three days before death.” The denominator is “all deceased cancer 

patients.” A high proportion (more than 8 %) of dying cancer patient being referred to hospice 

or palliative care less than three days before death indicates poor quality care.146 Support for the 

validity of this measure comes from literature reviews; focus groups of patients, caregivers, and 

providers; expert panel review; and quantitative analysis of the indicator in Medicare claims 

data. This measure was normed in a Medicare-SEER database, and using the highest performing 

decile of this measure, the data implied that in high performing HCSAs, fewer than 8% of dying 

patients would be admitted to hospice in the last three days of life. When a sample of 150 charts 

at Dana-Farber were abstracted as the gold standard against which the sensitivity and specificity 

of the measure was determined, this measure was determined to have a sensitivity of 0.97, 

specificity of 1.00, and accuracy of 0.97. As a measure of variability, investigators determined 

the 5th/95th percentile ratio (2.39) (range: 1.99-2.95). The Georgia Cancer Coalition has 

operationalized a closely related measure as cancer patients who receive hospice care for at least 

seven days. They propose to report this measure separately for inpatient and outpatient settings 

and to operationalize it using SEER Medicare data; the denominator is “the number of adults 

with cancer (ICD-10 codes C00-C97, ICD-9 140-208).”55 With the exception of the data from 

the Dana-Farber experience, no reliability or validity information is available. This item is not 

known to be in current use as a measure. 

Measure - Site of Death.150 This measure was developed by investigators at the Dana-Farber 

Cancer Institute146 and is also being used as an indicator by CCNS77 and as a measure by Cancer 

Care Ontario.119,120 In the case of the Dana-Farber measure, the numerator is “number of cancer 
 61 

patients who die in the hospital.” The denominator is “all deceased cancer patients.” This item 

focuses on high rates of hospital cancer deaths (greater than 17%)150 as indicating poor quality 

care. Support for this item comes from literature reviews; focus groups of patients, caregivers, 

and providers; expert panel review; and quantitative analysis of the indicator in Medicare claims 

data. The British Gold Standards Framework of dying has embraced a similar metric. This 

measure was normed in a Medicare-SEER database, and using the highest performing decile of 

this measure, the data implied that high performing HCSAs would have fewer than 17% 

inpatient deaths among cancer patients. When a sample of 150 charts at Dana-Farber were 

abstracted as the gold standard against which the sensitivity and specificity of the measure was 

determined, this measure was determined to have a sensitivity of 0.95, specificity of 1.00, and 

accuracy of 0.97. As a measure of variability, investigators determined the 5th/95th percentile 

ratio (2.49) (range: 2.05-3.12). Cancer Care Ontario is currently using this measure to evaluate 

provincial cancer care. About 56% of patients died in the hospital in Ontario, and rates varied 

between approximately 45% and 60% province-wide. Patients who died in the hospital were less 

likely to receive home care in the last six months, palliative care assessments, or physician house 

calls in the last two weeks of life.119 The same item is currently being operationalized as a set of 

comprehensive administrative data-based indicators for CCNS. 

Measure - Safe Dying in Hospice. This measure was developed by investigators at the 

NHPCO.

63 The numerator is “the number of deceased hospice patient’s caregivers with 

documentation in the patient’s medical record who were contacted and surveyed as to the quality 

of care provided by the hospice using the following question: “If you cared for the patient at 

home, did hospice increase your confidence to safely care for your loved one as death 

approached?” The denominator is “all caregivers of hospice patients.” This indicator is based 

on the assumption that caregivers who lack confidence or unsafe environments will aggravate 

dying or may hasten death. The “high risk, high volume, problem prone” home care 

environments are those in which there may be variable, uncertain, or absent caregiver 

competencies. Family caregivers, who may be uneducated in health care delivery, stressed by 

the anticipation of a loved one’s anticipated death, and fatigued by the labor associated with care 

of a physically dependent person, are given significant responsibility for the care of the patient. 

For the 116 agencies that voluntarily reported this information in 2002, the mean and median 

values were 98% for caregivers reporting safe dying experience.54,79

Indicator - Care Consistency With Documented Care Preferences. The numerator for this 

ACOVE measure is “the number of vulnerable elders whose specific treatment preferences were 

followed. The denominator is “all vulnerable elderly patients in any health care setting with 

specific treatment preferences documented in their medical record.”59 The assumption is that the 

patient’s preferences and designated surrogate will be elucidated and documented within the 

patient’s medical record, usually within a specified timeframe that depends on the site within the 

hospital, or there will be documentation as to why surrogate and preferences were not able to be 

obtained. Thus, the documentation in patients’ records of their preferences would provide the 

standard by which care consistency would be evaluated. Both patients and health care providers 

believe that end-of-life discussions are important and should be held while the patient is still able 

to meaningfully and actively participate in decision making. However, studies have shown that 

these conversations are not common.141 Moreover, research assessing the effectiveness of 

written advance directives in the care of seriously ill, hospitalized patients shows that even if the 
 62 

patient has completed an advance directive, and it is accessible in the patient’s chart, physicianpatient communication or decision making about resuscitation was not substantially enhanced. 

Current practice patterns indicate that increasing the frequency of advance directives is unlikely 

to be a substantial element in improving the care of seriously ill patients.151 Recent efforts to 

improve end-of-life care have focused on the application of documented preferences, as there is a 

strong theoretical basis and expert consensus that patient preferences should drive care. Danis 

and colleagues (1991)124 showed a high correlation between documented care preferences and 

life-sustaining treatments. However, similar work has not been performed in the outpatient or 

hospital settings. No trials have studied whether documentation improves the concordance of 

care with values and preferences. This indicator was not tested in ACOVE’s community-based 

sample and is not in current use. 

Potential Indicators and Measurement Gaps 

We reviewed a number of sets of proposed potential indicators, clinical practice guidelines, 

and quality improvement studies that did not meet the standards set for inclusion of relevant 

indicators. Overall, guidelines did not rise to the level of indicators because of the lack of 

specification of various elements required for this review: patient population, denominator, 

conceptual clarity, or data source information. We identified fourteen potential indicators. 

These potential indicators overlapped with those identified in the indicator table and also 

addressed a number of gaps in the indicator literature, including the special population of 

children with cancer.152-155

Although many of the issues in advance care planning in adults are similar to those in 

children, indicators for children may need to differ in some respects (Potential Indicators #1; #7, 

#8, #10, #11, #12). Potential indicators identified for children are derived from the Initiative for 

Pediatric Palliative Care (IPPC) Quality Domains of Children Living with Life-Threatening 

Conditions152 (Potential Indicator #1, #11, #12, #13), the Society of Industrial and Organization 

Psychology (SIOP) Working Committee on Psychosocial Issues in Pediatric Oncology154

(Potential Indicator #7), the American Academy of Pediatrics Committee on Bioethics and 

Committee on Hospital Care155 (Potential Indicator #8), and a literature review on key 

components of quality pediatric end-of-life care153 (Potential Indicator #11). These potential 

indicators overlap with those identified for adults but address children’s needs in light of their 

developmental level, and include assessment of needs, documentation of advance directives, 

support of the child and family, and continuity-of-care plans across provider sites. 

Another potential indicator focused on adults in the ICU135 (Potential Indicator #2). This 

indicator was derived from the American College of Physicians intervention literature and 

specified the components of the clinical identification of end-of-life ICU patients and the 

components of end-of-life care planning. 

A number of potential indicators were identified from palliative care guidelines (Indicator #3, 

#4, #5, #14)69,131,156 and focused on end-of-life palliative care for hospitalized patients. These 

indicators were derived from the National Consensus Project for Quality Palliative Care Clinical 

Practice Guidelines69,131 and the Palliative Care Australia Standards for Palliative Care 

Provision.156 Continuity of patient information across settings was addressed in Potential 

Indicator #6148 and was derived from the intervention literature. 

The use and withdrawal of feeding tubes in institutionalized patients (hospital, nursing home) 

is addressed in Potential Indicator #9.157 
 63 

A final potential indicator was identified from the National Consensus Project69,131 (Indicator 

#14) and focused on the treatment of the body in a culturally sensitive and respectful manner 

after a patient has died. 

Cancer Care Ontario119 is using several measures related to waiting times for chemotherapy 

and radiotherapy that could be adapted for supportive or palliative care with appropriate 

denominator adjustments. 

Measure That Crosses All Domains 

Follow-up. We identified one composite measure that crosses all domains.

Measure - Family Evaluation of Hospice Care (FEHC). The FEHC is a 61-item survey 

administered after the death of a hospice patient to assess family members’ perceptions of care 

provided by the hospice. The survey is based on the After-Death Bereaved Family Interview, 

which was developed based on a systematic review of the measurement literature relative to endof-life care, consensus conferences, and focus groups.158 Please refer to the Evidence Table in 

Appendix F5*. The numerator is “the percentage of respondents reporting opportunity for 

improvement in attention to family needs for support, attending to family needs for information, 

provision of desired physical comfort and emotional support, and mean overall satisfaction.” 

The denominator is “family members of hospice decedents.” Although there are individual 

questions that relate to pain, depression, dyspnea, advance care planning, and other domains, and 

might be used for independent performance measurement, the FEHC is intended to be 

administered as a complete instrument or as modules and is therefore described here as a 

composite measure. The measure is intended for a mixed-hospice population. Hospices submit 

data voluntarily to NHPCO. In the first half of 2004, 352 hospices submitted data on 29,292 

patients: 51% had cancer, 93% were white, 3% were African-American, and 3% were 

Hispanic.158 For pain, 6% of family members reported unmet need (25th and 75th percentiles 

among hospices with more than30 observations, 4% and 8%); for dyspnea, 5% (3% and 7%); 

and for emotional support, 9% (5% and 13%). Some questions related to advance care planning 

as well, such as satisfaction questions about respecting patient wishes and dying on one’s own 

terms, but these values are reported only as a composite with other questions. Validity and 

reliability information are not available for this measure, although it is derived from other 

measures that have undergone extensive psychometric testing. Finally, little variation was found 

among hospices on this survey. However, these results are from early adopters; thus, it is 

possible that more variation would have been found if the survey had been administered among 

all hospices. It continues to be in voluntary use by hospices and NHPCO. 

Summary 

We identified a number of measures and many indicators addressing the domains of pain, 

dyspnea, depression, and advance care planning within supportive and palliative cancer care. 

Many of these measures or indicators were developed and evaluated in populations that included 

cancer patients, although few were developed for quality assessment or improvement of cancer 

 

*

 Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/eolcanqm/eolcanqm.pdf 
 64 

care exclusively, and even fewer have proven their usefulness in application. Existing measures 

and indicators address both processes and outcomes of care. Canadian health care systems have 

been among the early adopters in initiating supportive and palliative cancer measurement, 

although ongoing US measurement efforts include the Georgia Cancer Coalition, and the VHA 

health care system, which are actively implementing measurement systems for cancer care and 

palliative care respectively. Some of these measures are mature enough for widespread 

application in quality improvement, but few have met all the criteria for public reporting and 

accountability. 
 65 

Chapter 4. Discussion 

We identified a remarkable number of measures or indicators that address aspects of pain, 

dyspnea, depression, and advance care planning. However, in all domains, we found very few 

items that had been developed for and tested specifically in a cancer population and none for 

pediatric care. We also found few that were developed for or had been tested across the full 

trajectory of cancer care. Early stage indicators or measures were particularly lacking. With 

regard to settings of care, the fewest measures were found for outpatient care, even though 

patients receive most of their care in that setting. For all symptom domains, the evidence base 

for measurement is certainly strongest in the area of assessment, with relative shortcomings in 

treatment and follow-up. Our systematic review should have also uncovered evidence that these 

measures promote quality assessment and improvement, but there has been little investment in 

implementing these measures. For example, although a few carefully conducted trials have been 

conducted for control of pain in seriously ill or cancer patients, only one has used quality 

indicators as the mechanism for quality improvement,62 and the generalizability of other studies 

to routine practice and to a cancer population159,160 is unclear, although these studies were 

effective in reducing the severity of pain in a population and do include potential indicators. 

An important limitation of our report is that, for the topics of pain and depression, we did not 

search for indicators for conditions other than cancer. The ability to use indicators or measures 

developed generically for these conditions and apply them to patients with cancer rests on the 

assumptions that the physiologic processes of pain and depression are essentially similar across 

disease conditions and that there are no particular measurement challenges in translating them 

from one population to another. This assumption requires certain caveats, such as the short time 

frames available for treatment in palliative care when these indicators or measures are applied to 

late-phase disease, and the fact that generic indicators or measures may not address short-term 

interventions. While these are empirically testable propositions, health care systems instituting 

measures for these conditions should be aware that they may not want to limit themselves to the 

disease-specific indicator or measures we have identified. NQF in particular has reviewed a 

large number of mental health indicators that may be readily applied to palliative conditions. 

Future Research 

Field Testing in Relevant Populations 

The lack of reported experience with these tools may not be surprising, as the measures or 

indicators we identified were largely developed in the last five years, reflecting the newness of 

the field of supportive cancer care.21 In addition, the science of quality improvement in 

particular has suffered from confusion about optimal methodological approaches.161 It is 

encouraging that some of the measures or indicators that we identified have been recently 

implemented or will be implemented in the near future for monitoring and improving population 

care (e.g., in Georgia, Ontario, Nova Scotia, and in the VHA system), which should offer 

opportunities for nurturing a fundamental knowledge base to foster improving the delivery of 

supportive or palliative cancer care. An especially important endeavor to foster progress in 
 66 

measurement would be to apply the better- developed general measure and indicator sets (e.g., 

ACOVE) to supportive or palliative care populations with cancer. 

An important objective of future testing is to obtain more robust information about the 

performance characteristics of the strongest measures. Even given limitations, many of them 

may be useful now for institutional-level quality assessment and improvement activities. 

However, most of these measures lack even basic published information about reliability and 

validity. Before they are deployed at higher levels of the health care system, information about 

their ability to discriminate quality performance and responsiveness to change will be required. 

In addition, testing in a variety of settings and institutions may also be needed. Differences in 

instrumentation, data source, and data collection protocols may significantly affect the 

performance and feasibility of measures, and research on these methodological issues and how 

they affect palliative care evaluation is needed. Furthermore, health care systems must consider 

the cost and feasibility of data collection and must be assured that measures are low risk for 

perverse incentives and that they can provide timely feedback on performance so that 

measurement can contribute to operational decisionmaking. 

The Need for Measures To Address Impaired Self-Report 

None of the indicators address how to evaluate symptoms among individuals with impaired 

self-report, which can be due to either temporary (e.g., delirium) or durable factors (e.g., 

dementia, brain metastasis). There is no doubt that delirium is common among cancer patients, 

although its incidence and prevalence are not well studied. Some recent reports noted a 

prevalence of 18-50% among patients with cancer in general, 162-164 and observed rates are even 

higher in certain treatment settings such as the intensive care unit or in the nursing home, where 

almost 60% of the population is also living with dementia. The clinical needs of these 

populations are undoubtedly profound; a recent description of the clinical profile of cancer 

patients in US nursing homes reveals that, of the 190,976 cases, 51% of cases surviving more 

than one assessment had persistent pain, 25% of all cases were using oxygen, only 45% had 

DNR orders, and only 29% of those who were defined as “terminally ill” received hospice 

services.165 A major effort to expand basic research in this area and ultimately address it through 

quality measurement would benefit many people at the end of life.166,167

Considering the example of pain, health professionals’ estimates of patient’s pain correlate 

poorly with patient self-reports, and discrepancies between patients and physicians in perceptions 

of pain severity are predictive of inadequate management.8

 Without regular screening for pain, 

many patients with significant pain do not have pain documented in the medical record and do 

not receive analgesia.85 Patient vulnerability makes data collection challenging even among 

patients capable of self-report, yet many patients who need palliative care are incapable of selfreport at least part of the time, often at times when they are also at great risk of pain. Research 

that addresses methodological challenges of obtaining patient reports and methods that allow 

monitoring of pain without self-report are an urgent need. All of the pain quality improvement 

interventions identified in this review that succeeded in reducing population pain intensity in 

different settings also included increasing the frequency of pain assessment as a key component 

of the intervention.62,159,160 
 67 

The Lack of Pediatric Measures 

A related and obvious shortcoming we noted is the complete absence of pediatric measures 

or indicators. As in the case of many adults, care assessment in young children is challenged by 

their inability to self report. Cancer represents the second leading cause of death among children 

ages 5-15 in the United States.168 However, profound challenges and serious deficiencies in the 

care of pediatric cancer cases have been noted.169,170 It is insufficient to propose that adult 

measures or indicators could simply be adopted for pediatric care, as the kinds of disease, 

challenges of symptom reporting and intervention, and basic approach to care differ so 

significantly between pediatric and adult oncology care. Many but not all of the adult measures 

have potentially useful analogues in the pediatric population,171 but substantial basic research 

will be needed to translate such efforts into useful pediatric tools. A comprehensive review in 

2003 found only 22 research studies and 6 guideline statements in the area,153 and most of this 

limited literature addresses pain, to the exclusion of other symptoms or experiences.152 In 

addition, a systematic review of measure and indicator sets relevant to pediatrics found no 

measures or indicators for end-of-life care.34,172-175

Defining the “End-of-Life” 

The lack of consensus on definitions of “end-of-life” presents what has been called the 

“denominator” problem. Of central importance is the continuing challenge of establishing a 

consistent and accepted definition of end-of-life that is not confined to that of the “imminently” 

dying. The correct definition of the end-of-life may well depend upon what use is to be made of 

the definition. For the measures reviewed, many of the indicators explicitly define end-of-life 

retrospectively (e.g., last six months of life). While appropriate for evaluating “aggressive 

treatment” in the last days before death, this definition does not provide guidance on important 

clinical services such as pain prevention or the timing of advance care planning. In addition, 

retrospective analyses may introduce a number of biases, since the timing of death is often not 

predictable. Prospective indicators, such as the quality of the discussion about whether the 

patient wants chemotherapy, would account for patient preferences. The ACOVE indicators 

come closer to a prospective method by identifying “vulnerable community-dwelling elders” for 

which the identification and documentation of life-sustaining treatment decisions may be most 

appropriate. Prospective analyses for cancer will require methodological work to define which 

cancer patients (certain types that are usually fatal, those at advanced stage) should be in the 

denominator for measures of the aggressiveness of care. 

The Publication Gap for Quality Measurement 

An important finding of our report was that much of the evidence on the measures that we 

identified was unpublished. Additional information that might inform the evidence base for 

these measures, (i.e., from local quality assessment and improvement efforts) probably exists but 

is likely unpublished and compiled by non-academic organizations. Further, much of the actual 

experience with measures and measurement probably exists in an anecdotal rather than archival 

form. We found that a strategy that consisted of an Internet search coupled with networking 

widely with organizations and individuals in the fields of oncology and supportive or palliative 
 68 

care was relatively effective for identifying such information. Future systematic reviews should 

be aware of this important issue and make an explicit effort to capture this unpublished 

information. Methods for systematically compiling and analyzing this type of data might help to 

improve the evidence base for quality measurement. Methods or other strategies to better 

promote the use of candidate measures in quality improvement work or to otherwise link 

measurement development work to quality improvement projects in cancer, such as the Veterans 

Administration Collaborative,160 might help to provide the information on measure 

responsiveness that is needed. 

Strengthening Quality Measurement of Depression in Cancer 

Given the strength of the literature on measurement in mental health in general and the 

recognized importance of depression in the care of cancer patients, surprisingly few indicators 

have been evaluated for depression in advanced cancer. This lack of indicators is striking 

because indicators for mental health are well represented within available measure sets in 

general. Indicators developed for other settings may be appropriate but have not been tested in 

this population. Furthermore, depression is widely recognized to be an important influence on 

the experience of cancer. Certainly, many of the drugs used to treat depression are relevant 

within the scope of cancer care, especially for earlier stage disease. More fundamental clinical 

research is needed to address late stage disease – such as the optimal approach to screening and 

the usefulness of short-term treatment, which has not, for the most part, been addressed by this 

literature.21 Further research on similarities between measurements in cancer populations and 

those in non-cancer populations might allow elements of this work to be used as supporting 

evidence for particular measures. This effort may be particularly important for areas where there 

is significant non-cancer-related evidence, such as in patients with cognitive impairment. 

Building an Evidence Base To Compare Population Subgroups 

We found insufficient evidence to comment on important differences by population. This 

finding is not surprising, as the tools for examining these differences are limited by the lack of 

basic knowledge.21 However, an important priority for future research is evaluating measures in 

important sub-populations (i.e., by race/ethnicity, age, language, and gender). With regard to 

pain, members of minority groups with cancer have higher rates of inadequate pain assessment 

and treatment than others.86, 87 A recent nonsystematic review of the topic of cancer pain found 

that differences in pain management persist for both assessment and treatment, across multiple 

settings and types of pain.88 Older cancer patients8

 and cancer patients in nursing homes87 are 

also less likely to receive adequate pain treatment. This issue was addressed in some of the 

guidelines we reviewed (e.g., the Oncology Nursing Society Position in Cancer Pain 

Management)82 but has not been operationalized into indicators. It is important to define 

subgroups of cancer by stage of illness and to understand the performance of indicators among 

major subgroups. 

Durable improvement in symptom assessment will require measures that are capable of being 

implemented with other routine metrics of performance in important care settings. For example, 

improvements in pain management often decay over time. In one study to increase daily pain 

assessment, performance on the indicator was 82% after an educational intervention but 

decreased to 59% over a seven-month period.9

 Evaluations of the use of pain assessment in 
 69 

clinical settings have shown good feasibility with minimal burden for staff.9

 However, in one 

study with a high prevalence of unreported pain, training of home health aides to document pain 

scores resulted in a large increase in screening; however, physician documentation increased by 

only 5- to 8%.85 In addition, although most of the indicators or measures that we identified were 

focused on the inpatient setting, much routine cancer care occurs in the ambulatory setting, and 

we found fewer indicators or measures to address outpatient performance. 

Conclusions 

Over time, measures should become available for an increasing number of areas of clinical 

performance in supportive or palliative cancer care. Research is needed to ensure that valid and 

reliable measures are available to address the full spectrum of palliative domains, including the 

most important symptoms that patients face: spiritual well-being and caregiver burdens, 

including bereavement. Measures are needed for the major settings where patients receive care 

(i.e., the outpatient setting) as well as those that follow patients across care venues. More 

patient-centered measures that address the actual processes important to high quality symptom 

control and care planning are needed. Especially in terms of treatment, current indicators do not 

fully reflect progress in adult supportive or palliative care, particularly with regard to pain, 

dyspnea, and depression care.21 However, a thoughtful application of existing measurement will 

ensure that some palliative measures are useful now for most (adult) cancer patients. 

Unlike other tools for quality evaluation where incentives are relatively strong in the private 

sector for improved quality management (e.g., those for prevention or treatment of hypertension 

or diabetes), the adoption of quality metrics for supportive and palliative care is particularly 

dependent on federal endorsement. Almost all elder care is the ultimate responsibility of the 

federal government through Medicare, Medicaid, or the VA systems. The absence of supportive 

or palliative measures within the recent National Healthcare Quality Report is particularly 

notable in that regard.176 Strong federal leadership of both research and the application of 

measurement in clinical care will be needed to make progress in this important area. However, 

the number of measures and indicators that we have identified and the recency of progress in this 

area suggest that now is a promising time to initiate these efforts within our health care system. 
 
 71 

References Cited in the Evidence Report 

1. American Cancer Society. Cancer facts & figures 

2004. available at URL: 

http://www.cancer.org;docroot/STT/stt_0.asp. 

Accessed 7/10/2005. 

2. McGlynn EA. Applying the strategic framework 

board’s model to select national goals and core 

measures for stimulating improved care for cancer. 

2002;RAND Health. 

3. Jemal A, Clegg LX, Ward E, et al. Annual report to 

the nation on the status of cancer, 1975-2001, with a 

special feature regarding survival. Cancer 

2004;101(1):3-27. 

4. Higginson IJ, Rajasekaran M, Davies E, Harding R, 

Gysels M. Improving supportive and palliative care 

for adults with cancer: research evidence manual. 

Accessed 7/10/2005. 

5. Multinational association of supportive care in 

cancer. Accessed 7/10/2005. 

6. WHO definition of palliative care. Accessed 

7/10/2005. 

7. Carr DB, Goudas LC, Balk EM, et al. Evidence 

report on the treatment of pain in cancer patients. J 

Natl Cancer Inst Monogr 2004;(32):23-31. 

8. Cleeland CS, Gonin R, Hatfield AK, et al. Pain and 

its treatment in outpatients with metastatic cancer. 

N Engl J Med 1994;330(9):592-6. 

9. de Rond M, de Wit R, van Dam F. The 

implementation of a Pain Monitoring Programme 

for nurses in daily clinical practice: results of a 

follow-up study in five hospitals. J Adv Nurs 

2001;35(4):590-8. 

10. Carr D, Goudas L, Lawrence D, et al. Management 

of cancer symptoms: pain, depression, and fatigue. 

Rockville, MD: Agency for Healthcare Research 

and Quality. 2002. 

11. Anderson KO, Richman SP, Hurley J, et al. Cancer 

pain management among underserved minority 

outpatients: perceived needs and barriers to optimal 

control. Cancer 2002;94(8):2295-304. 

12. Spector N, Klein D. Chronic critically ill dyspneic 

patients: mechanisms and clinical measurement. 

AACN Clinical Issues: Advanced Practice in Acute 

and Critical Care 2001;12(2):220-33. 

13. Silvestri GA. Palliation for the dying patient with 

lung cancer. Respir Care 2000;45(12):1490-6; 

discussion 1497-500. 

14. Dudgeon DJ, Lertzman M. Dyspnea in the 

advanced cancer patient. J Pain Symptom Manage 

1998;16(4):212-9. Comment in: J Pain Symptom 

Manage. 1999 Nov;18(5):313-5. PMID: 10584452. 

15. Sorenson HM. Dyspnea assessment. Respir Care 

2000;45(11):1331-8;discussion 1338-41. 

16. O’Driscoll M, Corner J, Bailey C. The experience 

of breathlessness in lung cancer. Eur J Cancer Care 

(Engl) 1999;8(1):37-43. 

17. Dudgeon DJ, Kristjanson L, Sloan JA, et al. 

Dyspnea in cancer patients: prevalence and 

associated factors. Journal of Pain & Symptom 

Management 2001;21(2):95-102. 

18. Ahmedzai SH, Muers MFe. Supportive care in 

respiratory disease. Oxford University Press; 2005. 

19. Hollen PJ, Gralla RJ, Kris MG, et al. Normative 

data and trends in quality of life from the Lung 

Cancer Symptom Scale (LCSS). Support Care 

Cancer 1999;7(3):140-8. 

20. Mancini I, Body JJ. Assessment of dyspnea in 

advanced cancer patients. Support Care Cancer 

1999;7(4):229-32. 

21. Lorenz K, Lynn J, Morton SC, et al. End-of-life 

care and outcomes. Evid Rep Technol Assess 

(Summ) 2004;(110):1-6. 

22. Polsky D, Doshi JA, Marcus S, et al. Long-term risk 

for depressive symptoms after a medical diagnosis. 

Arch Intern Med 2005;165(11):1260-6. 

23. Aukst-Margetic B, Jakovljevic M, Margetic B, et al. 

Religiosity, depression and pain in patients with 

breast cancer. Gen Hosp Psychiatry 

2005;27(4):250-5. 

24. Koenig HG, George LK, Peterson BL. Religiosity 

and remission of depression in medically ill older 

patients. Am J Psychiatry 1998;155(4):536-42. 

25. Fehring RJ, Miller JF, Shaw C. Spiritual well-being, 

religiosity, hope, depression, and other mood states 

in elderly people coping with cancer. Oncol Nurs 

Forum 1997;24(4):663-71. 

26. Johnson KS, Elbert-Avila KI, Tulsky JA. The 

influence of spiritual beliefs and practices on the 

treatment preferences of African Americans: a 

review of the literature. J Am Geriatr Soc 2005;53 

(4):711-9. 
 72 

27. Teno JM, Lynn J. Putting advance-care planning 

into action. J Clin Ethics 1996;7(3):205-13. 

28. Fisher ES, Wennberg DE, Stukel TA, et al. The 

implications of regional variations in Medicare 

spending. Part 1: the content, quality, and 

accessibility of care. Ann Intern Med 

2003;138(4):273-87. 

29. Fisher ES, Wennberg DE, Stukel TA, et al. The 

implications of regional variations in Medicare 

spending. Part 2: health outcomes and satisfaction 

with care. Ann Intern Med 2003;138(4):288-98. 

30. Wennberg JE, Fisher ES, Stukel TA, et al. Use of 

hospitals, physician visits, and hospice care during 

last six months of life among cohorts loyal to highly 

respected hospitals in the United States. BMJ 

2004;328(7440):607. 

31. Steinhauser KE, Christakis NA, Clipp EC, et al. 

Factors considered important at the end of life by 

patients, family, physicians, and other care 

providers. JAMA 2000;284(19):2476-82. 

32. Donabedian A. The criteria and standards for 

quality. Chicago, IL: Health Administration Press. 

1981. 

33. Merrian-Webster online. available at URL: 

www.mirriam-webster.com. Accessed 7/13/2005. 

34. World Health Organization (WHO). Cancer pain 

relief and palliative care in children. 1998. 

35. Chodosh J, Buckwalter JG, Blazer DG, et al. How 

the question is asked makes a difference in the 

assessment of depressive symptoms in older 

persons. Am J Geriatr Psychiatry 2004;12(1):75-83. 

36. Lorenz KA, Shapiro MF, Asch SM, et al. 

Associations of symptoms and health-related 

quality of life: findings from a national study of 

persons with HIV infection. Ann Intern Med 

2001;134(9 Pt 2):854-60. 

37. Sackett DL. The rational clinical examination. A 

primer on the precision and accuracy of the clinical 

examination. JAMA 1992;267(19):2638-44. 

38. Walter LC, Davidowitz NP, Heineken PA, et al. 

Pitfalls of converting practice guidelines into 

quality measures: lessons learned from a VA 

performance measure. JAMA 2004;291(20):2466-

70. 

39. Dubey S, Brown RL, Esmond SL, et al. Patient 

preferences in choosing chemotherapy regimens for 

advanced non-small cell lung cancer. J Support 

Oncol 2005;3(2):149-54. 

40. Ferrell BA, Ferrell BR, Rivera L. Pain in 

cognitively impaired nursing home patients. J Pain 

Symptom Manage 1995;10(8):591-8. 

41. Emanuel EJ, Fairclough DL, Emanuel LL. Attitudes 

and desires related to euthanasia and physicianassisted suicide among terminally ill patients and 

their caregivers. JAMA 2000;284(19):2460-8. 

42. Ditto PH, Smucker WD, Danks JH, et al. Stability 

of older adults’ preferences for life-sustaining 

medical treatment. Health Psychol 2003;22(6):605-

15. 

43. Danis M, Garrett J, Harris R, et al. Stability of 

choices about life-sustaining treatments. Ann Intern 

Med 1994;120(7):567-73. 

44. Dept. of Clinical Quality Improvement. Clinical 

practice guidelines directory. Chicago, IL: 

American Medical Association. 1999. 

45. U.S. Preventive Services Task Force. Guide to 

clinical and preventive services. Alexandria, VA: 

International Medical Pub. 1996. 

46. Committee on Performance Measurement. HEDIS 

2005: Health plan employer data & information set. 

Washington, DC: National Committee for Quality 

Assurance. 2005. 

47. 1998 The medical outcomes and guidelines 

sourcebook. New York, NY: Faulkner & Gray; 

1997. 

48. Clinical process and outcomes measurement 

directory, 1999 edition. Chicago, IL: American 

Medical Association; 1999. 

49. Behavioral outcomes and guidelines sourcebook, 

2000 ed. New York, NY: Faulkner & Gray; 1999. 

50. Appraisal of guidelines research & evaluation. 

available at URL: www.agreecollaboration.org. 

Accessed 6/30/2005. 

51. Shaneyfelt TM, Mayo-Smith MF, Rothwangl J. Are 

guidelines following guidelines? The 

methodological quality of clinical practice 

guidelines in the peer-reviewed medical literature. 

JAMA 1999;281(20):1900-5. 

52. McGlynn EA, Malin J. Selecting national goals and 

core measures of cancer care quality (Background 

paper #2). RAND 2002. 

53. Connor SR, Teno J, Spence C, et al. Family 

evaluation of hospice care: results from voluntary 

submission of data via website. J Pain Symptom 

Manage 2005;30(1):9-17. 
 73 

54. Ryndes T, Connor S, Cody C, Merriman M, Bruno 

S, Fine P, et al. Report on the alpha and beta pilots 

of end result outcome measures constructed by the 

outcomes forum. 

55. National Cancer Policy Board (NCPB). Assessing 

the quality of cancer care: an approach to 

measurement in Georgia.: The National Academies 

Press. 2005. 

56. Greenberg A, Angus H, Sullivan T, et al. 

Development of a set of strategy-based system-level 

cancer care performance indicators in Ontario, 

Canada. Int J Qual Health Care 2005. 

57. Cancer Care Ontario. Cancer system quality index: 

satisfaction with pain management. Ontario, 

Canada: Cancer Care Ontario. 2005. 

58. Chodosh J, Solomon DH, Roth CP, et al. The 

quality of medical care provided to vulnerable older 

patients with chronic pain. J Am Geriatr Soc 

2004;52(5):756-61. 

59. Wenger NS, Rosenfeld K. Quality indicators for 

end-of-life care in vulnerable elders. Ann Intern 

Med 2001;135(8 Pt 2):677-85. 

60. Saliba D, Solomon D, Rubenstein L, et al. 

Feasibility of quality indicators for the management 

of geriatric syndromes in nursing home residents. J 

Am Med Dir Assoc 2004;5(5):310-9. 

61. Erdek MA, Pronovost PJ. Improving assessment 

and treatment of pain in the critically ill. Int J Qual 

Health Care 2004;16(1):59-64. 

62. Pronovost PJ , Berenholtz SM. A practical guide to 

measuring performance in the intensive care unit.: 

VHA. 2002. 

63. National Hospice and Palliative Care Organization 

(NHPCO). Instructions for use of the six measures. 

64. In: Asch SM , Kerr EA, Hamilton EG, Reifel JL, 

McGlynn EA. Quality of care for oncologic 

condition and HIV: a review of the literature and 

quality indicators. RAND; 2000. 

65. McGlynn EA, Asch SM, Adams J, et al. The quality 

of health care delivered to adults in the United 

States. N Engl J Med 2003;348(26):2635-45. 

66. Asch SM, McGlynn EA, Hogan MM, et al. 

Comparison of quality of care for patients in the 

Veterans Health Administration and patients in a 

national sample. Ann Intern Med 

2004;141(12):938-45. 

67. Maxwell T, Cuny J. The UHC palliative care 

benchmarking project. University Health System 

Consortium 2004;1-35. 

68. VHA. Transformation of the Intensive Care Unit. 

Palliative Care Bundle: Care and Communication 

Measurement Guide. 2005. 

69. Clinical practice guidelines for quality palliative 

care.: National Consensus Project for Quality 

Palliative Care. 2004. 

70. Kress JP, Pohlman AS, O’Connor MF, et al. Daily 

interruption of sedative infusions in critically ill 

patients undergoing mechanical ventilation. N Engl 

J Med 2000;342(20):1471-7. 

71. Neuss MN, Desch CE, McNiff KK, et al. A process 

for measuring the quality of cancer care: the Quality 

Oncology Practice Initiative. J Clin Oncol 

2005;23(25):6233-9. 

72. Wenger NS, Solomon DH, Roth CP, et al. The 

quality of medical care provided to vulnerable 

community-dwelling older patients. Ann Intern Med 

2003;139(9):740-7. 

73. McNicol E, Horowicz-Mehler N, Fisk RA, et al. 

Management of opioid side effects in cancer-related 

and chronic noncancer pain: a systematic review. J 

Pain 2003;4(5):231-56. 

74. Gordon DB, Dahl JL, Miaskowski C, et al. 

American pain society recommendations for 

improving the quality of acute and cancer pain 

management: American Pain Society Quality of 

Care Task Force. Arch Intern Med 

2005;165(14):1574-80. 

75. Wu JS, Wong R, Johnston M, Bezjak A, Whelan T, 

Laetsch N , et al. Radiotherapy fractionation for the 

palliation of uncomplicated painful bone 

metastases.: Program in Evidence-Based Care. 

2003. 

76. Sze WM, Shelley MD, Held I, et al. Palliation of 

metastatic bone pain: single fraction versus 

multifraction radiotherapy--a systematic review of 

randomised trials. Clin Oncol (R Coll Radiol) 

2003;15(6):345-52. 

77. Grunfeld E, et al. Nova Scotia and Ontario quality 

end of life indicators project. 2005. 

78. Roque M, Martinez MJ, Alonso P, et al. 

Radioisotopes for metastatic bone pain. Cochrane 

Database Syst Rev 2003;(4):CD003347. 

79. Connor SR, Tecca M, LundPerson J, et al. 

Measuring hospice care: the National Hospice and 

Palliative Care Organization National Hospice Data 

Set. J Pain Symptom Manage 2004;28( 4):316-28. 

80. National Comprehensive Cancer Network (NCCN). 

Cancer pain. Clinical Practice Guidelines in 

Oncology 2004;(1) 
 74 

81. Maasilta P. Palliative treatment of a patient with 

lung cancer. EBM Guidelines. Duodecim Medical 

Publications 2004. 

82. Oncology Nursing Society. Cancer pain 

management. 2004. 

83. Quality improvement guidelines for the treatment of 

acute pain and cancer pain. American Pain Society 

Quality of Care Committee. JAMA 

1995;274(23):1874-80. 

84. NIH State-of-the-Science Statement on symptom 

management in cancer: pain, depression, and 

fatigue. NIH Consens State Sci Statements 

2002;19(4):1-29. 

85. Rhodes DJ, Koshy RC, Waterfield WC, et al. 

Feasibility of quantitative pain assessment in 

outpatient oncology practice. J Clin Oncol 

2001;19(2):501-8. 

86. Cleeland CS, Gonin R, Baez L, et al. Pain and 

treatment of pain in minority patients with cancer. 

The Eastern Cooperative Oncology Group Minority 

Outpatient Pain Study. Ann Intern Med 

1997;127(9):813-6. 

87. Bernabei R, Gambassi G, Lapane K, et al. 

Management of pain in elderly patients with cancer. 

SAGE Study Group. Systematic Assessment of 

Geriatric Drug Use via Epidemiology. JAMA 

1998;279(23):1877-82. 

88. Green CR, Anderson KO, Baker TA, et al. The 

unequal burden of pain: confronting racial and 

ethnic disparities in pain. Pain Med 2003; 4(3):277-

94. 

89. Stolee P, Hillier LM, Esbaugh J, et al. Instruments 

for the assessment of pain in older persons with 

cognitive impairment. J Am Geriatr Soc 

2005;53(2):319-26. 

90. Duhn LJ, Medves JM. A systematic integrative 

review of infant pain assessment tools. Adv 

Neonatal Care 2004;4(3):126-40. 

91. Morrison RS, Magaziner J, Gilbert M, et al. 

Relationship between pain and opioid analgesics on 

the development of delirium following hip fracture. 

J Gerontol A Biol Sci Med Sci 2003;58(1):76-81. 

92. Lloyd-Williams M, Dennis M, Taylor F. A 

prospective study to determine the association 

between physical symptoms and depression in 

patients with advanced cancer. Palliat Med 

2004;18(6):558-63. 

93. Gordon DB, Pellino TA, Miaskowski C, et al. A 10-

year review of quality improvement monitoring in 

pain management: recommendations for 

standardized outcome measures. Pain Manag Nurs 

2002;3(4):116-30. 

94. Allard P, Maunsell E, Labbe J, et al. Educational 

interventions to improve cancer pain control: a 

systematic review. J Palliat Med 2001;4(2):191-

203. 

95. Marinangeli F, Ciccozzi A, Leonardis M, et al. Use 

of strong opioids in advanced cancer pain: a 

randomized trial. J Pain Symptom Manage 

2004;27(5):409-16. 

96. Jacox A, Carr DB, Payne R, et al. Management of 

cancer pain. Clinical practice guideline 9. 

Rockville, MD: US Department of Health and 

Human Services, Public Health Service, AHRQ. 

1994. 

97. American Geriatrics Society Clinical Practice C. 

Management of cancer pain in older patients. 

Journal of the American Geriatrics Society 

1997;45(10):1273-1276 URL http 

//www.blackwellscience.com/~cgilib/bsinc.bin?Journal=geriatrics. 

98. Foster RL, Varni JW. Measuring the quality of 

children’s postoperative pain management: initial 

validation of the Child/Parent Total Quality Pain 

Management (TQPM) instruments. Journal of Pain 

and Symptom Management 2002;23(3):201-10. 

99. Taddio A, Soin HK, Schuh S, et al. Liposomal 

lidocaine to improve procedural success rates and 

reduce procedural pain among children: a 

randomized controlled trial. CMAJ 

2005;172(13):1691-5. 

100. Faber-Langendoen K. The clinical management of 

dying patients receiving mechanical ventilation. A 

survey of physician practice. Chest 

1994;106(3):880-8. 

101. Wenger NS, Belamy PE, Phillips RS, et al. 

Variation in withdrawing the ventilator from dying 

patients. J Gen Intern Med 1997;12(Suppl 1):105. 

102. Brody H, Campbell ML, Faber-Langendoen K, et 

al. Withdrawing intensive life-sustaining treatment -

- recommendations for compassionate clinical 

management. N Engl J Med 1997;336(9):652-7. 

103. Campbell ML. Forgoing life-sustaining therapy. 

Aliso Viejo, CA: AACN Critical Care Publication. 

1998. 
 75 

104. Anonymous. Withholding and withdrawing lifesustaining therapy: this official statement of the 

American Thoracic Society was adopted by the 

ATS Board of Directors, March 1991. Am Rev 

Respir Dis 1991;144(3 Pt 1):726-31. 

105. Faber-Langendoen K, Lanken PN . Dying patients 

in the intensive care unit: forgoing treatment, 

maintaining care. Ann Intern Med 

2000;133(11):886-93. 

106. Malin JL, Asch SM, Kerr EA, et al. Evaluating the 

quality of cancer care: development of cancer 

quality indicators for a global quality assessment 

tool. Cancer 2000;88(3):701-7. 

107. Golomb B. Chronic obstructive pulmonary disease. 

In: Kerr EA, Asch SM, Hamilton EG, McGlynn 

EA. Quality of care for cardiopulmonary 

conditions: a review of the literature and quality 

indicators. RAND; 2000. p. 91-109. 

108. Booth S, Wade R, Johnson M, et al. The use of 

oxygen in the palliation of breathlessness. A report 

of the expert working group of the Scientific 

Committee of the Association of Palliative 

Medicine. Respir Med 2004;98(1):66-77. 

109. Dyspnea. Mechanisms, assessment, and 

management: a consensus statement. American 

Thoracic Society. Am J Respir Crit Care Med 

1999;159( 1):321-40. 

110. Kvale PA, Simoff M, Prakash UB. Lung cancer. 

Palliative care. Chest 2003;123(1 Suppl):284S311S. 

111. Ferguson GT, Enright PL, Buist AS, et al. Office 

spirometry for lung health assessment in adults: a 

consensus statement from the National Lung Health 

Education Program. Chest 2000;117(4):1146-61. 

112. Jennings AL, Davies AN, Higgins JP, et al. Opioids 

for the palliation of breathlessness in terminal 

illness. Cochrane Database Syst Rev 

2001;(4):CD002066. 

113. Pfister DG, Johnson DH, Azzoli CG, et al. 

American Society of Clinical Oncology treatment of 

unresectable non-small-cell lung cancer guideline: 

update 2003. J Clin Oncol 2004;22(2):330-53. 

114. Socinski MA, Morris DE, Masters GA, et al. 

Chemotherapeutic management of stage IV nonsmall cell lung cancer. Chest 2003;123(1 

Suppl):226S-243S. 

115. Lacasse Y, Brosseau L, Milne S, et al. Pulmonary 

rehabilitation for chronic obstructive pulmonary 

disease. Cochrane Database Syst Rev 

2002;(3):CD003793 . 

116. Jennings A-L, Davies AN, Higgins JPT, et al. A 

systematic review of the use of opioids in the 

management of dyspnea. Thorax 2002;(57):939-44. 

117. Nakajima GA, Wenger NS. Quality indicators for 

the care of depression in vulnerable elders. RAND. 

118. Chow TW, MacLean CH. Quality indicators for 

dementia in vulnerable community-dwelling and 

hospitalized elders. Ann Intern Med 2001;135(8 Pt 

2):668-76. 

119. Cancer Care Ontario. Cancer system quality index: 

patterns of end-of-life care. Ontario, Canada: 

Cancer Care Ontario. 2005. 

120. Cancer Care Ontario. Cancer system quality index: 

hospital resources used during each phase of cancer 

care. Ontario, Canada: Cancer Care Ontario. 2005. 

121. Hanson LC, Tulsky JA, Danis M . Can clinical 

interventions change care at the end of life? Ann 

Intern Med 1997;126(5):381-8. 

122. Fisher GS, Tulsky JA, Rose MR, et al. Patient 

knowledge and physician predictions of treatment 

preferences after discussion of advance directives . J 

Gen Intern Med 1998;13:447-54. 

123. White DB, Luce JM. Palliative care in the intensive 

care unit: barriers, advances, and unmet needs. Crit 

Care Clin 2004;20(3):329-43, vii. 

124. Wenger NS, Phillips RS, Teno JM, et al. Physician 

understanding of patient resuscitation preferences: 

insights and clinical implications. J Am Geriatr Soc 

2000;48(5 Suppl):S44-51. 

125. Teno JM, Hakim RB, Knaus WA, et al. A 

prospective study of advance directives for lifesustaining care. N Engl J Med 1991;324:882-8. 

126. Danis M, Southerland LI, Garrett JM, et al. A 

prospective study of advance directives for lifesustaining care. N Engl J Med 1991;324(13):882-8. 

127. Blasi ZV, Hurley AC, Volicer L. End-of-life care in 

dementia: a review of problems, prospects, and 

solutions in practice. Journal of the American 

Medical Directors Association 2002;3(2):57-65. 

128. Sulmasy DP, Dwyer M, Marx E. Do the ward notes 

reflect the quality of end-of-life care? J Med Ethics 

1996;22(6):344-8. 

129. Cherry DL, Vickrey BG, Schwankovsky L, et al. 

Interventions to improve quality of care: the Kaiser 

Permanente-alzheimer’s Association Dementia Care 

Project. Am J Manag Care 2004;10(8):553-60. 
 76 

130. Prendergast TJ, Luce JM. Increasing incidence of 

withholding and withdrawal of life support from the 

critically ill. Am J Respir Crit Care Med 

1997;155(1):15-20. 

131. National Comprehensive Cancer Network (NCCN). 

Palliative care. Clinical Practice Guidelines in 

Oncology 2004;(1). 

132. Cook DJ, Guyatt G, Rocker G, et al. 

Cardiopulmonary resuscitation directives on 

admission to intensive-care unit: an international 

observational study. Lancet 2001;358(9297):1941-

5. 

133. Teno JM, Fisher E, Hamel MB, et al. Decisionmaking and outcomes of prolonged ICU stays in 

seriously ill patients. J Am Geriatr Soc 2000; 48(5 

Suppl):S70-4. 

134. Treece PD, Engelberg RA, Crowley L, et al. 

Evaluation of a standardized order form for the 

withdrawal of life support in the intensive care unit. 

Crit Care Med 2004;32(5):1141-8. 

135. Lilly CM, De Meo DL, Sonna LA, et al. An 

intensive communication intervention for the 

critically ill. Am J Med 2000;109(6):469-75. 

136. Azoulay E, Pochard F, Chevret S, et al. Impact of a 

family information leaflet on effectiveness of 

information provided to family members of 

intensive care unit patients: a multicenter, 

prospective, randomized, controlled trial . Am J 

Respir Crit Care Med 2002;165(4):438-42. 

137. Volicer L. Management of severe Alzheimer’s 

disease and end-of-life issues. Clin Geriatr Med 

2001;17(2):377-91. 

138. Sulmasy DP, Sood JR, Ury WA. The quality of care 

plans for patients with do-not-resuscitate orders. 

Arch Intern Med 2004;164(14):1573-8. 

139. Prendergast TJ. Advance care planning: pitfalls, 

progress, promise. Crit Care Med 2001;29(2 

Suppl):N34-9. 

140. Bradley EH, Hallemeier AG, Fried TR, et al. 

Documentation of discussions about prognosis with 

terminally ill patients. Am J Med 2001;111(3):218-

23. 

141. Song MK. Effects of end-of-life discussions on 

patients’ affective outcomes. Nurs Outlook 

2004;52( 3):118-25. 

142. Virmani J, Schneiderman LJ, Kaplan RM. 

Relationship of advance directives to physicianpatient communication. Arch Intern Med 

1994;154(8):909-13. 

143. Goodman MD, Tarnoff M, Slotman GJ. Effect of 

advance directives on the management of elderly 

critically ill patients. Crit Care Med 

1998;26(4):701-4. 

144. Gregory JJ, Dunn D. Implementation of the Patient 

Self-Determination Act in a community hospital. N 

J Med 1995;92 (7):438-42. 

145. Morrison RS, Olson E, Mertz KR, et al. The 

inaccessibility of advance directives on transfer 

from ambulatory to acute care settings. JAMA 

1995;274(6):478-82. 

146. Earle CC, Park ER, Lai B, et al. Identifying 

potential indicators of the quality of end-of-life 

cancer care from administrative data. J Clin Oncol 

2003;21(6):1133-38. 

147. Hawryluck LA, Harvey WR, Lemieux-Charles L, et 

al. Consensus guidelines on analgesia and sedation 

in dying intensive care unit patients. BMC Med 

Ethics 2002;3(1):E3. 

148. Clarke EB, Curtis JR, Luce JM, et al. Quality 

indicators for end-of-life care in the intensive care 

unit. Crit Care Med 2003;31(9):2255-62. 

149. A controlled trial to improve care for seriously ill 

hospitalized patients. The study to understand 

prognoses and preferences for outcomes and risks 

of treatments (SUPPORT). The SUPPORT 

Principal Investigators. JAMA 1995;274(20):1591-

8. 

150. Earle CC, Neville BA, Landrum MB, et al. 

Evaluating claims-based indicators of the intensity 

of end-of-life cancer care. 

151. Teno J, Lynn J, Wenger N, et al. Advance directives 

for seriously ill hospitalized patients: effectiveness 

with the patient self-determination act and the 

SUPPORT intervention. SUPPORT Investigators. 

Study to Understand Prognoses and Preferences for 

Outcomes and Risks of Treatment. J Am Geriatr 

Soc 1997;45(4):500-7. Comment in: J Am Geriatr 

Soc. 1997 Apr;45(4):519-20. PMID: 9100724. 

152. Dokken DL, Heller KS, Levetown M, et al. Quality 

domains, goals and indicators for family-centered 

care of children living with life-threatening 

conditions. Education Development Center, Inc. 

2002. 

153. Widger KA, Wilkins K. What are the key 

components of quality perinatal and pediatric endof-life care? A literature review. J Palliat Care 

2004;20(2):105-12. 
 77 

154. Masera G, Spinetta JJ, Jankovic M, et al. Guidelines 

for assistance to terminally ill children with cancer: 

a report of the SIOP Working Committee on 

psychosocial issues in pediatric oncology. Med 

Pediatr Oncol 1999;32(1):44-8. 

155. American Academy of Pediatrics. Committee on 

Bioethics and Committee on Hospital Care. 

Palliative care for children. Pediatrics 2000;106(2 

Pt 1):351-7. 

156. Standards and Quality Committee. Standards for 

palliative care provision. Australia. 1999. 

157. Mitchell SL, Berkowitz RE, Lawson FME, et al. A 

cross-national survey of tube-feeding decisions in 

cognitively impaired older persons. Journal of the 

American Geriatrics Society 2000;48(4):391-7. 

158. Connor SR, Teno J, Spence C, Smith N. Family 

evaluation of hospice care: results from voluntary 

submission of data over website. 

159. Fortner BV, Okon TA, Ashley J, et al. The Zero 

Acceptance of Pain (ZAP) Quality Improvement 

Project: evaluation of pain severity, pain 

interference, global quality of life, and pain-related 

costs. J Pain Symptom Manage 2003;25(4):334-43. 

160. Cleeland CS, Reyes-Gibby CC, Schall M, et al. 

Rapid improvement in pain management: the 

Veterans Health Administration and the institute for 

healthcare improvement collaborative. Clin J Pain 

2003;19(5):298-305. 

161. Mittman BS. Creating the evidence base for quality 

improvement collaboratives. Ann Intern Med 

2004;140(11):897-901. 

162. Ljubisavljevic V, Kelly B. Risk factors for 

development of delirium among oncology patients. 

Gen Hosp Psychiatry 2003;25(5):345-52. 

163. Gagnon P, Allard P, Masse B, et al. Delirium in 

terminal cancer: a prospective study using daily 

screening, early diagnosis, and continuous 

monitoring. J Pain Symptom Manage 

2000;19(6):412-26. 

164. Lawlor PG, Gagnon B, Mancini IL, et al. 

Occurrence, causes, and outcome of delirium in 

patients with advanced cancer: a prospective study. 

Arch Intern Med 2000;160(6):786-94. 

165. Johnson VM, Teno JM, Bourbonniere M, et al. 

Palliative care needs of cancer patients in U.S. 

nursing homes. J Palliat Med 2005;8(2):273-9. 

166. Taylor LJ, Harris J, Epps CD, et al. Psychometric 

evaluation of selected pain intensity scales for use 

with cognitively impaired and cognitively intact 

older adults. Rehabil Nurs 2005;30(2):55-61. 

167. Scherder E, Oosterman J, Swaab D, et al. Recent 

developments in pain in dementia. BMJ 2005;330( 

7489):461-4. 

168. National Center for Health Statistics. Deaths, 

percentage of total deaths, and death rates for the 10 

leading causes of death in selected age groups by 

race and sex: United States, 2002. available at URL: 

http://www.cdc.gov/nchs/fastats/pdf/nvsr53_17t1.p

df. Accessed 7/21/2005. 

169. Wolfe J, Grier HE, Klar N, et al. Symptoms and 

suffering at the end of life in children with cancer. 

N Engl J Med 2000;342(5):326-33. 

170. Klopfenstein KJ, Hutchison C, Clark C, et al. 

Variables influencing end-of-life care in children 

and adolescents with cancer. J Pediatr Hematol 

Oncol 2001;23(8):481-6. 

171. Levetown M, Dokken D, Heller KS, et al. A 

Pediatric Palliative Care Institutional SelfAssessment Tool (ISAT). Education Development 

Center, Inc 2002. 

172. Beal AC, Co JP, Dougherty D, et al. Quality 

measures for children’s health care. Pediatrics 

2004;113(1 Pt 2):199-209. 

173. Hooke C, Hellsten MB, Stutzer C, et al. Pain 

management for the child with cancer in end-of-life 

care: APON position paper. Journal of Pediatric 

Oncology Nursing 2002;19(2):43-7. 

174. Treadwell MJ, Franck LS, Vichinsky E. Using 

quality improvement strategies to enhance pediatric 

pain assessment. Int J Qual Health Care 

2002;14(1):39-47. 

175. American Pain Society (APS). Guideline for the 

management of cancer pain in adults and children. 

2005. 

176. Agency for Healthcare Research and Quality. List 

of Measures: National Healthcare Quality Report 

2004. available at URL: 

http://www.qualitytools.ahrq.gov/qualityreport/bro

wse/browse.aspx. Accessed 7/21/2005. 
APPENDIXES 

to 

“Cancer Care Quality Measures: 

Symptoms and End of Life Care” 

Prepared by the Southern California Evidence-based Practice Center 

(Contract #290-02-003) 
A-1

Appendix A. Technical Expert Panel Members and 

Peer Reviewers 

Table A1. Technical Expert Panel members 

Name Institution

Steve Asch, M.D., M.P.H. VAGLHS 

Marilyn Bookbinder, R.N., Ph.D. Beth Israel Medical Center 

June Dahl, Ph.D. University of Wisconsin-Madison 

Perry Fine, M.D. University of Utah 

Laura Hanson, M.D., M.P.H. University of North Carolina at Chapel Hill 

Jimmie Holland, M.D. Memorial Sloan Kettering Cancer Center 

Joan Teno, M.D., M.S. Brown Medical School 

Jean Kutner, M.D., M.S.P.H. University of Colorado Health Sciences Center 

Charles Cleeland, Ph.D. UTMD Anderson Cancer Center 

Jennifer Malin, M.D. UCLA School of Medicine, RAND 

Molla Donaldson, Dr.P.H., M.S. National Cancer Institute (Fed Reviewer) 

Patricia Ganz, M.D. UCLA Schools of Medicine and Public Health 

Rodger Winn, Ph.D. National Quality Forum 

Table A2. Peer Reviewers 

Name Institution

Adalsteinn Brown, A.B., D.Phil. University of Toronto 

Ellen Stovall National Coalition for Cancer Survivorship 

(NCCS) 

Eva Grunfeld, M.D., D.Phil. Cancer Care Nova Scotia 

Joanne Wolfe, M.D. Harvard University-Pediatrics 

Judi L. Person, M.P.H. National Hospice and Palliative Care 

Organization (NHPCO) 

Lindsey Bramwell, M.P.H., R.N., CDR, U.S.P.H.S. CMS (Fed Reviewer) 

Anna Lythgoe, R.N., M.S.N. VHACO, Office of Quality and Performance 

Phil Madvig, M.D. Kaiser Permanente 
B-1

Appendix B. TEP Worksheet Indicator Worksheet—Cancer Care Quality Measures: Symptoms and End of Life Care Established Indicator Worksheet TEP Member 

Indicator statement Numerator Denominator Source 
C-1 

Appendix C. Literature Search Strategies 

SEARCH METHODOLOGIES – END OF LIFE CANCER MEASUREMENT 

 CINAHL 

DATABASE SEARCHED & TIME PERIOD COVERED: 

CINAHL – 1995-2005 

OTHER LIMITERS: 

 ENGLISH 

 HUMAN 

SEARCH STRATEGY #1 (PAIN NOT PALLIATIVE) 

(neoplasm$ or cancer$ or tumour$ or tumor$ or carcinoma$ or adenocarcinoma$ or 

malignan$).mp. [mp=title, cinahl subject headings, abstract, instrumentation] 

AND 

(guideline$ or audit$ or outcome$ or peer review$ or professional review$ or evaluat$ or 

benchmark$ or utilization review$ or consensus or perform$ or quality).mp. 

AND 

((quality or perform$) and (measur$ or indicator$ or assess$)).mp. 

AND 

pain.mp. 

NUMBER OF ITEMS RETRIEVED : 414 

 

------------------------------------------------------------------------------------------- 

 

SEARCH STRATEGY #2 (DEPRESSION): 

(neoplasm$ or cancer$ or tumour$ or tumor$ or carcinoma$ or adenocarcinoma$ or 

malignan$).mp. 

AND 

(guideline$ or audit$ or outcome$ or peer review$ or professional review$ or evaluat$ or 

benchmark$ or utilization review$ or consensus or perform$ or quality).mp. 

 

AND 

((quality or perform$) and (measur$ or indicator$ or assess$)).mp. 

AND 
C-2 

(depression or depressive or sadness).mp. 

NUMBER OF ITEMS RETRIEVED: 215 

----------------------------------------------------------------------------------------- 

DATABASE SEARCHED & TIME PERIOD COVERED: 

CINAHL – 1995-2005 

OTHER LIMITERS: 

 ENGLISH 

 HUMAN

SEARCH STRATEGY #3 (DYSPNEA) 

(dyspnea or shortness of breath or difficulty breathing or breathless$).mp. 

AND 

(guideline$ or audit$ or outcome$ or peer review$ or professional review$ or evaluat$ or 

benchmark$ or utilization review$ or consensus).mp. 

AND 

((quality or perform$) and (measur$ or indicator$ or assess$)).mp. 

NUMBER OF ITEMS RETRIEVED : 278 (78 were duplicates) 

------------------------------------------------------------------------------------------------------------------ 

 

DATABASE SEARCHED & TIME PERIOD COVERED: 

CINAHL – 1995-2005 

OTHER LIMITERS: 

 ENGLISH 

 HUMAN

SEARCH STRATEGY #4 (ADVANCE DIRECTIVES) 

(advance directive$ or withdraw$ care or living will or resuscitat$).mp. 

AND 

(guideline$ or audit$ or outcome$ or peer review$ or professional review$ or evaluat$ or 

benchmark$ or utilization review$ or consensus).mp. 

AND 

((quality or perform$) and (measur$ or indicator$ or assess$)).mp. 

NUMBER OF ITEMS RETRIEVED : 182 
C-3 

SEARCH METHODOLOGIES – END OF LIFE CANCER MEASUREMENT 

REVISIONS – DYSPNEA & ADVANCE DIRECTIVES “NOT” CANCER 

DATABASE SEARCHED & TIME PERIOD COVERED: 

PUBMED – 1995-2005 

OTHER LIMITERS: 

 ENGLISH 

 HUMAN 

SEARCH STRATEGY #1 (DYSPNEA) 

guideline*[tiab] OR quality of health care OR guideline adherence OR medical audit OR nursing audit OR 

outcome and process assessment, health care OR commission on professional and hospital activities OR 

peer review, health care OR professional review organizations OR program evaluation OR benchmarking 

OR quality assurance, health care OR guidelines[mh] OR practice guidelines OR total quality 

management OR quality indicators, health care OR utilization review OR practice guideline[pt] OR 

consensus development conference[pt] OR (communication AND physician* AND (patient* OR 

consumer*)) 

AND 

(quality OR perform*) AND (measur* OR indicator* OR assess*) 

AND 

dyspnea OR shortness of breath OR difficulty breathing OR breathless*

NOT 

case report* 

NUMBER OF ITEMS RETRIEVED : 887 

DATABASE SEARCHED & TIME PERIOD COVERED: 

PUBMED – 1995-2005 

OTHER LIMITERS: 

 ENGLISH 

 HUMAN 

SEARCH STRATEGY #2 (ADVANCE DIRECTIVES) 

guideline*{tiab} OR quality of health care OR guideline adherence OR medical audit OR nursing audit OR 

outcome and process assessment, health care OR commission on professional and hospital activities OR 

peer review, health care OR professional review organizations OR program evaluation OR benchmarking 

OR quality assurance, health care OR guidelines[mh] OR practice guidelines OR total quality 

management OR quality indicators, health care OR utilization review OR practice guideline[pt] OR 

consensus development conference[pt] OR (communication AND physician* AND (patient* OR 

consumer*)) 
C-4 

AND 

(quality OR perform*) AND (measur* OR indicator* OR assess*) 

AND 

advance care OR advance directive* OR withdrawal of care OR living will OR "do not resuscitate" 

NOT 

case report* 

NUMBER OF ITEMS RETRIEVED : 486 
C-5 

SEARCH METHODOLOGIES – END OF LIFE CANCER MEASUREMENT 

PAIN NOT PALLIATIVE/DEPRESSION – PUBMED 

SEARCH STRATEGY #1 (PAIN NOT PALLIATIVE): 

DATABASE SEARCHED & TIME PERIOD COVERED: 

 PUBMED 1995-2005 

OTHER LIMITERS: 

 ENGLISH 

 HUMAN 

 NOT CASE REPORTS 

SEARCH STRATEGY:

neoplasms[mh] OR neoplasm*[tiab] OR cancer* OR tumour* OR tumor* OR carcinoma* OR 

adenocarcinoma* OR malignan* 

AND 

pain[mh] OR pain[ti] 

AND 

quality of health care OR guideline adherence OR medical audit OR nursing audit OR outcome 

and process assessment, health care OR commission on professional and hospital activities OR 

peer review, health care OR professional review organizations OR program evaluation OR 

benchmarking OR quality assurance, health care OR guidelines[mh] OR practice guidelines OR 

total quality management OR quality indicators, health care OR utilization review OR practice 

guideline[pt] OR consensus development conference[pt] OR (communication AND physician* 

AND (patient* OR consumer*)) 

AND 

(quality OR perform*) AND (measur* OR indicator* OR assess*) 

NOT 

case report* 

NUMBER OF ITEMS RETRIEVED: 691 

------------------------------------------------------------------------------------------------------------------ 

SEARCH STRATEGY #2 (DEPRESSION): 

DATABASE SEARCHED & TIME PERIOD COVERED: 

 PUBMED 1995-2005 
C-6 

OTHER LIMITERS: 

 ENGLISH 

 HUMAN 

 NOT CASE REPORTS 

SEARCH STRATEGY: 

neoplasms[mh] OR neoplasm*[tiab] OR cancer* OR tumour* OR tumor* OR carcinoma* OR 

adenocarcinoma* OR malignan* 

AND 

depressive disorders OR depression OR sadness 

AND 

quality of health care OR guideline adherence OR medical audit OR nursing audit OR outcome 

and process assessment, health care OR commission on professional and hospital activities OR 

peer review, health care OR professional review organizations OR program evaluation OR 

benchmarking OR quality assurance, health care OR guidelines[mh] OR practice guidelines OR 

total quality management OR quality indicators, health care OR utilization review OR practice 

guideline[pt] OR consensus development conference[pt] OR (communication AND physician* 

AND (patient* OR consumer*)) 

AND 

(quality OR perform*) AND (measur* OR indicator* OR assess*) 

NOT 

case report* 

NUMBER OF ITEMS RETRIEVED: 468 
C-7 

SEARCH METHODOLOGY – END OF LIFE CANCER MEASUREMENT - DEPRESSION 

PSYCINFO 

DATABASE SEARCHED & TIME PERIOD COVERED: 

PSYCINFO – 1995-2005 

OTHER LIMITERS: 

 ENGLISH 

 HUMAN 

SEARCH STRATEGY 

(kw: neoplasm* OR kw: cancer* OR kw: tumour* OR kw: tumor* OR kw: carcinoma* OR kw: 

adenocarcinoma* OR kw: malignan*) 

AND 

(kw: guideline* OR kw: audit OR kw: outcome* OR (kw: peer and kw: review) OR (kw: 

professional and kw: review) OR kw: evaluat* OR kw: benchmark* OR (kw: utilization and kw: 

review) OR kw: consensus) 

AND 

((kw: quality OR kw: perform*) AND (kw: measur* OR kw: indicator* OR kw: assess*)) 

AND 

((kw: depressive and kw: disorders) OR kw: depression OR kw: sadness) 

NUMBER OF ITEMS RETRIEVED: 86
C-8 

SEARCH METHODOLOGIES – END OF LIFE CANCER MEASUREMENT 

 PSYCINFO 

DATABASE SEARCHED & TIME PERIOD COVERED: 

PSYCINFO – 1995-2005 

OTHER LIMITERS: 

 ENGLISH 

SEARCH STRATEGY #1 (PAIN NOT PALLIATIVE): 

(kw: neoplasm* OR kw: cancer* OR kw: tumour* OR kw: tumor* OR kw: carcinoma* OR kw: 

adenocarcinoma* OR kw: malignan*) 

AND 

(kw: guideline* OR kw: audit OR kw: outcome* OR (kw: peer and kw: review) OR (kw: 

professional and kw: review) OR kw: evaluat* OR kw: benchmark* OR (kw: utilization and kw: 

review) OR kw: consensus) 

AND 

((kw: quality OR kw: perform*) AND (kw: measur* OR kw: indicator* OR kw: assess*)) 

AND 

Kw: pain 

NUMBER OF ITEMS RETRIEVED: 92

------------------------------------------------------------------------------------ 

SEARCH STRATEGY #2 (DEPRESSION): 

DATABASE SEARCHED & TIME PERIOD COVERED: 

PSYCINFO – 1995-2005 

OTHER LIMITERS: 

 ENGLISH 

(kw: neoplasm* OR kw: cancer* OR kw: tumour* OR kw: tumor* OR kw: carcinoma* OR kw: 

adenocarcinoma* OR kw: malignan*) 

AND 

(kw: guideline* OR kw: audit OR kw: outcome* OR (kw: peer and kw: review) OR (kw: 

professional and kw: review) OR kw: evaluat* OR kw: benchmark* OR (kw: utilization and kw: 

review) OR kw: consensus) 

AND 
C-9 

((kw: quality OR kw: perform*) AND (kw: measur* OR kw: indicator* OR kw: assess*)) 

AND 

((kw: depressive and kw: disorders) OR kw: depression OR kw: sadness) 

NUMBER OF ITEMS RETRIEVED: 86

------------------------------------------------------------------------------------------- 

SEARCH STRATEGY #3 (DYSPNEA): 

DATABASE SEARCHED & TIME PERIOD COVERED: 

PSYCINFO- 1995-2005 

OTHER LIMITERS: 

 ENGLISH 

SEARCH STRATEGY: 

(kw: dyspnea OR (kw: shortness and kw: breath) OR (kw: difficulty and kw: breathing) OR kw: 

breathless*) 

AND 

(kw: guideline* OR kw: audit OR kw: outcome* OR (kw: peer and kw: review) OR (kw: 

professional and kw: review) OR kw: evaluat* OR kw: benchmark* OR (kw: utilization and kw: 

review) OR kw: consensus) 

AND 

((kw: quality OR kw: perform*) AND (kw: measur* OR kw: indicator* OR kw: assess*)) 

NUMBER OF ITEMS RETRIEVED: 25 

-------------------------------------------------------------------------------------------------------- 

SEARCH STRATEGY #4 (ADVANCE DIRECTIVES): 

DATABASE SEARCHED & TIME PERIOD COVERED: 

PSYCINFO- 1995-2005 

OTHER LIMITERS: 

 ENGLISH 

SEARCH STRATEGY: 

((kw: advance and kw: care) OR (kw: advance and kw: directive*) OR (kw: withdrawal and kw: 

care) OR (kw: living and kw: will) OR kw: do w1 resuscitate) 
C-10 

AND 

(kw: guideline* OR kw: audit OR kw: outcome* OR (kw: peer and kw: review) OR (kw: 

professional and kw: review) OR kw: evaluat* OR kw: benchmark* OR (kw: utilization and kw: 

review) OR kw: consensus) 

AND 

((kw: quality OR kw: perform*) AND (kw: measur* OR kw: indicator* OR kw: assess*)) 

NUMBER OF ITEMS RETRIEVED: 66 
D-1 

Appendix D. RAND SCEPC Cancer Quality Measures 

Project – Measure Developer Contact Form 
E-1 

Appendix E. Cancer Measure Review Article Screener 
Appendix F1. Quality Measures and Indicators for Supportive Can

Assessment 

F1-1 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease Patient Characteristics Set

1 QA Tools 

RAND1

McGlynn, E.A.2

Asch, S.M.3

(Measure) Regular assessment of Pain. 

All patients with metastatic cancer to bone. 

Patients with the presence or absence of pain noted at least every 6 months. 

Mixed cancer McGlynn, E. A.: 

 Adult  White  Other (Non-White)  60% Male  Average age is 45.5  Age ranges from 18 to 97 Asch, S. M.:  100% Male  Average age is 63.0 

McCoAs C (i ve

2 UHC 

Maxwell, T.4

(Measure) Routine inpatient pain assessment. 

Adults > or = 18 years of age, with admission for CHF (DRG 127), Cancer (DRG 82, 203, 172, 274, 346, 10) HIV (DRG 489) OR respiratory (DRG 475, 483) AND length of stay > 4 days, 2 prior admissions for any cause in preceding 12 months in the hospital. 

Pain assessment within 48 hours of admission on chart review. 

CHF Mixed cancer HIV Mixed respiratory diseases 

Adult Average age is 58.4 Lower boundary for age is 18 

HoHo

3 UHC 

Maxwell, T.4

(Measure) Routine inpatient pain assessment with a numeric scale. 

Adults > or = 18 years of age, with admission for CHF (DRG 127), Cancer (DRG 82, 203, 172, 274, 346, 10) HIV (DRG 489) OR respiratory (DRG 475, 483) AND length of stay > 4 days, 2 prior admissions for any cause in preceding 12 months in the hospital , AND reported pain within the 1st 48 hours of admission. 

Persons assessed with a numeric pain scale. 

CHF Mixed cancer HIV Mixed respiratory diseases 

Adult Average age is 58.4 Lower boundary for age is 18 

HoHo
Assessment 

F1-2 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease Patient Characteristics Set

4 VHA 

VHA Updated March 20055

Pronovost, P.J.6

Erdek, M.A.7

(Measure) Regular ICU pain assessment. 

4 hour periods during which the patients physically present in the ICU, including the initial period if at least present for 2 hours. 

Total number of 4-hour intervals (within the first 24 hours following ICU admission) for which pain was assessed and documented, using an appropriate rating scale. 

Mixed ICU Population 

Not Reported Ho

5 QOPI 

Neuss, M.N.8

(Measure) Assessing pain in patients close to death. 

Recently deceased patients from oncology practices. 

Medical records where there is an explicit practioner's notation quantifying their physical pain or lack thereof on their last visit to the office prior to death. 

Mixed cancer Not Reported Am

out

6 ACOVE End of Life 

Wenger, N.S.9

Saliba, D.10

(Indicator) Routine pain assessment in expected dying. IF a VE who was conscious during the last 3 days and died an expected death THEN the medical record should contain documentation about pain or lack of pain during the last 3 days of life. 

Vulnerable elders who are conscious during the last 3 days of life. 

Number of vulnerable elders with documentation of pain or lack of pain during the last 3 days of life. 

Not Reported Wenger, N. S.: 

 Adult  36% Male  Average age is 81.0 

We NSal N

7 Cancer Care Georgia 

National Cancer Policy Board (NCPB)11

(Indicator) Routine assessment of pain. 

Number of cancer patient encounters. 

Number of cancer patient encounters where patient where patient was assessed for pain. 

Mixed cancer Not Reported No
Treatment 

F1-3 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease Patient Characteristics Se

8 QA Tools 

RAND1

McGlynn, E.A.2

Asch, S.M.3

(Measure) Responsive pain treatment. 

All cancer patients whose pain is uncontrolled. 

Patients offered a change in pain management within 24hours of the pain complaint. 

Mixed cancer McGlynn, E. A.: 

 Adult  White  Other (Non-White)  60% Male  Average age is 45.5  Age ranges from 18 to 97 Asch, S. M.:  100% Male  Average age is 63.0 

MCoA C ( v

9 UHC 

Maxwell, T.4

(Measure) Regular prophylaxis of opiate-induced constipation. 

Adults > or = 18 years of age, with admission for CHF (DRG 127), Cancer (DRG 82, 203, 172, 274, 346, 10) HIV (DRG 489) OR respiratory (DRG 475, 483) AND length of stay > 4 days, 2 prior admissions for any cause in preceding 12 months in the hospital , AND treated with narcotics. 

Proportion with a bowel regimen ordered within 24 hours of the opioid ordered and not contraindicated. 

CHF Mixed cancer HIV Mixed respiratory diseases 

Average age is 58.4 Lower boundary for age is 18 

HICHo

10 Wu, J.S.12 (Indicator) Minimizing 

radiotherapy burden. 

All adult patients with single of multiple radiographically confirmed bone metastases of any histology corresponding to painful areas in previously nonirradiated areas without pathologic fractures or spinal cord/cauda equina compression. It does not apply to the management of malignant primary bone tumor. For patients receiving radiotherapy where the objective / intent is pain relief. 

% of those patients receiving single dose therapy as a single 8Gy treatment, prescribed to the appropriate target volume. 

Mixed cancer Adult N
Treatment 

F1-4 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease Patient Characteristics Se

11 Cancer Care Nova Scotia 

Grunfeld, E.13

(Indicator) Effective treatment for painful bony metastasis. 

Patients with uncontrolled bone pain. 

Patients who are offered within one week of the notation of pain: - radiation therapy to the sites of pain. 

Mixed cancer All patients in the 

population with cancer 

N
Follow-up 

F1-5 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease Patient Characteristics Se

12 NHPCO 

National Hospice and Palliative Care Organization (NHPCO)14

Connor, S.R.15

Ryndes, T.16

(Measure) Timely treatment of pain in hospice. 

Patients admitted to hospice who answer 'yes' when asked "Are you uncomfortable because of pain?". 

The proportion of patients between 48 and 120 hours after admission who report 'yes' when asked "Was your pain brought to an acceptable level within 48 hours of your admission to the hospice program?". 

Mixed hospice population 

NHPCO:  Adult Connor, S. R.:  Adult  White  African American  Hispanic  Asian  Native American / Eskimo  44% Male 

Ho

13 VHA 

Updated March 20055

Pronovost, P.J.6

Erdek, M.A.7

(Measure) Effective treatment of pain in the ICU. 

Total number of 4-hour intervals (within the first 24 hours following ICU admission) with numerical pain values of 1 to 10, for patients with an ICU length of stay >=24 hours. 

Total number of 4-hour intervals (within the first 24 hours following ICU admission) for which the documented pain score was <3. 

Mixed ICU Population 

Not Reported H

14 Cancer Care Ontario17 (Measure) Satisfaction with 

pain treatment. 

Outpatient cancer patients reporting mild to severe pain. 

Patients where the response to the question "Do you think staff did everything they could to control your pain or discomfort?" was (1) yes, completely; (2) yes, somewhat; or (3) no. 

Mixed cancer Not Reported A

ou

15 UHC 

Maxwell, T.4

(Measure) Timely treatment of inpatient pain. 

Adults > or = 18 years of age, with admission for CHF (DRG 127), Cancer (DRG 82, 203, 172, 274, 346, 10) HIV (DRG 489) OR respiratory (DRG 475, 483) AND length of stay > 4 days, 2 prior admissions for any cause in preceding 12 months in the hospital , AND reported pain within the 1st 48 hours of admission. 

Persons with pain relief or reduction within 48 hours of admission < or = 3 on 0-10 scale. 

CHF Mixed cancer HIV Mixed respiratory diseases 

Adult Average age is 58.4 Lower boundary for age is 18 

HICHo
Follow-up 

F1-6 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease Patient Characteristics Se

16 Cancer Care Georgia 

National Cancer Policy Board (NCPB)11

(Indicator) Effective treatment of pain. 

Number of cancer patients who are not comatose. 

Number of cancer patients who report being in more than minor pain. 

Mixed cancer Not Reported N
Potential Indicators 

Assessment 

F1-7 

No. 

Measure or Indicator Set/ Reference / Year 

Description / Original Source Wording Denominator Numerator 

17 Erdek, M.A.7 Physician documentation of 

pain assessment. 

All patients with cancer. 

Patients where a pain assessment is documented in the physician note. 

18 Green, C.R.18 Disparities in pain 

assessment. 

All patients with cancer who are assessed for pain. 

Non-white patients with cancer who are assessed for pain. 

19 Ferrell, B.R.19 Assessment in patients who 

cannot verbalize their pain. 

Cancer patients who have difficulty verbalizing their pain. 

Patients who are assessed for pain. 

20 Carr, D.B.20 Assessment for depression 

in patients with pain. 

All cancer patients with pain. 

Patients assessed for depression. 
Potential Indicators 

Treatment 

F1-8 

No. 

Measure or Indicator Set/ Reference / Year 

Description / Original Source Wording Denominator Numerator 

21 Allard, P.21

Gordon, D.B.22

Education about pain management. 

Cancer patients with pain. 

Patients who are educated about pain management. 

22 Marinangeli, F.23

Gordon, D.B.22

Use of opioids for sever, persistent pain. 

Cancer patients with severe, persistent pain. 

Patients who are treated with opioids. 

23 NIH State-of-the-Science 

Statement24

Use of long-acting pain medications. 

All adults with cancer who have persistent pain and are prescribed opioids. 

Adults who are prescribed long acting opioids. 

24 World Health Organization 

(WHO)25

Procedure-related pain in children. 

All children with cancer receiving painful procedures. 

Children who receive appropriate preparation and/or anesthesia. 
F1-9 

Reference List 

1. In: Asch SM, Kerr EA, Hamilton EG, Reifel 

JL, McGlynn EA. Quality of care for 

oncologic condition and HIV: a review of the 

literature and quality indicators. RAND; 2000. 

 

2. McGlynn EA, Asch SM, Adams J, et al. The 

quality of health care delivered to adults in the 

United States. N Engl J Med 

2003;348(26):2635-45. 

3. Asch SM, McGlynn EA, Hogan MM, et al. 

Comparison of quality of care for patients in 

the Veterans Health Administration and 

patients in a national sample. Ann Intern Med 

2004;141(12):938-45. 

4. Maxwell T, Cuny J. The UHC palliative care 

benchmarking project. University Health 

System Consortium 2004;1-35. 

5. VHA. Transformation of the Intensive Care 

Unit. Palliative Care Bundle: Care and 

Communication Measurement Guide. 2005. 

6. Pronovost PJ, Berenholtz SM. A practical 

guide to measuring performance in the 

intensive care unit.: VHA. 2002. 

7. Erdek MA, Pronovost PJ. Improving 

assessment and treatment of pain in the 

critically ill. Int J Qual Health Care 

2004;16(1):59-64. 

8. Neuss MN, Desch CE, McNiff KK, et al. A 

process for measuring the quality of cancer 

care: the Quality Oncology Practice Initiative. 

J Clin Oncol 2005;23(25):6233-9. 

9. Wenger NS, Rosenfeld K. Quality indicators 

for end-of-life care in vulnerable elders. Ann 

Intern Med 2001;135(8 Pt 2):677-85. 

10. Saliba D, Solomon D, Rubenstein L, et al. 

Feasibility of quality indicators for the 

management of geriatric syndromes in nursing 

home residents. J Am Med Dir Assoc 

2004;5(5):310-9. 

11. National Cancer Policy Board (NCPB). 

Assessing the quality of cancer care: an 

approach to measurement in Georgia.: The 

National Academies Press. 2005. 

12. Wu JS, Wong R, Johnston M, Bezjak A, 

Whelan T, Laetsch N , et al. Radiotherapy 

fractionation for the palliation of 

uncomplicated painful bone metastases.: 

Program in Evidence-Based Care. 2003. 

13. Grunfeld E, et al. Nova Scotia and Ontario 

quality end of life indicators project. 2005. 

14. National Hospice and Palliative Care 

Organization (NHPCO). Instructions for use of 

the six measures. 

15. Conor SR, Tecca M, Lund Person J, et al. 

Measuring hospice care: the national hospice 

and palliative care organization national 

hospice data set. J Pain Symptom Manage 

2004;28(4):316-28. 

16. Ryndes T, Connor S, Cody C, Merriman M, 

Bruno S, Fine P, et al. Report on the alpha and 

beta pilots of end result outcome measures 

constructed by the outcomes forum. 

17. Cancer Care Ontario. Cancer system quality 

index: satisfaction with pain management. 

Ontario, Canada: Cancer Care Ontario. 2005. 

18. Green CR, Anderson KO, Baker TA, et al. The 

unequal burden of pain: confronting racial and 

ethnic disparities in pain. Pain Med 2003; 

4(3):277-94. 

19. Ferrell BA, Ferrell BR, Rivera L. Pain in 

cognitively impaired nursing home patients. J 

Pain Symptom Manage 1995;10(8):591-8. 

20. Carr DB, Goudas LC, Balk EM, et al. 

Evidence report on the treatment of pain in 

cancer patients. J Natl Cancer Inst Monogr 

2004;(32):23-31. 

21. Allard P, Maunsell E, Labbe J, et al. 

Educational interventions to improve cancer 

pain control: a systematic review. J Palliat 

Med 2001;4(2):191-203. 

22. Gordon DB, Dahl JL, Miaskowski C, et al. 

American pain society recommendations for 

improving the quality of acute and cancer pain 

management: American Pain Society Quality 

of Care Task Force. Arch Intern Med 

2005;165(14):1574-80. 
F1-10 

23. Marinangeli F, Ciccozzi A, Leonardis M, et al. 

Use of strong opioids in advanced cancer pain: 

a randomized trial. J Pain Symptom Manage 

2004;27(5):409-16. 

24. NIH State-of-the-Science Statement on 

symptom management in cancer: pain, 

depression, and fatigue. NIH Consensus State 

Sci Statements 2002;19(4):1-29. 

 

25. World Health Organization (WHO). Cancer 

pain relief and palliative care in children. 

1998. 
Appendix F2. Quality Measures and Indicators for Supportive Cance

Assessment 

F2-1 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease 

Patient Characteristics Settin

1 UHC 

Maxwell, T.1

(Measure) Regular assessment of dyspnea. 

Adults > or = 18 years of age, with admission for CHF (DRG 127), Cancer (DRG 82, 203, 172, 274, 346, 10) HIV (DRG 489) OR respiratory (DRG 475, 483) AND length of stay > 4 days AND 2 prior admissions for any cause in preceding 12 months in the hospital. 

Dyspnea assessment within 48 hours of admission. 

CHF Mixed cancer HIV Mixed respiratory diseases 

Adult Average age is 58.4 Lower boundary for age is 18 

HospitHospit
Treatment 

F2-2 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease 

Patient Characteristics 

2 ACOVE End of Life 

Wenger, N.S.2

(Indicator) Effective dyspnea treatment in expected dying. IF a noncomatose vulnerable elder is not expected to survive and a mechanical ventilator is withdrawn or intubation is withheld, THEN the patient should receive (or have orders available for) an opiate of benzodiazepine or barbiturate infusion to reduce dyspnea, and the chart should document whether the patient has dyspnea. 

Noncomatose vulnerable elders who are not expected to survive and a mechanical ventilator is withdrawn or intubation is withheld. 

Patient received or had ordered an opiate or benzodiazepine or barbiturate infusion to reduce dyspnea and the chart documented whether the patient has dyspnea. 

Not Reported Adult 

3 ACOVE End of Life 

Wenger, N.S.2

(Indicator) Regular treatment and follow-up of dyspnea in expected dying. IF a vulnerable elder who had dyspnea in the last 7 days of life died an expected death, THEN the chart should document how the dyspnea was treated and follow-up should be documented about the dyspnea. 

Vulnerable elders who had dyspnea in the last 7 days of life and died expected deaths. 

Chart documents how dyspnea was treated and follow-up about the dyspnea. 

Not Reported Adult 

4 QA Tools 

Golomb, B.3

McGlynn, E.A.4

Asch, S.M.5

(Indicator) Treatment of dyspnea caused by hypoxia. 

Patients with dyspnea and baseline room air oxygen saturation < 88% at rest. 

Therapy with home oxygen was used. 

COPD Adult 
Follow-up 

F2-3 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease 

Patient Characteristics Setting 

5 UHC 

Maxwell, T.1

(Measure) Timely treatment of inpatient dyspnea. 

Adults > or = 18 years of age, with admission for CHF (DRG 127), Cancer (DRG 82, 203, 172, 274, 346, 10) HIV (DRG 489) OR respiratory (DRG 475, 483) AND length of stay > 4 days, 2 prior admissions for any cause in preceding 12 months in the hospital , AND reported dyspnea within the 1st 48 hours of admission. 

Dyspnea relief or reduction within 48 hours of admission. 

CHF Mixed cancer HIV Mixed respiratory diseases 

Adult Average age is 58.4 Lower boundary for age is 18 

HospitaHospita
Potential Indicators 

Assessment 

F2-4 

No. 

Measure or Indicator Set/ Reference / Year 

Description / Original Source Wording Denominator Numerator Disease 

Patient Characteristics 

6 Dudgeon, D.J.6

Kvale, P.A.7Dyspnea. Mechanisms, assessment, and management: a consensus statement. American Thoracic Society8

Ferguson, G.T.9

The optimal treatment for breathlessness is correction of the cause. In all patients with lung cancer, potentially correctable causes of dyspnea, such as localized obstruction of a major airway, a large pleural effusion, or an exacerbation of coexisting COPD, should be sought initially. Diagnostic testing ... to identify the specific nature of the disorder ... is the cornerstone of the assessment of dyspnea and leads to a correct diagnosis in many, but not all, cases. Correction or amelioration of the disorder follows and generally reduces the intensity of dyspnea, increases the comfort with which patients perform activities, and increases their capacity to exercise. Spirometry ... is recommended for patients with respiratory symptoms such as chronic cough, episodic wheezing, and exertional dyspnea in order to detect 

airways obstruction. 

Patients with dyspnea. Evaluation was 

performed to search for correctable conditions as causative for dyspnea. 

Mixed cancer Lung cancer COPD 

Adult 
Potential Indicators 

Treatment 

F2-5 

No. 

Measure or Indicator Set/ Reference / Year 

Description / Original Source Wording Denominator Numerator Disease 

Patient Characteristics 

7 Kvale, P.A.7 Patients with malignant pleural effusions that cause 

dyspnea initially should be drained by thoracentesis. Patients with NSCLC and better performance status and recurrent malignant pleural effusions, and whose lungs re-expand with initial thoracentesis or thoracoscopy, should be followed up by pleurodesis. In patients with SCLC, the treatment of choice for malignant effusions is systemic chemotherapy. 

Patients with cancer and dyspnea caused by malignant pleural effusion. 

Therapy with thorocentesis and appropriate subsequent therapy (pleurodesis, chemotherapy, or other) 

Lung cancer Mixed cancer 

Adult 

8 Kvale, P.A.7 For patients with central airway obstruction, 

bronchoscopy should be done to determine the type of airway obstruction (extraluminal tumor compression of the major airways, intraluminal tumor growth, or both). In patients with central airway obstruction, rapid relief of dyspnea can be accomplished via bronchoscopy with removal of intraluminal tumor (laser, electrocautery, APC) and/or by inserting a stent. 

Patients with cancer and dyspnea caused by central airway obstruction. 

Diagnostic bronchoscopy and bronchoscopic therapy (laser, electrocautery, stent, argon plasma coagulation, cryotherapy, brachytherapy, and/or photodynamic therapy) was offered or received. 

Mixed cancer Adult 

9 Kvale, P.A.7 For all lung cancer patients with dyspnea, 

nonpharmacologic, noninterventional treatments including patient education and intervention by allied health personnel should be used to help control dyspnea, including breathing control, activity pacing, relaxation techniques, fans, and psychosocial support. 

Patients with cancer and dyspnea. 

Therapy with nonpharmacologic and noninterventional approaches (including breathing control, activity pacing, relaxation techniques, fans, and psychosocial support), including patient education, was received. 

Mixed cancer Adult 
Potential Indicators 

Treatment 

F2-6 

No. 

Measure or Indicator Set/ Reference / Year 

Description / Original Source Wording Denominator Numerator Disease 

Patient Characteristics 

10 Socinski, M.A.10

Pfister, D.G.11

Data from case series and randomized trials show that chemotherapy can have a palliative effect on diseaserelated symptoms [especially dyspnea, cough, and chest pain] and can improve QOL compared to BSC in stage IV NSCLC patients who are deemed suitable for treatment. Brachytherapy, photodynamic therapy, and the laser ... have been found to produce effective palliation of symptomatic endobronchial disease, with palliation of cough, hemoptysis, and dyspnea. 

Patients with appropriate malignancies and dyspnea. 

Palliative chemotherapy or radiotherapy (external beam, brachytherapy, and photodynamic therapy) was offered or received for patients who are deemed suitable for treatment. 

Mixed cancer Not Reported 

11 Kvale, P.A.7

Dyspnea. Mechanisms, assessment, and management: a consensus statement. American Thoracic Society8

Jennings, A.L.12

Booth, S.13

For all lung cancer patients with dyspnea, pharmacologic approaches for the management of dyspnea may include oxygen, bronchodilators, corticosteroids, antibiotics, and opioids. It is reasonable to recommend a trial of anxiolytic therapy on an individual basis, particularly in those with morbid anxiety or respiratory panic attacks. Bronchodilators have been shown ... to improve dyspnea [in those with obstructive lung disease]. This review shows statistically strong evidence for a small and probably clinically significant effect of oral or parenteral opioids in the treatment of breathlessness. Opiates acutely relieve dyspnea and improve exercise performance ...; despite safety concerns, these drugs do have a place in the management of patients in the terminal phase of their illness. Oxygen therapy may be one part of the palliative or supportive care of patients with cancer. 

Patients with cancer and dyspnea. 

Therapy with pharmacologic approaches (including oxygen, bronchodilators, corticosteroids, antibiotics, anxiolytics, and opiates) were offered or received in appropriate conditions as part of the palliative management. 

Lung cancer Mixed cancer CHF COPD Infection 

Adult 
F2-7 

Reference List 

1. Maxwell T, Cuny J. The UHC palliative care 

benchmarking project. University Health System 

Consortium 2004;1-35. 

2. Wenger NS, Rosenfeld K. Quality indicators for 

end-of-life care in vulnerable elders. Ann Intern 

Med 2001;135(8 Pt 2):677-85. 

3. Golomb B. Chronic obstructive pulmonary disease. 

In: Kerr EA, Asch SM, Hamilton EG, McGlynn 

EA. Quality of care for cardiopulmonary 

conditions: a review of the literature and quality 

indicators. RAND; 2000. p. 91-109. 

4. McGlynn EA, Asch SM, Adams J, et al. The quality 

of health care delivered to adults in the United 

States. N Engl J Med 2003;348(26):2635-45. 

5. Asch SM, McGlynn EA, Hogan MM, et al. 

Comparison of quality of care for patients in the 

Veterans Health Administration and patients in a 

national sample. Ann Intern Med 

2004;141(12):938-45. 

6. Dudgeon DJ, Lertzman M. Dyspnea in the 

advanced cancer patient. J Pain Symptom Manage 

1998;16(4):212-9. Comment in: J Pain Symptom 

Manage. 1999 Nov;18(5):313-5. PMID: 10584452. 

7. Kvale PA, Simoff M, Prakash UB. Lung cancer. 

Palliative care. Chest 2003;123(1 Suppl):284S311S. 

 

8. Dyspnea. Mechanisms, assessment, and 

management: a consensus statement. American 

Thoracic Society. Am J Respir Crit Care Med 

1999;159( 1):321-40. 

9. Ferguson GT, Enright PL, Buist AS, et al. Office 

spirometry for lung health assessment in adults: a 

consensus statement from the National Lung Health 

Education Program. Chest 2000;117(4):1146-61. 

10. Socinski MA, Morris DE, Masters GA, et al. 

Chemotherapeutic management of stage IV nonsmall cell lung cancer. Chest 2003;123(1 

Suppl):226S-243S. 

11. Pfister DG, Johnson DH, Azzoli CG, et al. 

American Society of Clinical Oncology treatment of 

unresectable non-small-cell lung cancer guideline: 

update 2003. J Clin Oncol 2004;22(2):330-53. 

12. Jennings AL, Davies AN, Higgins JP, et al. Opioids 

for the palliation of breathlessness in terminal 

illness. Cochrane Database Syst Rev 

2001;(4):CD002066 . 

13. Booth S, Wade R, Johnson M, et al. The use of 

oxygen in the palliation of breathlessness. A report 

of the expert working group of the Scientific 

Committee of the Association of Palliative 

Medicine. Respir Med 2004;98(1):66-77. 
Appendix F3. Quality Measures and Indicators for Supportive Cancer 

Assessment 

F3-1 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease 

Patient Characteristics Settin

1 UHC 

Maxwell, T.1

(Measure) Regular assessment for psychosocial well-being. 

Adults > or = 18 years of age, with admission for CHF (DRG 127), Cancer (DRG 82, 203, 172, 274, 346, 10) HIV (DRG 489) OR respiratory (DRG 475, 483) AND length of stay > 4 days, 2 prior admissions for any cause in preceding 12 months in the hospital. 

Formal psychosocial assessment up to 1 year prior to admission during a previous hospitalization OR within 4 days of index admission. 

CHF Mixed cancer HIV Mixed respiratory diseases 

Adult Average age is 58.4 Lower boundary for age is 18 

HospitHospit

2 ACOVE End of Life 

Wenger, N.S.2

Saliba, D.3

(Indicator) Regular spiritual assessment in expected dying. IF a vulnerable elder who was conscious during the last 3 days of life died an expected death, THEN the medical record should contain documentation about spirituality or how the patient was dealing with death or religious feelings. 

Vulnerable elder who was conscious during the last 3 days of life. 

Vulnerable elder with documentation of spiritual assessment in the medical record. 

Not Reported Not Reported All set
Treatment 

F3-2 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease 

Patient Characteristics S

3 ACOVE 

Wenger, N.S.2

Saliba, D.3

Nakajima, G.A.4

(Measure) Regular assessment or treatment of depression in newly diagnosed cancer. 

Vulnerable elders presenting with onset or discovery malignancy (excluding skin cancer) and other conditions*. 

Patients asked about or treated for depression, or referred to a mental health professional within two months of diagnosis of a condition. 

Saliba, D.: 

Mixed nursing home population for 11 diagnoses, not cancer 

Adult A

4 ACOVE 

Wenger, N.S.2

Saliba, D.3

Nakajima, G.A.4

(Measure) Routine assessment or treatment of depression in symptomatic patients. 

Vulnerable elders presenting with new onset of one the following symptoms: sad mood, feeling down; insomnia or difficulties with sleep; apathy or loss of interest in pleasurable activities; complaints of memory loss; unexplained weight loss of greater than 5% in the past month or 10% over one year; or unexplained fatigue or low energy. 

Patients asked about or treated for depression, or referred to a mental health professional within two weeks of presentation. 

Saliba, D.: 

Mixed nursing home population for 11 diagnoses, not cancer 

Adult A

* Other conditions include stroke, myocardial infarction, dementia, chronic pain, alcohol or substance abuse or dependence, anxiety disorder, or per
Potential Indicators 

Assessment 

F3-3 

No. 

Measure or Indicator Set/ Reference / Year 

Description / Original Source Wording Denominator Numerator Disease 

Patient Characteristics S

5 ACOVE End of Life 

Nakajima, G.A.4

IF a vulnerable elder has thoughts of suicide THEN the medical record should document, on the same date, that the patient either has no immediate plan for suicide, or that the patient was referred for evaluation for psychiatric hospitalization. 

Vulnerable elders with thoughts of suicide. 

Patients where medical record documents, on the same date, that the patient either has no immediate plan for suicide, or that the patient was referred for evaluation for psychiatric hospitalization. 

Not Reported Adult A

6 ACOVE End of Life 

Nakajima, G.A.4

IF a vulnerable elder receives a diagnosis of a new depression episode THEN the medical record should document at least three of the nine Diagnostic and Statistical Manual (DSM) IV target symptoms for major depression within the first month of diagnosis. 

Vulnerable elders with diagnosis of a new depression episode. 

Patients where the medical record documents at least three of the nine DSM-IV target symptoms for major depression within the first month of diagnosis. 

Not Reported Adult A

7 ACOVE End of Life 

Nakajima, G.A.4

IF a vulnerable elder receives a diagnosis of a new depression episode THEN the medical record should document, on the day of diagnosis, the presence or absence of suicidal ideation. 

Vulnerable elders with diagnosis of a new depression episode. 

Patients where the medical record document on the day of diagnosis the presence or absence of suicidal ideation and psychosis (consisting of, at a minimum, auditory hallucinations or delusions). 

Not Reported Adult A
Potential Indicators 

Treatment 

F3-4 

No. 

Measure or Indicator Set/ Reference / Year 

Description / Original Source Wording Denominator Numerator Disease 

Patient Characteristics S

8 ACOVE End of Life 

Nakajima, G.A.4

IF a vulnerable elder is being treated for depression with antidepressants THEN the antidepressants should be prescribed at appropriate starting doses, and they should have an appropriate titration schedule to a therapeutic dose, therapeutic blood level, or remission of symptoms by 12 weeks. 

Vulnerable elders being treated for depression with antidepressants. 

Patients with antidepressants prescribed at appropriate starting doses, and with an appropriate titration schedule to a therapeutic dose, therapeutic blood level, or remission of symptoms by 12 weeks. 

Not Reported Adult A

9 ACOVE End of Life 

Nakajima, G.A.4

IF a vulnerable elder has no meaningful symptom response after 6 weeks of treatment THEN one of the following should be initiated by the eighth week of treatment: Optimization of medication (i.e., alter dose of initial medication; change to or add a different medication); psychiatric referral (if initial treatment was medication); or medication trial (if initial treatment was psychotherapy alone). 

Vulnerable elders with depression and no meaningful symptom response after 6 weeks of treatment. 

Patients with one of the following treatment options initiated by the 8th week of treatment: medication dose should be optimized or the patient should be referred to a psychiatrist (if initial treatment was medication); or medication should be initiated or referral to a psychiatrist should be offered (if initial treatment was psychotherapy alone). 

Not Reported Adult A
Potential Indicators 

Follow-up 

F3-5 

No. 

Measure or Indicator Set/ Reference / Year 

Description / Original Source Wording Denominator Numerator Disease 

Patient Characteristics S

10 ACOVE End of Life 

Nakajima, G.A.4

IF a vulnerable elder responds only partially after 12 weeks of treatment THEN at least one of the following treatment options should be instituted by the 16th week of treatment: switch to a different medication class or add a second medication to the first (if initial treatment includes medication); add psychotherapy (if the initial treatment was medication); a trial of medication (if initial treatment was psychotherapy without medication), consider electroconvulsive therapy, or referral to a psychiatrist. 

Vulnerable elders responding only partially after 12 weeks of treatment. 

Patients with one of the following treatment options instituted by the 16th week of treatment: switch to a different medication class or add a second medication to the first (if initial treatment includes medication); add psychotherapy (if the initial treatment was medication); try medication (if initial treatment was psychotherapy without medication); consider ECT; or refer to a psychiatrist. 

Not Reported Adult A

11 ACOVE End of Life 

Nakajima, G.A.4

IF a vulnerable elder has responded to antidepressant medication THEN he or she should be continued on the drug at the same dosage for at least 6 months, and at least one physician documentation about depression should occur during that time period 

Vulnerable elders responding to antidepressant medication. 

Patients continued on the drug at the same dose for at least six months, and making at least one clinician contact (office visit or phone) during that time period. 

Not Reported Adult A

12 ACOVE End of Life 

Nakajima, G.A.4

IF a vulnerable elder is diagnosed with depression THEN antidepressant treatment, psychotherapy, or electroconvulsive therapy should be offered within 2 weeks after diagnosis unless there is documentation within that period that the patient has improved, or unless the patient has substance abuse or dependence, in which case treatment could wait until 8 weeks after the patient is in a drug- or alcohol-free state. 

Vulnerable elders with diagnosis of a new depression episode. 

Patients with antidepressant treatment, psychotherapy, or electroconvulsive therapy (ECT) be offered within two weeks after diagnosis unless there is documentation within that period that the patient has improved, or unless the patient has substance abuse or dependence, in which case treatment may wait until eight weeks after the patient is in a drug or alcohol free state. 

Not Reported Adult A
Potential Indicators 

Follow-up 

F3-6 

No. 

Measure or Indicator Set/ Reference / Year 

Description / Original Source Wording Denominator Numerator Disease 

Patient Characteristics S

13 ACOVE End of Life 

Nakajima, G.A.4

IF a vulnerable elder is started on an antidepressant medication THEN the following medications should not be used as first- or second-line therapy: tertiary amine tricyclics (amitriptyline, imipramine, doxepin, clomipramine, trimipramine); monoamine oxiase inhibitors (MAOIs) (unless atypical depression is present); benzodiazepines; or stimulants (except methylphenidate). 

Vulnerable elders being treated for depression with antidepressants. 

Patients where the following medications are not used as first- or second-line therapy: tertiary amine tricyclics, monoamine oxidase inhibitors (unless atypical depression is present), benzodiazepines, or stimulants (except methylphenidate). 

Not Reported Adult A
F3-7 

Reference List 

1. Maxwell T, Cuny J. The UHC palliative care 

benchmarking project. University Health System 

Consortium 2004;1-35. 

2. Wenger NS, Solomon DH, Roth CP, et al. The 

quality of medical care provided to vulnerable 

community-dwelling older patients. Ann Intern 

Med 2003;139(9):740-7. 

3. Saliba D, Solomon D, Rubenstein L, et al. 

Feasibility of quality indicators for the management 

of geriatric syndromes in nursing home residents. J 

Am Med Dir Assoc 2004;5(5):310-9. 

4. Nakajima GA, Wenger NS. Quality indicators for 

the care of depression in vulnerable elders.: RAND. 
Appendix F4. Quality Measures and Indicators for Supportive C

Advanced Care Planning 

Assessment 

F4-1 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease 

Patient Characteristics Setting 

1 ACOVE End of Life 

Wenger, N.S.1

Wenger, N.S.2

(Measure) Regular identification of a surrogate in the outpatient setting. All vulnerable elders should have in their outpatient chart one of the following: 1) an advance directive indicating the patient's surrogate/proxy decision-maker, 2) documentation of a discussion about who would be a surrogate decision maker or a search for a surrogate, or 3) indication that there is no identified surrogate upon entry to facility or transfer to new unit or site. 

All vulnerable elderly patients in an outpatient setting. 

Number of vulnerable elderly patient's with documentation in their medical record of their proxy or surrogate decision-maker's name and contact information, or of a discussion with patient of who would be surrogate, or documentation of a search for a surrogate, or an indication that there is no identified surrogate. 

Mixed disease Adult 

 36% Male Average age is 81.0 

Home care Hospice Ambulatory / ouHospital (non-ICHospital (ICU) Home Community 

2 ACOVE End of Life 

Wenger, N.S.1

Wenger, N.S.2

(Measure) Regular identification of a surrogate among hospital admissions with impaired cognition. IF a vulnerable elder with dementia, coma, or altered mental status is admitted to the hospital, THEN within 48 hours of admission the medical record should 1) contain an advance directive indicating the patient's surrogate decision maker, 2) document a discussion about who would be a surrogate decision maker or a discussion about a search for a surrogate, or 3) indicate that there is no identified surrogate. 

All vulnerable elders with dementia, coma, or altered mental status admitted to the hospital. 

Number of vulnerable elders with dementia, coma, or altered mental status admitted to a hospital with documentation within 48 hours of admission of 1) an advance directive indicating the patient's surrogate decision maker, 2) a discussion about who would be a surrogate decision maker or a discussion about a search for a surrogate, or 3) indicate that there is no identified surrogate. 

Mixed disease Adult 

 36% Male Average age is 81.0 

Ambulatory / ouHospital (non-ICHospital (ICU) 
Assessment 

F4-2 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease 

Patient Characteristics Setting 

3 ACOVE End-of Life 

Wenger, N.S.1

Wenger, N.S.2

(Measure) Regular assessment of preferences among inpatients with dementia. IF a vulnerable elder has a diagnosis of severe dementia is admitted to the hospital and survives 48 hours, THEN within 48 hours of admission, the medical record should document that the patient's the patient's prior preferences for care have been considered or that these preferences for care could not be elicited or are unknown. 

All vulnerable elders with severe dementia admitted to a hospital and surviving 48 hours. 

Number of vulnerable elders with severe dementia admitted to a hospital and surviving 48 hours with documentation that the patient's prior preferences for care have been considered or that these preferences could not be elicited or are unknown.

Mixed disease Adult 

 36% Male Average age is 81.0 

Hospice Ambulatory / ouHospital (non-ICHospital (ICU) Home 

4 ACOVE End of Life 

Wenger, N.S.1

Wenger, N.S.2

(Measure) Regular assessment of preferences in an ICU. IF a vulnerable elder is admitted directly to the intensive care unit from an outpatient or ER setting and survives 48 hours, THEN within 48 hours of admission, the medical record should document that the patient's preferences for care have been considered or that these preferences could not be elicited or are unknown. 

All vulnerable elderly patients admitted directly to the ICU from an outpatient or ER setting and surviving 48 hours. 

Number of vulnerable elders admitted directly to the intensive care unit from an outpatient or ER setting and surviving 48 hours with documentation in the medical record that the patient's preferences for care have been considered or that these preferences could not be elicited or are unknown within 48 hours of admission. 

Mixed disease Adult 

 36% Male Average age is 81.0 

Hospice Ambulatory / ouHospital (non-ICHospital (ICU) 
Assessment 

F4-3 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease 

Patient Characteristics Setting 

5 Ryndes, T.3

National Hospice and Palliative Care Organization (NHPCO)4

(Measure) Regular assessment of preferences in hospice. 

All patients admitted to hospice. 

Number of hospice patients with documentation in their medical record that they were assessed upon admission including physical, functional, emotional and spiritual needs and preferences with full autonomy to make decisions about how the remainder of their life is to be spent; any living will or advance directive information, and/or an assigned power of attorney for health care. 

Mixed disease Adult Home 

6 ACOVE End of Life 

Wenger, N.S.1

Wenger, N.S.2

(Measure) Regular patient participation in decisions to limit treatment. IF a vulnerable elder with decision-making capacity has orders written in the hospital or the nursing home to withhold or withdraw a particular treatment (e.g., a do-not-resuscitate order or an order not to initiate dialysis), THEN the medical record should document 1) patient participation in the decision or 2) why the patient did not participate in the decision. 

All vulnerable elderly patients in the hospital or nursing home with written orders to withdraw or withhold a LST. 

Number of vulnerable elderly patients with decision-making capacity with written orders to withhold or withdraw a particular LST with documentation in the medical record of 1) patient participation in the decision or 2) why the patient did not participate in the decision. 

Mixed disease Adult 

 36% Male Average age is 81.0 

Hospital (non-ICHospital (ICU) 

7 UHC 

Maxwell, T.5

(Measure) Regular family meetings among hospitalized patients. 

All adults admitted to the hospital. 

Number of adults admitted to the hospital with documentation in the medical record that the health care team conducted a patient/family meeting within 1 week of admission that included discussion of the patient's treatment preferences and/or a plan for discharge disposition. 

Mixed disease Adult Hospital (non-IC

Hospital (ICU) 

8 UHC 

Maxwell, T.5

(Measure) Timely and effective discharge planning. 

All adults admitted to the hospital. 

Number of adults admitted to the hospital with documentation in the medical record of a plan for discharge disposition within 4 days of admission. 

Mixed disease Adult Hospital (non-IC

Hospital (ICU) 
Assessment 

F4-4 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease 

Patient Characteristics Setting 

9 VHA Update March 20056 (Indicator) Regular 

identification of a surrogate in the ICU. 

Total number of patients with an ICU length of stay > 24 hours. 

Number of ICU patients with documentation in the chart of status of identification of a health care proxy (or other appropriate surrogate decision-maker) including one or more of the following: 1) documentation that the patient has a health care proxy, 2) documentation that the patient declined to appoint a health care proxy, 3) documentation that the care team discussed health care proxy appointment with the patient and/or family, 4) documentation that the patient has an appropriate surrogate decision-maker other than a health care proxy. 

Not Reported Adult Hospital (ICU) 

10 VHA Update March 20056 (Indicator) Regular 

assessment of advance directives for ICU patients. 

Total number of patients with an ICU length of stay > than 24 hours. 

Number of patients admitted to the ICU with documentation of the patient's advance directive (written or oral instructions from the patient specifying the type of medical treatment that is desired if the patient becomes incapacitated, including a living will, durable power of attorney (note state-specific status) or any document that state law recognizes as an "advance directive") entered in the medical record within 24 hours following admission. 

Not Reported Adult Hospital (ICU) 

11 VHA Update March 20056 (Indicator) Regular 

assessment of specific resuscitation preferences in the ICU. 

Total number of patients with an ICU length of stay > than 24 hours. 

Number of patients admitted to the ICU with resuscitation status including one or more of the following: 1) documentation of a directive to attempt resuscitation in the event of an arrest, 2) documentation of a directive to forego attempts to resuscitate in the event of an arrest, 3) documentation that the care team discussed resuscitation status with the patient and/or family) documented in the medical record within 24 hours of admission. 

Not Reported Adult Hospital (ICU) 
Assessment 

F4-5 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease 

Patient Characteristics Setting 

12 VHA Update March 20056 (Indicator) Regular 

clinician-patient-family communication in the ICU. 

Total number of patients with an ICU length of stay > than 120 hours. 

Number of patients admitted to the ICU who survive > than 5 days (120 hours) with documentation in the medical record that an interdisciplinary team meeting (with at least the attending physician (primary care or ICU attending) and a nurse) and the patient and/or family was conducted within 120 hours of ICU admission and involved a discussion addressing each of the following topics: 1) the patient's condition (diagnosis and prognosis), 2) goals of treatment; 3) the patient's and family's needs and preferences, 4) the patient's and family's understanding of the patient's condition and goals of treatment at the conclusion of the meeting. 

Not Reported Adult Hospital (ICU) 
Application 

F4-6 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease 

Patient Characteristics Setting 

13 ACOVE End of Life 

Wenger, N.S.1

Wenger, N.S.2

(Measure) Documentation of care preferences across venues. IF a vulnerable elder has an advance directive in the outpatient, inpatient, or nursing home medical record or the patient reports the existence of an existing advance directive in an interview, and the patient receives care in a second venue, THEN 1) the existing advance directive should be present in the patient's medical record in the second venue or 2) documentation should acknowledge its existence, its contents, and the reason that it is not in the current medical record. 

All vulnerable elderly patients with existing advance directives in another setting admitted to an outpatient, inpatient, or nursing home setting. 

Number of vulnerable elderly patients admitted to the outpatient, inpatient, or nursing home and reporting the existence of an advance directive in another setting with documentation in their medical record of 1) the existing advance directive or 2) documentation acknowledging the existing advance directive, it contents, and the reason that it is not included in the medical record. 

Mixed disease Adult 

 36% Male Average age is 81.0 

Home care Hospice Ambulatory / ouHospital (non-ICHospital (ICU) Home Community 

14 ACOVE End of Life 

Wenger, N.S.1

Wenger, N.S.2

(Measure) Documentation of specific life sustaining preferences. IF a vulnerable elder indicates (during an interview) that he/she would rather die than live permanently comatose, ventilated, or tube fed, THEN 1) the chart should document a discussion of life-sustaining treatment preferences, 2) the chart should contain an advance directive, or 3) the patient should indicate (during the interview) that he/she discussed these topics with the physician or does not wish to discuss the topics. 

All vulnerable elders in any setting. 

Number of vulnerable elders in any setting indicating that he/she would rather die than live permanently comatose, ventilated, or tube fed with one of the following documented in their medical record: 1) a discussion of life-sustaining treatment preferences, 2) an advance directive, or 3) that the patient discussed this topic with the physician or does not wish to discuss this topic. 

Mixed disease Adult 

 36% Male Average age is 81.0 

Home care Hospice Ambulatory / ouHospital (non-ICHospital (ICU) Home Community 
Follow-up 

F4-7 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease 

Patient Characteristics Setting 

15 ACOVE End of Life 

Wenger, N.S.1

Wenger, N.S.2

(Measure) Consistency of preferences with use of ventilatory support. IF a vulnerable elder requires mechanical ventilation during a hospitalization (except short-term and post operative mechanical ventilation), THEN within 48 hours of the initiation of mechanical ventilation the medical record should document the goals of care for the patient and the patient's preference for mechanical ventilation or why this information is unavailable. 

All hospitalized vulnerable elders requiring mechanical ventilation (except short-term and post operative mechanical ventilation). 

Number of hospitalized vulnerable elders requiring mechanical ventilation (except short-term and post operative mechanical ventilation) with documentation in the patient record of the goals of care and the patient's decision for mechanical ventilation or why this information is unavailable. 

Mixed disease Adult 

 36% Male Average age is 81.0 

Hospital (non-ICHospital (ICU) 

16 Earle, C.C.7

Earle, C.C.8Grunfeld, E.9

(Measure) Late life hospital use. 

All deceased cancer patients. 

Number of dying cancer patients with > 1 hospital admission in the last month of life. 

Mixed cancer Adult 

Grunfeld, E.: 

All patients in the population with cancer 

Hospital (non-ICHospital (ICU) 

17 Earle, C.C.7

Earle, C.C.8Grunfeld, E.9

(Measure) Late life ICU use. 

All deceased cancer patients. 

Number of dying cancer patients with 1 ICU admission in the last month of life. 

Mixed cancer Adult 

Grunfeld, E.: 

All patients in the population with cancer 

Hospice 
Follow-up 

F4-8 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease 

Patient Characteristics Setting 

18 Dana Farber Cancer Center 

Earle, C.C.7

Earle, C.C.8Grunfeld, E.9

Cancer Care Ontario10

(Measure) Late life rate of emergency care. 

All deceased cancer patients. 

Number of dying cancer patients with > 1 emergency room visit in the last month of life. 

Mixed cancer Adult 

Grunfeld, E.: 

All patients in the population with cancer 

Hospital (non-ICHospital (ICU) 

19 Earle, C.C.7

Earle, C.C.8Grunfeld, E.9

(Measure) New chemotherapy regimen in last 30 days of life. 

All deceased cancer patients. 

Number of dying cancer patients started on a new chemotherapy regimen in the last 30 days of life,. 

Mixed cancer Adult 

Grunfeld, E.: 

All patients in the population with cancer 

Ambulatory / ouHospital (non-ICHospital (ICU) 

20 Earle, C.C.7

Earle, C.C.8Grunfeld, E.9

Cancer Care Ontario11

(Measure) Chemotherapy in last 14 days of life. 

All deceased cancer patients. 

Number of dying cancer patients receiving chemotherapy in the last 14 days of life. 

Mixed cancer Adult 

Grunfeld, E.: 

All patients in the population with cancer 

Hospital (non-ICHospital (ICU) 
Follow-up 

F4-9 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease 

Patient Characteristics Setting 

21 Earle, C.C.7

Earle, C.C.8Grunfeld, E.9

Cancer Care Ontario11

National Cancer Policy Board (NCPB)9

(Measure) Admission to hospice. 

All deceased cancer patients. 

Number of cancer patients who die in the hospital. 

Mixed cancer Adult 

Grunfeld, E.: 

All patients in the population with cancer 

Hospice 

22 Earle, C.C.7

Earle, C.C.8Grunfeld, E.9

National Cancer Policy Board (NCPB)9

(Measure) Late referral to hospice. 

All deceased cancer patients. 

IF a dying cancer patients referred to hospice or palliative care < 3 days before death. 

Mixed cancer Adult 

Grunfeld, E.: 

All patients in the population with cancer 

Hospice 

23 Earle, C.C.7

Earle, C.C.8Grunfeld, E.9

Cancer Care Ontario10

Cancer Care Ontario11

(Measure) Site of death. All deceased cancer 

patients. 

Number of cancer patients who die in the hospital. 

Mixed cancer Adult 

Grunfeld, E.: 

All patients in the population with cancer 

Hospital (non-ICHospital (ICU) 
Follow-up 

F4-10 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease 

Patient Characteristics Setting 

24 Ryndes, T.3

National Hospice and Palliative Care Organization (NHPCO)4

Connor, S.12

(Measure) Safe dying in hospice. 

All Caregiver's of Hospice patients who died while in hospice.

Number of deceased hospice patient's caregivers with documentation in the patient's medical record who were contacted and surveyed as to the quality of care provided by the hospice using the following question: If you cared for the patient at home, did hospice increase your confidence to safely care for your loved one as death approached?. 

Mixed disease Adult Home 

25 ACOVE End of Life 

Wenger, N.S.2

(Indicator) Care consistency with documented care preferences. If a vulnerable elder has specific treatment preferences (e.g., a do-notresuscitate order, no tube feeding, or no hospital transfer) documented in a medical record, THEN these treatment preferences should be followed. 

All vulnerable elderly patients in any health care setting with specific treatment preferences documented in their medical record. 

Number of vulnerable elders with specific treatment preferences documented in their medical record with those treatment preferences having been followed. 

Mixed disease Adult Hospice 

Ambulatory / ouHospital (non-ICHospital (ICU) 
Potential Indicators 

Assessment 

F4-11 

No. 

Measure or Indicator Set/ Reference / Year 

Description / Original Source Wording Denominator Numerator Disease 

Patient Characteristics Setting 

1 Dokken, D.L.13 IF a child with life threatening 

illness is admitted to any health care setting, THEN the needs of each child are supported through periodic assessment and reassessment of the child's physical, social, emotional, developmental, spiritual, and educational strengths and needs in light of the child's developmental level and diagnosis and, based on the assessed needs, development and implementation of interventions involving experts in relevant areas (e.g., child life, chaplancy, physical and occupational therapy, psychology, etc.) should be conducted. 

All Children With Life Threatening Illness Admitted to Any Health Care Setting. 

Number of children with life threatening illness admitted to any health care setting with documentation in the medical record that the needs of each child were periodically assessed (including child's physical, social, emotional, developmental, spiritual, and educational strengths and needs in light of the child's developmental level and diagnosis) and, based on the assessed needs, there is evidence of the development and implementation of interventions involving experts in relevant areas (e.g., child life, chaplaincy, physical and occupational therapy, psychology, etc.). 

Not Reported Children Hospital (non-IC

Hospital (ICU) 
Potential Indicators 

Application 

F4-12 

No. 

Measure or Indicator Set/ Reference / Year 

Description / Original Source Wording Denominator Numerator Disease 

Patient Characteristics Setting 

2 Lilly, C.M.14 IF an ICU patient has a poor 

prognosis (e.g., a condition associated with a high probability of death, who had a predicted length of ICU stay longer than 5 days, a predicted mortality of greater than 25% as estimated by the attending physician, or a change in functional status that is potentially irreversible and sufficient to preclude eventual return to home) or a deteriorating condition (a progressive worsening of a patient's condition that predisposes the patient to death), THEN the attending physician should assess the patient's medical condition, review the case with the patient's primary care physician and agree on a recommended care plan, review any known advance directives, and conduct a family meeting with the patient's family (including health care proxy), where possible, the patient, the patient's nurse, and the hospital house officer in order to 1) review the medical facts and options for treatment, 2) to discuss the patient's perspectives on death and dying, chronic dependence, loss of function, and the acceptability of the risks and discomforts of critical care; 3) to agree on a care plan; and 4) to agree on criteria by which the success or failure of this care plan would be judged using defined "clinical events" as milestones indicating that the care plan had been effective or not and the time frame for their occurrence. 

All ICU patients. umber of ICU patients with a 

poor prognosis (e.g., a condition associated with a high probability of death, who had a predicted length of ICU stay longer than 5 days, a predicted mortality of greater than 25% as estimated by the attending physician, or a change in functional status that is potentially irreversible and sufficient to preclude eventual return to home) or a deteriorating condition (a progressive worsening of a patient's condition that predisposes the patient to death) with documentation in the medical record that the attending physician assessed the patient's medical condition, reviewed the case with the patient's primary care physician, reviewed any known advance directives, and conducted a family meeting with the patient's family (including health care proxy). 

Mixed disease Adult 

White African American Hispanic  59% Male Average age is 59.0 Age ranges from 45 to 75 

Hospital (ICU) 
Potential Indicators 

Application 

F4-13 

No. 

Measure or Indicator Set/ Reference / Year 

Description / Original Source Wording Denominator Numerator Disease 

Patient Characteristics Setting 

3 National Comprehensive 

Cancer Network (NCCN)15

All cancer patients with advanced, symptomatic, progressive disease and/or those with a life expectancy of <12 months, or with serious comorbid conditions, or the patient or family requests palliative care should receive a palliative care assessment, including symptoms, advance care planning discussion (living will, power of attorney, DNR orders, hospice preference), resource management/social support (caregivers, family, financial, community, cultural, spiritual, religious), and the benefits and risks of anticancer therapy to identify patient and family goals and expectations, referral for complex palliative care, discuss palliative care options (including hospice), elicit personal values and preferences of patient and family for end of life care, encourage communication of wishes between patient and family, and encourage the designation of a health care proxy. 

All cancer patients with advanced, symptomatic, progressive disease and/or those with a life expectancy of <12 months, or with serious comorbid conditions, or the patient or family requests palliative care. 

Number of cancer patients with advanced, symptomatic, progressive disease and/or those with a life expectancy of <12 months, or with serious comorbid conditions, or the patient or family requests palliative care with documentation in the chart that they received a palliative care assessment (including symptoms and advance care planning discussions of living will, power of attorney, DNR orders, hospice preference, resource management/social support concerning caregivers, family, financial, community, cultural, spiritual, religious needs, and the benefits and risks of anticancer therapy) to identify patient and family goals and expectations, wishes for referral for complex palliative care, discussion of palliative care options (including hospice), documentation of the personal values and preferences of the patient and family for end of life care, communication of wishes between patient and family, and encourage the designation of a health care proxy. 

Mixed cancer Adult Hospice 

Ambulatory/outHospital (non-ICHospital (ICU) 
Potential Indicators 

Application 

F4-14 

No. 

Measure or Indicator Set/ Reference / Year 

Description / Original Source Wording Denominator Numerator Disease 

Patient Characteristics Setting 

4 Clinical practice guidelines 

for quality palliative care16

Standards and Quality Committee17

A plan of care for dying patients should be developed for the patient based on 1) a comprehensive interdisciplinary medical, psychosocial, spiritual/existential, cultural assessment of the patient and family; 2) the identified and expressed values, goals, and needs of the patient and family developed with professional guidance and support for decision making; the plan is regularly reviewed and is responsive to the patient's changing illness and the family's changing needs; 4) where identified, the patient/family should be referred to external specialist services; and 5) the services provided by the Palliative Care Team are consistent with the care plan and the patient's and family wishes. 

All patients admitted to any health care setting and determined to need palliative care. 

Number of dying patients admitted to any health care setting and needing palliative care with documentation in the patient's medical record of: 1) a comprehensive interdisciplinary medical, psycho-social, spiritual/existential, cultural assessment of the patient and family; 2) documentation that the expressed values, goals, and needs of the patient and family were developed with professional guidance and support for decision making; 3) the plan was regularly reviewed and was responsive to the patient's changing illness and the family's changing needs; 4) the patient/family were referred to external specialist services when appropriate; and 5) the services provided by the Palliative Care Team were consistent with the care plan and the patient's and family wishes. 

Not Reported Adult Home care 

Hospice Ambulatory/outHospital (non-ICHospital (ICU) 
Potential Indicators 

Application 

F4-15 

No. 

Measure or Indicator Set/ Reference / Year 

Description / Original Source Wording Denominator Numerator Disease 

Patient Characteristics Setting 

5 Standards and Quality 

Committee17

All patients, in any health care setting, determined to be dying or whose death is expected, THEN the patient and family should have their psychological, emotional, social, cultural, and spiritual needs assessed on an ongoing basis by an interdisciplinary health care team, and where psychological, emotional, and/or spiritual needs are identified or expressed, should be referred to internal and external specialist services, including bereavement services and these assessments and referral/receipt of supportive services should be documented in the patient's medical record or why these needs were not assessed and/or addressed. 

All patients, in any health care setting, who died with an expected death. 

Number of deceased patients, in any health care setting, who died an expected death with documentation in the patient's medical record of ongoing assessment of the patient and family's psychological, emotional, social, cultural, and spiritual needs by an interdisciplinary health care team, and where psychological, emotional, and/or spiritual needs are identified or expressed, documentation of referral to internal and external specialist services, including bereavement services, and receipt of supportive services or why these needs were not assessed and/or addressed. 

Mixed disease Adult Other 

6 Clarke, E.B.18 IF a vulnerable elder's care, in any 

health care setting, will be transferred to another health care provider (individual, team, setting), THEN the current provider should maximize continuity of care across clinicians, consultants, and settings by orienting new clinicians regarding the patient and family health care preferences and plan of care and prepare patient and family for a change of clinician(s) and introduce new clinicians. 

All vulnerable elders, in any health care setting, who were transferred to new clinician(s) or settings. 

% of vulnerable elder's care, in any health care setting, who were transferred to another health care provider (individual, team, setting), with documentation in the patient's medical record that the current provider oriented new clinicians regarding the patient and family's treatment preferences and plan of care, prepared the patient and family for a change of clinician(s), and introduced new clinicians to the patient and family. 

Mixed disease Adult Hospital (ICU) 

7 Masera, G.19

American Academy of Pediatrics. Committee on Bioethics and Committee on Hospital Care. Palliative care for children20

IF a child with life threatening illness is admitted to any health care setting, THEN a discussion of an advance directive discussion should be conducted within 48 hours of admission to ensure that treatments that have become burdensome are not used. 

All children with life threatening illness admitted to any health care setting. 

% of children with life threatening illness admitted to any health care setting with documentation in the medical record of a discussion with the child's parents of advance directives with LST decisions documented within 48 hours of admission. 

Mixed cancer Adult/Children Home care 

Hospital (ICU) 
Potential Indicators 

Application 

F4-16 

No. 

Measure or Indicator Set/ Reference / Year 

Description / Original Source Wording Denominator Numerator Disease 

Patient Characteristics Setting 

8 Masera, G.19

American Academy of Pediatrics. Committee on Bioethics and Committee on Hospital Care. Palliative care for children20

IF a child with cancer cannot be successfully treated by presently available therapies, and the child needs specific treatments identified to be palliative and not curative for mental or physical distress, THEN the focus of care should be on key issues, including communication, social, psychological, emotional support, symptom control, etc. and a advance directive discussion should be conducted with the child's parents within 48 hours of admission. 

All children with life threatening illness admitted to any health care setting. 

% of children with cancer that cannot be successfully treated by presently available therapies needing specific treatments identified to be palliative and not curative for mental or physical distress with documentation in their medical record that the focus of care is on key issues, including communication, social, psychological, emotional support, symptom control, etc. and a advance directive discussion should be conducted with the child's parents within 48 hours of admission. 

Mixed cancer Children Home care 

Hospital (ICU) 

9 Mitchell, S.L.21 IF a patient entering any health care 

facility is determined to have dementia and has a feeding tube inserted and there is no accompanying documentation of the patient's advance directive or treatment preferences, THEN the medical record should document that the health care team has conferred with the patient's family members and/or proxy regarding sustaining or withdrawing the feeding tube (including a discussion of the evidence regarding the lack of benefit, improved survival, or prevention of aspiration of feeding tubes) within 48 hours of admission. 

Total number of patients entering any health care facility with dementia, a feeding tube, and no accompanying documentation of the patient's advance directive or treatment preferences with LOS > 24 hours. 

Number of patients entering any health care facility with dementia, a feeding tube, and no accompanying documentation of the patient's advance directive or treatment preferences with documentation in the medical record that the health care team conferred with the patient's family members and/or proxy regarding sustaining or withdrawing the feeding tube (including a discussion of the evidence regarding the lack of benefit, improved survival, or prevention of aspiration of feeding tubes) within 48 hours of admission. 

Mixed disease Adult 

Lower boundary for age is 65 

Hospital (non-ICHospital (ICU) 
Potential Indicators 

Application 

F4-17 

No. 

Measure or Indicator Set/ Reference / Year 

Description / Original Source Wording Denominator Numerator Disease 

Patient Characteristics Setting 

10 Dokken, D.L.13 IF a child with life threatening 

illness is admitted to any health care facility, THEN the child and his/her family should be regarded as the unit of care and 1) any concerns raised by the family about the child's condition should be acknowledged and addressed in a timely and respectful manner, 2) the family's cultural values, beliefs, views about quality of life and customary modes of interaction among family and community should be respected by health care providers, 3) the family is present with the child and involved in his/her care to the extent they desire and degree feasible based on the child's condition and health care setting, and 4) the impact of the child's illness on the parental relationship, siblings, and family coping strategies and finances should be routinely assessed, and a range of practical (including financial), emotional, and spiritual supports should be made available to meet family-identified needs by the health care institution and/or through the community. 

All children with a life threatening illness admitted to any health care facility. 

Number of children with life threatening illness is admitted to any health care facility with documentation in the patient's medical record that the child and his/her family were regarded as the unit of care and 1) any concerns raised by the family about the child's condition were acknowledged and addressed in a timely and respectful manner by the health care team, 2) the family's cultural values, beliefs, views about quality of life and customary modes of interaction among family and community were respected by health care providers, 3) the family was present with the child and involved in his/her care to the extent they desired and degree feasible based on the child's condition and health care setting, and 4) the impact of the child's illness on the parental relationship, siblings, and family coping strategies and finances was routinely assessed and a range of practical (including financial), emotional, and spiritual supports were made available to meet familyidentified needs by the health care institution and/or through the community. 

Not Reported Children Hospital (non-IC

Hospital (ICU) 
Potential Indicators 

Application 

F4-18 

No. 

Measure or Indicator Set/ Reference / Year 

Description / Original Source Wording Denominator Numerator Disease 

Patient Characteristics Setting 

11 Widger, K.A.22 Key components of quality care for 

children include information transmission, preparation for death and care at death, bereavement care, interpersonal aspects and competence of health professionals, clarity and honesty of communication, parent role, support for the family, pain and symptom management, decision making, psychosocial and spiritual needs of the child, availability and accessibility of services, coordination and integration of services, respite care for families, and financial support. 

All children with a life threatening illness admitted to any health care facility. 

Number of children with a lifethreatening illness admitted to any health care facility with documentation in the patient's medical record that the key components of quality terminal care for children with cancer and their families. 

Mixed cancer Children Not Reported 
Potential Indicators 

Application 

F4-19 

No. 

Measure or Indicator Set/ Reference / Year 

Description / Original Source Wording Denominator Numerator Disease 

Patient Characteristics Setting 

12 Dokken, D.L.13 IF a child with life threatening 

illness is admitted to any health care setting, THEN each child should have a designated individual, a physician or other health care providers, identified as responsible for coordinating the child's overall care and who should 1)should be involved in care planning and decision-making with the child, family, and other health care providers, 2) is responsible for ongoing communication among the designated health care provider, the child and family, and the child's other health care providers, and the health care setting should develop and implement a process for the child and family to articulate concerns about the choice of designated health care provider and to explore the possibility of changing providers. 

All children with a life threatening illness admitted to any health care facility. 

Number of children with life threatening illness admitted to any health care setting with documentation in the medical record that a designated individual, a physician or other health care providers, was identified and was identified as being responsible for the coordinating the child's overall care and who was involved in care planning and decisionmaking with the child, family, and other health care providers, 2) was identified to the child and family as responsible for ongoing communication among the designated health care provider, the child and family, and the child's other health care providers, and there is documentation in the medical record that the child and family were informed about the health care setting's process for the child and family to articulate concerns about the choice of designated health care provider and to the method established for the child and family to change designated providers if they chose to do so. 

Not Reported Children Hospital (non-IC

Hospital (ICU) 
Potential Indicators 

Application 

F4-20 

No. 

Measure or Indicator Set/ Reference / Year 

Description / Original Source Wording Numerator Denominator Disease 

Patient Characteristics Setting 

13 Dokken, D.L.13 IF a child with life threatening illness 

is admitted to any health care setting, THEN each child should have a comprehensive and appropriate written plan of care documented in their medical record and should be developed with the child's (to the extent appropriate), the family, and their health care providers involvement and that is 1) periodically reviewed, 2) that outlines the roles of family caregivers and the relationships between them, and 3) that includes discharging and receiving health care providers, within and across care settings, and 4) that is communicated in a timely manner across all care settings. 

Number of children with life threatening illness admitted to any health care setting with documentation in their medical record of a comprehensive and appropriate written plan of care developed with the child's (to the extent appropriate), the family, and their health care providers involvement that is 1) periodically reviewed, 2) that outlines the roles of family caregivers and the relationships between them, and that is communicated in a timely manner across all care settings, including discharging and receiving health care providers and within and across care settings. 

All children with a life threatening illness admitted to any health care facility. 

Not Reported Children Hospital (non-IC

Hospital (ICU) 

14 National Comprehensive 

Cancer Network (NCCN)15

All families and next of kin of cancer patients who have died SHOULD receive culturally sensitive and respectful after death care, including respectful treatment of the patient's body, family time with the body, address concerns about organ donation or autopsy, facilitate funeral arrangements, inform insurance companies, and the offer and referral of family/next of kin to grief and bereavement services. 

Number of families and next of kin of cancer patients who died with documentation of receiving culturally sensitive and respectful after death care, including respectful treatment of the patient's body, family time with the body, address concerns about organ donation or autopsy, facilitate funeral arrangements, inform insurance companies, and the offer and referral of family/next of kin to grief and bereavement services. 

All families and next of kin of patients who died in any health care setting. 

Mixed cancer Adult Home care 

Hospice Ambulatory/outpHospital (non-ICHospital (ICU) 
F4-21 

Reference List 

1. Wenger NS, Solomon DH, Roth CP, et al. The 

quality of medical care provided to vulnerable 

community-dwelling older patients. Ann Intern 

Med 2003;139(9):740-7. 

2. Wenger NS, Rosenfeld K. Quality indicators for 

end-of-life care in vulnerable elders. Ann Intern 

Med 2001;135(8 Pt 2):677-85. 

3. Ryndes T, Connor S, Cody C, Merriman M, Bruno 

S, Fine P, et al. Report on the alpha and beta pilots 

of end result outcome measures constructed by the 

outcomes forum. 

4. National Hospice and Palliative Care Organization 

(NHPCO). Instructions for use of the six measures. 

5. Maxwell T, Cuny J. The UHC palliative care 

benchmarking project. University Health System 

Consortium 2004;1-35. 

6. VHA. Transformation of the Intensive Care Unit. 

Palliative Care Bundle: Care and Communication 

Measurement Guide. 2005. 

7. Earle CC, Park ER, Lai B, et al. Identifying 

potential indicators of the quality of end-of-life 

cancer care from administrative data. J Clin Oncol 

2003;21(6):1133-38. 

8. Earle CC, Neville BA, Landrum MB, et al. 

Evaluating claims-based indicators of the intensity 

of end-of-life cancer care. 

9. Grunfeld E, et al. Nova Scotia and Ontario quality 

end of life indicators project. 2005. 

10. National Cancer Policy Board (NCPB). Assessing 

the quality of cancer care: an approach to 

measurement in Georgia.: The National Academies 

Press. 2005. 

11. Wu JS, Wong R, Johnston M, Bezjak A, Whelan T, 

Laetsch N , et al. Radiotherapy fractionation for the 

palliation of uncomplicated painful bone 

metastases.: Program in Evidence-Based Care. 

2003. 

12. Conor SR, Tecca M, Lund Person J, et al. 

Measuring hospice care: the national hospice and 

palliative care organization national hospice data 

set. J Pain Symptom Manage 2004;28(4):316-28. 

13. Dokken DL, Heller KS, Levetown M, et al. Quality 

domains, goals and indicators for family-centered 

care of children living with life-threatening 

conditions. Education Development Center, Inc. 

2002. 

14. Lilly CM, De Meo DL, Sonna LA, et al. An 

intensive communication intervention for the 

critically ill. Am J Med 2000;109(6):469-75. 

15. National Comprehensive Cancer Network (NCCN). 

Palliative care. Clinical Practice Guidelines in 

Oncology 2004;(1) 

16. Clinical practice guidelines for quality palliative 

care.: National Consensus Project for Quality 

Palliative Care. 2004. 

17. Standards and Quality Committee. Standards for 

palliative care provision. Australia. 1999. 

18. Clarke EB, Curtis JR, Luce JM, et al. Quality 

indicators for end-of-life care in the intensive care 

unit. Crit Care Med 2003;31(9):2255-62. 

19. Masera G, Spinetta JJ, Jankovic M, et al. Guidelines 

for assistance to terminally ill children with cancer: 

a report of the SIOP Working Committee on 

psychosocial issues in pediatric oncology. Med 

Pediatr Oncol 1999;32(1):44-8. 

20. American Academy of Pediatrics. Committee on 

Bioethics and Committee on Hospital Care. 

Palliative care for children. Pediatrics 2000;106(2 

Pt 1):351-7. 

21. Mitchell SL, Berkowitz RE, Lawson FME, et al. A 

cross-national survey of tube-feeding decisions in 

cognitively impaired older persons. Journal of the 

American Geriatrics Society 2000;48(4):391-7. 

22. Widger KA, Wilkins K. What are the key 

components of quality perinatal and pediatric endof-life care? A literature review. J Palliat Care 

2004;20(2):105-12. 
Appendix F5. Quality Measures for Cancer Palliation: Measures That C

All Domains 

Pain, Depression, Dyspnea, and Advance Care Planning 

F5-1 

No. 

Measure or Indicator Set/ Reference / Year 

Description of measure or indicator Denominator Numerator Disease 

Patient Characteristics Setting 

1 Connor, S. R.1 (Measure) Family 

evaluation of hospice care 

Family members of hospice decedents 

Percentage of respondents reporting opportunity for improvement in attention to family needs for support, attending to family needs for information, provision of desired physical comfort and emotional support; and mean overall satisfaction 

Mixed hospice population 

Adult/Children White African American Hispanic Asian 

Hospice 
F5-2 

Reference List 

1. Connor SR, Teno J, Spence C, Smith N. Family 

evaluation of hospice care: results from voluntary 

submission of data over website. 
Appendix G. Peer Reviewer Comments 

G-1 

Item Name Institution Role

Page Section Comment 

1 

Adalsteinn Brown, AB, D.Phil 

University of Toronto Reviewer 2 Introduction 

When we review performance measures with complex continuing (chronic) care hospitals and rehabilitation hospitals in Ontario they often ask us to think about measures of caregiver burden and health status. When I read your comments about depression on page 2 and 

how both patients and care-givers maintain hope, I thought it might be important to point out that measures of how care-givers are doing might be important (at least as predictors of patients’ capacities) but that they would be difficult to incorporate into the assessment, treatment, and follow-up paradigm and might be also be subject to a lot of exclusions in measurement. 

2 

Adalsteinn Brown, AB, D.Phil 

University of Toronto Reviewer 

We have just completed an assessment of recall or reliability in complex continuing care hospital patients in relation to their ability to report on their perceptions of care. It was awful. But the important thing out of the study was the fact that repeated assessment, in this 

case by a trained interviewer, actually caused some of the patients anxiety as they worried that they “hadn’t gotten it right the first time” owouldn’t be able to answer the same again. I think it might be useful tpoint out some of the issues associated with repeated measurement that might happen in some surveying programs in chronically ill patients. 
G-2 

Item Name Institution Role

Page Section Comment 

3 

Adalsteinn Brown, AB, D.Phil 

University of Toronto Reviewer 30 CCO- Measure 

CCO patient satisfaction survey. All the NRC surveys go through a very primitive form of psychometric validation (alpha) before they go into widespread use. I don’t know whether you believe the Picker surveys are reflexive surveys or not, but you might want to note that 

there has been some validation. Also, we have used a cut-off of 100 responses for a long time in Ontario for virtually all our other surveys (for no better reason that than the confidence intervals were reasonable at that point). I wouldn’t be surprised that CCO (you note 300 surveys) goes that route shortly as well. We will be publishing twpapers on oncology satisfaction shortly (one has been accepted at Annals of Oncology and the other is under revision) but I do not believe they will be of much use to you. Let me know if you want to see them confidentially regardless. 

6 

Ellen Stovall, PhD/ Mark Gorman 

National Coalition for Cancer Survivorship (NCCS) 

Reviewer Stage of Illness 

Evidence discussed is derived from end of life, hospice or a narrowly defined "palliative" stage of a cancer patient's illness. Numerous IOM 

evidence-based findings point out, cancer patients need comprehensive symptom management at all phases of their treatmenbeginning with their diagnosis and continuing throughout the remainder of their lives. 

7 

Ellen Stovall, PhD/ Mark Gorman 

National Coalition for Cancer Survivorship (NCCS) 

Reviewer Settings of Care 

Many measures discussed in this report are derived from an inpatient setting and some are derived from the highly specialized setting of an ICU. A majority of US cancer patients are treated in ambulatory settings, and most of those are treated completely outside a hospital setting in private community oncology practices. Therefore, there is a need for development of measures and indicators relevant to physiciaoffice settings where most of cancer care is provided. 
G-3 

Item Name Institution Role

Page Section Comment 

8 

Ellen Stovall, PhD/ Mark Gorman 

National Coalition for Cancer Survivorship (NCCS) 

Reviewer Sources 

One source of data that does not appear, namely information from the 

60 NCI designated cancer centers. Most if not all of these centers have formal guidelines pertaining to at least some of the domains deawith in this report. 

9 

Ellen Stovall, PhD/ Mark Gorman 

National Coalition for Cancer Survivorship (NCCS) 

Reviewer Task Order 

Processes vs. Outcomes: Task order for this study directed a focus oprocesses, from the patient's perspective outcomes are more important. While it is useful that the study has included some outcommeasures for "completeness," assigning such secondary importance outcomes measures represents a misplacement of priorities. Particularly in the domains of symptom management, which so dramatically impact a cancer patient's quality of life, having good measures of whether particular interventions are resulting in better patient outcomes is vital. 

10 

Ellen Stovall, PhD/ Mark Gorman 

National Coalition for Cancer Survivorship (NCCS) 

Reviewer Overall 

Progress in setting quality measures fro cancer will need to focus on both process and patient outcomes, be based on the real world settings where most of cancer care is provided, and be applicable to all stages of patient's treatment. 

11 

Eva Grunfeld, MD, Dphil 

Cancer Care Nova Scotia Reviewer 10 Methods Search strategy: what was the rationale for starting the search in 1995? Some justification is warranted. 
G-4 

Item Name Institution Role

Page Section Comment 

12 

Eva Grunfeld, MD, Dphil 

Cancer Care Nova Scotia Reviewer 16 Results re statement on ORYX, JCAHO, NCQA – should you at least include them in Table 1? 

13 

Eva Grunfeld, MD, Dphil 

Cancer Care Nova Scotia Reviewer 18 Table 1 

Description of CCNS qualitative phase – the most important component of this phase is the focus groups with patients with metastatic cancer and bereaved caregivers, and the modified Delphi process with health care providers (oncologists, palliative care physicians, cancer nurses, social workers, community family physicians) to determine their views on the 19 quality indicators being measured in the quantitative component of the study. You may want to add this to your description of the qualitative phase of this project. (NB: These have now all been conducted and analysis is nearing completion.) 

14 

Eva Grunfeld, MD, Dphil 

Cancer Care Nova Scotia Reviewer 28 Pain 

re: treatment for painful bony metastases. We have now examined this indicator both in terms of its acceptability to stakeholders (i.e., through the Delphi with health care providers) and through attempting to calculate it using administrative datasets. Based on these two processes, it is now clear that we will not adopt this as a useful QI despite the evidence that it is an effective treatment. The primary difficulty is that we are not able to measure when the pain is no longecontrolled by medications and requires radiotherapy. The one week window was considered appropriate by the stakeholder group but thedid not agree with our denominator for the same reason given above. 
G-5 

Item Name Institution Role

Page Section Comment 

15 

Eva Grunfeld, MD, Dphil 

Cancer Care Nova Scotia Reviewer 46-50 ACP 

the measures included here (those used by Dana Farber and CCNS) do not appropriately fit under ACP since they do not measure when these aspects of care are in keeping with the patients’/caregivers’ preferences. In fact, in our Delphi process the principal objection was that measuring these indictors in isolation of knowledge of patients’ preferences does not necessarily reflect good quality of care. 

16 

Eva Grunfeld, MD, Dphil 

Cancer Care Nova Scotia Reviewer 54, Para1 ACP I think this is a very important point. Where are these indictors being used to actually improve quality of care? 

17 

Eva Grunfeld, MD, Dphil 

Cancer Care Nova Scotia Reviewer Appe ndix Reviewer list Please note the correction to your reviewer list: Dr. Grunfeld, MD, DPhil 

18 

Eva Grunfeld, MD, Dphil 

Cancer Care Nova Scotia Reviewer Depression Consideration might be given to include an indicator for screening/treatment of depression in caregivers 
G-6 

Item Name Institution Role

Page Section Comment 

19 

Eva Grunfeld, MD, Dphil 

Cancer Care Nova Scotia Reviewer Title 

I find the title not sufficiently specific. Suggest either end-of-life cancemeasures (as originally termed) or measures of cancer symptom management. 

20 

Eva Grunfeld, MD, Dphil 

Cancer Care Nova Scotia Reviewer 

In some instances you discuss the issue of benchmarking which essentially aims to define how one concludes whether an institution/organization/region is doing a good job. When is it enoughwhen is it too much, or too little? A general discussion about this poinand potential approaches to benchmarking would enhance the usefulness of the report. 

21 

Eva Grunfeld, MD, Dphil 

Cancer Care Nova Scotia Reviewer Suggestions Perhaps a table summarizing those indicators you conclude are most helpful could be added. 

22 

Eva Grunfeld, MD, Dphil 

Cancer Care Nova Scotia Reviewer Evidence Tables 

Tables – description of CCNS indictors: For all indicators - Patient characteristics should read “all patients in the population with cancer”The data sources are: cancer registry, administrative claims data, hospital discharge abstracts. 

23 

Eva Grunfeld, MD, Dphil 

Cancer Care Nova Scotia Reviewer Chapter 4 

The major gap is raised by your statement in the first paragraph of Chapter 4: where is the evidence that quality measures are actually being used to measure and promote quality of care. Two possible examples are the National Comprehensive Cancer Network (Weeks eal) which uses a measurement and feedback system with the goal of improving adherence to guidelines (as a process measure of quality ocare); and the Cancer Care Ontario quality initiative (your references 54, 113 and 114).. 
G-7 

Item Name Institution Role

Page Section Comment 

24 

Eva Grunfeld, MD, Dphil 

Cancer Care Nova Scotia Reviewer 

Measurement tools and data capture methods. 

The principal point of QIs is that they are measurable. The report doenot specifically discuss the issue of measurement instruments (for example, is asking the patient if they have pain equal in value to usina validated pain assessment tool?) or data capture methods (chart abstraction vs prospective data capture using validate instruments vs administrative datasets). There is a general discussion within the texof some of the QIs but no overarching discussion. The validity, reliability and feasibility of measuring QIs will vary with each of the different data capture techniques and their useful is contingent on how the data is captured and how accessible it is to use. Just to give one example, the QI ‘effective treatment of pain’ (p.81). The denominator for this indicator is “number of cancer patients who are not comatose”The rationale for this definition is self-evident. However, precisely hoan institution would capture the information is questionable. Institutions may be able to know precisely how many cancer patients they see in a given period of time. They are much less likely to be able to subtract from the denominator the number that are comatose (or, at least not without a detailed chart review). Hence the value of thintended precision of this QI is reduced by the difficulty of actually capturing the data. 

25 

Eva Grunfeld, MD, Dphil 

Cancer Care Nova Scotia Reviewer Scope of the report 

The questions posed in the task order indicate that the scope of the report is at any point of the cancer trajectory (i.e., not just end-of-lifeThis is a very wide scope. However, the review seems to focus on end-of-life/palliative care (EOL/PC). For example the majority of the indicator sets you have identified relate to EOL/PC. This raises some confusion in my mind whether the broader scope is being adequately addressed or whether, in fact, it is the narrower scope that is really intended for the report. If EOL/PC really is the focus, it should be specified clearly (for example, in the title of the report). If it is intendeto be the entire cancer trajectory then the other periods (e.g., diagnosis, treatment, and survivorship) have not be adequately addressed or a statement should be made that no quality indicators (QIs) were identified for these periods. 
G-8 

Item Name Institution Role

Page Section Comment 

26 

Eva Grunfeld, MD, Dphil 

Cancer Care Nova Scotia Reviewer 

While a comprehensive discussion about measurement is probably beyond the scope of the task order, you do touch on it in a general way on pages 54-55. I think the usefulness of the report would be enhanced by a slightly more detailed discussion about measurement within each of the domains. 

27 

Eva Grunfeld, MD, Dphil 

Cancer Care Nova Scotia Reviewer Suggestions 

You identify many of the indicators as ‘not in current use’. A separate table listing those indicators that are currently being used would be helpful 

28 

Joanne Wolfe, MD, MPH 

Harvard UniversityPediatrics Reviewer 11 Methods 

With regard to the Grey Literature - Internet Search the following sites could have also been included: American Society of Pediatric Hematology/Oncology www.aspho.org The Association for Children with Life-Threatening or Terminal Conditions and Their Families (ACT) www.act.org.uk Children's Oncology Group http://www.childrensoncologygroup.org/ Children's Hospice International www.chionline.org/ http://www.apon.org/files/public/last_acts_precepts.pdf http://www.aps-spr.org 

29 

Joanne Wolfe, MD, MPH 

Harvard UniversityPediatrics Reviewer 

13, para 4 

Methods I suspect you intended to include "Canada" inside parentheses. 

30 

Joanne Wolfe, MD, MPH 

Harvard UniversityPediatrics Reviewer Gaps/article s Consider including the attached article (Beal A. Quality Measures for Children's Health Care. Pediatrics 2004;113(1) 199-209 
G-9 

Item Name Institution Role

Page Section Comment 

31 

Judi L. Person, MPH 

National Hospice and Palliative Care Organization 

Reviewer 3 Summary/ Conclusion 

The sentence: "Palliative care is a term that is sometimes used interchangeably, except that it is focused on living with the more 

advanced stages of illness. " I would question this description of 

palliative care- is a component of care from diagnosis and increasing in focus. 

32 

Judi L. Person, MPH 

National Hospice and Palliative Care Organization 

Reviewer 19 Table 1 VHA- Spell out VHA, no reference found 

33 

Judi L. Person, MPH 

National Hospice and Palliative Care Organization 

Reviewer 30 First paragraph "...PE.1.4; and the National Consensus Project for Quality Palliative..." add Project 

34 

Judi L. Person, MPH 

National Hospice and Palliative Care Organization 

Reviewer 32 

Under Treatment, second paragraph, last sentence 

"...the reviewers concluded that improved adherence with long-acting drugs in clinical practice improved outcomes." is an incomplete sentence 

35 

Judi L. Person, MPH 

National Hospice and Palliative Care Organization 

Reviewer 35 Dyspnea QA Tools end of paragraph, " ...Association of Palliative Medicine." Reference country here- could be confused with AAHPM. 

36 

Judi L. Person, MPH 

National Hospice and Palliative Care Organization 

Reviewer 36 

DyspneaPIAssessment ACCP, spell out somewhere, found on page 37 but should also be reference here, should also be listed on page 11. 

37 

Judi L. Person, MPH 

National Hospice and Palliative Care Organization 

Reviewer 48 

ACPChemo in last 14 days measure 

"A high proportion (> 10%) of cancer patients receiving chemotherapy in the last 14 days of life indicates poor quality care. This is the 

indicator? 
G-10 

Item Name Institution Role

Page Section Comment 

38 

Judi L. Person, MPH 

National Hospice and Palliative Care Organization 

Reviewer 50 

ACP- safe dying in hospice measure 

"This indicator is based on the assumption that caregivers who lack 

confidence or..." consider adding this to your sentence. This is also an indicator of caregiver competence. 

39 

Judi L. Person, MPH 

National Hospice and Palliative Care Organization 

Reviewer 52 ACP - PI 

What about POLST (Physicians Order for Life Sustaining Treatment) In use in Oregon and West Virginia, close to being used in NY. Nationwide rollout in 2006+. Probably should have some mention. 

40 

Judi L. Person, MPH 

National Hospice and Palliative Care Organization 

Reviewer Reviewer list Organization incorrect, change to National Hospice and Palliative Ca

Organization. Also, Kaiser is spelled incorrectly. 

43 

Lindsey Bramwell, MPH, RN, CDR, USPHS 

CMS (Fed Reviewer) Reviewer 

Symptom Management 

Symptom management measures for pain, dyspnea, and depression disappoint in their failure to adequately examine patients freed from these symptoms, as well as the other processes of care such as assessment, treatment, and follow-up. 

44 

Lindsey Bramwell, MPH, RN, CDR, USPHS 

CMS (Fed Reviewer) Reviewer General 

The Report identifies gaps in the research--- the majority of the measures and indicators which are mentioned in this report have not specified the reliability and/or validity of these measures and have onbeen tested, not implemented. 

45 

Lindsey Bramwell, MPH, RN, CDR, USPHS 

CMS (Fed Reviewer) Reviewer Dyspnea Measures 

Your innovative approach of using dyspnea measures developed for chronic obstructive pulmonary disease in view of the lack of specific cancer population dyspnea research is to be commended. You have identified the need for new Cancer Care measurement research, but the meanwhile, you have promoted the concept of cross-cutting measures which apply to multiple diseases to move towards broadening the applicability and aligning Cancer Care measures with measurement research projects and guidelines. 
G-11 

Item Name Institution Role

Page Section Comment 

46 

Anna M. Lythgoe, RN, MSN 

VHACO Reviewer 

The review of the literature for quality indicators related to cancer carwas broad enough to capture relevant information but stringent enough to focus on particular areas of interest: pain, depression, dyspnea, and advance care planning/patient as decision-maker. It was surprising that there were not more quality of care indicators related to cancer care noted in the literature. 

47 

Anna M. Lythgoe, RN, MSN 

VHACO Reviewer 8 

Symptom management included assessment of “frequency, severity, and bother or functional impact” (pg 8). Recently reviewed studies fodevelopment of home care indicators indicate patients feel that loss oindependence/function is critical to their perceptions of the quality of their lives. Clinical review of functional impact of symptoms (pain, depression, dyspnea) would potentially improve symptom assessmenand impact of treatment for those with cancer. The VHA “pain management workgroup” is currently discussing how measure improvement in function as pain management improves. It would seem that this would also be relevant for the cancer patient. 

48 

Anna M. Lythgoe, RN, MSN 

VHACO Reviewer 

Potential indicators discussed that could be used to address the gaps in the literature were fairly global as currently stated. However, with refinement, they may be well utilized to drive ongoing assessment antreatment of symptoms related to cancer care. Many could be readily operationalized (almost) as written (e.g. physician documentation of pain assessment, patient education regarding pain management, routine assessment for dyspnea) while others may require clinical evidence to support change in practice (e.g. use of long acting opiatewith as-needed opiates, indicators supporting specific interventional modalities). In summary, the suggested indicators based on the literature reviewed were more hopeful than the indicators currently in place. 
G-12 

Item Name Institution Role

Page Section Comment 

49 

Anna M. Lythgoe, RN, MSN 

VHACO Reviewer 

Mention is made of the burden of cancer on patients and caregivers (pg 1) but the caregivers are not addressed in the review. The significant fiscal impact of (unpaid) family caregivers on the healthcarsystem begs their inclusion in the quality of care review. Also, the clinical responsibility to ensure that the family caregiver has the education, support, and resources needed to successfully provide thicare is heavy and must be addressed at some level. 

50 

Anna M. Lythgoe, RN, MSN 

VHACO Reviewer 

Table 2 (Processes potentially relevant to symptom care and advance care planning) – Assessment – Advance Care Planning: If a family caregiver will be providing care, assessing his/her ability and willingness to provide care, educational needs, support systems, and identifying the limitations of their abilities may need to be assessed. 

53 

Phil Madvig, MD(Paul Feigenbaum and Jann Dorman) 

Kaiser Permanente Reviewer 

For those metrics where the denominator is a population associated with an institution or delivery system, e.g., "all deceased cancer patients" at Dana Farber there may be a concern that the metric woube driven by the unique characteristics of the institutional population iaddition to the practice patterns applied to the numerator population.  For example, the patient population attracted to an academic medicacenter with a reputation for clinical trials may be different than the patient population at a regional medical center. How would these 2 institutions compare their performance on a single metric? 

54 

Phil Madvig, MD (Paul Feigenbaum and Jann Dorman) 

Kaiser Permanente Reviewer 

The listings of measures and indicators appeared relatively complete to us. We are not aware of additional sources of measures or indicators which you have not included. 
G-13 

Item Name Institution Role

Page Section Comment 

55 

Phil Madvig, MD (Paul Feigenbaum and Jann Dorman) 

Kaiser Permanente Reviewer Comment 

We face several challenges in developing quality metrics at Kaiser Permanente. We need metrics that are: *Feasible to collect on a scale basis. Metrics which are dependent upon chart review present tremendous resource challenges to measure in a large population. *Low risk for perverse incentives. For example, the percentage of patients referred to hospice in the last 3 days of life can be decreased by not referring patients to hospice in the last 3 days of life. *Timely. We need performance feedback that can be used to managoperations. The metric must be reported within a time frame that is meaningful to make operational improvements. These dimensions represent the major gaps and priorities we face. 

56 

Phil Madvig, MD (Paul Feigenbaum and Jann Dorman) 

Kaiser Permanente Reviewer ACP 

We question if the logic of the conceptual framework for the processeof care used for symptom management holds up for advance care planning. As an example, you have labeled the second step in the conceptual framework "application", but refer to "application of preferences..." in the third step for ACP. Given the well known challenges in operationalizing advance care planning on a scale basian accurate framework for the processes of advance care planning would be a great asset in understanding how to be successful. 

59 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 2 Last paragraph "...varies significantly among elderly cancer patients.." only among elderly, or all cancer patients? 
G-14 

Item Name Institution Role

Page Section Comment 

60 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 2 Last paragraph "Cancer treatment at any stage commonly requires potentially difficu

interventions such as chemotherapy or surgery." Not sure what "difficult means. 

61 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 2 Last paragraph 

"Now, quality practice endorses advance care planning for a number reasons: to allow patients to shape care to their preferences, to avoid imposition of [unwanted?] treatment in urgent situations, to relieve 

patient and family anxiety..." 

62 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 3 Summary/ Conclusion "...that is sometimes used interchangeably, except that it is focused..." Interchangeably with what? Supportive care? 

63 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 5 Overview Staff conducted an extensive grey literature and Web search..." Wil

the readers know what "grey literature" is? 

64 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 6 Third paragraph "Concerns were related relative to the feasibility of conducting a systematic review of distress." Consider "Concerns were expressed relative to the feasibility of conducting a systematic review of distress

65 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 9 Last paragraph "Elements of care planning include prediction of the clinical situation,.." Not sure what this means? Prognostication? 
G-15 

Item Name Institution Role

Page Section Comment 

66 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 10 

Table 2, first row under Symptom Management 

"...preferences for evaluation and treatment;..." Does this include patients desired level of relief? 

67 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 17 Table 1, Dana Farber 

The Dana-Farber Cancer Center at Harvard University has pursued aeffort to develop a set of administrative data-based indicators for palliative cancer care as part of its recent research agenda.- Are thesintended as quality measures, or for some other purpose? 

68 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 19 Table 1, GCC 

1. Pain assessment and management. 2. Use of hospice care. Why only mention these 2 of the 80 measures? Are none other relevant to this project? 

69 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 19 Table 1, CCO Measures relevant to palliative [cancer?] care include those that address: 

70 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 20 Table 1, VHA 

o Assess pain regularly o Manage pain optimally Do these not continue across the ICU stay/measurement period? Are they only measured at day 1? 

71 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 21 Literature flow "...we found 389 articles from the database..." Unique from the 4580
G-16 

Item Name Institution Role

Page Section Comment 

72 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 21 Table 1, NHPCO Quality measures included: 1. Comfortable dying 2. Self-determined life closure 3. Safe dying 4. Effective grieving 5. Family evaluation of hospice care Any need to define any of these? 

73 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 24 Pain, Introduction- Two last paragraphs of the introduction, Can this be presented as a table instead? Somewhat difficult to follow in text? 

74 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 25 

Measures and Indicators 

Section is somewhat difficult to follow in text but is a nice description these measures. 

75 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 25 

Measures and Indicators 

Suggest add, Measures and Indicators: Pain as the section header. Did I miss somewhere a statement of the structure for this section (re 

measures/indicators in italics) 

76 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 31 

Potential Indicators and Measurement Gaps 

Any data that physician documentation actually translates into improved pain management? 
G-17 

Item Name Institution Role

Page Section Comment 

77 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 31 

Potential Indicators and Measurement Gaps 

Suggest add Potential Indicators and Measurement Gaps: Pain as section title 

78 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 32 

Potential Indicators and Measurement Gaps 

First paragraph "Disparities in pain assessment" What about age related differences/disparities? Are also data that geriatrics populatio(especially in [illegible]) receives inadequate pain control. 

79 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 32 

TreatmentMeasurement gap 

Treatment. "Procedure-related pain in children paragraph." Should thone be limited to children? Is it not a relevant issue for adults? 

80 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 32 Treatment 

Treatment. Education. "The numerator is patients who are educated about pain management.." Does this need to be defined any more (e.g.. Person is printed materials) 

81 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 33 Dyspnea- Introduction First paragraph. Again, can this be formatted differently for ease of following? 

82 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 33 

Measures and Indicators 

Measures and Indicators: Dyspnea. Suggest adding Dysnea to section title 
G-18 

Item Name Institution Role

Page Section Comment 

83 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 34 

TreatmentMeasurement gap 

Treatment: Pain. Suggest adding Dysnea to section title 

84 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 35 Follow-up Follow-up: Dysnea. Suggest adding Dyspnea to section title. 

85 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 36 

Potential Indicators and Measurement Gaps 

"Assessment for treatable causes of dyspnea" I like the measure, buam concerned that there are sufficient data to support a list of defined "treatable" causes. 

86 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 36 

Potential Indicators and Measurement Gaps 

footnote 1, Nice summary and important to include 

87 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 36 

Potential Indicators and Measurement Gaps 

Suggest adding Dyspnea to section title 

88 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 37 Treatment 

"Treatment for malignant pleural effusions. The numerator is patients who have ..." "Bronchoscopic therapy: The numerator is patients who have had..." "Palliative radiation and chemotherapy. The numerator is patients whhave received..." Would it be "who have been offered" to respect patients choice? 
G-19 

Item Name Institution Role

Page Section Comment 

89 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 37 Treatment Treatment: Dyspnea. Suggest adding Dyspnea to title section 

90 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 39 

Measures and Indicators 

Suggest adding Depression to title section. 

91 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 41 

Measures and Indicators 

Suggest adding Advance Care Planning to title section. 

92 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 45 Application Suggest adding Advance Care Planning to title section. 

93 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 46 Follow-up Suggest adding Advance Care Planning to title section. 

94 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 50 First paragraph (Measure- site of death) paragraph. "...(greater than 4%)... Is this number correct? 
G-20 

Item Name Institution Role

Page Section Comment 

95 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 51 Potential Indicators Not ruse what is being addressed here- is this the "gaps" section? Where are the actual potential indicators? Did I miss something? 

96 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 51 Potential Indicators Suggest adding Advance Care Planning to title section. 

97 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 54 Chapter 4 

Last paragraph. " Symptom assessment requires efficient indicators that can become part of the dashboard of quality monitors for care 

settings. Is this a comment enough terminology? 

98 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 55 Chapter 4 

Second Paragraph. Consider the following revision: Pediatric [cancercases represent the second leading cause of death among children 

ages 5-15 in the United States. 

99 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 55 

Chapter 4, Last paragraph 

there is an extra closing parenthesis after dying that need not be ther

100 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 56 

Chapter 4, Second to last paragraph 

It is important to define subgroups of cancer by stage of illness and to 

understand the performance of indicators among major subgroups. Issues such as timeliness of care and how side effects are tolerated omanaged may vary in importance across groups distinguished by typor severity of illness. This seems to be a separate issue than the disparities addressed above, $ 
G-21 

Item Name Institution Role

Page Section Comment 

101 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 56 Chapter 4 

Third paragraph, first sentence. "We found insufficient evidence to comment on important difference by gender, race/ethnicity..." Age, should geriatric population have separate consideration? What about language issues? If measures that require patients repor

are only available in English, are less useful? 

102 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 57 Chapter 4 

Last paragraph. Starting with the sentence "However, unlike other tools..." Should this be separate paragraph? Is really a distinct issue from measurement availability. 

103 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 

Conceptual Issues and processes of care? Quite complete. One area 

that I did not see mentioned (expect in an ACOVE measure related to advance care planning) is the coordination of these symptom management and advance care planning areas across specialty and primary care. This may be beyond the scope of this project, but perhaps worth addressing or at least acknowledging. 

104 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 

How complete is the listing of relevant measures or indicators? Quite complete. Appropriate attention is paid to the need for pediatricspecific measures and those that address potential racial. ethnic disparities. I question whether specific attention needs to e paid to geriatric-specific issues as well. 

105 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP 

The "take home" message being that there are sufficient measures to 

begin assessment/monitoring in these areas, but that additional research is necessary to fill in the gaps. 

106 Jean Kutner, MD, MSPH 

University of Colorado Health Sciences Center 

TEP Overall The Evidence tables are especially well-constructed and useful 
G-22 

Item Name Institution Role

Page Section Comment 

107 Jennifer Malin, MD 

UCLA School of Medicine, RAND 

TEP 32 

Proposed indicator for patient education regarding pain management how would "education" be operationalized? Would this be based upophysician/nurse documentation in the chart? Or would patients' and o

their caregivers be assessed for their knowledge? 

108 Jennifer Malin, MD 

UCLA School of Medicine, RAND 

TEP 32 Why is the procedure-related pain indicator limited to children? Seem

like it should apply to adults also. 

109 Jennifer Malin, MD 

UCLA School of Medicine, RAND 

TEP 36 

Routine dyspnea assessment in cancer - the denominator "all cancer encounters" is too broad - many patients with a diagnosis of cancer wnot have active disease (i.e. routine follow-up of early stage breast cancer treated 5 years ago). Even limiting this indicator to "all encounters in patients with metastatic disease" is probably too broadPerhaps "all encounters in patients with lung involvement with cancer or a prior complaint of dyspnea" would be a reasonable starting place for routine assessment of dyspnea. What would constitute assessment? Is recording of a respiratory rate in the chart assessment or does the clinician need to question the patient regarding the presence or absence of dyspnea? 

110 Jennifer Malin, MD 

UCLA School of Medicine, RAND 

TEP 36 

Assessment for treatable causes of dyspnea - consider making this numerator more specific - perhaps # of patients who have had all of 

the following performed within xx weeks of the complaint of dyspnea: vital signs, lung exam, O2 saturation, Hb, chest imaging, DVT/PE assessment (documentation of physical exam of legs or Doppler/UTZ lower extremities or CT angio or VQ or D-dimer). The other challenge here is longitudinal assessment - what if patient has dyspnea in July and is evaluated and treated for pneumonia with resolution of symptoms and then in September has dyspnea again? 
G-23 

Item Name Institution Role

Page Section Comment 

111 Jennifer Malin, MD 

UCLA School of Medicine, RAND 

TEP 37 

Treatment of malignant pleural effusions. I would urge you to conside

being more cautious in the recommendation of pleurodesis as a qualiindicator. In my own experience, there tends to be overuse not underuse of pleurodesis - use in patients who are not symptomatic or where the production of malignant fluid is occurring at such a rapid rate that pleurodesis cannot be successful. Many of the published reports tend to overstate the effectiveness observed in actual practice and rarely provide the rates of continued success 1-2 months later. I'm attaching a survey of pulmonologists regarding their views of the procedure which I think echoes the uncertainty regarding its effectiveness. 

112 Jennifer Malin, MD 

UCLA School of Medicine, RAND 

TEP 37 

Broncoschopic therapy - other treatment modalities for central airway 

obstruction are not mentioned. Brachytherapy and external beam radiation can often be quite effective and may be more widely available than bronchoscopic therapy. In selected cancers (small cellung cancer, lymphoma), chemotherapy is the most effective approach. Also, steroids may provide significant relief of symptoms and if patients are in a terminal phase of their disease, this may be enough to provide them palliation until death. 

113 Jennifer Malin, MD 

UCLA School of Medicine, RAND 

TEP 37 Patient education - same comment as pain indicator regarding education. 
G-24 

Item Name Institution Role

Page Section Comment 

114 Jennifer Malin, MD 

UCLA School of Medicine, RAND 

TEP 46 

Late life hospital use - it may be worth explaining under what 

circumstances this may or may not be a good quality measure. It likereflects structural as much if not more than process quality - without high quality in home care or inpatient hospices or nursing homes that really address the needs of the dying, hospitalization may actually provide patients with a more comfortable death experience and more support for families. Given the daily capitation rate for hospice, currehome hospice provides very little in the way of support - and is really not viable for patients who do not have informal caregivers or the resources to pay for round the clock care. In addition, there is a catch-22 that as not been fully addressed by the work of the proponents of this and other similar measures: patients hospitalized late in life and/or who die in the hospital is asked may have had a preference for hospitalization at that time. Those same patients may also have had a preference to die at home. However, athe time of hospitalization, they may not yet accepted that death was imminent even if this was communicated to them. I cannot tell you how many times in my own practice, I have talked with patients and ofamily of members of patients at home receiving hospice care while the paramedics were waiting to take them to ER (even after making a home visit earlier in the day...). The issue of the limitations posed by the retrospective "look back" of number of these quality measures should also be addressed (see article by Bach et al.). Research efforts should focus on developing quality measures that allow prospective identification of the denominator, for example by using performance status or some other predictor of death. My own experience caring for patients at the endof-life has been limited to patients with breast cancer. And, except in rare case, I have been struck by how patients have lived with good quality of life and excellent performance status for several years befohaving quite a precipitous decline over 3-6 weeks prior to dying. I imagine that different cancers have different trajectories and that for our quality measures to improve the care of patients we need to be able to account for these differences in disease trajectory -- as well apatient preferences - neither of which can be accomplished with these global "look back" measures. Finally, there may be cultural differences regarding preferences for place of death that have yet to be fully explored. 
G-25 

Item Name Institution Role

Page Section Comment 

115 Jennifer Malin, MD 

UCLA School of Medicine, RAND 

TEP 8 para 2 "Self report" may be jargon and not readily understood by all readers 

it may be worth explaining a few more words the first time you use it. 

116 Jennifer Malin, MD 

UCLA School of Medicine, RAND 

TEP 11 para 2 

We also searched for 'related articles' for citations that described measures..." -- I found this sentence confusing. I was not sure what 'related articles' were or how they pertained to ACOVE. 

117 Jennifer Malin, MD 

UCLA School of Medicine, RAND 

TEP 12 Last paragraph It might be clearer to state "We searched the database of the End of Life Care and Outcomes Project, a systematic review recently completed by this report's authors, for articles coded as 'cancer." 

118 Jennifer Malin, MD 

UCLA School of Medicine, RAND 

TEP 23 Figure only mentions indicators and potential indicators not measures 

119 Jennifer Malin, MD 

UCLA School of Medicine, RAND 

TEP 25 

Consider reminding reader of the distinction between measure and indicator. Also I found the use of the parentheses in the headings somewhat distracting - consider using heading and subheadings: Measures Identified Regular Assessment of Pain: Quality Assessment Tools proposed 

120 Jennifer Malin, MD 

UCLA School of Medicine, RAND 

TEP 25 last paragraph refers to the second UHC "pain assessment indicator

but this appears to be a measure? 

121 Jennifer Malin, MD 

UCLA School of Medicine, RAND 

TEP 26 para 3 

"As with other pain outcome measures..." - Is the proposed measure 

"regular pain assessment," which would be a process measure, or the score (average, final, change in?), which would be an outcome measure? 
G-26 

Item Name Institution Role

Page Section Comment 

122 Jennifer Malin, MD 

UCLA School of Medicine, RAND 

TEP 31 para 2 Have the numerator and denominator been reversed? 

124 Joan Teno, MD, MS Brown Medical School TEP Indicators You need to move away chart based indicators 125 Joan Teno, MD, MS Brown Medical School TEP Literature Search 

The literature searching strategy that restricted everything to cancer 

was unneeded for symptoms, pain, and cancer. They do not need to be cancer specific processes. There is nothing that is inherent to the disease trajectory that warrant a different process of care. So you indicators that could go forward that has been used in other research and quality improvement efforts. 
G-27 

Item Name Institution Role

Page Section Comment 

126 Joan Teno, MD, MS Brown Medical School TEP Measureme nt Set 

My overall assessment is that the current measurement set is very weak and potentially, should not go forward - information on reliability and validity is missing. Even more important, I did not see information on responsiveness, discriminate validity.. all things that are very important prior to the use of accountability. 

127 June Dahl, PhD University of Wisconsin- Madison TEP 31 Pain 

I found the comment at the end of page 31: "improvements in pain assessment by non-physician staff may not lead to improved outcomes because the physician may not attend to the report of pain

I need to explore that reference source because if that statement is true, we are in deep trouble. Physicians rarely see patients in LTC anin the post-op period it is obviously non-physician clinicians who have the responsibility to assess and document pain. 

128 June Dahl, PhD University of Wisconsin- Madison TEP 32 Pain there is a statement: "the reviewers concluded that improved adherence with long-acting drugs in clinical practice improved outcomes." Is there any rationale for this conclusion? What outcome

are the reviewers referring to? 

129 June Dahl, PhD University of Wisconsin- Madison TEP 43 General Edits Data is a plural noun and yet sometimes use as a singular noun, e.g.

line 2 page 43 ...."validity data is available." 
G-28 

Item Name Institution Role

Page Section Comment 

130 June Dahl, PhD University of Wisconsin- Madison TEP General Edits 

 Dangling participles are another issue. I am a purist about those, bu

only mention two. On page 26, line 5: what or who is "using an appropriate rating scale?" On page 49, 10th line from bottom, who is "using the highest performing decile of this measure?" I am certain you will catch the others as you edit. 

131 June Dahl, PhD University of Wisconsin- Madison TEP Pain An important measure identified: how many patients being treated wit

opioids are on a bowel regimen was also identified by the JCAHO- NCQA-AMA Task Force. I think JACHO's field trial showed there walots of room for improvement. 

132 June Dahl, PhD University of Wisconsin- Madison TEP Dana Farber Guidelines 

I was very interested in measures developed at Dana Farber: of new chemotherapy in the last 30 days of life, of patients receiving 

chemotherapy in the last 14 days of life, and late referral to hospice. Unfortunately, in our area, some of the more recently developed chemotherapies are being offered to patients with very advanced disease which results in a delay in their admission to hospice. My comment adds nothing to your review - it is just a reflection of our realities and makes me question how implementing these measures could improve quality given the forces in the oncology profession that are pushing for more therapy. 
G-29 

Item Name Institution Role

Page Section Comment 

133 June Dahl, PhD University of Wisconsin- Madison TEP General Edits Inconsistency in use of terms related to opioid analgesics. Opioid would be preferred, but narcotic appears (e.g., page 27) as does the term opiate (e.g., page 32). 134 June Dahl, PhD University of Wisconsin- Madison TEP Numeric rating scales Is it intended that numeric rating scales be 1-10 (page 26)? The numeric pain rating scale is 0-10. On page 33, there is reference to a 10-point dyspnea scale. Is it indeed a 10 point scale or an 11 point scale? 135 June Dahl, PhD University of Wisconsin- Madison TEP Pain It is interesting that none of the studies you cited used pharmacy department data. If pain is the symptom of interest, one could measure the percentage of patients with a diagnosis of cancer and moderate to severe pain who are/were being treated with opioids. 136 June Dahl, PhD University of Wisconsin- Madison TEP Future research Obviously data collection is a huge challenge. The data have to be easy to obtain. The advent of electronic medical records, should enable collection of accurate data so as to make measurement possible. 137 June Dahl, PhD University of Wisconsin- Madison TEP Pain The cognitively impaired are a special assessment challenge. We need to develop measures that don't rely on self reports if we are to assess pain in that population. We are working on a scale based on items in MDS 2.0; certain behaviors suggest the presence of pain. 
G-30 

Item Name Institution Role

Page Section Comment 

138 June Dahl, PhD University of Wisconsin- Madison TEP Pain 

The critical first step is assessment. And as your report emphasizes, waiting 48 hours to do the assessment makes no sense. I would prefer the JCAHO standard: ask about pain on admission: All patients 

with a diagnosis of cancer who are admitted to hospital should be screened for pain on admission regardless of the reason for their admission. If pain is present, the intensity and quality of the pain should be assessed.The focus in all of the literature available is on assessment of pain intensity. Quality should be assessed as well so as to determine whether the patient has neuropathic pain. A measure of the presence of neuropathic pain is critical to development of an effective treatment plan. Krause and Backonja (Clinical J of Pain, 2003) have developed a short form of a neuropathic pain questionnaire which involves rating of numbness, tingling, increased pain due to touch. A comprehensive pain assessment involves asking questions about quality; should be possible to add two or three boxes to check or numbers to circle to provide data for this assessment. We haven't made an effort to create appropriate forms to make this additional component of assessment a practical reality. 

139 June Dahl, PhD University of Wisconsin- Madison TEP Sources 

The recent recommendations from the American Pain Society (Archives Int Med 2005; 165: 1574-1580) were published after you finished your intensive review. They deal with acute and cancer painthe recommendations for acute pain reflect what was in reference #87 of your draft. 
G-31 

Item Name Institution Role

Page Section Comment 

140 

Laura Hanson, MD, MPH 

University of North Carolina at Chapel Hill 

TEP 

Process vs. Outcome Measures 

Consider adding to the report whether a measure is a process measure which primarily addresses quality of care, vs. an outcome measure which directly addresses the patient experience. 

141 

Laura Hanson, MD, MPH 

University of North Carolina at Chapel Hill 

TEP 

I would encourage addition of the absence of reliable and valid measures for cognitively impaired patients to your lists of gaps in 

quality measurement. 

142 

Laura Hanson, MD, MPH 

University of North Carolina at Chapel Hill 

TEP 

In addition to reporting reliability and validity of individual measures, iwould be very valuable to report on the responsiveness of these measures. It will not be true for all measures, but for any that have repeated measures over time and/or in relations to interventions, adding evidence that a measure has dynamic properties adds its relevance to clinical quality improvement. 
G-32 

Item Name Institution Role

Page Section Comment 

143 

Laura Hanson, MD, MPH 

University of North Carolina at Chapel Hill 

TEP Pain 

The text would benefit from an introduction to measurement issues from pain, the best studied of all these symptoms, so that the measures are in a broader context of certain limitations. I would 

include the following aspects: Measurement systems need to specify exclusion or use of alternative measures for patients unable to self-rate symptoms due to cognitive impairment Effect of depression, symptom interpretation and addiction history mamodify treatment approaches; settings may vary in the frequency with which these effects influence measurement Pain literature shows a discrepancy between pain severity scores and patient satisfaction with pain treatment- perhaps due to low expectations for relief, OR due to positive responses to attention to pain even when it is not fully relieved. You might want to address the dilemma of symptom scores as measures vs. satisfaction with care. 

144 

Marilyn Bookbinder, RN, PhD 

Beth Israel Medical Center 

TEP Future research 

If benchmarks are to represent best practice and serve as the “gold standard” for practitioners to emulate – more rigorous research designs and larger homogeneous samples are needed to strengthen the evidence base and persuade those allocating funds to increase “quality” care research dollars. 

145 

Marilyn Bookbinder, RN, PhD 

Beth Israel Medical Center 

TEP Sources 

Revised indicators for measuring outcomes of pain management havrecently been published (Arch Intern Med. 2005 Jul 25;165(14):1574-80) by the American Pain Society Task Force on Quality. They includitems such as how often the patient had moderate to severe pain in the last 24 hours; how much interference did pain have with their function (able to do what they needed to do) interference with sleep, interference with mood. No data has been published to date, althougwe at Beth Israel and colleagues at University of Wisconsin have data on selected indicators. 
G-33 

Item Name Institution Role

Page Section Comment 

146 

Marilyn Bookbinder, RN, PhD 

Beth Israel Medical Center 

TEP Potential Indicators 

The potential indicators lists are relevant and are an excellent approach to using data available (not meeting the criteria). The indicators selected and approach used to target vulnerable population(DRG, LOS, Adm) is excellent and I think will give others the guidancand tools needed to measure and improve symptom management anACP in problem- prone high risk groups. 

147 

Molla Donaldson, DrPH, MS 

National Cancer Institute (Fed Reviewer) 

TEP 0 Title 

Make the title more specific as there are two other EPC reports on cancer measures - perhaps along the line of Cancer Quality of Care 

Measures for Supportive care and End of Life 

148 

Molla Donaldson, DrPH, MS 

National Cancer Institute (Fed Reviewer) 

TEP 3 Introduction 

The term "utilization outcomes" is a little strange without explanation think of utilization as just that (e.g., admissions, length of stay, hospic

use), so that sentence could be clarified. 

149 

Molla Donaldson, DrPH, MS 

National Cancer Institute (Fed Reviewer) 

TEP 5 Chapter 2 

Under the 1. RFTO question, you might explain that after further discussion distress was dropped in favor of dyspnea - something alonthose lines so that it is clear you were responsive to the task order. 
G-34 

Item Name Institution Role

Page Section Comment 

150 Patricia Ganz, MD 

UCLA Schools of Medicine and Public Health 

TEP 

It looks like you have found quite a few measures, but overall, I was very disappointed to see that so few had been implemented or tested

 Have you contacted the measure developers in follow-up to see if they can give you any information on their use? For that reason, I worry that many of the measures may look good from an academic omeasurement perspective, but may not be ready for prime time. I see this as a real limitation. 

151 Patricia Ganz, MD 

UCLA Schools of Medicine and Public Health 

TEP 

Did you find any references to the pain thermometer scales that are being widely used in hospitals as required for hospital accreditation? We also use them in our outpatient department. I don’t know how reliable and valid they are as measures, but they are now in wide use and seem practical. 

152 Patricia Ganz, MD 

UCLA Schools of Medicine and Public Health 

TEP 

I think Chapter 4 does a good job talking about the limitations of the 

measures and indeed the issue of obtaining self-report data from seriously ill patients is a real challenge. 

153 Patricia Ganz, MD 

UCLA Schools of Medicine and Public Health 

TEP In summary, I think you have done a fine job, but I have concerns about how practical a lot of these measures will be for implementation
G-35 

Item Name Institution Role

Page Section Comment 

154 Perry Fine, MD University of Utah TEP 11 Internet Search Under the section listing "Studies or Internet documents....", the first bullet point omits Canadian references. Was this purposeful or an unintended omission? 155 Perry Fine, MD University of Utah TEP 32 Treatment- Measureme nt gap 

Might I suggest that a more broadly applicable potential indicator 

(measurement gap) would be (numerator) patients with moderatesevere pain who receive prescriptions for opioid analgesics (denominator = all patients with moderate-severe pain scores). This might be simplified further to the % of all cancer patients in a given practice who are prescribed opioids, with a benchmark extrapolated from exigent literature on the prevalence of moderate-severe pain in cancer patients. 

156 Perry Fine, MD University of Utah TEP 40 Depression I was very surprised to see that there were no potential indicators/ measurement gaps enumerated for depression. Is there really no bas

to elaborate upon this section (assessment, treatment and follow-up)? 

157 Perry Fine, MD University of Utah TEP Chapter 4 

You rightly elaborate upon the clinically significant and problematic area of pain assessment in non-self-reporting patients. There may be the opportunity to round out the pain assessment section with additional potential indicators from (non-cancer) dementia studies. I would refer you to Dr. Keela Herr, who chairs the gerontological nursing division in the U. of Iowa College of Nursing, whose area of 

focus is pain assessment in cognitively impaired older patients. She may be able to point quickly to a rich little vein of literature that was neasily uncovered in your search. Her email address is keelaherr@uiowa.edu 
G-36 

Item Name Institution Role

Page Section Comment 

158 Roger Winn, PhD National Quality Forum TEP Overall 

I think the reviewers have done an admirable job of collecting availabmeasures and analyzing them in a well conceptualized framework. I found the division between measures, indicators, and potential indicators useful and operationally sound. The analyses of the specifimeasures appears to be balanced and to target the major strengths 

and weaknesses. The conclusions of the report are justified by the body of evidence reported. 

159 Roger Winn, PhD National Quality Forum TEP 1 1 Mention scope: limited to pain, dyspnea, depression, advanced directives 160 Roger Winn, PhD National Quality Forum TEP 3 para 2, line 13 toxic instead of taxing 161 Roger Winn, PhD National Quality Forum TEP 7 para 2, line 2-3 I think this statement is a little too strong, For example the number of trained social workers or psychologists might have an impact on depression. I think it’s more the case that it’s difficult to directly establish the linkages between structure and outcomes. 
G-37 

Item Name Institution Role

Page Section Comment 

162 Roger Winn, PhD National Quality Forum TEP 7 para 2 

Although I think it’s implied, you might want to point out that for symptom-related measures, the data collection is usually a patient response instrument, which from a measure perspective is much harder than trying to capture data from administrative sources or evethe medical record. {I see you make the point later in the report} 

163 Roger Winn, PhD National Quality Forum TEP 16 1 I don't understand the decision not to abstract JCAHO, ORYX, or NCQA measures of relevance 164 Roger Winn, PhD National Quality Forum TEP 34 ACOVE effective dyspnea measure: I’m confused about the numerator: shouldn’t the denominator be all patients with dyspnea? 165 Roger Winn, PhD National Quality Forum TEP 39 ACOVE spiritual assessment: Numerator and denominator definitions are reversed 
G-38 

Item Name Institution Role

Page Section Comment 

166 Roger Winn, PhD National Quality Forum TEP 41 

ACOVE surrogates for poor cognition measure: how does the measuoperationalize “dementia and altered mental state” to determine the denominator? How is impairment assessed, i.e standard tests, etc? 

Similar consideration for regular assessment of inpatients with severe dementia (p 42). 

167 Roger Winn, PhD National Quality Forum TEP 46 ACOVE documentation of specific life sustaining preferences. Why is the denominator restricted to those who state they do not want life- prolonging measures? Shouldn’t the preference of those who do wan

aggressive therapy also be documented? 

168 Roger Winn, PhD National Quality Forum TEP 46 

ACOVE consistency of preferences and mechanical ventilation. 1) As 

stated, the numerator doesn’t seem to get at the consistency (concordance) issue: isn’t the measure all those who are receiving ventilation in the denominator and all those who stated they did not want it in the numerator? 2) The discussion appears to be about withdrawal of ventilation not its institution. 

169 Roger Winn, PhD National Quality Forum TEP 47 Dana Farber hospital use and ICU use measures: both discussions quote the norm in terms of use of emergency visits. This applies only to the ER measure. 
G-39 

Item Name Institution Role

Page Section Comment 

170 Roger Winn, PhD National Quality Forum TEP 48 Dana Farber new chemo measure: does it exclude acute leukemias (10-15%) die from drug toxicity and hospitals with large leukemia practices might therefore suffer from this measure. 171 Roger Winn, PhD National Quality Forum TEP 49 Dana Farber admission to hospice measure: the Earle paper gives reasons for the low sensitivity (24%) and this should probably be included. 172 Roger Winn, PhD National Quality Forum TEP 49 

 Site of death Dana Farber late referral to hospice: does this need to be a composite measure with the % hospice admission? If two hospitals both have 9% late admissions of all the patients who die, i.epoor quality, is this the same measure of quality for hospital A who has a 50% hospice admission rate and hospital B who has a (% admission rate (all late)? 

173 Roger Winn, PhD National Quality Forum TEP 50 Dana Farber site of death , line 7 should be “(greater than 17%)” not 4% 174 Roger Winn, PhD National Quality Forum TEP 51 Need NEJM reference for Danis. 
G-40 

Item Name Institution Role

Page Section Comment 

175 Roger Winn, PhD National Quality Forum TEP 54 para 4 line 1: I really have trouble understanding: “Symptom assessment requires efficient indicators that can become part of the dashboard of quality monitors for care settings". What are efficient indicators? 176 Steve Asch, MD, MPH VAGLHS TEP No comments received. 177 

William Lawrence, MD 

AHRQ TOO 5 Task Order questions 

 I think this would read easier to start with the final key questions in th

task order questions section (if you’d like, you can even change the subhead to final questions). I’d rather the emphasis for the report were placed on the questions you actually answer in the report, not thones that you were originally asked. If you’d like, I think it would be quite reasonable to put the initial questions (unbolded) in the approacto the report section, if you want to detail how we arrived at the final questions. At the moment, the only stand-alone key questions sectiolists questions that you only answer some of. 

178 

William Lawrence, MD 

AHRQ TOO 24 Pain the mention of an XRT measure in relation to pain was a bit confusing – I presume this was palliative XRT specifically? 

179 

William Lawrence, MD 

AHRQ TOO 25 Pain 

First measure (as general comment). “The measure is not in use” – I 

appreciate having this bit of data, but this means not in use anywhereNot in use by the organization that has proposed it? 
G-41 

Item Name Institution Role

Page Section Comment 

180 

William Lawrence, MD 

AHRQ TOO 37 Dyspnea 

 treatment of malignant pleural effusions. Despite having worked in a cancer center, I don’t have enough clinical experience here to judge, so I’ll ask you – does this happen with enough frequency that you thinit will be usable as a quality measure (i.e. will the rates be made from enough cases for a particular entity, be it practice, hospital, health plan, that the rate is stable enough to judge quality from the number)? This would get at the importance of the measure. 

181 

William Lawrence, MD 

AHRQ TOO 39 Depression 

assessment of psychosocial well-being. Is “formal assessment” defined? Is a screening survey sufficient? Must it be an assessment by a psychologic specialist? 

182 

William Lawrence, MD 

AHRQ TOO 39 Depression 

Spiritual assessment. Looks like you have your numerator and denominator backward. Also, I presume the numerator is specifically those with a spiritual assessment and have died? Otherwise, numerator is not a subset of the denominator. 

183 

William Lawrence, MD 

AHRQ TOO 40 ACP first measure. Watch the measure/indicator terminology – you refer 

this first as a measure, then as an indicator. 

184 

William Lawrence, MD 

AHRQ TOO 40 Gaps 

 Would appreciate this section sooner or later, preferably sooner. While I’m thinking about it, I thought your gaps sections (at least the ones that exist) were excellent in both content and writing. 
G-42 

Item Name Institution Role

Page Section Comment 

185 

William Lawrence, MD 

AHRQ TOO 41 ACP 

regular identification of surrogate. Numerator is “vulnerable” elderly. How do we define vulnerable? This section in general seems to have a several terms with a similar degree of vagueness. 

186 

William Lawrence, MD 

AHRQ TOO 42 ACP 2nd regular identification of surrogate. Is the numerator a subset of th

denominator here? 
G-43 

Item Name Institution Role

Page Section Comment 

187 

William Lawrence, MD 

AHRQ TOO 42 ACP last measure – how do we assess that a patient had care consistent with expressed preferences? 

188 

William Lawrence, MD 

AHRQ TOO 46 ACP 

documentation of specific life sustaining preferences. Not sure why the denominator is only those who don’t want therapies (although, if this is what the measure says, we go with it, but might deserve comment). Also, numerator and denominator could be from different sets. 
G-44 

Item Name Institution Role

Page Section Comment 

189 

William Lawrence, MD 

AHRQ TOO 47 ACP 

DFCC measure set – you refer to ER visits for discussions of hospitalizations and ICU episodes. Also, while I could grasp your ratof the 5th percentile/95th percentile for a measure of variability of the 

measure, I think it would be helpful to put in the raw rates here also. 

190 

William Lawrence, MD 

AHRQ TOO 48 ACP chemotherapy in last 30 days of life. Does this include Phase I trials? Should it? Is palliative chemotherapy used this late? 

191 

William Lawrence, MD 

AHRQ TOO 49 ACP 

admission to hospice and others. Is there a definition of “deceased cancer patients”? Is a breast cancer survivor NED for 5 years who dies of a heart attack a deceased cancer patient? 

192 

William Lawrence, MD 

AHRQ TOO 51 ACP potential indicators – a reference to the appendix would be nice here. 
G-45 

Item Name Institution Role

Page Section Comment 

193 

William Lawrence, MD 

AHRQ TOO 54 

Chapter 4, 3rd paragraph 

Can we say anything from the extensive amount of data that you 

reviewed concerning the frequency with which pain is assessed and appropriately treated? I realize that by virtue of the settings being quality measure testing settings, they might not be generalizable, but presumably they could at least be considered an upper bound of whahappens in reality. 

194 

William Lawrence, MD 

AHRQ TOO 55 

Chapter 4, second paragraph 

Agree with the issue of no measures in pediatric cancer. It might be worthwhile echoing this in some of the gaps sections in the prior paragraphs. 
G-46 

Item Name Institution Role

Page Section Comment 

195 

William Lawrence, MD 

AHRQ TOO 56 

Chapter 4, 1st paragraph 

Also agreed on the issue of the measures not providing guidance. M

question for you is whether this is a downside of the measure. Ideallya quality measure is linked to a leverage point that clinicians, health systems, or someone else should be able to impact upon both to change the measure but ultimately to improve outcomes. For me as clinician, it would be much easier to implement change if I have parameters with which to act in accordance with the quality measure and presumably in accordance with quality care. If the trigger is that should take action 3 weeks prior to the person dying, it is somewhat hard to operationalize in my practice. I agree that the ACOVE measures have a better conceptual basis here, although again it woube helpful to understand what defines a vulnerable elder (if a definitiohas been specified in the measure). 

196 

William Lawrence, MD 

AHRQ TOO 56 

Chapter 4, 3rd paragraph 

The lack of information on race, gender, etc, is not particularly surprising, but does anyone at all publish rates on their measures by these categories? If so, it would be worth mentioning. 

197 

William Lawrence, MD 

AHRQ TOO 

25 and on 

Minor editorial comment – could you reference the appropriate appendix containing the evidence tables at the intro of the sections, 

simply to note that there is more detailed info on the discussed measures. That way, if people want a formal definition of numerator, denominator, or want to know how data are abstracted, they’ll know where to look. 

198 

William Lawrence, MD 

AHRQ TOO 

Front Matter 

Don’t forget abstract, executive summary, etc. Would also appreciate an acknowledgement of the Federal Partners – in addition to AHRQ, they include NCI, CMS, and CDC. 
G-47 

Item Name Institution Role

Page Section Comment 

199 

William Lawrence, MD 

AHRQ TOO Measures 

Also, general issue on the reports of measures – at times, you put in rates for the measures (e.g. second measure on page 25), which is aissue in testing (has someone actually put the measure out in the field 

and tested it enough to be able to get a rate?). It might be reasonablto consistently comment on rates, or might want to put up front that if you don’t comment on rates, there aren’t any (or whatever the conditions are for your not including these data). Also, should they appear in the evidence tables? 

200 

William Lawrence, MD 

AHRQ TOO Evidence Tables could you spell out Advanced Care Planning in the title? To me, ACP is American College of Physicians 

201 

William Lawrence, MD 

AHRQ TOO Organization 

I liked the organization of the measures/indicators by Assessment/Treatment/Follow-up – that seems a very intuitive approach to dividing the measures. 

202 

William Lawrence, MD 

AHRQ TOO 

Defining difference between measures and indicators 

In the results section, you divide things into measures and indicators, but I haven’t found where you define the difference between the two. Could you make this clear in text? Similarly, you have in your abstraction sheets a working definition of a quality measure, which I didn’t see in text in the intro – might be worth adding. 

203 

William Lawrence, MD 

AHRQ TOO Evidence Tables 

Incidental note after comparing page 39 text to evidence tables – it would be helpful if you could have the order of measures in the evidence table match the order in text – much easier to find a measuin the appendix that way. 
G-48 

Item Name Institution Role

Page Section Comment 

204 

William Lawrence, MD 

AHRQ TOO Table 1 

Table 1. I like this table. Having just finished commenting on the draof colorectal measures report recently, they found a lot of literature reports on one or two measures (e.g. percent of stage III colorectal cancer patients receiving adjuvant chemotherapy) in a report. In yours, it appears that most or all of the measures/indicators are individual items from much broader measure sets. Is this true? Don’necessarily need to comment on this in the report, but I found the difference quite interesting 

205 

William Lawrence, MD 

AHRQ TOO Evidence Tables 

While it is out of chronologic order, would suggest for the evidence tables that you list denominator first, then numerator, since typically 

the numerator should be a subset of the denominator. This will a) save you repeating the denominator in the numerator cell in the tableand b) help make it easier to spot when the numerator isn’t a subset (happened at least once – I’ll find it in my notes), which should at leasraise a red flag. 

206 Castillo, DonnaRae AHRQ Title The title is not descriptive enough. Suggest a change to “Quality Measures for Supportive Cancer Care” or something very similar as you mention in your text. 207 Castillo, DonnaRae AHRQ Title page It is unclear who the actual authors are. I am assuming the two PIs and the two directors. This needs to be clear (for reference purposes)208 Castillo, DonnaRae AHRQ Contents 

This needs to be restructured. List the chapters in order, then 

References, Figures, Tables, Appendixes. (The last are merely listed with their titles; no page numbers for Appendixes (note it is “Appendixes” not Appendices] as these are only online.) Note also figures and tables are not listed within chapters. 
G-49 

Item Name Institution Role

Page Section Comment 

209 Castillo, DonnaRae AHRQ Contents We normally only use the first and, if necessary, second level of headings. Otherwise it becomes too unwieldy (see reworking of Chapter 3 contents, below). 210 Castillo, DonnaRae AHRQ Contents 

Remove the line “Areas of Study” from the Contents for Chapter 3. It is not in text and is superfluous. Here is Chapter 3 using 1st and 2nd level heads: Chapter 3: Results..................................................  Overview...........................................................  Literature Flow..................................................  Pain....................................................................  Introduction................................................... 

 Measures and Indicators................................  Potential Indicators and Measurement Gaps..  Dyspnea................................................................  Introduction.................................................... 

 Measures and Indicators.................................  Potential Indicators and Measurement Gaps..  Depression............................................................  Introduction.................................................... 

 Measures and Indicators.................................  Potential Indicators and Measurement Gaps..  Advance Care Planning........................................  Introduction.................................................... 

 Measures and Indicators.................................  Potential Indicators and Measurement Gaps..  Summary............................................................... You may want to only use the chapter title and 1st level since the sections are short. 
G-50 

Item Name Institution Role

Page Section Comment 

211 Castillo, DonnaRae AHRQ General Format and Style Report proper begins on page 1 (arabic not roman). All chapters open on a right hand page 212 Castillo, DonnaRae AHRQ General Format and Style First level headings (next level down from the chapter title) are centered 213 Castillo, DonnaRae AHRQ General Format and Style Appendixes, tables and figures are numbered in the order they are cited in the text. 214 Castillo, DonnaRae AHRQ General Format and Style Please do not leave space between paragraphs. The indention is enough to mark the beginning of new paragraph. The extra leading wastes space. 215 Castillo, DonnaRae AHRQ General Format and Style 

Do not use bold to emphasize a word in text; italics is sufficient and it 

should only be used when defining a term e.g., “An indicator was defined as a descriptive statement...”). Italics should not be used repeatedly. For example, ACOVE or Dana Farber should not be emphasized whenever they are noted as the measure source and so forth. 

216 Castillo, DonnaRae AHRQ General Format and Style 

Titles of published books listed in running text should be in italics. (Sefor example p. xxxi, 1st full paragraph and p. xv, 2nd full paragraph.) 

This includes the title of your evidence report/summary in the run-on sentence at the bottom of p. xv; suggest you rewrite as: The authors had recently conducted a systematic review, End-of-Life Care and Outcomes.21 The project’s database was searched for articles coded 

as “cancer”—regardless of type—by our topics of interest.) 

217 Castillo, DonnaRae AHRQ General Format and Style First word of a bulleted item is capitalized. Generally, periods end eac

bulleted item (this is mainly a Web convention for compliance and readability). 
G-51 

Item Name Institution Role

Page Section Comment 

218 Castillo, DonnaRae AHRQ General Format and Style If quotations are necessary they should be double, not single marks (e.g., “living wills” on p. vi). Often, however, these marks are not needed. 219 Castillo, DonnaRae AHRQ General Format and Style As a courtesy to the reader, spell out any acronym when it is introduced in the report (e.g., SCEPC on p. viii, DNR and DNI on p. xv). Do not, however, spell out acronyms in every chapter or in every section within chapters. 220 Castillo, DonnaRae AHRQ General Format and Style The following are printed solid: followup (used as a noun or adjective), contraindicated, decisionmaking 221 Castillo, DonnaRae AHRQ General Format and Style When used as a reflexive pronoun, self is hyphenated (e.g., self- expression, self-reported, etc.) 222 Castillo, DonnaRae AHRQ General Format and Style Health care is two words (e.g. “health care measurement” or “health care system”) unless used as an established title (e.g., “Agency for Healthcare Research and Quality”). 223 Castillo, DonnaRae AHRQ General Format and Style There is no space on either side of the solidus mark; i.e., Summary/Conclusion and 5/95 are correct (not Summary / Conclusion or 5 / 95). 224 Castillo, DonnaRae AHRQ General Format and Style Use the TM and ® symbols when required; for example, GoogleTM CINAHL® PsychINFO® HEDIS® 225 Castillo, DonnaRae AHRQ General Format and Style If two or more numerals are used in conjunction and at least 1 of them is 10 or more, make them all numerals (as in this sentence). Thus we would not say “In nine of 10 trials...” 
G-52 

Item Name Institution Role

Page Section Comment 

226 Castillo, DonnaRae AHRQ General Format and Style The word State is initial cap when referring to a U.S. State (this is U.S

govt. style). 

227 Castillo, DonnaRae AHRQ General Format and Style It is not necessary to name books or articles if you are providing reference citations (see p. xxxi for example; you already cite the reference #71.) 228 Castillo, DonnaRae AHRQ Chapter 1 I suggest that either here (or alternatively at the beginning of the next chapter), you distinguish between measure and indicator as you are using them. This is not totally clear. 229 Castillo, DonnaRae AHRQ Chapter 2 

On current p. viii where you list your two key questions, I suggest you either put the a/b/c of question 1 with the main part of the question or break out the two parts of question 2 after the word “including” as “a” and “b” in order to be consistent in format with question 1. In other words, question 2 formatted like question 1 would read:2. What gaps in knowledge about quality measurement are evident from the currently available literature, including: a. Absence of measures or b. 

Measures lacking evidence of their scientific soundness, whether for the population of cancer patients as a whole or for specific populations. 

230 Castillo, DonnaRae AHRQ Chapter 1 

Table 1 is mentioned at top of current p. x as a list of organizations. I 

am assuming this is the Table 1, now in Chapter 3, that lists the organizations and the measures they are supplying. thus this table should be moved to Chapter 2. Table 1 and Table 2 can be placed athe end of this chapter so as not to interfere with the running text. If this current Table 1 is not the one you reference on p. x, that table needs to be supplied. 
G-53 

Item Name Institution Role

Page Section Comment 

231 Castillo, DonnaRae AHRQ Chapter 2 

As noted above, appendixes are numbered in the order cited. As cite

in the text of Chapter 2, the order should be: A. Technical Expert Panel and Peer Reviewers B. TEP Worksheet C. Literature Search Strategies (NOTE: This is one appendix, Appendix C, not Appendixes C; the individual databases are not C-1, C-2 etc.). D. Contact Form E. Article Screener 

232 Castillo, DonnaRae AHRQ Chapter 2 

At the top of p. xiv, to make more typographically consistent, I suggesyou reformat as the end of the sentence just before the bulleted items as follows: The main search strategy included an extensive list of terms intended to identify research publications associated with all of the following: · Each of the domains of interest (pain, dyspnea, depression, advanccare planning). · Quality assessment or improvement. · Quality measurement. 

233 Castillo, DonnaRae AHRQ Chapter 2 The hyperlinks on pp. xiv and xv should be removed. (okay to leave the underlining). 234 Castillo, DonnaRae AHRQ Chapter 3 

Headings should be consistent in wording and format. For example, the Pain section, one of the headings is “Potential Measures & 

Measurement Gaps”; in the Dyspnea section, the equivalent heading reads “Potential Measures / Measurement Gaps”; and in the Advance Care section it is just “Potential Measures.” They should all read Potential Measures and Measurement Gaps 
G-54 

Item Name Institution Role

Page Section Comment 

235 Castillo, DonnaRae AHRQ Chapter 3 

As noted above, if the reference to Table 1 is to the Table 1 mentionein Chapter 2, it needs to be moved and the title should reflect the measure source; for example: Table 1. Sources and attributes of 

identified measures and indicators 

236 Castillo, DonnaRae AHRQ Chapter 3 

Run-in headings should be bold, not italics or parenthesized, and followed by a period. Do not make the beginning of the sentence bolFor example, in the Pain section, the first paragraph under “Measures and Indicators” in Chapter 3, p. xxviii, begins with the following heading and 1st sentence:(Measure - regular assessment of pain) Quality Assessment Tools (QA Tools), proposed in 200062 includes one measure relevant to pain assessment.This is awkward and not according to our required style. It should read:Measure: Regular assessment of pain. Quality Assessment Tools (QA Tools), proposed in 2000,62 includes one measure relevant to pain assessment.All the measure or indicator names would be set up the same way. 

237 Castillo, DonnaRae AHRQ Chapter 3 

Move the name of the second UHC measure on p. xxviii to make it a heading, for typographical consistency. For example: Measure: Routine inpatient pain assessment. (for the first measure) 

Measure: Routine inpatient pain assessment with a numeric scale. (for the second measure) This would apply also on p. xliii to the second ACOVE measure undeTreatment of Depression. 

238 Castillo, DonnaRae AHRQ Chapter 3 

Under the “Treatment” section on p. xxx, the text mentions “3 measures and 1 indicator”; however the text in that section (see pp. xxx and xxxi” presents 2 measures and 2 indicators. This discrepancmust be reconciled; either the intro sentence or a run-in heading is wrong. 
G-55 

Item Name Institution Role

Page Section Comment 

239 Castillo, DonnaRae AHRQ Chapter 3 

Under the “Followup” section of Advance Care Planning, the intro sentence mentions 14 measures in 10 categories and 7 indicators in 

categories. The 10 general measure categories are easily seen; the indicators are not. For those measures that also that include a similaindicator, we should modify the run-in heading to include the notationFor example the measure noted as “Measure: late life hospital use” includes a similar indicator. Let’s change the heading to: Measure/Indicator: Late life hospital use. (or alternatively, but less preferable, Measure and indicator: Late life hospital use.) 

240 Castillo, DonnaRae AHRQ Chapter 3 

In addition, I count 8, not 7, indicators listed within this section as 

follows (page nos. in parens):  1) late life hospital use (p. xlix)  2) late life ICU use (p. l)  3) late life rate of emergency care (also p. l)  4) new chemotherapy regimen in last 30 days of life (p. li)  5) chemotherapy in last 14 days of life (also p. li)  6) admission to hospice (p. lii)  7) site of death (p. liii) The above 7 are within the “measure” paragraphs; but there is also  8) care consistency with documented care preferences (p liii) If this is correct, please change the intro paragraph to read 8 indicatoin 8 categories. 

241 Castillo, DonnaRae AHRQ Chapter 4 This should include Discussion as part of the chapter title. 242 Castillo, DonnaRae AHRQ References References is the correct title of this section 
G-56 

Item Name Institution Role

Page Section Comment 

243 Castillo, DonnaRae AHRQ Excluded Studies Since this is one of the last reports under the “old” style guide, it is appropriate to list alphabetically the excluded studies (your current Appendix J as listed in the Contents could fill this need here.) 244 Castillo, DonnaRae AHRQ Appendixes 

As noted these should be reordered and numbered in the order of thecitation in text. As of now, this is the order of their call-out in the text: 

A. Technical Expert Panel and Peer Reviewers B. TEP Worksheet C. Literature Search Strategies (NOTE: This is one appendix, Appendix C, not Appendixes C; the individual databases are not C-1, C-2 etc.). D. Contact Form E. Article Screener F. Evidence Tables 

245 Castillo, DonnaRae AHRQ Appendix A: Please make sure you have permission from the individuals to be listed to include their names in the report. 246 Castillo, DonnaRae AHRQ Appendix B: Title should be Appendix C: Literature Search Strategies. 247 Castillo, DonnaRae AHRQ Evidence Tables 

 Title should be Appendix F. Evidence Tables. Table titles should be where the distinctions are made. For example: Evidence Table F.1. Quality Measures and Indicators for Supportive Cancer Care: Pain. 
G-57 

Item Name Institution Role

Page Section Comment 

248 Castillo, DonnaRae AHRQ Other Appendixes 

· Perhaps I overlooked them but I could not find any text reference to several appendixes (G-L) currently listed in your Contents table. These appendixes are not here. · Is the “Cambridge Ballot” necessary? · I have already mentioned Appendix J above which can become the List of Excluded Studies. · What is the purpose of separating observational evidence tables 

currently noted as Appendix L from the others currently noted as Appendix E (which will become App F)? Can these be combined? Whare the “Observational Studies” necessary anyway if they are not discussed? · Appendix H should be one appendix, not two, entitled Appendix H: Methodological Issues in Measurement or, if you prefer, Appendix H: Reliabiliy,Validity, and Other Measurement Issues. Include the table within the appendix. · Delete Appendix K. Including peer review comments is unnecessarThis is prepublication information. They belong in your file, not with a published report. 
H-1 

Appendix H. Excluded Studies 

Rejected: Duplicate Data 

 1. Appendix A: panel rating sheets by condition. Kerr, E. A.; Asch, S. M.; Hamilton, E. G., and McGlynn, E. A. 

Quality of care for cardiopulmonary conditions: a review of the literature and quality indicators. 

RAND; 2000; pp. 251-84. 

Rec #: 739 

 2. Appendix B: crosswalk table to original and final indicators. Kerr, E. A.; Asch, S. M.; Hamilton, E. G., and 

McGlynn, E. A. Quality of care for cardiopulmonary conditions: a review of the literature and quality 

indicators . RAND; 2000; pp. 285-327. 

Rec #: 740 
H-2 

Rejected: Non-Western Location

 1. Auchus, A. P. and Chen, C. P. Asia-Pacific Consensus Statement on dementia. Alzheimer Dis Assoc Disord. 

2001 Apr-2001 Jun 30; 15(2):63-5. 

Rec #: 635 

 2. Howland SC. Managing the Alzheimer's patient. Emerg Med Serv. 2000 Sep; 29(9):89-92. 

Rec #: 32435 

 3. Ministry of Health, Singapore. Cancer pain. 2003 Mar(Clnical Practice Guildelines). 

Rec #: 207 

 4. National Comprehensive Cancer Network (NCCN). Colon cancer. Clinical Practice Guidelines in Oncology. 

2005; (2). 

Rec #: 128 

 5. Palliative Care and Critical Care Committees. Removal of mechanical ventilation in the dying patient. 

Guidelines for Physician Staff Froedtert Hospital, Milwaukee, Wisconsin :1-4. 

Rec #: 136 

 6. Tomita T and Kitamura T. Clinical and research measures of grief: a reconsideration. Compr Psychiatry. 2002 

Mar-2002 Apr 30; 43(2):95-102. 

Rec #: 15879 
H-3 

Rejected: No Domain of Interest

 1. American Academy of Hospice and Palliative Medicine (AAHPM). Position statements: comprehensive endof-life care and physician-assisted suicide [Web Page]. 1997 Jun 25; Accessed 2005 Jan 24. 

Rec #: 483 

 2. American Academy of Hospice and Palliative Medicine (AAHPM). Position statements: statement on the 

ethics of palliative care research [Web Page]. 2002 Jan 30; Accessed 2005 Jan 24. 

Rec #: 482 

 3. American Academy of Hospice and Palliative Medicine (AAHPM). Position statements: statement on the use 

of nutrition and hydration [Web Page]. 2001 Nov 16; Accessed 2005 Jan 24. 

Rec #: 481 

 4. American Academy of Hospice and Palliative Medicine (AAHPM). Positions statements: statement on 

sedation at the end-of-life [Web Page]. 2002 Sep 13; Accessed 2005 Jan 24. 

Rec #: 480 

 5. American Board of Family Practice. Carcinoma of the breast. Reference Guide. 2001. 

Rec #: 465 

 6. Bast, R. C. Jr; Ravdin, P.; Hayes, D. F.; Bates, S.; Fritsche, H. Jr; Jessup, J. M.; Kemeny, N.; Locker, G. Y.; 

Mennel, R. G., and Somerfield, M. R. 2000 update of recommendations for the use of tumor markers 

in breast and colorectal cancer: clinical practice guidelines of the American Society of Clinical 

Oncology. J Clin Oncol. 2001 Mar 15; 19(6):1865-78. 

Rec #: 174 

 7. Benson, A. B. 3rd; Schrag, D.; Somerfield, M. R.; Cohen, A. M.; Figueredo, A. T.; Flynn, P. J.; 

Krzyzanowska, M. K.; Maroun, J.; McAllister, P.; Van Cutsem, E.; Brouwers, M.; Charette, M., and 

Haller, D. G. American Society of Clinical Oncology recommendations on adjuvant chemotherapy 

for stage II colon cancer. J Clin Oncol. 2004 Aug 15; 22(16):3408-19. 

Rec #: 170 

 8. Bramwell, V. H. C.; Anderson, D.; Charette, M. L., and Sarcoma Disease Site Group. Doxorubicin-based 

chemotherapy for the palliative treatment of adult patients with locally advanced or metastatic soft 

tissue sarcoma. Practice Guideline Report #11-1. 1999(Program in Evidence-Based Care). 

Rec #: 152 

 9. Browning, D. M. and Solomon, M. Z. The initiative for pediatric palliative care: an interdisciplinary 

educational approach for health care professionals. 

Rec #: 755 

 10. Burt, R. A. The limitations of protocols for end-of-life care. Respir Care. 2000 Dec; 45(12):1523-9; 

discussion 1529-33. 

Rec #: 223 

 11. Byock, I.; Norris, K.; Curtis, J. R., and Patrick, D. L. Improving end-of-life experience and care in the 

community: a conceptual framework. J Pain Symptom Manage. 2001 Sep; 22(3):759-72. 

Rec #: 642 

 12. Cherny, N. I.; Catane, R.; Kosmidis, P. A., and Members of the ESMO Palliative Care Working Group. 

ESMO policy on supportive and palliative care. European Society for Medical Oncology (ESMO). 

2003:1-5. 

Rec #: 139 
H-4 

 13. Chow, T. W. and MacLean, C. H. Quality indicators for dementia in vulnerable community-dwelling and 

hospitalized elders. Ann Intern Med. 2001 Oct 16; 135(8 Pt 2):668-76. 

Rec #: 645 

 14. Cluzeau, F. A.; Littlejohns, P.; Grimshaw, J. M.; Feder, G., and Moran, S. E. Development and application of 

a generic methodology to assess the quality of clinical guidelines. Int J Qual Health Care. 1999 Feb; 

11(1):21-8. 

Rec #: 432 

 15. Cummings, J. L.; Frank, J. C.; Cherry, D.; Kohatsu, N. D.; Kemp, B.; Hewett, L., and Mittman, B. Guidelines 

for managing Alzheimer's disease: part I. Assessment. Am Fam Physician. 2002 Jun 1; 65(11):2263-

72. 

Rec #: 633 

 16. Daly, B. J. Organizational change and delivery of multidisciplinary palliative care. Respir Care. 2000 Dec; 

45(12):1501-10; discussion 1510-2. 

Rec #: 221 

 17. Davidson, K. Family bereavement support before and after the death of a nursing home resident. University 

of Iowa Gerontological Nursing Interventions Research Center (RDC); 2002. 

Rec #: 713 

 18. Dudley, R. A.; Bowers, L. V., and Luft, H. S. Reconciling quality measurement with financial risk adjustment 

in health plans. Jt Comm J Qual Improv. 2000 Mar; 26(3):137-46. 

Rec #: 427 

 19. European Society for Medical Oncology (ESMO). Minimum clinical recommendations for diagnosis, 

adjuvant treatment and follow-up of primary breast cancer. ESMO Guidelines Task Force; 2003. 

Rec #: 117 

 20. --. Minimum clinical recommendations for diagnosis, treatment and follow-up of advanced colorectal 

cancer. ESMO Guidelines Task Force; 2002. 

Rec #: 115 

 21. European Society for Medical Oncology (ESMO). Minimum clinical recommendations for diagnosis, 

treatment and follow-up of Ewing's sarcoma of bone. ESMO Guidelines Task Force; 2002. 

Rec #: 118 

 22. European Society for Medical Oncology (ESMO). Minimum clinical recommendations for diagnosis, 

treatment and follow-up of locally recurrent or metastatic breast cancer (MBC): ESMO Guidelines 

Task Force; 2003. 

Rec #: 114 

 23. --. Minimum clinical recommendations for diagnosis, treatment and follow-up of non-small-cell lung cancer 

(NSCLC). ESMO Guidelines Task Force; 2003. 

Rec #: 116 

 24. --. Minimum clinical recommendations for diagnosis, treatment and follow-up of osteosarcoma : ESMO 

Guidelines Task Force; 2002. 

Rec #: 111 

 25. European Society for Medical Oncology (ESMO). Minimum clinical recommendations for diagnosis, 

treatment and follow-up of small-cell lung cancer (SCLC): ESMO Guidelines Task Force; 2003. 

Rec #: 112 
H-5 

 26. European Society for Medical Oncology (ESMO). Minimum clinical recommendations for the diagnosis, 

treatment and follow-up of prostate cancer: ESMO Guidelines Task Force; 2003. 

Rec #: 113 

 27. Fahey, T.; Montgomery, A. A.; Barnes, J., and Protheroe, J. Quality of care for elderly residents in nursing 

homes and elderly people living at home: controlled observational study. BMJ. 2003 Mar 15; 

326(7389):580. 

Rec #: 631 

 28. Fowler, F. J. Jr.; Coppola, K. M., and Teno, J. M. (Center for Survey Research, University of Massachusetts 

at Boston, 100 Morrissey Boulevard, Boston, MA 02125-3393.). Methodological challenges for 

measuring quality of care at the end of life. Journal of Pain and Symptom Management. 1999 Feb; 

17(2):114-9. 

Rec #: 343 

 29. Gagliardi, A. R.; Fung, M. F. K.; Langer, B.; Stern, H., and Brown, A. D. Development of ovarian cancer 

surgery quality indicators using a modified delphi approach. Ottawa, Ontario, Canada. 

Rec #: 726 

 30. Giordano, M. The respiratory therapist and palliative care. Respir Care. 2000 Dec; 45(12):1468-74; 

discussion 1475-7. 

Rec #: 218 

 31. Gralla, R. J.; Osoba, D.; Kris, M. G.; Kirkbride, P.; Hesketh, P. J.; Chinnery, L. W.; Clark-Snow, R.; Gill, D. 

P.; Groshen, S.; Grunberg, S.; Koeller, J. M.; Morrow, G. R.; Perez, E. A.; Silber, J. H., and Pfister, 

D. G. Recommendations for the use of antiemetics: evidence-based, clinical practice guidelines. 

American Society of Clinical Oncology. J Clin Oncol. 1999 Sep; 17(9):2971-94. 

Rec #: 162 

 32. Hillner, B. E.; Ingle, J. N.; Chlebowski, R. T.; Gralow, J.; Yee, G. C.; Janjan, N. A.; Cauley, J. A.; 

Blumenstein, B. A.; Albain, K. S.; Lipton, A., and Brown, S. American Society of Clinical Oncology 

2003 update on the role of bisphosphonates and bone health issues in women with breast cancer. J 

Clin Oncol. 2003 Nov 1; 21(21):4042-57. 

Rec #: 462 

 33. Holtzman, J.; Degelau, J.; Meyers, R.; Christianson, C., and Lurie, N. (Institute of Health Services Research, 

School of Public Health, Box 197 Mayo, 420 Delaware Street S.E., Minneapolis, MN 55455.). 

Development and testing of a process measure of nursing home quality of care. Journal of the 

American Geriatrics Society. 1997 Oct; 45(10):1203-7. 

Rec #: 397 

 34. Hunt et al. ACC/AHA guidelines for the evaluation and management of chronic heart failure in the adult: a 

report of the American College of Cardiology/American Heart Association task force on practice 

guidlines (Committee to reise the 1995 guidelines for the evaluation and management of heart 

failure). ACC/AHA Practice Guidelines. 2001:1-55. 

Rec #: 120 

 35. Jacobsen, P. B.; Donovan, K. A.; Trask, P. C.; Fleishman, S. B.; Zabora, J.; Baker, F., and Holland, J. C. 

Screening for psychological distress in ambulatory cancer patients: a multi-center evaluation of the 

distress thermomenter. 

Rec #: 487 

 36. Jennings, B. M.; Loan, L. A.; DePaul, D.; Brosch, L. R., and Hildreth, P. Lessons learned while collecting 

ANA indicator data. J Nurs Adm. 2001 Mar; 31(3):121-9. 

Rec #: 554 
H-6 

 37. Jett, J. R.; Scott, W. J.; Rivera, M. P., and Sause, W. T. Guidelines on treatment of stage IIIB non-small cell 

lung cancer. Chest. 2003 Jan; 123(1 Suppl):221S-225S. 

Rec #: 164 

 38. Kahn, K. L.; Malin, J. L.; Adams, J., and Ganz, P. A. Developing a reliable, valid, and feasible plan for 

quality-of-care measurement for cancer: how should we measure? Med Care. 2002 Jun; 40(6 

Suppl):III73-85. 

Rec #: 643 

 39. Levetown, M. ; Dokken, D.; Heller, K. S., and et al. for IPPC. A Pediatric Palliative Care Institutional SelAssessment Tool (ISAT). Education Development Center, Inc. 2002. 

Rec #: 749 

 40. Loblaw, D. A.; Mendelson, D. S.; Talcott, J. A.; Virgo, K. S.; Somerfield, M. R.; Ben-Josef, E.; Middleton, 

R.; Porterfield, H.; Sharp, S. A.; Smith, T. J.; Taplin, M. E.; Vogelzang, N. J.; Wade, J. L. Jr; 

Bennett, C. L., and Scher, H. I. American Society of Clinical Oncology recommendations for the 

initial hormonal management of androgen-sensitive metastatic, recurrent, or progressive prostate 

cancer. J Clin Oncol. 2004 Jul 15; 22(14):2927-41. 

Rec #: 475 

 41. Lynn, J.; Nolan, K.; Kabcenell, A.; Weissman, D.; Milne, C., and Berwick, D. M. Reforming care for persons 

near the end of life: the promise of quality improvement. Ann Intern Med. 2002 Jul 16; 137(2):117-

22. 

Rec #: 637 

 42. MacIntyre, N. R.; Cook, D. J.; Ely, E. W. Jr; Epstein, S. K.; Fink, J. B.; Heffner, J. E.; Hess, D.; Hubmayer, 

R. D., and Scheinhorn, D. J. Evidence-based guidelines for weaning and discontinuing ventilatory 

support: a collective task force facilitated by the American College of Chest Physicians; the 

American Association for Respiratory Care; and the American College of Critical Care Medicine. 

Chest. 2001 Dec; 120(6 Suppl):375S-95S. 

Rec #: 470 

 43. Malin, J. L.; Asch, S. M.; Kerr, E. A., and McGlynn, E. A. Evaluating the quality of cancer care: development 

of cancer quality indicators for a global quality assessment tool. Cancer. 2000 Feb 1; 88(3):701-7. 

Rec #: 641 

 44. Multinational Association of Supportive Care in Cancer (MASCC). Consensus Conference on Antiemetic 

Therapy. Perugia International Cancer Conference VII; Perugia. 2004 Jun. 

Rec #: 169 

 45. National Comprehensive Cancer Network (NCCN). Acute Myeloid Leukemia. National Comprehensive 

Cancer Network; 2005 Mar 4(Practice Guidelines in Oncology; version 2.2005). 

Rec #: 658 

 46. National Comprehensive Cancer Network (NCCN). Antiemesis. National Comprehensive Cancer Network 

(NCCN); 2005 Jan 6(Practice Guidelines in Oncology; version 1.2005). 

Rec #: 699 

 47. --. Basal cell and squamous cell skin cancers. National Comprehensive Cancer Network (NCCN); 2004 Jan 

5(Practice Guidleines in Oncology; version 1.2004). 

Rec #: 685 

 48. --. Bladder cancer. National Comprehensive Cancer Network; 2005 Mar 4(Practice Guidelines in Oncology; 

version 1.2005). 

Rec #: 659 
H-7 

 49. ---. Bone cancer. Clinical Practice Guidelines in Oncology. 2005; (1). 

Rec #: 122 

 50. ---. Breast cancer. Clinical Practice Guidelines in Oncology. 2005; (1). 

Rec #: 126 

 51. --. Breast cancer risk reduction. National Comprehensive Cancer Network (NCCN); 2004 May 3(Practice 

Guidleines in Oncology; version 1.2004). 

Rec #: 692 

 52. --. Breat cancer screening and diagnosis. National Comprehensive Cancer Network (NCCN); 2004 Jul 

13(Practice Guidleines in Oncology; version 1.2004). 

Rec #: 693 

 53. --. Cancer and treatment-related anemia. National Comprehensive Cancer Network (NCCN); 2004 Dec 

21(Practice Guidleines in Oncology; version 2.2004). 

Rec #: 698 

 54. --. Cancer-related fatigue. National Comprehensive Cancer Network (NCCN); 2004 Apr 19(Practice 

Guidleines in Oncology; version 1.2004). 

Rec #: 700 

 55. National Comprehensive Cancer Network (NCCN). Cervical cancer. National Comprehensive Cancer 

Network; 2004 Oct 28(Practice Guidelines in Oncology; version 1.2004). 

Rec #: 663 

 56. National Comprehensive Cancer Network (NCCN). Cervical screening. National Comprehensive Cancer 

Network (NCCN); 2004 Oct 1(Practice Guidelines in Oncology; version 1.2005). 

Rec #: 694 

 57. --. Colorectal screening. National Comprehensive Cancer Network (NCCN); 2004 Aug 27(Practice 

Guidleines in Oncology; version 1.2004). 

Rec #: 695 

 58. --. Dermatofibrosarcoma protuberans. National Comprehensive Cancer Network (NCCN); 2004 Jan 

5(Practice Guidleines in Oncology; version 1.2004). 

Rec #: 686 

 59. National Comprehensive Cancer Network (NCCN). Distress management. 2005(Clinical Practice 

Guidelines in Oncology). 

Rec #: 707 

 60. ---. Distress management: clinical practice guidelines. JNCCN. 2003; 1:344-74. 

Rec #: 486 

 61. National comprehensive Cancer Network (NCCN). Esophageal cancer. National Comprehensive Cancer 

Network; 2005 Jan 1(Practice Guidleines in Oncology; version 1.2005). 

Rec #: 665 

 62. National Comprehensive Cancer Network (NCCN). Fever and Neutropenia. National Comprehensive 

Cancer Network (NCCN); 2004 Nov 1(Practice Guidelines in Oncology; version 1.2004). 

Rec #: 703 
H-8 

 63. --. Genetic/familial high-risk assessment. National Comprehensive Cancer Network (NCCN); 2005 Feb 

9(Practice Guidleines in Oncology; version 1.2005). 

Rec #: 696 

 64. National Comprehensive Cancer Network (NCCN). Head and neck cancers. National Comprehensive 

Cancer Network; 2005 Feb 7(Practice Guidelines in Oncology; version 1.2005). 

Rec #: 668 

 65. National Comprehensive Cancer Network (NCCN). Hepatobiliary cancers. National Comprehensive Cancer 

Network; 2005 Feb 1(Practice Guidelines in Oncology; version 1.2005). 

Rec #: 669 

 66. --. Hodgkin's disease. National Comprehensive Cancer Network; 2004 Aug 10(Practice Guidelines in 

Oncology; version 1.2004). 

Rec #: 670 

 67. National Comprehensive Cancer Network (NCCN). Melanoma. National Comprehensive Cancer Network; 

2004 Nov 1(Practice Guidelines in Oncology; version 1.2004). 

Rec #: 672 

 68. National Comprehensive Cancer Network (NCCN). Merkel cell carcinoma. National Comprehensive 

Cancer Network (NCCN); 2004 Jun(Practice Guidleines in OncologyPractice Guidleines in 

Oncology; version 1.2004). 

Rec #: 687 

 69. National Comprehensive Cancer Network (NCCN). Myelodysplastic syndoromes. National Comprehensive 

Cancer Network; 2004 Sep 9(Practice Guidelines in Oncology; version 1.2005). 

Rec #: 673 

 70. National Comprehensive Cancer Network (NCCN). Neuroendocrine tumors. National Comprehensive 

Cancer Network (NCCN); 2004 Oct 6(Practice Guidelines in Oncology; version 1.2004). 

Rec #: 677 

 71. --. Non-Hodgkin's Lymphoma. National Comprehensive Cancer Network (NCCN); 2005 Feb 8(Practice 

Guidelines in Oncology; version 1.2005). 

Rec #: 678 

 72. ---. Non-small cell lung cancer. Clinical Practice Guidelines in Oncology. 2005; (2). 

Rec #: 124 

 73. --. Occult Primary. National Comprehensive Cancer Network (NCCN); 2004 Nov 8(Practice Guidleines in 

Oncology; version 1.2005). 

Rec #: 680 

 74. --. Ovarian cancer . National Comprehensive Cancer Network (NCCN); 2004 Nov 1(Practice Guidleines in 

Oncology; version 1.2005). 

Rec #: 681 

 75. --. Pancreatic Adenocarcinoma. National Comprehensive Cancer Network (NCCN); 2004 Nov 1(Practice 

Guidleines in Oncology; version 1.2004 ). 

Rec #: 682 

 76. ---. Prostate cancer. Clinical Practice Guidelines in Oncology. 2004; (1). 

Rec #: 125 
H-9 

 77. --. Prostate cancer early detection. National Comprehensive Cancer Network (NCCN); 2004 Mar 3(Practice 

Guidleines in Oncology; version 1.2004). 

Rec #: 697 

 78. ---. Rectal cancer. Clinical Practice Guidelines in Oncology. 2005; (1). 

Rec #: 127 

 79. ---. Small cell lung cancer. Clinical Practice Guidelines in Oncology. 2005; (1). 

Rec #: 123 

 80. --. Soft tissue sarcoma. National Comprehensive Cancer Network (NCCN); 2005 Feb 16(Practice 

Guidleines in Oncology; version1.2005). 

Rec #: 684 

 81. --. Testicular Cancer. National Comprehensive Cancer Network (NCCN); 2004 Oct 18(Practice Guidleines 

in Oncology; version 1.2005). 

Rec #: 689 

 82. --. Thyroid carcinoma. National Comprehensive Cancer Network (NCCN); 2005 Jan 31(Practice Guidelines 

in Oncology; version 1.2005). 

Rec #: 690 

 83. --. Uterine cancer. National Comprehensive Cancer Network (NCCN); 2004 Oct 20(Practice Guidleines in 

Oncology; version 1.2004). 

Rec #: 691 

 84. National Hospice and Palliative Care Organization (NHPCO). Patient core measure sheet. 

http://www.nhpco.org/files/public/patientcoremeasuresheet.pdf. 

Rec #: 748 

 85. National Institute for Clinical Excellence. Improving supportive and palliative care for adults with cancer. 

National Institute for Clinical Excellence; 2004 Mar(Guidance on Cancer Services). 

Rec #: 466 

 86. Nelson, J. F. and Meier, D. E. Palliative care in the intensive care unit: part I. J Intensive Care Med. 1999; 

14:130-9. 

Rec #: 484 

 87. Oncology Nursing Society and Geriatric Oncology Consortium. Oncology nursing society and geriatric 

oncology consortium joint position on cancer care in the older adult. Oncology Nursing Forum. 

2004; 31(3):1-2. 

Rec #: 119 

 88. Oncology Nusing Society. Oncology nursing society and association of oncology social work joint position 

on end-of-life care; 2003(Oncology Nursing Society Position). 

Rec #: 141 

 89. Ozer, H.; Armitage, J. O.; Bennett, C. L.; Crawford, J.; Demetri, G. D.; Pizzo, P. A.; Schiffer, C. A.; Smith, 

T. J.; Somlo, G.; Wade, J. C.; Wade, J. L. 3rd; Winn, R. J.; Wozniak, A. J., and Somerfield, M. R. 

2000 update of recommendations for the use of hematopoietic colony-stimulating factors: evidencebased, clinical practice guidelines. American Society of Clinical Oncology Growth Factors Expert 

Panel. J Clin Oncol. 2000 Oct 15; 18(20):3558-85. 

Rec #: 159 
H-10 

 90. Palliative Care and Ethics Committees and Department/Divisions of Neurology, Geriatrics Radiology 

Gastroenterology. Non-oral hydration and feeding in advanced dementia or at the end of life. 

Guidlines for Physician Staff Froedtert Hospital, Milwaukee, Wisconsin:1-5. 

Rec #: 132 

 91. Palliative Care Working Group (Canadian Strategy for Cancer Control). Canadian strategy for cancer 

control: palliative care working group final report. 2002 Jan. 

Rec #: 140 

 92. Recht, A.; Edge, S. B.; Solin, L. J.; Robinson, D. S.; Estabrook, A.; Fine, R. E.; Fleming, G. F.; Formenti, S.; 

Hudis, C.; Kirshner, J. J.; Krause, D. A.; Kuske, R. R.; Langer, A. S.; Sledge, G. W. Jr; Whelan, T. 

J., and Pfister, D. G. Postmastectomy radiotherapy: clinical practice guidelines of the American 

Society of Clinical Oncology. J Clin Oncol. 2001 Mar 1; 19(5):1539-69. 

Rec #: 463 

 93. Reifel, J. L. Colorectal cancer treatment. Asch, S. M.; Kerr, E. A.; Hamilton, E. G.; Reifel, J. L., and 

McGlynn, E. A. Quality of care for oncologic condition and HIV: a review of the literature and 

quality indicators. RAND; 2000; pp. 77-111. 

Rec #: 705 

 94. ---. Prostate cancer treatment. Asch, S. M.; Kerr, E. A.; Hamilton, E. G.; Reifel, J. L., and McGlynn, E. A. 

Quality of care for oncologic condition and HIV: a review of the literature and quality indicators. 

RAND; 2000; pp. 189-215. 

Rec #: 706 

 95. Rizzo, J. D.; Lichtin, A. E.; Woolf, S. H.; Seidenfeld, J.; Bennett, C. L.; Cella, D.; Djulbegovic, B.; Goode, 

M. J.; Jakubowski, A. A.; Lee, S. J.; Miller, C. B.; Rarick, M. U.; Regan, D. H.; Browman, G. P., 

and Gordon, M. S. Use of epoetin in patients with cancer: evidence-based clinical practice guidelines 

of the American Society of Clinical Oncology and the American Society of Hematology. J Clin 

Oncol. 2002 Oct 1; 20(19):4083-107. 

Rec #: 161 

 96. Robinson, L. A.; Wagner, H. Jr, and Ruckdeschel, J. C. Treatment of stage IIIA non-small cell lung cancer. 

Chest. 2003 Jan; 123(1 Suppl):202S-220S. 

Rec #: 165 

 97. Rubenfeld, G. D. Withdrawing life-sustaining treatment in the intensive care unit. Respir Care. 2000 Nov; 

45(11):1399-407, discussion 1408-10. 

Rec #: 216 

 98. Rubenstein, E. B.; Peterson, D. E.; Schubert, M.; Keefe, D.; McGuire, D.; Epstein, J.; Elting, L. S.; Fox, P. 

C.; Cooksley, C., and Sonis, S. T. Clinical practice guidelines for the prevention and treatment of 

cancer therapy--induced oral and gastronintestinal mucositis. Cancer. 2004; 100(9 Suppl):2026-46. 

Rec #: 168 

 99. Schuchter, L. M.; Hensley, M. L.; Meropol, N. J., and Winer, E. P. 2002 update of recommendations for the 

use of chemotherapy and radiotherapy protectants: clinical practice guidelines of the American 

Society of Clinical Oncology. J Clin Oncol. 2002 Jun 15; 20(12):2895-903. 

Rec #: 160 

 100. Scottish Intercollegiate Guidelines Network (SIGN). Breast cancer in women. National Clincal Guidelines. 

1998 Oct; 29. 

Rec #: 464 
H-11 

 101. Sessa, C.; Pampallona, S.; Carobbio, M.; Neuenschwander, H., and Cavalli, F. Palliative care of cancer 

patients: audit of current hospital procedures. Support Care Cancer. 1998 May; 6(3):266-72. 

Rec #: 445 

 102. Shaw, P. and Agarwal, R. Pleurodesis for malignant pleural effusions. Cochrane Database Syst Rev. 2004; 

(1):CD002916. 

Rec #: 403 

 103. Simon, G. R. and Wagner, H. Small cell lung cancer. Chest. 2003 Jan; 123(1 Suppl):259S-271S. 

Rec #: 166 

 104. Solomon, D. H.; Wenger, N. S.; Saliba, D.; Young, R. T.; Adelman, A. M.; Besdine, R. K.; Blazer, D. G.; 

Cassell, C. K.; Cummings, J. L.; Katz, P. R.; Kitzman, D. W.; Lavizzo-Mourey, R. J.; Mondoux, L. 

C.; Popovich, R.; Pories, W. J., and Wenger, N. Appropriateness of quality indicators for older 

patients with advanced dementia and poor prognosis. J Am Geriatr Soc. 2003 Jul; 51(7):902-7. 

Rec #: 632 

 105. Stuart B; D'Onofrio CN; Boatman S, and Feigelman G. CHOICES: promoting early access to end-of-life care 

through home-based transition management. J Palliat Med. 2003 Aug; 6(4):671-83. 

Rec #: 7422 

 106. Takada, K.; Aksentijevich, I.; Mahadevan, V.; Dean, J. A.; Kelley, R. I., and Kastner, D. L. (National 

Institute of Arthritis and Musculoskeletal and Skin Diseases, NIH, Bethesda, MD.). Favorable 

preliminary experience with etanercept in two patients with the hyperimmunoglobulinemia D and 

periodic fever syndrome. Arthritis & Rheumatism. 2003 Sep; 48(9):2645-51. 

Rec #: 324 

 107. Taylor, A. and Box, M. Multicultural palliative care guidelines. Australia: Palliative Care Australia; 

1999(Palliative Care Australia). 

Rec #: 476 

 108. The National Quality Forum (NQF). A comprehensive framework for hospital care performance evaluation: 

a consensus report. National Quality Forum; 2003. 

Rec #: 489 

 109. Thompson, B. T.; Cox, P. N.; Antonelli, M.; Carlet, J. M.; Cassell, J.; Hill, N. S.; Hinds, C. J.; Pimentel, J. 

M.; Reinhart, K., and Thijs, L. G. Challenges in end-of-life care in the ICU: statement of the 5th 

International Consensus Conference in Critical Care: Brussels, Belgium, April 2003: executive 

summary. Crit Care Med. 2004 Aug; 32(8):1781-4. 

Rec #: 638 

 110. Tolson, D.; Smith, M., and Knight, P. An investigation of the components of best nursing practice in the care 

of acutely ill hospitalized older patients with coincidental dementia: a multi-method design. J Adv 

Nurs. 1999 Nov; 30(5):1127-36. 

Rec #: 578 

 111. Viles, L. Death and the practitioner. Respir Care. 2000 Dec; 45(12):1513-9; discussion 1520-2. 

Rec #: 222 

 112. Wallace SK; Ewer MS; Price KJ, and Feeley TW. Outcome and cost implications of cardiopulmonary 

resuscitation in the medical intensive care unit of a comprehensive cancer center. Support Care 

Cancer. 2002 Jul; 10(5):425-9. 

Rec #: 622 
H-12 

 113. Wu, A. W.; Gifford, A.; Asch, S.; Cohn, S. E.; Bozzette, S. A., and Yurk, R. Quality-of-care indicators for 

HIV/AIDS. Dis Manage Health Outcomes. 2000 Jun; 7(6):315-30. 

Rec #: 731 
H-13 

Rejected: Disease Not Cancer

 1. Beal, A. C.; Co, J. P.; Dougherty, D.; Jorsling, T.; Kam, J.; Perrin, J., and Palmer, R. H. Quality measures for 

children's health care. Pediatrics. 2004 Jan; 113(1 Pt 2):199-209. 

Rec #: 757 

 2. Buchanan RJ; Choi M; Wang S, and Huang C. Analyses of nursing home residents in hospice care using the 

minimum data set. Palliat Med. 2002 Nov; 16(6):465-80. 

Rec #: 354 

 3. Chodosh, J.; Ferrell, B. A.; Shekelle, P. G., and Wenger, N. S. Quality indicators for pain management in 

vulnerable elders. Ann Intern Med. 2001 Oct 16; 135(8 Pt 2):731-5. 

Rec #: 647 

 4. Chodosh, J.; Solomon, D. H.; Roth, C. P.; Chang, J. T.; MacLean, C. H.; Ferrell, B. A.; Shekelle, P. G., and 

Wenger, N. S. The quality of medical care provided to vulnerable older patients with chronic pain. J 

Am Geriatr Soc. 2004 May; 52(5):756-61. 

Rec #: 627 

 5. Chou, R.; Clark, E., and Helfand, M. Comparative efficacy and safety of long-acting oral opioids for chronic 

non-cancer pain: a systematic review. J Pain Symptom Manage. 2003 Nov; 26(5):1026-48. 

Rec #: 238 

 6. Foster, R. L. and Varni, J. W. (School of Nursing, C288-10, University of Colorado, Health Sciences Center, 

4200 East Ninth Avenue, Denver, CO 80262.). Measuring the quality of children's postoperative pain 

management: initial validation of the Child/Parent Total Quality Pain Management (TQPM) 

instruments. Journal of Pain and Symptom Management. 2002 Mar; 23(3):201-10. 

Rec #: 310 

 7. Kalso, E.; Allan, L.; Dellemijn, P. L.; Faura, C. C.; Ilias, W. K.; Jensen, T. S.; Perrot, S.; Plaghki, L. H., and 

Zenz, M. Recommendations for using opioids in chronic non-cancer pain. Eur J Pain. 2003; 7(5):381-

6. 

Rec #: 243 

 8. Standards and Quality Committee and Council of Palliative Care Australia. Resource manual for palliative 

care performance indicators in Australia. Australia; 1998 Jun(Palliaitve Care Australia). 

Rec #: 478 
H-14 

Rejected: No Indicators

 1. Cancer care during the last phase of life. J Clin Oncol. 1998 May; 16(5):1986-96. 

Rec #: 106 

 2. Definitions related to the use of opioids for the treatment of pain. WMJ. 2001; 100(5):28-9. 

Rec #: 202 

 3. Organizational assessment: personnel competency/performance. Supportive Care of the Dying: A Coalition 

for Compassionate Care:1-4. 

Rec #: 468 

 4. Organizational Assessment: system grid and assumptions. Supportive Care of the Dying: A Coalition for 

Compassionate Care. 2001:1-9. 

Rec #: 467 

 5. Quality improvement guidelines for the treatment of acute pain and cancer pain. American Pain Society 

Quality of Care Committee. JAMA. 1995 Dec 20; 274(23):1874-80. 

Rec #: 296 

 6. . Questions and answers on breast cancer: guideline 10- the management of persistant pain. 2001 Jul 3. 

Rec #: 727 

 7. Abrahm, J. L. Management of pain and spinal cord compression in patients with advanced cancer. ACPASIM End-of-life Care Consensus Panel. American College of Physicians-American Society of 

Internal Medicine. Ann Intern Med. 1999 Jul 6; 131(1):37-46. 

Rec #: 278 

 8. Addington-Hall, J.; Altmann, D., and McCarthy, M. Which terminally ill cancer patients receive hospice inpatient care? Social Science & Medicine. 1998; 46(8):1011-6. 

Rec #: 623 

 9. Addington-Hall, J. and McCarthy, M. Dying from cancer: results of a national population-based 

investigation. Palliat Med. 1995 Oct; 9(4):295-305. 

Rec #: 301 

 10. Ahmedzai, S. Recent clinical trials of pain control: impact on quality of life. Eur J Cancer. 1995; 31A Suppl 

6:S2-7. 

Rec #: 306 

 11. Ahronheim, J. C.; Morrison, R. S.; Baskin, S. A.; Morris, J., and Meier, D. E. Treatment of the dying in the 

acute care hospital. Advanced dementia and metastatic cancer. Arch Intern Med. 1996 Oct 14; 

156(18):2094-100. 

Rec #: 606 

 12. Allcock, N.; McGarry, J., and Elkan, R. Management of pain in older people within the nursing home: a 

preliminary study. Health Soc Care Community. 2002 Nov; 10(6):464-71. 

Rec #: 251 

 13. Allen, R. S.; DeLaine, S. R.; Chaplin, W. F.; Marson, D. C.; Bourgeois, M. S.; Dijkstra, K., and Burgio, L. D. 

Advance care planning in nursing homes: correlates of capacity and possession of advance 

directives. Gerontologist. 2003 Jun; 43(3):309-17. 

Rec #: 524 
H-15 

 14. Alpert, H. R.; Hoijtink, H.; Fischer, G. S., and Emanuel, L. Psychometric analysis of an advance directive. 

Med Care. 1996 Oct; 34(10):1057-65. 

Rec #: 607 

 15. American Academy of Pain Medicine (AAPM). Acute pain and cancer pain. 1998. 

Rec #: 204 

 16. American Academy of Pain Medicine (AAPM). Basic principles of ethics for the practice of pain medicine. 

1999. 

Rec #: 203 

 17. American Acaemy of Hospice and Pallliative Medicine (AAHPM). Position statements: definitions of 

palliative care and palliative medicine [Web Page]. Accessed 2005 Jan 24. 

Rec #: 479 

 18. American Geriatrics Society Clinical Practice, Committee. Management of cancer pain in older patients. 

Journal of the American Geriatrics Society. 1997; 45(10):1273-1276 URL http //www.blackwellscience.com/~cgilib/bsinc.bin?Journal=geriatrics. 

Rec #: 375 

 19. American Pain Society (APS). Guideline for the management of cancer pain in adults and children. 2005. 

Rec #: 651 

 20. --. Principles of analgesic use in the treatment of acute pain and cancer pain. 5th Ed. ed.; 2003. 

Rec #: 650 

 21. Arcangeli, G.; Giovinazzo, G.; Saracino, B.; D'Angelo, L.; Giannarelli, D.; Arcangeli, G., and Micheli, A. 

Radiation therapy in the management of symptomatic bone metastases: the effect of total dose and 

histology on pain relief and response duration. Int J Radiat Oncol Biol Phys. 1998 Dec 1; 

42(5):1119-26. 

Rec #: 285 

 22. Bach, P. B.; Schrag, D., and Begg, C. B. Resurrecting treatment histories of dead patients: a study design that 

should be laid to rest. JAMA. 2004 Dec 8; 292(22):2765-70. 

Rec #: 758 

 23. Baer, W. M. and Hanson, L. C. (Division of General Medicine, School of Medicine, University of North 

Carolina, Chapel Hill, North Carolina.). Families' perception of the added value of hospice in the 

nursing home. Journal of the American Geriatrics Society. 2000 Aug; 48(8):879-82. 

Rec #: 338 

 24. Bailly, D. J. and DePoy, E. Older people's responses to education about advance directives. Health Soc Work. 

1995 Aug; 20(3):223-8. 

Rec #: 615 

 25. Barnato, A. E. and Angus, D. C. Value and role of intensive care unit outcome prediction models in end-oflife decision making. Crit Care Clin. 2004 Jul; 20(3):345-62, vii-viii. 

Rec #: 512 

 26. Barton, M. B.; Dawson, R.; Jacob, S.; Currow, D.; Stevens, G., and Morgan, G. Palliative radiotherapy of 

bone metastases: an evaluation of outcome measures. J Eval Clin Pract. 2001 Feb; 7(1):47-64. 

Rec #: 269 

 27. BC Cancer Agency. Cancer management guidelines [Web Page]. Accessed 2005 May 3. 

Rec #: 744 
H-16 

 28. Beeber, L. S.; Shea, J., and McCorkle, R. The Center for Epidemiologic Studies Depression Scale as a 

measure of depressive symptoms in newly diagnosed patients. Journal of Psychosocial Oncology. 

1998; 16(1):1-20 URL http //www.haworthpressinc.com/store/product.asp?sku=J077. 

Rec #: 388 

 29. Bell, R.; Eccleston, C., and Kalso, E. Ketamine as an adjuvant to opioids for cancer pain. Cochrane Database 

Syst Rev. 2003; (1):CD003351. 

Rec #: 250 

 30. Benditt, J. O. Noninvasive ventilation at the end of life. Respir Care. 2000 Nov; 45(11):1376-81; discussion 

1381-4. 

Rec #: 214 

 31. Bentley, D. W.; Bradley, S.; High, K.; Schoenbaum, S.; Taler, G., and Yoshikawa, T. T. Practice guideline 

for evaluation of fever and infection in long-term care facilities. J Am Geriatr Soc. 2001 Feb; 

49(2):210-22. 

Rec #: 556 

 32. Berghmans, R. L. Ethical hazards of the substituted judgement test in decision making concerning the end of 

life of dementia patients. Int J Geriatr Psychiatry. 1997 Mar; 12(3):283-7. 

Rec #: 602 

 33. Bernabei, R.; Gambassi, G.; Lapane, K.; Landi, F.; Gatsonis, C.; Dunlop, R.; Lipsitz, L.; Steel, K., and Mor, 

V. Management of pain in elderly patients with cancer. SAGE Study Group. Systematic Assessment 

of Geriatric Drug Use via Epidemiology. JAMA. 1998 Jun 17; 279(23):1877-82. 

Rec #: 286 

 34. Berry, P. H. Getting ready for JCAHO--just meeting the standards or really improving pain management. Clin 

J Oncol Nurs. 2001 May-2001 Jun 30; 5(3):110-2. 

Rec #: 255 

 35. Blasi, Z. V.; Hurley, A. C., and Volicer, L. (Northeastern University, School of Nursing, Boston, MA.). Endof-life care in dementia: a review of problems, prospects, and solutions in practice. Journal of the 

American Medical Directors Association. 2002 Mar-2002 Apr 30; 3(2):57-65. 

Rec #: 557 

 36. Bookbinder, M.; Coyle, N.; Kiss, M.; Goldstein, M. L.; Holritz, K.; Thaler, H.; Gianella, A.; Derby, S.; 

Brown, M.; Racolin, A.; Ho, M. N., and Portenoy, R. K. Implementing national standards for cancer 

pain management: program model and evaluation. J Pain Symptom Manage. 1996 Dec; 12(6):334-

47; discussion 331-3. 

Rec #: 292 

 37. Bottomley, A. (Coordinator, Quality of Life, European Organization for Research and Treatment of Cancer, 

avenue E. Mounier 83 Bte 11, 1200 Brussels, Belgium.). Depression in cancer patients: a literature 

review. European Journal of Cancer Care. 1998 Sep; 7(3):181-91. 

Rec #: 414 

 38. Bradley EH; Hallemeier AG; Fried TR; Johnson-Hurzeler R; Cherlin EJ; Kasl SV, and Horwitz SM. 

Documentation of discussions about prognosis with terminally ill patients. Am J Med. 2001 Aug 15; 

111(3):218-23. 

Rec #: 520 

 39. Breen, C. M.; Abernethy, A. P.; Abbott, K. H., and Tulsky, J. A. Conflict associated with decisions to limit 

life-sustaining treatment in intensive care units. J Gen Intern Med. 2001 May; 16(5):283-9. 

Rec #: 546 
H-17 

 40. Briggs, L. A.; Kirchhoff, K. T.; Hammes, B. J.; Song, M., and Colvin, E. R. (Assistant Director for Advance 

Care Planning, Gundersen Lutheran Medical Foundation, 1836-1910 South Ave., La Crosse, 

Wisconsin 54601; labriggs@gundluth.org.). Patient-centered advance care planning in special 

patient populations: a pilot study. Journal of Professional Nursing. 2004 Jan-2004 Feb 28; 20(1):47-

58. 

Rec #: 540 

 41. Butow, P. N.; Maclean, M.; Dunn, S. M.; Tattersall, M. H., and Boyer, M. J. The dynamics of change: cancer 

patients' preferences for information, involvement and support. Annals of Oncology. 1997; 8(9):857-

63. 

Rec #: 50312 

 42. Campbell, M. L. and Guzman, J. A. A proactive approach to improve end-of-life care in a medical intensive 

care unit for patients with terminal dementia. Crit Care Med. 2004 Sep; 32(9):1839-43. 

Rec #: 499 

 43. Cancer Care Ontario. Cancer system quality index: cancer in Ontario- the facts. Ontario, Canada: Cancer 

Care Ontario; 2005. 

Rec #: 734 

 44. Caraceni, A.; Cherny, N.; Fainsinger, R.; Kaasa, S.; Poulain, P.; Radbruch, L., and De Conno, F. (Neurology 

Unit, Rehabilitation and Palliative Care Unit, National Cancer Institute of Milan, via Venezian 1, 

20133 Milan, Italy.). Pain measurement tools and methods in clinical research in palliative care: 

recommendations of an expert working group of the European Association of Palliative Care. 

Journal of Pain and Symptom Management. 2002 Mar; 23(3):239-55. 

Rec #: 309 

 45. Carlson, J.; Youngblood, R.; Dalton, J. A.; Blau, W., and Lindley, C. (School of Nursing, CB# 7460, 

Carrington Hall, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599-7460.). Is 

patient satisfaction a legitimate outcome of pain management? Journal of Pain and Symptom 

Management. 2003 Mar; 25(3):264-75. 

Rec #: 327 

 46. Carr, D. B. The development of national guidelines for pain control: synopsis and commentary. Eur J Pain. 

2001; 5 Suppl A:91-8. 

Rec #: 259 

 47. Centeno-Cortes C and Nunez-Olarte JM. Questioning diagnosis disclosure in terminal cancer patients: a 

prospective study evaluating patients' responses. Palliat Med. 1994; 8(1):39-44. 

Rec #: 10138 

 48. Center for Practical Bioethics. Guidelines for the determination of decisional incapacity. Center for 

Practical Bioethcis; 1996. 

Rec #: 710 

 49. Chan A and Woodruff RK. Comparison of palliative care needs of English- and non-English-speaking 

patients. J Palliat Care. 1999 Spring; 15(1):26-30. 

Rec #: 409 

 50. Chang, S. M.; Parney, I. F.; Huang, W.; Anderson, F. A. Jr; Asher, A. L.; Bernstein, M.; Lillehei, K. O.; 

Brem, H.; Berger, M. S., and Laws, E. R. Patterns of care for adults with newly diagnosed malignant 

glioma. JAMA. 2005 Feb 2; 293(5):557-64. 

Rec #: 716 
H-18 

 51. Chang, V. T.; Hwang, S. S., and Kasimis, B. Longitudinal documentation of cancer pain management 

outcomes: a pilot study at a VA medical center. J Pain Symptom Manage. 2002 Nov; 24(5):494-505. 

Rec #: 249 

 52. Cherry, D. L.; Vickrey, B. G.; Schwankovsky, L.; Heck, E.; Plauchm, M., and Yep, R. Interventions to 

improve quality of care: the Kaiser Permanente-alzheimer's Association Dementia Care Project. Am 

J Manag Care. 2004 Aug; 10(8):553-60. 

Rec #: 626 

 53. Chochinov, H. M.; Wilson, K. G.; Enns, M., and Lander, S. "Are you depressed?" Screening for depression in 

the terminally ill. Am J Psychiatry. 1997 May; 154(5):674-6. 

Rec #: 439 

 54. Clarke, E. B.; Curtis, J. R.; Luce, J. M.; Levy, M.; Danis, M.; Nelson, J., and Solomon, M. Z. Seven end-oflife care domains associated quality indicators and related clinician and organizational 

itnerventions/behaviors . 2003. 

Rec #: 104 

 55. Cleeland, C. S.; Reyes-Gibby, C. C.; Schall, M.; Nolan, K.; Paice, J.; Rosenberg, J. M.; Tollett, J. H., and 

Kerns, R. D. Rapid improvement in pain management: the Veterans Health Administration and the 

institute for healthcare improvement collaborative. Clin J Pain. 2003 Sep-2003 Oct 31; 19(5):298-

305. 

Rec #: 242 

 56. Cohen, L. M.; Germain, M.; Poppel, D. M.; Woods, A., and Kjellstrand, C. M. Dialysis discontinuation and 

palliative care. Am J Kidney Dis. 2000 Jul; 36(1):140-4. 

Rec #: 564 

 57. Cohen, L. M. and Germain, M. J. Measuring quality of dying in end-stage renal disease. Semin Dial. 2004 

Sep-2004 Oct 31; 17(5):376-9. 

Rec #: 498 

 58. Cohen-Mansfield, J.; Libin, A., and Lipson, S. Differences in presenting advance directives in the chart, in the 

minimum data set, and through the staff's perceptions. Gerontologist. 2003 Jun; 43(3):302-8. 

Rec #: 1376 

 59. Covinsky KE; Fuller JD; Yaffe K; Johnston CB; Hamel MB; Lynn J; Teno JM, and Phillips RS. 

Communication and decision-making in seriously ill patients: findings of the SUPPORT project. The 

Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments. J Am 

Geriatr Soc. 2000 May; 48(5 Suppl):S187-93. 

Rec #: 1501 

 60. Crispell, K. A. Common cardiovascular issues encountered in geriatric critical care. Crit Care Clin. 2003 Oct; 

19(4):677-91. 

Rec #: 521 

 61. Cull, A.; Gould, A.; House, A.; Smith, A.; Strong, V.; Velikova, G.; Wright, P., and Selby, P. Validating 

automated screening for psychological distress by means of computer touchscreens for use in routine 

oncology practice. Br J Cancer. 2001 Dec 14; 85(12):1842-9. 

Rec #: 417 

 62. Cullen, D. L. and Rodak, B. Clinical utility of measures of breathlessness. Respir Care. 2002 Sep; 47(9):986-

93. 

Rec #: 423 
H-19 

 63. Curtis, J. R. Communicating with patients and their families about advance care planning and end-of-life 

care. Respir Care. 2000 Nov; 45(11):1385-94; discussion 1394-8. 

Rec #: 215 

 64. Curtis, J. R.; Engelberg, R. A.; Nielsen, E. L.; Au, D. H., and Patrick, D. L. Patient-physician communication 

about end-of-life care for patients with severe COPD. Eur Respir J. 2004 Aug; 24(2):200-5. 

Rec #: 502 

 65. Curtis, J. R.; Patrick, D. L.; Caldwell, E.; Greenlee, H., and Collier, A. C. The quality of patient-doctor 

communication about end-of-life care: a study of patients with advanced AIDS and their primary 

care clinicians. AIDS. 1999 Jun 18; 13(9):1123-31. 

Rec #: 584 

 66. Curtis, J. R.; Patrick, D. L.; Caldwell, E. S., and Collier, A. C. Why don't patients and physicians talk about 

end-of-life care? Barriers to communication for patients with acquired immunodeficiency syndrome 

and their primary care clinicians. Arch Intern Med. 2000 Jun 12; 160(11):1690-6. 

Rec #: 1587 

 67. Curtis, J. R.; Patrick, D. L.; Engelberg, R. A.; Norris, K.; Asp, C., and Byock, I. A measure of the quality of 

dying and death. Initial validation using after-death interviews with family members. J Pain 

Symptom Manage. 2002 Jul; 24(1):17-31. 

Rec #: 1588 

 68. Dalrymple, J. L.; Levenback, C.; Wolf, J. K.; Bodurka, D. C.; Garcia, M., and Gershenson, D. M. Trends 

among gynecologic oncology inpatient deaths: is end-of-life care improving? Gynecol Oncol. 2002 

May; 85(2):356-61. 

Rec #: 533 

 69. Dalton, J. A.; Blau, W.; Lindley, C.; Carlson, J.; Youngblood, R., and Greer, S. M. (School of Nursing, CB# 

7460, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599.). Changing acute pain 

management to improve patient outcomes: an educational approach. Journal of Pain and Symptom 

Management. 1999 Apr; 17(4):277-87. 

Rec #: 347 

 70. Dawson, R.; Spross, J. A.; Jablonski, E. S.; Hoyer, D. R.; Sellers, D. E., and Solomon, M. Z. Probing the 

paradox of patients' satisfaction with inadequate pain management. Journal of Pain & Symptom 

Management. 2002; 23(3):211-20. 

Rec #: 367 

 71. de Graeff, A.; de Leeuw, J. R.; Ros, W. J.; Hordijk, G. J.; Blijham, G. H., and Winnubst, J. A. 

Sociodemographic factors and quality of life as prognostic indicators in head and neck cancer. Eur J 

Cancer. 2001 Feb; 37(3):332-9. 

Rec #: 421 

 72. de Rond, M.; de Wit, R., and van Dam, F. The implementation of a Pain Monitoring Programme for nurses in 

daily clinical practice: results of a follow-up study in five hospitals. J Adv Nurs. 2001 Aug; 

35(4):590-8. 

Rec #: 265 

 73. de Wit R; van Dam F; Abu-Saad HH; Loonstra S; Zandbelt L; van Buuren A; van der Heijden K, and 

Leenhouts G. Empirical comparison of commonly used measures to evaluate pain treatment in 

cancer patients with chronic pain. J Clin Oncol. 1999 Apr; 17(4):1280. 

Rec #: 356 
H-20 

 74. de Wit, R.; van Dam, F.; Loonstra, S.; Zandbelt, L.; van Buuren, A.; van der Heijden, K.; Leenhouts, G.; 

Duivenvoorden, H., and Huijer Abu-Saad, H. Improving the quality of pain treatment by a tailored 

pain education programme for cancer patients in chronic pain. Eur J Pain. 2001; 5(3):241-56. 

Rec #: 264 

 75. de Wit, R.; van Dam, F.; Vielvoye-Kerkmeer, A.; Mattern, C., and Abu-Saad, H. H. (Division of 

Psychosocial Research and Epidemiology, Netherlands Cancer Institute/Antoni van Leeuwenhoek 

Hospital, Amsterdam, The Netherlands.). The treatment of chronic cancer pain in a cancer hospital 

in the Netherlands. Journal of Pain and Symptom Management. 1999 May; 17(5):333-50. 

Rec #: 341 

 76. de Wit, R.; van Dam, F.; Zandbelt, L.; van Buuren, A.; van der Heijden, K.; Leenhouts, G., and Loonstra, S. 

A pain education program for chronic cancer pain patients: follow-up results from a randomized 

controlled trial. Pain. 1997 Oct; 73(1):55-69. 

Rec #: 288 

 77. DeLaGarza, V. W.; Andersen, R. E.; Mach, J., and Bennett, R. G. (Johns Hopkins University School of 

Medicine, Division of Geriatric Medicine and Gerontology, Johns Hopkins Geriatrics Center, 

Baltimore, MD.). Improving rates of advance directive discussions among managed care nursing 

home enrollees. Journal of the American Medical Directors Association. 2001 May-2001 Jun 30; 

2(3):105-9. 

Rec #: 566 

 78. Della Santina, C. and Bernstein, R. H. Whole-patient assessment, goal planning, and inflection points: their 

role in achieving quality end-of-life care. Clin Geriatr Med. 2004 Nov; 20(4):595-620, v. 

Rec #: 496 

 79. Desbiens, N. A.; Mueller-Rizner, N.; Connors, A. F. Jr.; Wenger, N. S., and Lynn, J. (University of 

Tennessee College of Medicine, Chattanooga Unit, 921 East Third St., Suite 400, Chattanooga, TN 

37403.). The symptom burden of seriously ill hospitalized patients. Journal of Pain and Symptom 

Management. 1999 Apr; 17(4):248-55. 

Rec #: 346 

 80. Detterbeck, F. C.; Jones, D. R.; Kernstine, K. H., and Naunheim, K. S. Lung cancer. Special treatment issues. 

Chest. 2003 Jan; 123(1 Suppl):244S-258S. 

Rec #: 167 

 81. Devine, E. C. Meta-analysis of the effect of psychoeducational interventions on pain in adults with cancer. 

Oncol Nurs Forum. 2003 Jan-2003 Feb 28; 30(1):75-89. 

Rec #: 365 

 82. Devlen, J.; Maguire, P.; Phillips, P., and Crowther, D. Psychological problems associated with diagnosis and 

treatment of lymphomas. II: Prospective study. British Medical Journal Clinical Research Ed. 1987; 

295(6604):955-7. 

Rec #: 455 

 83. Devlen, J.; Maguire, P.; Phillips, P.; Crowther, D., and Chambers, H. Psychological problems associated with 

diagnosis and treatment of lymphomas. I: Retrospective study. British Medical Journal Clinical 

Research Ed. 1987; 295(6604):953-4. 

Rec #: 456 

 84. Dolan, C. S. Putting guidelines to work in an ambulatory care setting. Oncol Nurs Forum. 1995 Oct; 

22(9):1436. 

Rec #: 302 
H-21 

 85. Donner, C. F. and Muir, J. F. Selection criteria and programmes for pulmonary rehabilitation in COPD 

patients. Rehabilitation and Chronic Care Scientific Group of the European Respiratory Society. Eur 

Respir J. 1997 Mar; 10(3):744-57. 

Rec #: 447 

 86. Donovan KA; Greene PG; Shuster JL; Partridge EE, and Tucker DC. Treatment preferences in recurrent 

ovarian cancer. Gynecol Oncol. 2002 Aug; 86(2):200-11. 

Rec #: 1899 

 87. Du Pen, A. R.; Du Pen, S.; Hansberry, J.; Miller-Kraybill, B.; Millen, J.; Everly, R.; Hansen, N., and Syrjala, 

K. An educational implementation of a cancer pain algorithm for ambulatory care. Pain Manag Nurs. 

2000 Dec; 1(4):116-28. 

Rec #: 261 

 88. Du Pen, S. L.; Du Pen, A. R.; Polissar, N.; Hansberry, J.; Kraybill, B. M.; Stillman, M.; Panke, J.; Everly, R., 

and Syrjala, K. Implementing guidelines for cancer pain management: results of a randomized 

controlled clinical trial. J Clin Oncol. 1999 Jan; 17(1):361-70. 

Rec #: 277 

 89. Dudgeon, D. J.; Kristjanson, L.; Sloan, J. A.; Lertzman, M., and Clement, K. Dyspnea in cancer patients: 

prevalence and associated factors. Journal of Pain & Symptom Management. 2001; 21(2):95-102. 

Rec #: 457 

 90. Dufault, M. A. and Willey-Lessne, C. (Associate Professor, College of Nursing, University of Rhode Island, 

Kingston, Rhode Island.). Using a collaborative research utilization model to develop and test the 

effects of clinical pathways for pain management. Journal of Nursing Care Quality. 1999 Apr; 

13(4):19-33. 

Rec #: 345 

 91. Dy, S. and Teno, J. Palliative care/end of life measures background paper. IOM. 

Rec #: 750 

 92. Emery, C.; Gallagher, R.; Hugi, M., and Levine, M. Clinical practice guidelines for the care and treatment of 

breast cancer: the management of chronic pain in patients with breast cancer (summary of the 2001 

update). CMAJ. 2001 Oct 30; 165(9):1218-9. 

Rec #: 720 

 93. Evers, M. M.; Meier, D. E., and Morrison, R. S. Assessing differences in care needs and service utilization in 

geriatric palliative care patients. J Pain Symptom Manage. 2002 May; 23(5):424-32. 

Rec #: 532 

 94. Extreme Care, Humane Options ECHO. Recommendations for improving end-of-life care for persons 

residing in California skilled nursing and intermediate care facilities. Sacramento, CA: California 

Coalition for Compassionate Care; 2000 Jan(Nursing Faclity Recommendations). 

Rec #: 474 

 95. Farrar, J. T.; Berlin, J. A., and Strom, B. L. Clinically important changes in acute pain outcome measures: a 

validation study. J Pain Symptom Manage. 2003 May; 25(5):406-11. 

Rec #: 247 

 96. Fellowes, D.; Barnes, K., and Wilkinson, S. Aromatherapy and massage for symptom relief in patients with 

cancer. Cochrane Database Syst Rev. 2004; (2):CD002287. 

Rec #: 232 
H-22 

 97. Ferrell, B. R.; Juarez, G., and Borneman, T. (Nursing Research and Education Department, City of Hope 

National Medical Center, Duarte, CA.). Use of routine and breakthrough analgesia in home care. 

Oncology Nursing Forum. 1999 Nov-1999 Dec 31; 26(10):1655-60. 

Rec #: 340 

 98. Fischer, G. S.; Alpert, H. R.; Stoeckle, J. D., and Emanuel, L. L. Can goals of care be used to predict 

intervention preferences in an advance directive? Arch Intern Med. 1997 Apr 14; 157(7):801-7. 

Rec #: 600 

 99. Fortner, B. V.; Okon, T. A.; Ashley, J.; Kepler, G.; Chavez, J.; Tauer, K.; Clements-Thompson, M.; 

Schwartzberg, L.; Demarco, G., and Houts, A. C. The Zero Acceptance of Pain (ZAP) Quality 

Improvement Project: evaluation of pain severity, pain interference, global quality of life, and painrelated costs. J Pain Symptom Manage. 2003 Apr; 25(4):334-43. 

Rec #: 248 

 100. Fried, T. R. and Mor, V. Frailty and hospitalization of long-term stay nursing home residents. J Am Geriatr 

Soc. 1997 Mar; 45(3):265-9. 

Rec #: 41137 

 101. Fried TR and Bradley EH. What matters to seriously ill older persons making end-of-life treatment 

decisions?: A qualitative study. J Palliat Med. 2003 Apr; 6(2):237-44. 

Rec #: 2435 

 102. Fried TR; Bradley EH, and O'Leary J. Prognosis communication in serious illness: perceptions of older 

patients, caregivers, and clinicians. J Am Geriatr Soc. 2003 Oct; 51(10):1398-403. 

Rec #: 2436 

 103. Fried TR; Bradley EH, and Towle VR. Assessment of patient preferences: integrating treatments and 

outcomes. J Gerontol B Psychol Sci Soc Sci. 2002 Nov; 57(6):S348-54. 

Rec #: 2437 

 104. Fried TR; Bradley EH; Towle VR, and Allore H. Understanding the treatment preferences of seriously ill 

patients. N Engl J Med. 2002 Apr 4; 346(14):1061-6. Comment in: N Engl J Med. 2002 Apr 

4;346(14):1087-9. PMID: 11932479. Comment in: N Engl J Med. 2002 Aug 15;347(7):533-5; 

author reply 533-5. PMID: 12181411. Comment in: N Engl J Med. 2002 Aug 15;347(7):533-5; 

author reply 533-5. PMID: 12182184. Co. 

Rec #: 2439 

 105. Gessert, C. E.; Mosier, M. C.; Brown, E. F., and Frey, B. (Office of Primary Care, University of Kansas 

Medical Center, Kansas City, Kansas.). Tube feeding in nursing home residents with severe and 

irreversible cognitive impairment. Journal of the American Geriatrics Society. 2000 Dec; 

48(12):1593-600. 

Rec #: 582 

 106. Gifford, D. and Schmidt, L. Breast cancer diagnosis and treatment. Asch, S. M.; Kerr, E. A.; Hamilton, E. 

G.; Reifel, J. L., and McGlynn, E. A. Quality of care for oncologic condition and HIV: a review of 

the literature and quality indicators. RAND; 2000; pp. 33-45. 

Rec #: 704 

 107. Golant, M.; Altman, T., and Martin, C. (Wellness Community-National, 2716 Ocean Park Blvd, Suite 1040, 

Santa Monica, CA 90405; golant@la.wellnesscommunity.org.). Managing cancer side effects to 

improve quality of life: a cancer psychoeducation program. Cancer Nursing. 2003 Feb; 26(1):37-46. 

Rec #: 400 
H-23 

 108. Golin, C. E.; Wenger, N. S.; Liu, H.; Dawson, N. V.; Teno, J. M.; Desbiens, N. A.; Lynn, J.; Oye, R. K., and 

Phillips, R. S. A prospective study of patient-physician communication about resuscitation. J Am 

Geriatr Soc. 2000 May; 48(5 Suppl):S52-60. 

Rec #: 570 

 109. Goodlin, S. J.; Zhong, Z.; Lynn, J.; Teno, J. M.; Fago, J. P.; Desbiens, N.; Connors, A. F. Jr; Wenger, N. S., 

and Phillips, R. S. Factors associated with use of cardiopulmonary resuscitation in seriously ill 

hospitalized adults. JAMA. 1999 Dec 22-1999 Dec 29; 282(24):2333-9. 

Rec #: 577 

 110. Goodman, M. D.; Tarnoff, M., and Slotman, G. J. Effect of advance directives on the management of elderly 

critically ill patients. Crit Care Med. 1998 Apr; 26(4):701-4. 

Rec #: 591 

 111. Gordon, D. B.; Pellino, T. A.; Miaskowski, C.; McNeill, J. A.; Paice, J. A.; Laferriere, D., and Bookbinder, 

M. A 10-year review of quality improvement monitoring in pain management: recommendations for 

standardized outcome measures. Pain Manag Nurs. 2002 Dec; 3(4):116-30. 

Rec #: 252 

 112. Grande GE; McKerral A, and Todd CJ. Which cancer patients are referred to Hospital at Home for palliative 

care? Palliat Med. 2002 Mar; 16(2):115-23. 

Rec #: 2777 

 113. Grant, M.; Ferrell, B. R.; Rivera, L. M., and Lee, J. Unscheduled readmissions for uncontrolled symptoms. A 

health care challenge for nurses. Nurs Clin North Am. 1995 Dec; 30(4):673-82. 

Rec #: 299 

 114. Grant, M.; Rivera, L. M.; Alisangco, J., and Francisco, L. Improving cancer pain management using a 

performance improvement framework. J Nurs Care Qual. 1999 Apr; 13(4):60-72. 

Rec #: 280 

 115. Grant, S.; Aitchison, T.; Henderson, E.; Christie, J.; Zare, S.; McMurray, J., and Dargie, H. (Institute of 

Biomedical and Life Sciences, University of Glasgow, 64 Oakfield Ave, Glasgow, G12 8LT, UK; email: S.Grant@bio.gla.ac.uk.). A comparison of the reproducibility and the sensitivity to change of 

visual analogue scales, Borg scales, and Likert scales in normal subjects during submaximal 

exercise. Chest. 1999 Nov; 116(5):1208-17. 

Rec #: 395 

 116. Greenberg, A.; Angus, H.; Sullivan, T., and Brown, A. D. Development of a set of strategy-based systemlevel cancer care performance indicators in Ontario, Canada. Int J Qual Health Care. 2005 Jan 21. 

Rec #: 209 

 117. Greenberg, D. B. Barriers to the treatment of depression in cancer patients. J Natl Cancer Inst Monogr. 2004; 

(32):127-35. 

Rec #: 391 

 118. Griffin, J. P.; Nelson, J. E.; Koch, K. A.; Niell, H. B.; Ackerman, T. F.; Thompson, M., and Cole, F. H. Jr. 

End-of-life care in patients with lung cancer. Chest. 2003 Jan; 123(1 Suppl):312S-331S. 

Rec #: 101 

 119. Grond S; Zech D; Lynch J; Diefenbach C; Schug SA, and Lehmann KA. Validation of World Health 

Organization guidelines for pain relief in head and neck cancer. A prospective study. Ann Otol 

Rhinol Laryngol. 1993 May; 102(5):342-8. 

Rec #: 357 
H-24 

 120. Grond S; Zech D; Schug SA; Lynch J, and Lehmann KA. Validation of World Health Organization 

guidelines for cancer pain relief during the last days and hours of life. J Pain Symptom Manage. 

1991 Oct; 6(7):411-22. 

Rec #: 358 

 121. Gunnarsdottir, S.; Donovan, H. S.; Serlin, R. C.; Voge, C., and Ward, S. Patient-related barriers to pain 

management: the Barriers Questionnaire II (BQ-II). Pain. 2002 Oct; 99(3):385-96. 

Rec #: 253 

 122. Guyatt, G. H.; King, D. R.; Feeny, D. H.; Stubbing, D., and Goldstein, R. S. Generic and specific 

measurement of health-related quality of life in a clinical trial of respiratory rehabilitation. J Clin 

Epidemiol. 1999 Mar; 52(3):187-92. 

Rec #: 444 

 123. Hagan, S. Using guidelines for education, policy making, and assessing quality of care. Oncol Nurs Forum. 

1995 Oct; 22(9):1435-6. 

Rec #: 303 

 124. Haidet, P.; Hamel, M. B.; Davis, R. B.; Wenger, N.; Reding, D.; Kussin, P. S.; Connors, A. F. Jr; Lynn, J.; 

Weeks, J. C., and Phillips, R. S. Outcomes, preferences for resuscitation, and physician-patient 

communication among patients with metastatic colorectal cancer. SUPPORT Investigators. Study to 

Understand Prognoses and Preferences for Outcomes and Risks of Treatments. Am J Med. 1998 

Sep; 105(3):222-9. 

Rec #: 587 

 125. Hakim, R. B.; Teno, J. M.; Harrell, F. E. Jr; Knaus, W. A.; Wenger, N.; Phillips, R. S.; Layde, P.; Califf, R.; 

Connors, A. F. Jr, and Lynn, J. Factors associated with do-not-resuscitate orders: patients' 

preferences, prognoses, and physicians' judgments. SUPPORT Investigators. Study to Understand 

Prognoses and Preferences for Outcomes and Risks of Treatment. Ann Intern Med. 1996 Aug 15; 

125(4):284-93. 

Rec #: 608 

 126. Hanks GW ; Robbins M ; Sharp D ; Forbes K ; Done K ; Peters TJ ; Morgan H ; Sykes J ; Baxter K ; Corfe F 

, and Bidgood C. The imPaCT study: a randomised controlled trial to evaluate a hospital palliative 

care team. Br J Cancer. 2002 Sep 23; 87(7):733-9. 

Rec #: 425 

 127. Hann, D.; Winter, K., and Jacobsen, P. Measurement of depressive symptoms in cancer patients: evaluation 

of the Center for Epidemiological Studies Depression Scale (CES-D). J Psychosom Res. 1999 May; 

46(5):437-43. 

Rec #: 433 

 128. Hansen-Flaschen, J. H. Dyspnea in the ventilated patient: a call for patient-centered mechanical ventilation. 

Respir Care. 2000 Dec; 45(12):1460-4, discussion 1464-7. 

Rec #: 217 

 129. ---. Palliative home care for advanced lung disease. Respir Care. 2000 Dec; 45(12):1478-86; discussion 1486-

9. 

Rec #: 219 

 130. Hanson, L. C.; Henderson, M., and Menon, M. As individual as death itself: a focus group study of terminal 

care in nursing homes. J Palliat Med. 2002 Feb; 5(1):117-25. 

Rec #: 536 
H-25 

 131. Hardin, S. B. and Yusufaly, Y. A. Difficult end-of-life treatment decisions: do other factors trump advance 

directives? Arch Intern Med. 2004 Jul 26; 164(14):1531-3. 

Rec #: 505 

 132. Harter, M.; Reuter, K.; Aschenbrenner, A.; Schretzmann, B.; Marschner, N.; Hasenburg, A., and Weis, J. 

Psychiatric disorders and associated factors in cancer: results of an interview study with patients in 

inpatient, rehabilitation and outpatient treatment. Eur J Cancer. 2001 Jul; 37(11):1385-93. 

Rec #: 419 

 133. Hawryluck, L. A.; Harvey, W. R.; Lemieux-Charles, L., and Singer, P. A. Consensus guidelines on analgesia 

and sedation in dying intensive care unit patients. BMC Med Ethics. 2002 Aug 12; 3(1):E3. 

Rec #: 469 

 134. Heffner, J. E. Role of pulmonary rehabilitation in palliative care. Respir Care. 2000 Nov; 45(11):1365-71; 

discussion 1371-5. 

Rec #: 213 

 135. Heiman, H.; Bates, D. W.; Fairchild, D.; Shaykevich, S., and Lehmann, L. S. Improving completion of 

advance directives in the primary care setting: a randomized controlled trial. Am J Med. 2004 Sep 1; 

117(5):318-24. 

Rec #: 501 

 136. Hester, N. O.; Miller, K. L.; Foster, R. L., and Vojir, C. P. Symptom management outcomes. Do they reflect 

variations in care delivery systems? Med Care. 1997 Nov; 35(11 Suppl):NS69-83. 

Rec #: 289 

 137. Higginson, I. J.; Finlay, I.; Goodwin, D. M.; Cook, A. M.; Hood, K.; Edwards, A. G. K.; Douglas, H., and 

Norman, C. E. (Department of Palliative Care & Policy, King's College London, Weston Education 

Center, Cutcombe Road, Denmark Hill, London SE5 9RJ, United Kingdom.). Do hospital-based 

palliative teams improve care for patients or families at the end of life? Journal of Pain and 

Symptom Management. 2002 Feb; 23(2):96-106. 

Rec #: 314 

 138. Hinkka H; Kosunen E; Metsanoja R; Lammi UK, and Kellokumpu-Lehtinen P. Factors affecting physicians' 

decisions to forgo life-sustaining treatments in terminal care. J Med Ethics. 2002; 28(2):109-14. 

Rec #: 3231 

 139. Hofmann, J. C.; Wenger, N. S.; Davis, R. B.; Teno, J.; Connors, A. F. Jr; Desbiens, N.; Lynn, J., and Phillips, 

R. S. Patient preferences for communication with physicians about end-of-life decisions. SUPPORT 

Investigators. Study to Understand Prognoses and Preference for Outcomes and Risks of Treatment. 

Ann Intern Med. 1997 Jul 1; 127(1):1-12. 

Rec #: 599 

 140. Holland, J. C. and Chertkov, L. Clinical practice guidelines for the management of psychosocial and physical 

symptoms of cancer. Improving Palliative Care for Cancer. Washington, DC: National Academy 

Press; 2001. 

Rec #: 485 

 141. Hollen PJ; Gralla RJ; Kris MG; Eberly SW, and Cox C. Normative data and trends in quality of life from the 

Lung Cancer Symptom Scale (LCSS). Support Care Cancer. 1999 May; 7(3):140-8. 

Rec #: 452 

 142. Holzheimer, A.; McMillan, S. C., and Weitzner, M. Improving pain outcomes of hospice patients with 

cancer. Oncology Nursing Forum. 1999 Oct; 26(9):1499-504. 

Rec #: 339 
H-26 

 143. Hooke, C.; Hellsten, M. B.; Stutzer, C., and Forte, K. Pain management for the child with cancer in end-oflife care: APON position paper. Journal of Pediatric Oncology Nursing. 2002 Mar 8-2002 Apr 8; 

19(2):43-7. 

Rec #: 308 

 144. Hopwood, P.; Howell, A., and Maguire, P. Psychiatric morbidity in patients with advanced cancer of the 

breast: prevalence measured by two self-rating questionnaires. British Journal of Cancer. 1991; 

64(2):349-52. 

Rec #: 458 

 145. ---. Screening for psychiatric morbidity in patients with advanced breast cancer: validation of two self-report 

questionnaires. British Journal of Cancer. 1991; 64(2):353-6. 

Rec #: 459 

 146. Hopwood P and Stephens RJ. Depression in patients with lung cancer: prevalence and risk factors derived 

from quality-of-life data. J Clin Oncol. 2000 Feb; 18(4):893-903. 

Rec #: 431 

 147. Huang, H.; Wilkie, D. J.; Zong, S.; Berry, D.; Hairabedian, D.; Judge, M. K.; Farber, S., and Chabal, C. (Dept 

of Biobehavioral Nursing and Health Systems, Box 357266, University of Washington, Seattle, WA 

98195-7266; hyhuang@u.washington.edu.). Developing a computerized data collection and decision 

support system for cancer pain management. CIN: Computers, Informatics, Nursing. 2003 Jul-2003 

Aug 31; 21(4):206-17. 

Rec #: 329 

 148. Hwang, S. S.; Chang, V. T.; Cogswell, J.; Alejandro, Y.; Osenenko, P.; Morales, E.; Srinivas, S., and 

Kasimis, B. Study of unmet needs in symptomatic veterans with advanced cancer: incidence, 

independent predictors and unmet needs outcome model. J Pain Symptom Manage. 2004 Nov; 

28(5):421-32. 

Rec #: 385 

 149. Hwang, S. S.; Chang, V. T.; Fairclough, D. L.; Cogswell, J., and Kasimis, B. (Section of 

Hematology/Oncology (111), VA New Jersey Health Care System, 385 Tremont Ave., East Orange, 

NJ 07018.). Longitudinal quality of life in advanced cancer patients: pilot study results from a VA 

Medical Cancer Center. Journal of Pain and Symptom Management. 2003 Mar; 25(3):225-35. 

Rec #: 328 

 150. Hwang, S. S.; Chang, V. T.; Fairclough, D. L., and Kasimis, B. (Section of Hematology/Oncology (111), VA 

New Jersey Health Care System at East Orange, 385 Tremont Avenue, East Orange, NJ 07018.). 

Development of a cancer pain prognostic scale. Journal of Pain and Symptom Management. 2002 

Oct; 24(4):366-78. 

Rec #: 334 

 151. Hwang, S. S.; Chang, V. T., and Kasimis, B. (Section Hematology/Oncology (111), VA New Jersey Health 

Care System at East Orange, 385 Tremont Avenue, East Orange, NJ 07018.). Dynamic cancer pain 

management outcomes: the relationship between pain severity, pain relief, functional interference, 

satisfaction and global quality of life over time. Journal of Pain and Symptom Management. 2002 

Mar; 23(3):190-200. 

Rec #: 311 

 152. Ibbotson, T.; Maguire, P.; Selby, P.; Priestman, T., and Wallace, L. Screening for anxiety and depression in 

cancer patients: the effects of disease and treatment. European Journal of Cancer. 1994; 30A(1):37-

40. 

Rec #: 460 
H-27 

 153. Ingram, S. S.; Seo, P. H.; Martell, R. E.; Clipp, E. C.; Doyle, M. E.; Montana, G. S., and Cohen, H. J. (Box 

3085, Dept of Radiation Oncology, Duke University Medical Center, Durham, NC 27710; 

ingram@radonc.duke.edu.). Comprehensive assessment of the elderly cancer patient: the feasibility 

of self-report methodology. Journal of Clinical Oncology. 2002 Feb 1; 20(3):770-5. 

Rec #: 406 

 154. Jacobs, L. G.; Bonuck, K.; Burton, W., and Mulvihill, M. (Department of Medicine, Division of Geriatrics, 

Montefiore Medical Center, 111 East 210th Street, Bronx, New York 10467.). Hospital care at the 

end of life: an institutional assessment. Journal of Pain and Symptom Management. 2002 Sep; 

24(3):291-8. 

Rec #: 333 

 155. Jadad, A. R. and Browman, G. P. The WHO analgesic ladder for cancer pain management. Stepping up the 

quality of its evaluation. JAMA. 1995 Dec 20; 274(23):1870-3. 

Rec #: 297 

 156. Jadlos, M. A.; Kelman, G. B.; Marra, K., and Lanoue, A. (- Samaritan Hospital, Troy, NY). - 

Documentation. A pain management documentation tool. - Oncology Nursing Forum. 23(9):1451-4, 

1996 Oct. (7 Ref). 1996 Oct. 

Rec #: 307 

 157. Jefford, M.; Mileshkin, L.; Richards, K.; Thomson, J.; Matthews, J. P.; Zalcberg, J.; Jennens, R.; McLachlan, 

S. A.; Wein, S.; Antill, Y., and Clarke, D. M. Rapid screening for depression--validation of the Brief 

Case-Find for Depression (BCD) in medical oncology and palliative care patients. Br J Cancer. 2004 

Aug 31; 91(5):900-6. 

Rec #: 390 

 158. Jennings, A-L; Davies, A. N.; Higgins, J. P. T.; Gibbs, J. S. R., and Broadley, K. E. A systematic review of 

the use of opioids in the management of dyspnoea. Thorax. 2002; (57):939-44. 

Rec #: 157 

 159. Jensen, M. P. (Department of Rehabilitation Medicine, Box 356490, University of Washington School of 

Medicine, Seattle, WA 98195-6490; mjensen@u.washington.edu.). The validity and reliability of 

pain measures in adults with cancer. Journal of Pain. 2003 Feb; 4(1):2-21. 

Rec #: 330 

 160. Johnston, S. C.; Pfeifer, M. P., and McNutt, R. The discussion about advance directives. Patient and 

physician opinions regarding when and how it should be conducted. End of Life Study Group. Arch 

Intern Med. 1995 May 22; 155(10):1025-30. 

Rec #: 617 

 161. Joint Commission on Accreditation of Healthcare Organizations (JCAHO) and National Pharmaceutical 

Council, Inc. NPC. Improving the quality of pain management through measurement and action. 

2003 Mar. 

Rec #: 745 

 162. Kaiser Permanente, Care Management Institute's Cancer Initiative Guidelines Workgroup and Care 

Management Institute Kaiser Permanente. Evidence-based guidelines for the management of 

chemotherapy-induced nausea & vomiting and moderate to severe cancer-associated pain. 2003 

Mar. 

Rec #: 175 

 163. Kane, R. S. and Burns, E. A. Cardiopulmonary resuscitation policies in long-term care facilities. J Am Geriatr 

Soc. 1997 Feb; 45(2):154-7. 

Rec #: 41723 
H-28 

 164. Keay, T. J. ; Taler, G. A.; Fredman, L., and Levenson, S. A. Assessing medical care of dying residents in 

nursing homes. Am J Med Qual. 1997 Fall; 12(3):151-6. 

Rec #: 3779 

 165. Keenan, S. P.; Busche, K. D.; Chen, L. M.; McCarthy, L.; Inman, K. J., and Sibbald, W. J. A retrospective 

review of a large cohort of patients undergoing the process of withholding or withdrawal of life 

support. Crit Care Med. 1997 Aug; 25(8):1324-31. 

Rec #: 50514 

 166. Introduction. Kerr, E. A.; Asch, S. M.; Hamilton, E. G., and McGlynn, E. A. Quality of care for 

cardiopulmonary conditions: a review of the literature and quality indicators. RAND; 2000; pp. 1-

27. 

Rec #: 737 

 167. Kerridge, I. H.; Pearson, S. A.; Rolfe, I. E., and Lowe, M. Decision making in CPR: attitudes of hospital 

patients and healthcare professionals. Med J Aust. 1998 Aug 3; 169(3):128-31. 

Rec #: 3855 

 168. Kirchhoff, K. T.; Anumandla, P. R.; Foth, K. T.; Lues, S. N., and Gilbertson-White, S. H. Documentation on 

withdrawal of life support in adult patients in the intensive care unit. Am J Crit Care. 2004 Jul; 

13(4):328-34. 

Rec #: 503 

 169. Kurtz, M. E.; Kurtz, J. C.; Stommel, M.; Given, C. W., and Given, B. (Professor, Department of Family and 

Community Medicine, Michigan State University, B211 West Fee Hall, East Lansing, MI 48824.). 

Physical functioning and depression among older persons with cancer. Cancer Practice: A 

Multidisciplinary Journal of Cancer Care. 2001 Jan-2001 Feb 28; 9(1):11-8. 

Rec #: 411 

 170. Laakkonen, M. L.; Pitkala, K. H.; Strandberg, T. E.; Berglind, S., and Tilvis, R. S. Living will, resuscitation 

preferences, and attitudes towards life in an aged population. Gerontology. 2004 Jul-2004 Aug 31; 

50(4):247-54. 

Rec #: 506 

 171. Lacasse, Y.; Brosseau, L.; Milne, S.; Martin, S.; Wong, E.; Guyatt, G. H., and Goldstein, R. S. Pulmonary 

rehabilitation for chronic obstructive pulmonary disease. Cochrane Database Syst Rev. 2002; 

(3):CD003793. 

Rec #: 428 

 172. Lacasse, Y.; Wong, E.; Guyatt, G. H.; King, D.; Cook, D. J., and Goldstein, R. S. Meta-analysis of respiratory 

rehabilitation in chronic obstructive pulmonary disease. Lancet. 1996 Oct 26; 348(9035):1115-9. 

Rec #: 448 

 173. Lamy, O.; Jenzer-Closuit, A., and Burckhardt, P. Hypercalcaemia of malignancy: an undiagnosed and 

undertreated disease. J Intern Med. 2001 Jul; 250(1):73-9. 

Rec #: 267 

 174. Landi, F.; Onder, G.; Cesari, M.; Gambassi, G.; Steel, K.; Russo, A.; Lattanzio, F., and Bernabei, R. Pain 

management in frail, community-living elderly patients. Arch Intern Med. 2001 Dec 10-2001 Dec 

24; 161(22):2721-4. 

Rec #: 628 

 175. Larue, F.; Colleau, S. M.; Brasseur, L., and Cleeland, C. S. Multicentre study of cancer pain and its treatment 

in France. BMJ. 1995 Apr 22; 310(6986):1034-7. 

Rec #: 304 
H-29 

 176. Larue, F.; Colleau, S. M.; Fontaine, A., and Brasseur, L. Oncologists and primary care physicians' attitudes 

toward pain control and morphine prescribing in France. Cancer. 1995 Dec 1; 76(11):2375-82. 

Rec #: 298 

 177. Lee, M. A.; Brummel-Smith, K.; Meyer, J.; Drew, N., and London, M. R. Physician orders for life-sustaining 

treatment (POLST): outcomes in a PACE program. Program of All-Inclusive Care for the Elderly. J 

Am Geriatr Soc. 2000 Oct; 48(10):1219-25. 

Rec #: 561 

 178. Lee, Y. C.; Baumann, M. H.; Maskell, N. A.; Waterer, G. W.; Eaton, T. E.; Davies, R. J.; Heffner, J. E., and 

Light, R. W. Pleurodesis practice for malignant pleural effusions in five English-speaking countries: 

survey of pulmonologists. Chest. 2003 Dec; 124(6):2229-38. 

Rec #: 759 

 179. Levy, C. R.; Fish, R., and Kramer, A. M. (Division of Health Care Policy and Research, University of 

Colorado, 13611 E. Colfax Ave., Suite 100, Aurora, CO 80011; cari.levy@uchsc.edu.). Site of death 

in the hospital versus nursing home of Medicare skilled nursing facility residents admitted under 

Medicare's Part A Benefit. Journal of the American Geriatrics Society. 2004 Aug; 52(8):1247-54. 

Rec #: 542 

 180. Lorenz, K. A. and Lynn, J. Oregon's lessons for improving advance care planning. J Am Geriatr Soc. 2004 

Sep; 52(9):1574-5. 

Rec #: 500 

 181. Mancini, I. and Body, J. J. Assessment of dyspnea in advanced cancer patients. Support Care Cancer. 1999 

Jul; 7(4):229-32. 

Rec #: 443 

 182. Manfredi, P. L.; Chandler, S.; Pigazzi, A., and Payne, R. Outcome of cancer pain consultations. Cancer. 

2000; 89(4):920-4. 

Rec #: 368 

 183. Manning, H. L. Dyspnea treatment. Respir Care. 2000 Nov; 45(11):1342-50; discussion 1350-4. 

Rec #: 211 

 184. Marik, P. E. and Craft, M. An outcomes analysis of in-hospital cardiopulmonary resuscitation: the futility 

rationale for do not resuscitate orders. J Crit Care. 1997 Sep; 12(3):142-6. 

Rec #: 594 

 185. Markson, L.; Clark, J.; Glantz, L.; Lamberton, V.; Kern, D., and Stollerman, G. The doctor's role in 

discussing advance preferences for end-of-life care: perceptions of physicians practicing in the VA. J 

Am Geriatr Soc. 1997 Apr; 45(4):399-406. 

Rec #: 4791 

 186. Martinez, M. J.; Roque, M.; Alonso-Coello, P.; Catala, E.; Garcia, J. L., and Ferrandiz, M. Calcitonin for 

metastatic bone pain. Cochrane Database Syst Rev. 2003; (3):CD003223. 

Rec #: 244 

 187. Maunsell, E.; Allard, P.; Dorval, M., and Labbe, J. A brief pain diary for ambulatory patients with advanced 

cancer: acceptability and validity. Cancer. 2000 May 15; 88(10):2387-97. 

Rec #: 275 

 188. McDonald, M. V.; Passik, S. D.; Dugan, W.; Rosenfeld, B.; Theobald, D. E., and Edgerton, S. Nurses' 

recognition of depression in their patients with cancer. Oncol Nurs Forum. 1999 Apr; 26(3):593-9. 

Rec #: 434 
H-30 

 189. McGlynn, E. A. Applying the strategic framework board's model to select national goals and core measures 

for stimulating improved care for cancer. 2002 Aug 21; RAND Health . 

Rec #: 105 

 190. McGlynn, E. A. and Malin, J. Selecting national goals and core measures of cancer care quality (Background 

paper #2). RAND. 2002 Aug 21. 

Rec #: 103 

 191. McLachlan, S.; Allenby, A.; Matthews, J.; Wirth, A.; Kissane, D.; Bishop, M.; Beresford, J., and Zalcberg, J. 

(Family Cancer Centre, Peter MacCallum Cancer Institute, Locked Bag 1, A'Beckett St PO, 

Melbourne, Victoria 8006, Australia; mclachsa@svhm.org.au.). Randomized trial of coordinated 

psychosocial interventions based on patient self-assessments versus standard care to improve the 

psychosocial functioning of patients with cancer. Journal of Clinical Oncology. 2001 Nov 1; 

19(21):4117-25. 

Rec #: 408 

 192. McMillan, S. C. (University of South Florida, College of Nursing, MDC 22, 12901 Bruce B Downs 

Boulevard, Tampa, FL 33612.). Pain and pain relief experienced by hospice patients with cancer. 

Cancer Nursing. 1996 Aug; 19(4):298-307. 

Rec #: 336 

 193. McNicol, E.; Horowicz-Mehler, N.; Fisk, R. A.; Bennett, K.; Gialeli-Goudas, M.; Chew, P. W.; Lau, J., and 

Carr, D. Management of opioid side effects in cancer-related and chronic noncancer pain: a 

systematic review. J Pain. 2003 Jun; 4(5):231-56. 

Rec #: 237 

 194. McParland, E.; Likourezos, A.; Chichin, E.; Castor, T., and Paris, B. E. BE. Stability of preferences regarding 

life-sustaining treatment: a two-year prospective study of nursing home residents. Mt Sinai J Med. 

2003 Mar; 70(2):85-92. 

Rec #: 42225 

 195. McQuay, H. J.; Moore, R. A.; Eccleston, C.; Morley, S., and Williams, A. C. Systematic review of outpatient 

services for chronic pain control. Health Technol Assess. 1997; 1(6):i-iv, 1-135. 

Rec #: 287 

 196. Mercadante, S. Pain treatment and outcomes for patients with advanced cancer who receive follow-up care at 

home. Cancer. 1999 Apr 15; 85(8):1849-58. 

Rec #: 282 

 197. Mercadante, S.; Agnello, A.; Armata, m. G., and Pumo, S. (Pain Relief and Palliative Care, SAMOT, via 

Liberta 191, 90141, Palermo, Italy.). Clinical note. The inappropriate use of the epidural route in 

cancer pain. Journal of Pain and Symptom Management. 1997 Apr; 13(4):233-7. 

Rec #: 351 

 198. Mercadante, S.; Radbruch, L.; Caraceni, A.; Cherny, N.; Kaasa, S.; Nauck, F.; Ripamonti, C., and De Conno, 

F. Episodic (breakthrough) pain: consensus conference of an expert working group of the European 

Association for Palliative Care. Cancer. 2002 Feb 1; 94(3):832-9. 

Rec #: 258 

 199. Meuser, T.; Pietruck, C.; Radbruch, L.; Stute, P.; Lehmann, K. A., and Grond, S. Symptoms during cancer 

pain treatment following WHO-guidelines: a longitudinal follow-up study of symptom prevalence, 

severity and etiology. Pain. 2001 Sep; 93(3):247-57. 

Rec #: 266 
H-31 

 200. Meyer, H. A.; Sinnott, C., and Seed, P. T. Depressive symptoms in advanced cancer. Part 2. Depression over 

time; the role of the palliative care professional. Palliat Med. 2003 Oct; 17(7):604-7. 

Rec #: 450 

 201. Mezey, M.; Kluger, M.; Maislin, G., and Mittelman, M. Life-sustaining treatment decisions by spouses of 

patients with Alzheimer's disease. J Am Geriatr Soc. 1996 Feb; 44(2):144-50. 

Rec #: 613 

 202. Mezey, Mathy; Teresi, Jeanne; Ramsey, Gloria; Mitty, Ethel, and Bobrowitz, Tracy. Decision-making 

capacity to execute a heath care proxy: Development and testing of guidelines. Journal of the 

American Geriatrics Society. 2000; 48(2):179-187 URL http //www.blackwellscience.com/~cgilib/bsinc.bin?Journal=geriatrics. 

Rec #: 493 

 203. Miaskowski, C. New approaches for evaluating the quality of cancer pain management in the outpatient 

setting. Pain Manag Nurs. 2001 Mar; 2(1):7-12. 

Rec #: 262 

 204. Miaskowski, C.; Mack, K. A.; Dodd, M.; West, C.; Paul, S. M.; Tripathy, D.; Koo, P.; Schumacher, K., and 

Facione, N. (School of Nursing, University of California, San Francisco, CA, 

Chris.Miaskowski@nursing.ucsf.edu.). Oncology outpatients with pain from bone metastasis require 

more than around-the-clock dosing of analgesics to achieve adequate pain control. Journal of Pain. 

2002 Feb; 3(1):12-20. 

Rec #: 313 

 205. Midewest Bioethics Center. Health care treatment decision-making guidelines for minors. Midwest 

Bioethics Center. 

Rec #: 711 

 206. Miller, E. H.; Belgrade, M. J.; Cook, M.; Portu, J. B.; Shepherd, M.; Sierzant, T.; Sallmen, P., and Fraki, S. 

Institutionwide pain management improvement through the use of evidence-based content, 

strategies, resources, and outcomes. Qual Manag Health Care. 1999 Winter; 7(2):28-40. 

Rec #: 279 

 207. Mitchell, S. L. and Kiely, D. K. (Division of Geriatrics, Loeb Health Research Institute, Ottawa Hospital, 

University of Ottawa, Ottawa, Ontario.). A cross-national comparison of institutionalized tube-fed 

older persons: the influence of contrasting healthcare systems. Journal of the American Medical 

Directors Association. 2001 Jan-2001 Feb 28; 2(1):10-4. 

Rec #: 568 

 208. Morita, T.; Hirai, K.; Sakaguchi, Y.; Maeyama, E.; Tsuneto, S., and Shima, Y. Measuring the quality of 

structure and process in end-of-life care from the bereaved family perspective. J Pain Symptom 

Manage. 2004 Jun; 27(6):492-501. 

Rec #: 392 

 209. Morrison, R. S.; Siu, A. L.; Leipzig, R. M.; Cassel, C. K., and Meier, D. E. The hard task of improving the 

quality of care at the end of life. Arch Intern Med. 2000 Mar 27; 160(6):743-7. 

Rec #: 13876 

 210. Murdaugh, C.; Parsons, M.; Gryb-Wysocki, T.; Palmer, J.; Glasby, C.; Bonner, J., and Tavakoli, A. (College 

of Nursing, University of South Carolina.). Implementing a quality of care model in a restructured 

hospital environment. National Academies of Practice Forum: Issues in Interdisciplinary Care 

(NAPF). 1999 Jul; 1(3):219-26. 

Rec #: 342 
H-32 

 211. Mystakidou K; Parpa E; Tsilika E; Katsouda E; Kouloulias V; Kouvaris J; Georgaki S, and Vlahos L. Pain 

management of cancer patients with transdermal fentanyl: A study of 1828 step I, II, & III transfers. 

J Pain. 2004 Mar; 5(2):119-32. 

Rec #: 360 

 212. National Comprehensive Cancer Network (NCCN). Cancer pain. Clinical Practice Guidelines in Oncology. 

2004(1). 

Rec #: 108 

 213. National Comprehensive Cancer Network (NCCN). Central nervous system cancers . National 

Comprehensive Cancer Network; 2005 Jan 1(Practice Guidelines in Oncology; version 1.2204 ). 

Rec #: 662 

 214. National Comprehensive Cancer Network (NCCN). Chronic Myelogenous Leukemia . National 

Comprehensive Cancer Network (NCCN); 2004 Nov 23(Practice Guidelines in Oncology; version 

2.2005). 

Rec #: 664 

 215. National Comprehensive Cancer Network (NCCN). Gastric cancer. National Comprehensive Cancer 

Network; 2005 Jan 12(Practice Guidleines in Oncology; version 1.2005). 

Rec #: 667 

 216. National Comprehensive Cancer Network (NCCN). Kidney cancer. National Comprehensive Cancer 

Network; 2005 Feb 7(Practice Guidelines in Oncology; version 2.2005). 

Rec #: 671 

 217. --. Multiple Myeloma. National Comprehensive Cancer Network; 2004 Nov 1(Practice Guidelines in 

Oncology; version 1.2005). 

Rec #: 674 

 218. National Comprehensive Cancer Network (NCCN). Senior adult oncology. Clinical Practice Guidelines in 

Oncology. 2004(1). 

Rec #: 107 

 219. National Institute for Clinical Excellence. Improving outcomes in breast cancer manual update. London: 

National Institute for Clinical Excellence; 2002 Aug(Guidance on Cancer Services). 

Rec #: 171 

 220. --. Improving outcomes in colorectal cancers manual update. London: National Institute for Clinical 

Excellence; 2004 May(Guidance on Cancer Services). 

Rec #: 172 

 221. National Pharmaceutical Council, Inc. NPC and Joint Commission on Accreditation of Healthcare 

Organizations (JCAHO). Pain: current understanding of assessment, management and treatments. 

2001 Dec. 

Rec #: 746 

 222. Nicholson, A. B. Methadone for cancer pain. Cochrane Database Syst Rev. 2004; (2):CD003971. 

Rec #: 231 

 223. Norris, P. Discontinuing life support in the midst of doubt. Health Care Ethics USA. 1996 Spring; 4(2):4-5. 

Rec #: 538 
H-33 

 224. Nowels D and Lee JT. Cancer pain management in home hospice settings: a comparison of primary care and 

oncologic physicians. J Palliat Care. 1999 Autumn; 15(3):5-9. 

Rec #: 361 

 225. O'Driscoll, M; Corner, J, and Bailey, C. The experience of breathlessness in lung cancer. Eur J Cancer Care 

(Engl). 1999 Mar; 8(1):37-43. 

Rec #: 453 

 226. O'Shea, E. M.; Penson, R. T.; Stern, T. A.; Younger, J.; Chabner, B. A., and Lynch, T. J. Jr. (HematologyOncology Department, Massachusetts General Hospital Cancer Center, Boston, MA.). Schwartz 

Center rounds. A staff dialogue on do not resuscitate orders: psychosocial issues faced by patients, 

their families, and caregivers. Oncologist. 1999 Jul; 4(3):256-62. 

Rec #: 586 

 227. Oliver, D. P.; Porock, D., and Zweig, S. End-of-life care in U.S. nursing homes: a review of the evidence. J 

Am Med Dir Assoc. 2004 May-2004 Jun 30; 5(3):147-55. 

Rec #: 515 

 228. Oncology Nursing Society. Cancer pain management; 2004 Oct(Oncology Nursing Society Position). 

Rec #: 134 

 229. Osse, B. H.; Vernooij, M. J.; Schade, E., and Grol, R. P. Towards a new clinical tool for needs assessment in 

the palliative care of cancer patients: the PNPC instrument. J Pain Symptom Manage. 2004 Oct; 

28(4):329-41. 

Rec #: 227 

 230. Owen, J. E.; Klapow, J. C.; Hicken, B., and Tucker, D. C. Psychosocial interventions for cancer: review and 

analysis using a three-tiered outcomes model. Psychooncology. 2001 May-2001 Jun 30; 10(3):218-

30. 

Rec #: 420 

 231. Pacala, J. T.; Kane, R. L.; Atherly, A. J., and Smith, M. A. Using structured implicit review to assess quality 

of care in the Program of All-Inclusive Care for the Elderly (PACE). J Am Geriatr Soc. 2000 Aug; 

48(8):903-10. 

Rec #: 636 

 232. Paice, J. A.; Toy, C., and Shott, S. (Department of Neurosurgery, Rush Medical Center, 1725 West Harrison 

Street, Suite 755, Chicago, IL 60612.). Barriers to cancer pain relief: fear of tolerance and addiction. 

Journal of Pain and Symptom Management. 1998 Jul; 16(1):1-9. 

Rec #: 349 

 233. Palliative Care and Ethics Committees. Do not resusciate orders. Guidelines for Physician Staff Froedtert 

Hospital, Milwaukee, Wisconsin:1-5. 

Rec #: 138 

 234. ---. Managing conflicts concerning requests to withhold or withdraw life-sustaining medical treatment. 

Guidelines for Physician Staff Froedtert Hospital, Milwaukee, Wisconsin:1-5. 

Rec #: 137 

 235. Palliative Care Committee. Use of pentobarbital for total sedation and ventilator withdrawal. Guidelines for 

Physicia Staff Froedtert Hospital, Milwaukee, Wisconsin:1-3. 

Rec #: 135 
H-34 

 236. Pasacreta, J. V. (Associate Professor, Yale University School of Nursing, New Haven, Connecticut.). Review: 

psychoeducational interventions reduce depressive symptoms in cancer. Evidence-Based Nursing. 

2002 Oct; 5(4):113. 

Rec #: 398 

 237. Passik, S. D.; Dugan, W.; McDonald, M. V.; Rosenfeld, B.; Theobald, D. E., and Edgerton, S. Oncologists' 

recognition of depression in their patients with cancer. J Clin Oncol. 1998 Apr; 16(4):1594-600. 

Rec #: 438 

 238. Passik, S. D.; Kirsh, K. L.; Theobald, D.; Donaghy, K.; Holtsclaw, E.; Edgerton, S., and Dugan, W. 

(Symptom Management and Palliative Care Program, University of Kentucky College of Medicine, 

800 Rose Street, Lexington, KY 40536-0093.). Use of a depression screening tool and a fluoxetinebased algorithm to improve the recognition and treatment of depression in cancer patients: a 

demonstration project. Journal of Pain and Symptom Management. 2002 Sep; 24(3):318-27. 

Rec #: 399 

 239. Patrick, D. L.; Ferketich, S. L.; Frame, P. S.; Harris, J. J.; Hendricks, C. B.; Levin, B.; Link, M. P.; Lustig, C.; 

McLaughlin, J.; Ried, L. D.; Turrisi, A. T. 3rd; Unutzer, J., and Vernon, S. W. National Institutes of 

Health State-of-the-Science Conference Statement: Symptom Management in Cancer: Pain, 

Depression, and Fatigue, July 15-17, 2002. J Natl Cancer Inst. 2003 Aug 6; 95(15):1110-7. 

Rec #: 245 

 240. Patrick, D. L.; Pearlman, R. A.; Starks, H. E.; Cain, K. C.; Cole, W. G., and Uhlmann, R. F. Validation of 

preferences for life-sustaining treatment: implications for advance care planning. Ann Intern Med. 

1997 Oct 1; 127(7):509-17. 

Rec #: 5877 

 241. Payne, R. Practice guidelines for cancer pain therapy. Issues pertinent to the revision of national guidelines. 

Oncology (Huntingt). 1998 Nov; 12(11A):169-75. 

Rec #: 284 

 242. Percy, M. E. and Llewellyn-Thomas, H. Assessing preferences about the DNR order: does it depend on how 

you ask? Med Decis Making. 1995 Jul-1995 Sep 30; 15(3):209-16. 

Rec #: 616 

 243. Phillips, R. S.; Wenger, N. S.; Teno, J.; Oye, R. K.; Youngner, S.; Califf, R.; Layde, P.; Desbiens, N.; 

Connors, A. F. Jr, and Lynn, J. Choices of seriously ill patients about cardiopulmonary resuscitation: 

correlates and outcomes. SUPPORT Investigators. Study to Understand Prognoses and Preferences 

for Outcomes and Risks of Treatments. Am J Med. 1996 Feb; 100(2):128-37. 

Rec #: 612 

 244. Phipps, E. J.; True, G., and Murray, G. F. Community perspectives on advance care planning: report from the 

Community Ethics Program. J Cult Divers. 2003 Winter; 10(4):118-23. 

Rec #: 6016 

 245. Portenoy, R. K. and Itri, L. M. (Department of Pain Medicine and Palliative Care, Beth Israel Medical Center, 

New York, NY 10003; e-mail: RPortenoy@BethIsraelNY.org.). Cancer-related fatigue: guidelines 

for evaluation and management. Oncologist. 1999; 4(1):1-10. 

Rec #: 412 

 246. Prendergast, T. J. Advance care planning: pitfalls, progress, promise. Crit Care Med. 2001 Feb; 29(2 

Suppl):N34-9. 

Rec #: 555 
H-35 

 247. Program in Evidence-Based Care. Radiopharmaceuticals for the palliation of painful bone metastases. 

Practice Guideline Report #14-1. Program in Evidence-Based Care; 2004 Jun 15(Cancer Care 

ONtario Practice Guidelines Initiative). 

Rec #: 206 

 248. Puchalski, C. M.; Zhong, Z.; Jacobs, M. M.; Fox, E.; Lynn, J.; Harrold, J.; Galanos, A.; Phillips, R. S.; Califf, 

R., and Teno, J. M. Patients who want their family and physician to make resuscitation decisions for 

them: observations from SUPPORT and HELP. Study to Understand Prognoses and Preferences for 

Outcomes and Risks of Treatment. Hospitalized Elderly Longitudinal Project. J Am Geriatr Soc. 

2000 May; 48(5 Suppl):S84-90. 

Rec #: 569 

 249. Quigley, C. Hydromorphone for acute and chronic pain. Cochrane Database Syst Rev. 2002; (1):CD003447. 

Rec #: 257 

 250. Quigley, C. and Wiffen, P. A systematic review of hydromorphone in acute and chronic pain. Journal of Pain 

& Symptom Management. 2003; 25(2):169-78. 

Rec #: 369 

 251. Ramsey, G. and Mitty, E. L. Advance directives: protecting patient's rights. Mezey, M.; Fulmer, T.; Abraham, 

I., and Zwicker, D. A. Geriatric nursing protocols for best practice. New York (NY): Springer 

Publishing Company, Inc.; 2003; pp. 265-291. 

Rec #: 649 

 252. Ratanatharathorn, V.; Powers, W. E.; Moss, W. T., and Perez, C. A. Bone metastasis: review and critical 

analysis of random allocation trials of local field treatment. Int J Radiat Oncol Biol Phys. 1999 Apr 

1; 44(1):1-18. 

Rec #: 283 

 253. Ratner, E.; Norlander, L., and McSteen, K. Death at home following a targeted advance-care planning process 

at home: the kitchen table discussion. J Am Geriatr Soc. 2001 Jun; 49(6):778-81. 

Rec #: 6278 

 254. Redman, B. K. Ethical issues in the development and use of guidelines for clinical practice. J Clin Ethics. 

1996 Fall; 7(3):251-6. 

Rec #: 295 

 255. Reifel, J. Cancer pain and palliation. Asch, S. M.; Kerr, E. A.; Hamilton, E. G.; Reifel, J. L., and McGlynn, E. 

A. Quality of care for oncologic condition and HIV: a review of the literature and quality indicators. 

RAND; 2000; pp. 227-33. 

Rec #: 656 

 256. Reifel, J. L. Lung cancer. Asch, S. M.; Kerr, E. A.; Hamilton, E. G.; Reifel, J. L., and McGlynn, E. A. Quality 

of care for oncologic condition and HIV: a review of the literature and quality indicators. RAND; 

2000; pp. 133-71. 

Rec #: 657 

 257. Reilly, B. M.; Wagner, M.; Magnussen, C. R.; Ross, J.; Papa, L., and Ash, J. Promoting inpatient directives 

about life-sustaining treatments in a community hospital. Results of a 3-year time-series intervention 

trial. Arch Intern Med. 1995 Nov 27; 155(21):2317-23. 

Rec #: 614 
H-36 

 258. Renal Physicians Association. Shared decision-making in the appropriate initiation of and withdrawal from 

dialysis: Renal Physicians Association and American Society of Nephrology; 2000 Feb(Clinical 

Practice Guideline; 2). 

Rec #: 751 

 259. Rischer, J. B. and Childress, S. B. Cancer pain management: pilot implementation of the AHCPR guideline in 

Utah. Jt Comm J Qual Improv. 1996 Oct; 22(10):683-700. 

Rec #: 293 

 260. Rita, J. Palliative treatment of cancer. Duodecim Medical Publications Ltd.; 2003 May 30(EBM 

Guidelines). 

Rec #: 644 

 261. Roos, D. E. Continuing reluctance to use single fractions of radiotherapy for metastatic bone pain: an 

Australian and New Zealand practice survey and literature review. Radiother Oncol. 2000 Sep; 

56(3):315-22. 

Rec #: 273 

 262. Roque, M.; Martinez, M. J.; Alonso-Coello, P.; Catala, E. ; Garcia, J. L., and Ferrandiz, M. Radioisotopes for 

metastatic bone pain. The Cochrane Library, (Oxford) ** (4)2004. 2004; (ID #CD003347). 

Rec #: 320 

 263. Roque, M.; Martinez, M. J.; Alonso, P.; Catala, E.; Garcia, J. L., and Ferrandiz, M. Radioisotopes for 

metastatic bone pain. Cochrane Database Syst Rev. 2003; (4):CD003347. 

Rec #: 239 

 264. Rosenfeld, K. and Wenger, N. S. (Dept of General Medicine (111G), VA Greater Los Angeles Healthcare 

System, 11301 Wilshire Blvd., Los Angeles, CA 90012.). Measuring quality in end-of-life care. 

Clinics in Geriatric Medicine. 2000 May; 16(2):387-400. 

Rec #: 576 

 265. Rosenfeld, K. E.; Wenger, N. S.; Phillips, R. S.; Connors, A. F.; Dawson, N. V.; Layde, P.; Califf, R. M.; Liu, 

H.; Lynn, J., and Oye, R. K. Factors associated with change in resuscitation preference of seriously 

ill patients. The SUPPORT Investigators. Study to Understand Prognoses and Preferences for 

Outcomes and Risks of Treatments. Arch Intern Med. 1996 Jul 22; 156(14):1558-64. 

Rec #: 610 

 266. Roter, D. L.; Larson, S.; Fischer, G. S.; Arnold, R. M., and Tulsky, J. A. Experts practice what they preach: A 

descriptive study of best and normative practices in end-of-life discussions. Arch Intern Med. 2000 

Dec 11-2000 Dec 25; 160(22):3477-85. 

Rec #: 6541 

 267. Roth, A. J.; Kornblith, A. B.; Batel-Copel, L.; Peabody, E.; Scher, H. I., and Holland, J. C. Rapid screening 

for psychologic distress in men with prostate carcinoma: a pilot study. Cancer. 1998 May 15; 

82(10):1904-8. 

Rec #: 437 

 268. Rustoen, T.; Fossa, S. D.; Skarstein, J., and Moum, T. (Oslo University College, Faculty of Nursing, Falbesgt 

5, 0167 Oslo, Norway.). The impact of demographic and disease-specific variables on pain in cancer 

patients. Journal of Pain and Symptom Management. 2003 Aug; 26(2):696-704. 

Rec #: 331 
H-37 

 269. Saliba, D.; Kington, R.; Buchanan, J.; Bell, R.; Wang, M.; Lee, M.; Herbst, M.; Lee, D.; Sur, D., and 

Rubenstein, L. (Geriatric Research Education and Clinical Center, VA Greater Los Angeles Health 

Care System, CA.). Appropriateness of the decision to transfer nursing facility residents to the 

hospital. Journal of the American Geriatrics Society. 2000 Feb; 48(2):154-63. 

Rec #: 581 

 270. Saliba, D.; Solomon, D.; Rubenstein, L.; Young, R.; Schnelle, J.; Roth, C., and Wenger, N. Quality indicators 

for the management of medical conditions in nursing home residents. J Am Med Dir Assoc. 2004 

Sep-2004 Oct 31; 5(5):297-309. 

Rec #: 625 

 271. Santa-Emma, P. H.; Roach, R.; Gill, M. A.; Spayde, P., and Taylor, R. M. Development and implementation 

of an inpatient acute palliative care service. Journal of Palliative Medicine. 2002; 5(1):93-100. 

Rec #: 370 

 272. Sarna, L. Effectiveness of structured nursing assessment of symptom distress in advanced lung cancer. Oncol 

Nurs Forum. 1998 Jul; 25(6):1041-8. 

Rec #: 435 

 273. Schmidt, K. L.; Alpen, M. A., and Rakel, B. A. Implementation of the Agency for Health Care Policy and 

Research Pain Guidelines. AACN Clin Issues. 1996 Aug; 7(3):425-35. 

Rec #: 294 

 274. Schonwetter, R. S.; Walker, R. M.; Solomon, M.; Indurkhya, A., and Robinson, B. E. Life values, 

resuscitation preferences, and the applicability of living wills in an older population. J Am Geriatr 

Soc. 1996 Aug; 44(8):954-8. 

Rec #: 609 

 275. Schwartz, C. E.; Merriman, M. P.; Reed, G. W., and Hammes, B. J. Measuring patient treatment preferences 

in end-of-life care research: applications for advance care planning interventions and response shift 

research. J Palliat Med. 2004 Apr; 7(2):233-45. 

Rec #: 514 

 276. Scottish Intercollegiate Guidelines Network (SIGN) and Scottish Cancer Therapy Network. Control of pain 

in patients with cancer. 2000. 

Rec #: 717 

 277. Sheard, T. and Maguire, P. The effect of psychological interventions on anxiety and depression in cancer 

patients: results of two meta-analyses. Br J Cancer. 1999 Aug; 80(11):1770-80. 

Rec #: 430 

 278. Shega, J. W.; Levin, A.; Hougham, G. W.; Cox-Hayley, D.; Luchins, D.; Hanrahan, P.; Stocking, C., and 

Sachs, G. A. Palliative Excellence in Alzheimer Care Efforts (PEACE): a program description. J 

Palliat Med. 2003 Apr; 6(2):315-20. 

Rec #: 50732 

 279. Silvestri, G. A. Palliation for the dying patient with lung cancer. Respir Care. 2000 Dec; 45(12):1490-6; 

discussion 1497-500. 

Rec #: 220 

 280. Sloan, P. A.; Vanderveer, B. L.; Snapp, J. S.; Johnson, M., and Sloan, D. A. (Department of Anesthesiology, 

University of Kentucky Hospital, 800 Rose St., Lexington, KY 40536.). Cancer pain assessment and 

management recommendations by hospice nurses. Journal of Pain and Symptom Management. 1999 

Aug; 18(2):103-10. 

Rec #: 344 
H-38 

 281. Smith, E. L.; Whedon, M. B., and Bookbinder, M. (Dartmouth-Hitchcock Medical Center/Norris Cotton 

Cancer Center, One Medical Center Dr, Lebanon, NH 03756.). Quality improvement of painful 

peripheral neuropathy. Seminars in Oncology Nursing. 2002 Feb; 18(1):36-43. 

Rec #: 312 

 282. Somogyi-Zalud, E.; Zhong, Z.; Hamel, M. B., and Lynn, J. The use of life-sustaining treatments in 

hospitalized persons aged 80 and older. J Am Geriatr Soc. 2002 May; 50(5):930-4. 

Rec #: 531 

 283. Somogyi-Zalud, E.; Zhong, Z.; Lynn, J., and Hamel, M. B. Elderly persons' last six months of life: findings 

from the Hospitalized Elderly Longitudinal Project. J Am Geriatr Soc. 2000 May; 48(5 Suppl):S131-

9. 

Rec #: 7221 

 284. Song, M. K. Effects of end-of-life discussions on patients' affective outcomes. Nurs Outlook. 2004 May-2004 

Jun 30; 52(3):118-25. 

Rec #: 510 

 285. Sorenson, H. M. Dyspnea assessment. Respir Care. 2000 Nov; 45(11):1331-8;discussion 1338-41. 

Rec #: 210 

 286. ---. Managing secretions in dying patients. Respir Care. 2000 Nov; 45(11):1355-63; discussion 1363-4. 

Rec #: 212 

 287. Spector, N. and Klein, D. (Assistant Professor, Loyola University, 6525 N Sheridan Road, Chicago, IL 

60626.). Chronic critically ill dyspneic patients: mechanisms and clinical measurement. AACN 

Clinical Issues: Advanced Practice in Acute and Critical Care. 2001 May; 12(2):220-33. 

Rec #: 383 

 288. Srebnik, D.; Appelbaum, P. S., and Russo, J. Assessing competence to complete psychiatric advance 

directives with the competence assessment tool for psychiatric advance directives. Compr 

Psychiatry. 2004 Jul-2004 Aug 31; 45(4):239-45. 

Rec #: 508 

 289. Stein, W. M. and Miech, R. P. Cancer pain in the elderly hospice patient. J Pain Symptom Manage. 1993 Oct; 

8(7):474-82. 

Rec #: 362 

 290. Sterman, E.; Gauker, S., and Krieger, J. (Clinical Nurse Specialist, Newark Beth Israel Medical Center, New 

Jersey; eers619@aol.com.). A comprehensive approach to improving cancer pain management and 

patient satisfaction. Oncology Nursing Forum. 2003 Sep-2003 Oct 31; 30(5):857-64. 

Rec #: 323 

 291. Stiefel, R.; Die Trill, M.; Berney, A.; Olarte, J. M., and Razavi, A. Depression in palliative care: a pragmatic 

report from the Expert Working Group of the European Association for Palliative Care. Support 

Care Cancer. 2001 Oct; 9(7):477-88. 

Rec #: 725 

 292. Strasser, F.; Sweeney, C.; Willey, J.; Benisch-Tolley, S.; Palmer, J. L., and Bruera, E. Impact of a half-day 

multidisciplinary symptom control and palliative care outpatient clinic in a comprehensive cancer 

center on recommendations, symptom intensity, and patient satisfaction: a retrospective descriptive 

study. J Pain Symptom Manage. 2004 Jun; 27(6):481-91. 

Rec #: 393 
H-39 

 293. Straton, J. B.; Wang, N. Y.; Meoni, L. A.; Ford, D. E.; Klag, M. J.; Casarett, D., and Gallo, J. J. Physical 

functioning, depression, and preferences for treatment at the end of life: the Johns Hopkins 

Precursors Study. J Am Geriatr Soc. 2004 Apr; 52(4):577-82. 

Rec #: 7406 

 294. Stromgren, A. S.; Goldschmidt, D.; Groenvold, M.; Petersen, M. A.; Jensen, P. T.; Pedersen, L.; Hoermann, 

L.; Helleberg, C., and Sjogren, P. Self-assessment in cancer patients referred to palliative care: a 

study of feasibility and symptom epidemiology. Cancer. 2002 Jan 15; 94(2):512-20. 

Rec #: 416 

 295. Stromgren, A. S.; Groenvold, M.; Pedersen, L.; Olsen, A. K.; Spile, M., and Sjogren, P. (Department of 

Palliative Medicine, Bispebjerg Hospital, DK-2400 Copenhagen NV, Denmark.). Does the medical 

record cover the symptoms experienced by cancer patients receiving palliative care? A comparison 

of the record and patient self-rating. Journal of Pain and Symptom Management. 2001 Mar; 

21(3):189-96. 

Rec #: 410 

 296. Stromgren, A. S.; Groenvold, M.; Petersen, M. A.; Goldschmidt, D.; Pedersen, L.; Spile, M.; IrmingPedersen, G., and Sjogren, P. (Department of Palliative Medicine, Bispebjerg Hospital, DK-2400 

Copenhagen NV, Denmark.). Pain characteristics and treatment outcome for advanced cancer 

patients during the first week of specialized palliative care. Journal of Pain and Symptom 

Management. 2004 Feb; 27(2):104-13. 

Rec #: 317 

 297. Stromgren, A. S.; Groenvold, M.; Sorensen, A., and Andersen, L. Symptom recognition in advanced cancer. 

A comparison of nursing records against patient self-rating. Acta Anaesthesiol Scand. 2001 Oct; 

45(9):1080-5. 

Rec #: 263 

 298. Sulmasy, D. P.; Dwyer, M., and Marx, E. Do the ward notes reflect the quality of end-of-life care? J Med 

Ethics. 1996 Dec; 22(6):344-8. 

Rec #: 15638 

 299. Sulmasy, D. P. and McIlvane, J. M. Patients' ratings of quality and satisfaction with care at the end of life. 

Arch Intern Med. 2002 Oct 14; 162(18):2098-104. 

Rec #: 7440 

 300. Sulmasy, D. P.; Sood, J. R., and Ury, W. A. The quality of care plans for patients with do-not-resuscitate 

orders. Arch Intern Med. 2004 Jul 26; 164(14):1573-8. 

Rec #: 504 

 301. Sulmasy, Daniel P.; McIlvane, Jessica M.; Pasley, Peter M., and Rahn, Maike. A scale for measuring patient 

perceptions of the Quality of End-of-Life Care and Satisfaction with Treatment: The reliability and 

validity of QUEST. Journal of Pain & Symptom Management. 2002; 23(6):458-470 URL http 

//www.elsevier.com/inca/publications/store/5/0/5/7/7/5/. 

Rec #: 492 

 302. Sze, W. M.; Shelley, M. D.; Held, I.; Wilt, T. J., and Mason, M. D. Palliation of metastatic bone pain: single 

fraction versus multifraction radiotherapy--a systematic review of randomised trials. Clin Oncol (R 

Coll Radiol). 2003 Sep; 15(6):345-52. 

Rec #: 240 

 303. Tarzian, A. J. and Hoffmann, D. E. Barriers to managing pain in the nursing home: findings from a statewide 

survey. J Am Med Dir Assoc. 2004 Mar-2004 Apr 30; 5(2):82-8. 

Rec #: 235 
H-40 

 304. Tattersall MH; Gattellari M; Voigt K, and Butow PN. When the treatment goal is not cure: are patients 

informed adequately? Support Care Cancer. 2002 May; 10(4):314-21. 

Rec #: 7561 

 305. Teno, J.; Lynn, J.; Wenger, N.; Phillips, R. S.; Murphy, D. P.; Connors, A. F. Jr; Desbiens, N.; Fulkerson, W.; 

Bellamy, P., and Knaus, W. A. Advance directives for seriously ill hospitalized patients: 

effectiveness with the patient self-determination act and the SUPPORT intervention. SUPPORT 

Investigators. Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatment. 

J Am Geriatr Soc. 1997 Apr; 45(4):500-7. Comment in: J Am Geriatr Soc. 1997 Apr;45(4):519-20. 

PMID: 9100724. 

Rec #: 43327 

 306. Teno, J. M.; Branco, K. J.; Mor, V.; Phillips, C. D.; Hawes, C.; Morris, J., and Fries, B. E. (Center for 

Gerontology and Health Care Research, Brown University, Providence, RI.). Changes in advance 

care planning in nursing homes before and after the Patient Self-Determination Act: report of a 10-

state survey. Journal of the American Geriatrics Society. 1997 Aug; 45(8):939-44. 

Rec #: 592 

 307. Teno, J. M.; Clarridge, B.; Casey, V.; Edgman-Levitan, S., and Fowler, J. Validation of Toolkit After-Death 

Bereaved Family Member Interview. J Pain Symptom Manage. 2001 Sep; 22(3):752-8. 

Rec #: 7592 

 308. Teno, J. M.; Kabumoto, G.; Wetle, T.; Roy, J., and Mor, V. Daily pain that was excruciating at some time in 

the previous week: prevalence, characteristics, and outcomes in nursing home residents. J Am 

Geriatr Soc. 2004 May; 52(5):762-7. 

Rec #: 234 

 309. The International Psychooncology Society (IPOS). Psychosocial component in cancer control. 2004 Aug. 

Rec #: 488 

 310. The Robert Wood Johnson Foundation. Completing the continuum of ALS care: a consensus document: The 

Robert Wood Johnson Foundation; 2004 Jan(Promoting Excellence in End-of-Life Care). 

Rec #: 752 

 311. Tilden, V. P.; Tolle, S. W.; Drach, L. L., and Perrin, N. A. Out-of-hospital death: advance care planning, 

decedent symptoms, and caregiver burden. J Am Geriatr Soc. 2004 Apr; 52(4):532-9. 

Rec #: 7664 

 312. Tilden, V. P.; Tolle, S. W.; Nelson, C. A., and Fields, J. Family decision-making to withdraw life-sustaining 

treatments from hospitalized patients. Nurs Res. 2001 Mar-2001 Apr 30; 50(2):105-15. 

Rec #: 50907 

 313. Todd J and Edmonds P. Caring for patients wanting a home death. Practitioner. 2000 Jun; 244(1611):580-6. 

Rec #: 15873 

 314. Tolle, S. W.; Tilden, V. P.; Nelson, C. A., and Dunn, P. M. A prospective study of the efficacy of the 

physician order form for life-sustaining treatment. J Am Geriatr Soc. 1998 Sep; 46(9):1097-102. 

Rec #: 588 

 315. Tolle SW; Rosenfeld AG; Tilden VP, and Park Y. Oregon's low in-hospital death rates: what determines 

where people die and satisfaction with decisions on place of death? Ann Intern Med. 1999 Apr 20; 

130(8):681-5. 

Rec #: 15876 
H-41 

 316. Tong E; McGraw SA; Dobihal E; Baggish R; Cherlin E, and Bradley EH. What is a good death? Minority 

and non-minority perspectives. J Palliat Care. 2003 Fall; 19(3):168-75. 

Rec #: 15884 

 317. Treadwell, M. J.; Franck, L. S., and Vichinsky, E. Using quality improvement strategies to enhance pediatric 

pain assessment. Int J Qual Health Care. 2002 Feb; 14(1):39-47. 

Rec #: 256 

 318. Treece, P. D.; Engelberg, R. A.; Crowley, L.; Chan, J. D.; Rubenfeld, G. D.; Steinberg, K. P., and Curtis, J. R. 

Evaluation of a standardized order form for the withdrawal of life support in the intensive care unit. 

Crit Care Med. 2004 May; 32(5):1141-8. 

Rec #: 511 

 319. Trowbridge, R.; Dugan, W.; Jay, S. J.; Littrell, D.; Casebeer, L. L.; Edgerton, S.; Anderson, J., and O'Toole, 

J. B. Determining the effectiveness of a clinical-practice intervention in improving the control of 

pain in outpatients with cancer. Acad Med. 1997 Sep; 72(9):798-800. 

Rec #: 290 

 320. Truog, R. D.; Cist, A. F.; Brackett, S. E.; Burns, J. P.; Curley, M. A.; Danis, M.; DeVita, M. A.; Rosenbaum, 

S. H.; Rothenberg, D. M.; Sprung, C. L.; Webb, S. A.; Wlody, G. S., and Hurford, W. E. 

Recommendations for end-of-life care in the intensive care unit: The Ethics Committee of the 

Society of Critical Care Medicine. Crit Care Med. 2001 Dec; 29(12):2332-48. 

Rec #: 471 

 321. Tulsky, J. A.; Chesney, M. A., and Lo, B. See one, do one, teach one? House staff experience discussing donot-resuscitate orders. Arch Intern Med. 1996 Jun 24; 156(12):1285-9. 

Rec #: 15939 

 322. Tulsky, J. A.; Fischer, G. S.; Rose, M. R., and Arnold, R. M. Opening the black box: how do physicians 

communicate about advance directives? Ann Intern Med. 1998 Sep 15; 129(6):441-9. 

Rec #: 50788 

 323. Uitterhoeve, R. J.; Vernooy, M.; Litjens, M.; Potting, K.; Bensing, J.; De Mulder, P., and van Achterberg, T. 

Psychosocial interventions for patients with advanced cancer - a systematic review of the literature. 

Br J Cancer. 2004 Sep 13; 91(6):1050-62. 

Rec #: 389 

 324. Vainio, A. and Auvinen, A. (Finnish Cancer Registry, Liisankatu 21B, 00170 Helsinki, Finland.). Prevalence 

of symptoms among patients with advanced cancer: an international collaborative study. Journal of 

Pain and Symptom Management. 1996 Jul; 12(1):3-10. 

Rec #: 335 

 325. Vigano, A.; Bruera, E., and Suarez-Almazor, M. E. Age, pain intensity, and opioid dose in patients with 

advanced cancer. Cancer. 1998; 83(6):1244-50. 

Rec #: 371 

 326. Volicer, L. Management of severe Alzheimer's disease and end-of-life issues. Clin Geriatr Med. 2001 May; 

17(2):377-91. 

Rec #: 7946 

 327. Volicer, L.; Hurley, A. C., and Blasi, Z. V. (Clinical Director, Geriatric Research Education Clinical Center, 

EN Rodgers Memorial Veteans Hospital, Bedford, Massachusetts.). Characteristics of dementia endof-life care across care settings. American Journal of Hospice and Palliative Care. 240 May-240 Jun 

30; 20(3):191-200. 

Rec #: 553 
H-42 

 328. Waldron, D.; O'Boyle, C. A.; Kearney, M.; Moriarty, M., and Carney, D. Quality-of-life measurement in 

advanced cancer: assessing the individual. J Clin Oncol. 1999 Nov; 17(11):3603-11. 

Rec #: 580 

 329. Wallace, J. and Desbiens, N. A. Evaluation of a labeling system to indicate the presence of an advance 

directive in a hospital medical record. Int J Qual Health Care. 2004 Aug; 16(4):333-5. 

Rec #: 43541 

 330. Walsh, D.; Donnelly, S., and Rybicki, L. The symptoms of advanced cancer: relationship to age, gender, and 

performance status in 1,000 patients. Support Care Cancer. 2000 May; 8(3):175-9. 

Rec #: 426 

 331. Ward, S.; Donovan, M., and Max, M. B. (University of Wisconsin-Madison, School of Nursing, K6/348, 600 

Highland Ave., Madison, WI 53792.). A survey of the nature and perceived impact of quality 

improvement activities in pain management. Journal of Pain and Symptom Management. 1998 Jun; 

15(6):365-73. 

Rec #: 348 

 332. Washington-Alaska Cancer Pain Initiative (WACPI). Joint commission on accreditation of healthcare 

organizations pain standards for 2001: pain assessment [Web Page]. Accessed 2005 Feb 3. 

Rec #: 205 

 333. Weiler, K. and Garand, L. Advance directives. University of Iowa Gerontological Nursing Interventions 

Research Center (RDC); 1999(Evidence-based protocol). 

Rec #: 712 

 334. Wenger, N. S.; Phillips, R. S.; Teno, J. M.; Oye, R. K.; Dawson, N. V.; Liu, H.; Califf, R.; Layde, P.; Hakim, 

R., and Lynn, J. Physician understanding of patient resuscitation preferences: insights and clinical 

implications. J Am Geriatr Soc. 2000 May; 48(5 Suppl):S44-51. 

Rec #: 8134 

 335. Wenzel, L. B.; Fairclough, D. L.; Brady, M. J.; Cella, D.; Garrett, K. M.; Kluhsman, B. C.; Crane, L. A., and 

Marcus, A. C. Age-related differences in the quality of life of breast carcinoma patients after 

treatment. Cancer. 1999 Nov 1; 86(9):1768-74. 

Rec #: 429 

 336. White, D. B. and Luce, J. M. Palliative care in the intensive care unit: barriers, advances, and unmet needs. 

Crit Care Clin. 2004 Jul; 20(3):329-43, vii. 

Rec #: 513 

 337. Wiffen, P.; Collins, S.; McQuay, H.; Carroll, D.; Jadad, A., and Moore, A. Anticonvulsant drugs for acute 

and chronic pain. Cochrane Database Syst Rev. 2000; (3):CD001133. 

Rec #: 274 

 338. Wiffen, P.; Collins, S.; McQuay, H.; Carroll, D.; Jadad, A., and Moore, A. Anticonvulsant drugs for acute 

and chronic pain. The Cochrane Library, (Oxford) ** (4)2004. 2004; (ID #CD001133). 

Rec #: 318 

 339. Wolfe, J.; Klar, N.; Grier, H. E.; Duncan, J.; Salem-Schatz, S.; Emanuel, E. J., and Weeks, J. C. 

Understanding of prognosis among parents of children who died of cancer: impact on treatment 

goals and integration of palliative care. JAMA. 2000 Nov 15; 284(19):2469-75. 

Rec #: 559 
H-43 

 340. Wong, R. and Wiffen, P. J. Bisphosphonates for the relief of pain secondary to bone metastases. Cochrane 

Database Syst Rev. 2002; (2):CD002068. 

Rec #: 254 

 341. Wong, R. and Wiffen, P. J. Bisphosphonates for the relief of pain secondary to bone metastases. The 

Cochrane Library, (Oxford) ** (4)2004 . 2004; (ID #CD002068). 

Rec #: 322 

 342. World Health Organization (WHO). Cancer pain relief: with a guide to opioid availability. 1996. 

Rec #: 719 

 343. Worthington, H. V.; Clarkson, J. E., and Eden, O. B. Interventions for treating oral mucositis for patients with 

cancer receiving treatment. Cochrane Database Syst Rev. 2004; (2):CD001973. 

Rec #: 233 

 344. Yurk, R.; Morgan, D.; Franey, S.; Stebner, J. B., and Lansky, D. Understanding the continuum of palliative 

care for patients and their caregivers. J Pain Symptom Manage. 2002 Nov; 24(5):459-70. 

Rec #: 8401 

 345. Zabora, J.; BrintzenhofeSzoc, K.; Curbow, B.; Hooker, C., and Piantadosi, S. The prevalence of 

psychological distress by cancer site. Psychooncology. 2001 Jan-2001 Feb 28; 10(1):19-28. 

Rec #: 424 

 346. Zech, D. F.; Grond, S.; Lynch, J.; Hertel, D., and Lehmann, K. A. Validation of World Health Organization 

Guidelines for cancer pain relief: a 10-year prospective study. Pain. 1995 Oct; 63(1):65-76. 

Rec #: 300 

 347. Zhukovsky DS; Gorowski E; Hausdorff J; Napolitano B, and Lesser M. Unmet analgesic needs in cancer 

patients. J Pain Symptom Manage. 1995 Feb; 10(2):113-9. 

Rec #: 364 
H-44 

Rejected: Guidelines With No Methiods

 1. American Medical Directors Association (AMDA) . Pain management in the long-term care setting. Clinical 

Practice Guideline. 2003. 

Rec #: 648 

 2. Caraceni, A.; Martini, C.; Zecca, E.; Portenoy, R. K.; Ashby, M. A.; Hawson, G.; Jackson, K. A.; Lickiss, N.; 

Muirden, N.; Pisasale, M.; Moulin, D.; Schulz, V. N.; Rico Pazo, M. A.; Serrano, J. A.; Andersen, H.; 

Henriksen, H. T.; Mejholm, I.; Sjogren, P.; Heiskanen, T.; Kalso, E.; Pere, P.; Poyhia, R.; Vuorinen, 

E.; Tigerstedt, I.; Ruismaki, P.; Bertolino, M.; Larue, F.; Ranchere, J. Y.; Hege-Scheuing, G.; Bowdler, 

I.; Helbing, F.; Kostner, E.; Radbruch, L.; Kastrinaki, K.; Shah, S.; Vijayaram, S.; Sharma, K. S.; Devi, 

P. S.; Jain, P. N.; Ramamani, P. V.; Beny, A.; Brunelli, C.; Maltoni, M.; Mercadante, S.; Plancarte, R.; 

Schug, S.; Engstrand, P.; Ovalle, A. F.; Wang, X.; Alves, M. F.; Abrunhosa, M. R.; Sun, W. Z.; Zhang, 

L.; Gazizov, A.; Vaisman, M.; Rudoy, S.; Gomez Sancho, M.; Vila, P.; Trelis, J.; Chaudakshetrin, P.; 

Koh, M. L.; Van Dongen, R. T.; Vielvoye-Kerkmeer, A.; Boswell, M. V.; Elliott, T.; Hargus, E., and 

Lutz, L. Breakthrough pain characteristics and syndromes in patients with cancer pain. An 

international survey. Palliat Med. 2004 Apr; 18(3):177-83. 

Rec #: 229 

 3. Jenkins, Catherine and Bruera, Eduardo. Assessment and management of medically ill patients who refuse lifeprolonging treatments: Two case reports and proposed guidelines. Journal of Palliative Care. 1998; 

14(1):18-24. 

Rec #: 495 

 4. Mahler, D. A.; Tomlinson, D.; Olmstead, E. M.; Tosteson, A. N., and O'Connor, G. T. Changes in dyspnea, 

health status, and lung function in chronic airway disease. Am J Respir Crit Care Med. 1995 Jan; 

151(1):61-5. 

Rec #: 449 

 5. Pain and Palliative Care Committees. Analagesic prescribing guidelines. Guidelines for Physician Staff 

Froedtert Hospital, Milwaukee, Wisconsin:1-4. 

Rec #: 131 
H-45 

Rejected: Duplicate Article

 1. Glinder, J. G. and Persons, J. B. (San Francisco Bay Area Center for Cognitive Therapy, Oakland, California.). 

Review: psychoeducational interventions reduce depressive symptoms in cancer. Evidence-Based 

Mental Health. 2002 Nov; 5(4):105. 

Rec #: 401 

 2. Wu, J. S.; Wong, R. K.; Lloyd, N. S.; Johnston, M.; Bezjak, A., and Whelan, T. Radiotherapy fractionation for 

the palliation of uncomplicated painful bone metastases - an evidence-based practice guideline. 

Program in Evidence-Based Care: A Cancer Care Ontario Program. 2003 Mar 14:1-25. 

Rec #: 461 
H-46 

Rejected: Useful for Background Only

 1. Farrar, J. T.; Portenoy, R. K.; Berlin, J. A.; Kinman, J. L., and Strom, B. L. Defining the clinically important 

difference in pain outcome measures. Pain. 2000 Dec 1; 88(3):287-94. 

Rec #: 271 

 2. University of Ottawa Evidence-based Practice Center. Measuring the quality of breast cancer care in women. 

Agency of Healthcare Research and Quality; 2004 Oct. 

Rec #: 743