Evidence Report/Technology Assessment 

Number 138 

Cancer Care Quality Measures: Diagnosis and 

Treatment of Colorectal Cancer 

Prepared for: 

Agency for Healthcare Research and Quality 

U.S. Department of Health and Human Services 

540 Gaither Road 

Rockville, MD 20850 

www.ahrq.gov 

Contract No. 290-02-0025 

Prepared by:

Duke Evidence-based Practice Center, Durham, NC 

Investigators 

Meenal B. Patwardhan, M.D., M.H.S.A., Principal Investigator 

Gregory P. Samsa, Ph.D., EPC Investigator 

Douglas C. McCrory, M.D., M.H.S., EPC Investigator 

Deborah A. Fisher, M.D., M.H.S., Clinical Investigator 

Christopher R. Mantyh, M.D., Clinical Investigator 

Michael A. Morse, M.D., M.H.S., Clinical Investigator 

Robert G. Prosnitz, M.D., M.P.H., Clinical Investigator 

Kathryn E. Cline, M.H.S., Project Manager 

Rebecca N. Gray, D.Phil., Editor 

 

 

AHRQ Publication No. 06-E002 

May 2006
This report is based on research conducted by the Duke Evidence-based Practice Center (EPC) 

under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD 

(Contract No. 290-02-0025). The findings and conclusions in this document are those of the 

author(s), who are responsible for its contents, and do not necessarily represent the views of 

AHRQ. No statement in this report should be construed as an official position of AHRQ or of the 

U.S. Department of Health and Human Services. 

The information in this report is intended to help clinicians, employers, policymakers, and others 

make informed decisions about the provision of health care services. This report is intended as a 

reference and not as a substitute for clinical judgment. 

This report may be used, in whole or in part, as the basis for development of clinical practice 

guidelines and other quality enhancement tools, or as a basis for reimbursement and coverage 

policies. AHRQ or U.S. Department of Health and Human Services endorsement of such 

derivative products may not be stated or implied. 
ii

This document is in the public domain and may be used and reprinted without permission except 

those copyrighted materials noted for which further reproduction is prohibited without the 

specific permission of copyright holders. 

Suggested Citation: 

Patwardhan MB, Samsa GP, McCrory DC, Fisher DA, Mantyh CR, Morse MA, Prosnitz RG, 

Cline KE, Gray RN. Cancer Care Quality Measures: Diagnosis and Treatment of Colorectal 

Cancer. Evidence Report/Technology Assessment No. 138. (Prepared by the Duke Evidencebased Practice Center under Contract No. 290-02-0025.) AHRQ Publication No. 06-E002. 

Rockville, MD: Agency for Healthcare Research and Quality. May 2006. 

None of the investigators has any affiliations or financial 

involvement that conflicts with the material presented in 

this report. 
iii

Preface 

 The Agency for Healthcare Research and Quality (AHRQ), through its Evidence-Based 

Practice Centers (EPCs), sponsors the development of evidence reports and technology 

assessments to assist public- and private-sector organizations in their efforts to improve the 

quality of health care in the United States. This report on Cancer Care Quality Measures: 

Diagnosis and Treatment of Colorectal Cancer was requested and funded by the National 

Cancer Institute (NCI), the Centers for Medicare & Medicaid Services (CMS), and the Centers 

for Disease Control and Prevention (CDC). The reports and assessments provide organizations 

with comprehensive, science-based information on common, costly medical conditions and new 

health care technologies. The EPCs systematically review the relevant scientific literature on 

topics assigned to them by AHRQ and conduct additional analyses when appropriate prior to 

developing their reports and assessments. 

 To bring the broadest range of experts into the development of evidence reports and health 

technology assessments, AHRQ encourages the EPCs to form partnerships and enter into 

collaborations with other medical and research organizations. The EPCs work with these partner 

organizations to ensure that the evidence reports and technology assessments they produce will 

become building blocks for health care quality improvement projects throughout the Nation. The 

reports undergo peer review prior to their release. 

 AHRQ expects that the EPC evidence reports and technology assessments will inform 

individual health plans, providers, and purchasers as well as the health care system as a whole by 

providing important information to help improve health care quality. 

 We welcome comments on this evidence report. They may be sent by mail to the Task Order 

Officer named below at: Agency for Healthcare Research and Quality, 540 Gaither Road, 

Rockville, MD 20850, or by e-mail to epc@ahrq.gov. 

Carolyn M. Clancy, M.D. 

Director 

Agency for Healthcare Research and Quality 

Andrew C. von Eschenbach, M.D. 

Director, National Cancer Institute 

National Institutes of Health 

Acting Commissioner of Food and Drugs 

U.S. Food and Drug Administration 

Mark B. McClellan, M.D., Ph.D. 

Administrator 

Centers for Medicare & Medicaid Services 

Julie Louise Gerberding, M.D., Ph.D. 

Director 

Centers for Disease Control and Prevention 

Jean Slutsky, P.A., M.S.P.H. 

Director, Center for Outcomes and Evidence 

Agency for Healthcare Research and Quality 

Beth A. Collins Sharp, Ph.D., R.N. 

Acting Director, EPC Program 

Agency for Healthcare Research and Quality 

William Lawrence, M.D., M.S. 

EPC Program Task Order Officer 

Agency for Healthcare Research and Quality 
iv

Acknowledgments 

The authors gratefully acknowledge Robert Sandler, M.D., M.P.H., Morris Weinberger, Ph.D., 

and Christopher Willett, M.D., for methodological and clinical advice, and for comments on 

drafts of material included in this report, and Udita Patel, M.S.P.H., for assistance in drafting 

parts of the report.
v

Structured Abstract 

Objectives. To identify measures that are currently available to assess the quality of care 

provided to patients with colorectal cancer (CRC), and to assess the extent to which these 

measures have been developed and tested. 

Data Sources. Published and unpublished measures identified through a computerized search 

of English-language citations in MEDLINE® (1966-January 2005), the Cochrane Database of 

Systematic Reviews, and the National Guideline Clearinghouse; through review of reference lists 

contained in seed articles, all included articles, and relevant review articles; and through searches 

of the grey literature (institutional or government reports, professional society documents, 

research papers, and other literature, in print or electronic format, not controlled by commercial 

publishing interests). Sources for grey literature included professional organization websites and 

the Internet. 

Review Methods. Measures were selected by reviewers according to standardized criteria 

relating to each question, and were then rated according to their importance and usability, 

scientific acceptability, and extent of testing; each domain was rated from 1 (poor) to 5 (ideal). 

Results. We identified a number of well-developed and well-tested CRC-related quality-ofcare measures, both general process-of-care measures (on a broader scale) and technical 

measures (pertaining to specific details of a procedure). At least some process measures are 

available for diagnostic imaging, staging, surgical therapy, adjuvant chemotherapy, adjuvant 

radiation therapy, and colonoscopic surveillance. Various technical measures were identified for 

quality of colonoscopy (e.g., cecal intubation rate, complications) and staging (adequate lymph 

node retrieval and evaluation). These technical measures were guideline-based and well 

developed, but less well tested, and the linkage between them and patient outcomes, although 

intuitive, was not always explicitly provided. For some elements of the care pathway, such as 

operative reports and chemotherapy reports, no technical measures were found. 

Conclusions. Some general process measures have a stronger evidence base than others. 

Those based on guidelines have the strongest evidence base; those derived from basic first 

principles supported by some research findings are relatively weaker, but are often sufficient for 

the task at hand. A consistent source of tension is the distinction between the clinically derived 

fine-tuning of the definition of a quality measure and the limitations of available data sources 

(which often do not contain sufficient information to act on such distinctions). Although some 

excellent technical measures were found, the overall development of technical measures seems 

less advanced than that of the general process measures. 

vii

Contents 

Executive Summary.........................................................................................................................1 

Evidence Report.............................................................................................................................9 

Chapter 1. Introduction .................................................................................................................11 

 General Background ................................................................................................................11 

 Care of Patients With Colorectal Cancer.................................................................................11 

 Burden of Disease..............................................................................................................11 

 Conceptual Model..............................................................................................................12 

 Diagnosis of CRC ..............................................................................................................15 

 Staging ...............................................................................................................................15 

 Management: Surgery........................................................................................................16

 Management: Chemotherapy.............................................................................................16 

 Management: Radiation Therapy.......................................................................................17 

 Surveillance........................................................................................................................17 

 Quality Measures for Patients With Colorectal Cancer...........................................................17 

 

Chapter 2. Methods.......................................................................................................................19 

 Topic Assessment and Refinement..........................................................................................19 

 Analytic Framework ................................................................................................................20 

 Literature Search and Review..................................................................................................22 

 Sources...............................................................................................................................22 

 Search Strategies................................................................................................................22 

 Abstract and Full-Text Screening ......................................................................................22 

 Full-Text Screening Criteria ..............................................................................................23 

 Data Abstraction and Development of Evidence Tables .........................................................24 

 Quality Assessment Criteria ....................................................................................................25 

 Additional Analyses.................................................................................................................28 

 Peer Review Process................................................................................................................28 

Chapter 3. Results .........................................................................................................................31 

 Question 1a: Colon Imaging, Endoscopic Visualization, and Biopsy....................................31 

 Background........................................................................................................................31 

 Results................................................................................................................................31 

 General Process Measures ...........................................................................................32 

 Technical Process Measures ........................................................................................34 

 Other Measures ............................................................................................................35

 Conclusions........................................................................................................................35 

 Future Research .................................................................................................................39 

 Question 1b: Pathologic Staging.............................................................................................40 

 Background........................................................................................................................40 

 Results................................................................................................................................41 
viii

 Conclusions........................................................................................................................42 

 Future Research .................................................................................................................43 

 Question 2a: Polypectomy for Malignant Polyps ...................................................................44 

 Background........................................................................................................................44 

 Results................................................................................................................................45 

 Conclusions........................................................................................................................45 

 Future Research .................................................................................................................45 

 Question 2b: Surgical Therapy ...............................................................................................45 

 Background........................................................................................................................45 

 Results................................................................................................................................47 

 Measures That do not Account for Stage.....................................................................47 

 Measures That Assess Appropriate Receipt (or Non-Receipt) of Surgery for 

 Colon or Rectal Cancer for Patients Having Cancers of Various 

 Locations and Stage ...............................................................................................47 

 Sphincter-Saving Surgery for Rectal Cancer...............................................................49 

 Measures That Assess the Quality of Surgery for Colon and/or Rectal Cancer..........50 

 Measures That Assess the Complication Rates of Colon and/or Rectal 

Cancer Surgery.......................................................................................................50 

 Measures That Assess Intermediate Outcomes of Surgery for Colon 

and/or Rectal Cancer..............................................................................................50 

 Conclusions........................................................................................................................51 

 Future Research .................................................................................................................54 

 Staging Workup ...........................................................................................................54 

 Appropriate Primary Therapy for Stage IV CRC ........................................................54 

 Improvement in Sphincter-Preserving Data.................................................................54 

 Anastomotic Leak Rates ..............................................................................................55 

 Recurrence Rates Following Resection of CRC..........................................................55 

 Improvement in Quality of Rectal Surgery..................................................................55 

 Complications of Rectal Cancer Surgery Due to the Use of Neoadjuvant 

Therapy ..................................................................................................................55 

 Appropriate Treatment for T1 Rectal Cancer ..............................................................55 

 Question 2c: Adjuvant Chemotherapy....................................................................................55 

 Background........................................................................................................................56 

 Results................................................................................................................................57 

 Measures That do not Specify Stage and/or Location .................................................57 

 Measures That Use Suboptimal Combinations of Stage and/or Location ...................57 

 Measures for Stage IV Colorectal Cancer ...................................................................58 

 Measures for Stage III Colon Cancer...........................................................................58 

 Measures for Stages II and III Rectal Cancer ..............................................................60 

 Reports That are Disaggregated by Stage and Location (or Stage Only)....................61 

 Other: Intermediate Process Measures........................................................................61 

 Conclusions........................................................................................................................62 

 Future Research .................................................................................................................65 

 Question 2d: Radiation Therapy .............................................................................................66

 Background........................................................................................................................66 
ix

 Results................................................................................................................................67 

 Conclusions........................................................................................................................69 

 Future Research .................................................................................................................71 

 Question 3: Colonoscopic Surveillance..................................................................................72 

 Background........................................................................................................................73 

 Results................................................................................................................................74 

 Postoperative (Surveillance) Colonoscopy..................................................................74 

 Perioperative (Clearing) Colonoscopy.........................................................................75 

 Conclusions........................................................................................................................75 

 Future Research .................................................................................................................78 

 Question 4: Pathology, Operative, and Chemotherapy Reports .............................................79 

 Background........................................................................................................................79 

 Results................................................................................................................................80 

 Conclusions........................................................................................................................81 

 Future Research .................................................................................................................84 

 Question 5a: Patient Populations and Purposes......................................................................84 

 Question 5b: Age, Race/Ethnicity, and Socioeconomic Status ..............................................85 

 Background........................................................................................................................85 

 Results................................................................................................................................86 

 Conclusions........................................................................................................................95 

Chapter 4. Future Research...........................................................................................................97 

Chapter 5. Conclusions .................................................................................................................99 

References and Included Studies .................................................................................................103 

Acronyms and Abbreviations ......................................................................................................115 

Figures 

Figure 1. Diagnosis and management of colon cancer .................................................................13 

Figure 2. Diagnosis and management of rectal cancer .................................................................14 

Tables 

Table 1. Results of abstract screening and full-text review..........................................................24 

Table 2. Included full-text articles by research question ..............................................................24 

Table 3. Sample quality measure ratings table .............................................................................28 

Table 4. Quality measure ratings – Question 1a...........................................................................37 

Table 5. Quality measure ratings – Question 1b...........................................................................43 

Table 6. Quality measure ratings – Question 2b...........................................................................52 

Table 7. Quality measure ratings – Question 2c...........................................................................64 

Table 8. Quality measure ratings – Question 2d...........................................................................70 
x

Table 9. Guidelines for colonoscopic surveillance after curative resection of CRC....................73 

Table 10. Quality measure ratings – Question 3...........................................................................77 

Table 11. Quality measure ratings – Question 4...........................................................................82 

Table 12. Quality measures used to assess differences in quality of care by patients’ age..........86 

Table 13. Quality measures used to assess differences in quality of care by patients’ sex ..........89 

Table 14. Quality measures used to assess differences in quality of care by patients’ 

race/ethnicity............................................................................................................................90 

Table 15. Quality measures used to assess differences in quality of care by patients’ 

socioeconomic status ...............................................................................................................93 

Appendixes 

Appendix A: Quality of Life 

Appendix B: Exact Search String 

Appendix C: List of Excluded Studies 

Appendix D: Sample Data Abstraction Forms 

Appendix E: Evidence Tables 

Appendix F: Criteria Used to Assign Ratings to Quality Measures 

The Appendixes and Evidence Tables cited in this report are provided electronically at

http://www.ahrq.gov/pub/evidence/pdf/colocanqm/colocanqm.pdf
1

Executive Summary

Introduction 

The burden of illness from colorectal cancer (CRC), the second leading cause of cancer death 

in the U.S., is substantial. One way to reduce the burden of illness is to improve the quality of 

care for patients with CRC. Measuring quality of care can help identify specific areas in need of 

improvement. Sponsored by the Agency for Healthcare Research and Quality (AHRQ), the 

National Cancer Institute (NCI), the Centers for Medicare & Medicaid Services (CMS), and the 

Centers for Disease Control and Prevention (CDC), the Duke Evidence-base Practice Center 

undertook this evidence report to identify measures that are currently available to assess the 

quality of care provided to patients with CRC, and to assess the extent to which these measures 

have been developed and tested. With this basis, we also report on areas for future research with 

an explicit aim of developing a set of measures that can effectively evaluate the process of care 

provided to CRC patients. Information from this evidence report will be used by technical 

panels convened by the National Quality Forum (NQF) to consider cancer care quality measures. 

Quality measures may be classified as structure, process, and outcome measures. While all 

three types are considered here, the primary focus is on process measures, whether or not such 

measures have been formally applied within the context of quality improvement. 

Methods 

Five key questions are addressed by this report: 

Question 1: What quality-of-care measures are available and what evidence is available for 

these measures to assess the quality of diagnosis of colorectal cancer, including: (a) appropriate 

use of colon imaging, endoscopic visualization, and biopsy; and (b) availability and accuracy of 

pathologic staging? 

Question 2: As appropriate to specific stages of colorectal cancer, what quality-of-care 

measures are available and what evidence is available for measures of quality of care of 

treatment of colorectal cancer, including: (a) polypectomy for malignant polyps, including 

evaluation of surgical margins; (b) surgical therapy for colon and rectal cancers; (c) appropriate 

use of adjuvant chemotherapy and adjuvant radiation therapy, including for patients with 

metastatic but potentially curable (hepatic/pulmonary-resectable) disease; and (d) appropriate use 

of radiation therapy for either curative or palliative therapy, specifically for rectal cancers? 

Question 3: What quality-of-care measures are available and what evidence is available for 

measures of colonoscopic surveillance for colorectal cancer? 

Question 4: What measures are available and what evidence is available for measures to 

assess the adequacy and completeness of documentation of pathology, operative, and 

chemotherapy reports? 
2

Question 5: For Questions 1-4 above: (a) in what patient populations and for what purposes 

have these quality of care measures been used; and (b) does evidence support the use of any of 

these measures to assess differences in quality of care across patients’ age, race/ethnicity, and/or 

socioeconomic status? 

 After finalizing the key questions in consultation with study sponsors, NQF, and a panel of 

national experts, we identified measures through a computerized search of English-language 

citations in MEDLINE® (1966-January 2005), the Cochrane Database of Systematic Reviews, 

and the National Guideline Clearinghouse. We also reviewed reference lists from all relevant 

articles and made special attempts to retrieve grey literature, loosely defined as institutional or 

government reports, professional society documents, research papers, fact sheets, and other 

literature, in print or electronic format, that is not controlled by commercial publishing interests. 

The basic MEDLINE search strategy was adapted for use in the other databases. The final 

search yielded 3,771 citations published since 1990. 

Measures described in individual studies were selected by reviewers according to 

standardized criteria relating to each question, and were then rated according to their importance 

and usability (I), scientific acceptability (S), and extent of testing (T); each domain was scored 

from 1 (poor) to 5 (ideal). A final “I-S-T” rating was assigned to each quality measure across 

studies. 

Results 

Colon Imaging, Endoscopic Visualization, and Biopsy (Question 1a) 

The best-developed measure identified was percentage of patients who underwent 

appropriate evaluation for a positive fecal occult blood test [FOBT] (I5S4T4), which is linked to 

an outcome of interest (reduced mortality from CRC), based on an evidence-based 

recommendation, and fairly well tested. The leverage points are awareness of the positive FOBT 

and referral for colonoscopy. 

Although the benefits of expeditious action are intuitive, the measure time from presentation 

to diagnosis (I3S1T1) has not been explicitly linked to an outcome of interest, is not based on any 

evidence-based guidelines, and can therefore be considered to be relatively less well developed 

and untested. The same can be said for proportion of colonoscopies that were completed in a 

timely fashion (I5S4T-). 

Any measure that relates to miss rates (I5S3T3) is unlikely to be useful to compare quality of 

care, since the methodological gold standard for determining a miss rate is tandem or closely 

timed colonoscopies, and such a gold standard procedure is not realistic within general clinical 

practice. Computed tomographic (CT) colonography, or “virtual colonoscopy,” appears to be a 

better alternative, but is not yet part of mainstream evaluation considering the flux in the 

technology utilized. 

Also important were technical process measures of colonoscopy such as cecal intubation rate 

(I5S4T4), intraprocedural complication rate (I5S4T-), and postprocedural complication rate

(I5S4T4). These technical measures are endorsed by guidelines and are meaningful indicators of 

quality. These measures tend to be fairly well developed, but poorly tested. 

Complication rate of colonoscopy (I5S4T4) is an important technical measures; however, an 

accurate definition of the measure is required. 
3

Pathologic Staging (Question 1b) 

Our literature search retrieved two process measures that have been used to assess the quality 

of staging of CRC: 

• Adequate lymph node retrieval and evaluation (I5S4T5). 

• Percentage of newly diagnosed CRC cases who were staged using the American Joint 

Committee on Cancer (AJCC) system (I4S3T3). 

Adequate lymph node retrieval and evaluation is linked to an outcome of interest (better 

staging, leading to appropriate management and better survival) and based on evidence (the 1999 

consensus statement of the College of American Pathologists (CAP) recommends evaluating 10-

15 lymph nodes in node-negative patients). It is well suited as a quality measure in a broader 

context, and is well developed and tested. Percentage of newly diagnosed CRC cases that were 

staged using the AJCC system is another measure that is well developed, has face validity, is 

related to an outcome of interest, and represents an important leverage point for improving the 

quality of CRC care. However, it does not distinguish between pathology staging and clinical 

staging. 

Polypectomy for Malignant Polyps (Question 2a) 

No quality measures were found addressing polypectomy for malignant polyps 

Surgical Therapy (Question 2b) 

 The most basic measure of the quality of surgical management is whether surgical therapy 

was provided as appropriate to the location and stage of the cancer. Results could potentially be 

broken out by stage or presented as a single summary measure, surgical therapy appropriate to 

the location and stage of the cancer, for example, appropriate primary therapy for CRC as 

defined by the NCI guidelines (I4S4T4). The former approach provides more information and is 

preferable. Regarding the denominator of this measure, patients for whom the therapy is 

inappropriate should be excluded. This point is particularly problematic for patients with stage 

IV colon or rectal cancer, for whom the intent of the surgery is palliation rather than cure. 

Ideally, the measure should also distinguish between patients who were offered “appropriate” 

surgery and refused, and those patients who were not offered such surgery. Making these 

distinctions increases the data collection demands. 

Surgical complication rates provide another very natural measure. Most variations on this 

measure focus on mortality rates (e.g., 30-day mortality [I5S4T5] or in-hospital mortality

[I5S3T3]), although a delineation of the actual complications would be helpful as well, this latter 

type of measure being relatively underdeveloped. Presumably, such a measure should be both 

stratified by stage and location, and risk adjusted to take into account other clinical 

characteristics of the patient. 

For patients with rectal cancer, various measures have been proposed regarding the rate of 

sphincter-saving surgery (I5S4T4). This is an outcome of importance to patients. However, 
4

quality measures should consider detailed information about the anatomical location and other 

characteristics of the tumor, which is not yet the case in practice. 

Technical measures associated with surgery are underdeveloped. 

Adjuvant Chemotherapy (Question 2c) 

We considered measures for stage III colon cancer, stages II and III rectal cancer, and stage 

IV colorectal cancer. Roughly speaking, these measures were defined as the proportion of 

patients with stage III colon cancer receiving adjuvant chemotherapy (I5S4T5), the proportion of 

patients with stages II or III rectal cancer receiving chemoradiotherapy (I5S3T2), and the 

proportion of patients with stage IV colon cancer or stage IV rectal cancer receiving palliative 

chemotherapy (I5S3T2). 

Of these measures, perhaps the best-developed and best-tested measure applies to patients 

with stage III colon cancer. It is derived from an evidence-based recommendation, is an 

important leverage point, and is well tested. 

Also evidence-based and well validated is the measure applying to patients with stages II or 

III rectal cancer. Several different versions of the measure have been created, but the ideal 

version of the measure is the one that most closely matches the most recent recommendations 

from guidelines, which specify that chemoradiotherapy is the preferred method of management 

for essentially all such patients. 

The measure regarding palliative chemotherapy for stage IV colon cancer or stage IV rectal 

cancer is reasonable in principle, but suffers from the difficulty that the decision to provide this 

therapy should very much be made on a case-by-case basis, thus implying that prospective data 

collection is probably necessary. 

For all the above measures, patient preference needs to be taken into account. However, this 

data element is particularly difficult to collect. 

The measure on the proportion of patients referred to an oncologist (I5S3T2) pertains to an 

important leverage point in practice, although it is possible that this is neither a necessary nor a 

sufficient condition for the provision of high-quality care. 

Radiation Therapy (Question 2d) 

We identified an important process measure: the percentage of patients with stage II or III 

rectal cancer receiving adjuvant chemoradiotherapy (I5S3T2). A well-developed and tested 

measure, it needs to account for reasons for non-receipt of therapy (e.g., insurance coverage or 

patient preference). 

We also identified two potential quality measures for radiation therapy related to the 

technical quality of its administration, with reference to two different guidelines: adherence to 

radiotherapy management treatment guidelines for patients with adenocarcinoma of the rectum 

or sigmoid colon (I5S4T4) and rate of use of modern radiation therapy techniques and adherence 

to recommendations of NCI-sponsored randomized controlled trials in rectal cancer patients

(I3S3T2). However, neither of the two technical measures is appropriate for current use because 

they are based on outdated guidelines. These measures could be modified to make them relevant 

to current evidence and practice standards. 
5

Colonoscopic Surveillance (Question 3) 

Measures describing the performance of postoperative surveillance colonoscopies were fairly 

well developed and well motivated: their ratings ranged from I5S4T5 for the measure percentage 

of patients with CRC receiving postoperative (surveillance) colonoscopy to I4S3T2 for percentage 

of patients with local or regional CRC who had colonoscopy or flexible sigmoidoscopy with 

barium enema. The most problematic point was the lack of consensus regarding the optimal 

scheduling of this procedure. The numerator should include colonoscopies only, and not other 

tests such as barium enema or flexible sigmoidoscopy. The measure could also benefit by 

explicitly specifying whether patients without clearing colonoscopy should be included. 

“Clearing colonoscopy” (1) refers to a sufficiently comprehensive examination of the colon to 

exclude other significant neoplasia, such as adenomatous polyps or synchronous cancers; and (2) 

signifies that all polyps were removed. Technical criteria should be considered as well; for 

example, whether the colonoscopy was complete according to the standard of documented cecal 

intubation. 

A measure describing the rate of perioperative colonoscopy (I5S4T4) was similarly well 

motivated, but suffered from a lack of precision in distinguishing clearing colonoscopies from 

similar procedures pre- and postsurgery. Including this additional detail would be substantively 

helpful, although potentially increasing the data collection requirements, as not all databases 

(particularly administrative databases) will contain sufficient information to make this 

distinction. 

Pathology, Operative, and Chemotherapy Reports (Question 4) 

The quality measure proportion of CRC cases in which pathologic staging preceded 

chemotherapy and radiation treatment (I5S4T-) is well developed, free from confounders, and is 

an important leverage point, but needs testing. 

The quality measure pathology report in concordance with CAP guidelines (I5S4T-) is well 

developed, free from confounders, and is an important leverage point, but needs testing. Not all 

items included in the CAP guidelines are related to outcomes of interest, and it may be useful to 

select those that address a leverage point, for example, local extent of tumor, regional lymph 

node metastases, or residual tumor at resected margin. This aspect is considered by the measure 

adequacy of pathology reports on CRC (I5S4T-), which limits measurement to scientifically 

validated elements of the CAP guidelines, and is therefore a potentially ideal measure; however 

it needs field-testing. 

The other measures (each being an item of the Association of Directors of Anatomic and 

Surgical Pathology [ADASP] guideline) take more of a micro-level perspective, addressing 

individual elements of the pathology report. These are well developed and address some 

important leverage points for improving quality of colon and rectum cancer pathology reporting 

standards. The most important ones are: local extent of tumor, regional lymph node metastases, 

and residual tumor at surgical resection margin (all I5S5T4). 

Technical measures relating to operative and chemotherapy reports were underdeveloped. 
6

Patient Populations and Purposes; Age, Race/Ethnicity, and 

Socioeconomic Status (Questions 5a and 5b) 

A number of studies have measured differences in process measures related to quality of care 

based on age, racial/ethnic, or socioeconomic differences; however, few studies have evaluated 

potential confounders that may explain the differences observed. More refined measures that 

evaluate a variety of covariates involved in clinical decisionmaking may be necessary to 

understand and explain these apparent quality-of-care differences. 

Conclusions 

Although we also considered measures of structure and outcome, the primary focus of this 

report was on measures of CRC-related processes of care. Here, the main distinction is between 

those measures that are general (dealing with processes of care on a broader scale) versus 

technical (pertaining to specific details of a procedure). 

The formula for creating a general process measure is to take the sequence of steps 

describing the optimal pattern of care for patients with CRC, identify each of these steps as both 

a leverage point and a potential quality measure, and then define such a measure. For example, 

one of the earliest steps in the process involves the proper diagnosis of patients with a suspicion 

of CRC. A well-developed and well-tested measure associated with this step is percentage of 

patients who underwent appropriate evaluation for a positive FOBT (I5S4T4). As with all ratiobased measures, it is critical to carefully define both the numerator and denominator of the 

measure. Here, the numerator can be based on a guideline and involves colonoscopy with 

complete visualization of the colon, although the literature did discuss various nuances, such as 

the precise definition of what complete visualization entails, how soon the colonoscopy should 

be performed after the positive FOBT, and so forth. The numerators of some general process 

measures will have a stronger evidence base than others, with those that are based on guidelines 

(especially those guidelines that are consistent across professional societies, when multiple such 

guidelines have been proffered) having the strongest evidence base, and those that are based on 

basic first principles supported by some research findings being relatively weaker, but often quite 

sufficient for the task at hand. 

Regarding the denominator, “patients with positive FOBT” comprise only a subset of those 

patients of ultimate interest, namely, “all patients with sufficiently high clinical suspicion of 

CRC to require followup testing.” These patients would include those with positive tests 

according to other modalities (e.g., barium enema, flexible sigmoidoscopy), and also those who 

come to the provider’s attention because of symptoms such as bleeding. Often, a clinically 

precise statement of the denominator involved deleting various patients; for example, those that 

would be ineligible because of comorbid conditions or other clinical issues, patients that refuse a 

form of therapy such as chemotherapy, and so forth. 

A consistent source of tension is the distinction between the clinically derived fine-tuning of 

the definition of the denominator of a quality measure and the limitations of available data 

sources (which often do not contain sufficient information to act on such distinctions). Our 

approach here has been to encourage the elicitation of denominators that are as precise as 

possible, even if the available data sources might not meet all their requirements. In practice, it 

is crucially important to assess both the strengths and weaknesses of available data sources. 
7

Formal validation efforts are encouraged, but are not always strictly necessary. The assessment 

of data sources should include the differential impact of data quality on patient subgroups. For 

example, older patients tend to have more comorbidity and might thus be less likely to tolerate 

adjuvant chemoradiation therapy; an analysis that does not take this tendency into account could 

incorrectly lead to the conclusion that an age-related bias exists in the provision of such therapy. 

A similar caveat applies to analyses comparing patients by race and ethnicity, hospitals 

according to type, and so forth. Another example of bias is the inability of data sources to 

account for patient preferences. 

Overall, the coverage of general process measures (i.e., across the various leverage points in 

the sequence of care) is extensive. At least some process measures are available for the steps 

involving diagnostic imaging, staging, surgical therapy, adjuvant chemotherapy, adjuvant 

radiation therapy, and colonoscopic surveillance. Additional general process measures might be 

developed for polyp assessment and removal, although it must be recognized that the proper 

clinical course of action when presented with a malignant polyp is not always clear. While the 

above formula could be followed to generate even more general process measures within the 

above categories, perhaps the task of greater importance is to continue to refine those measures 

that already exist. The continued expansion of the evidence base, and the continued updating 

guidelines, should assist this process. Moreover, the basic structure of general process measures 

facilitates such updating. For example, if future research suggests that the optimal interval for 

colonoscopic surveillance should be changed, then the structure of the existing quality measures 

related to surveillance need not be changed; all that would require change is the replacement of 

one recommended interval with another. 

In contrast to general process measures, as might be anticipated, technical measures tended to 

be used in a formal quality improvement context. Although some excellent technical measures 

were found, the overall development of technical measures seems somewhat less advanced than 

that of the general process measures. In particular, for some elements of the care pathway, such 

as operative reports and chemotherapy reports, no technical measures were found. Various 

technical measures for quality of colonoscopy were found (e.g., cecal intubation rate). These 

were guideline-based, well developed, but less well tested, and the linkage between the technical 

measures and patient outcomes, although intuitive, was not always explicitly provided. To the 

extent that complication rate of colonoscopy (I5S4T4) can be considered a technical measure, this 

area of inquiry is relatively well developed, but needs further testing. Adequate lymph node 

retrieval and evaluation (I5S4T5) is an excellent technical measure related to disease staging, 

although residual areas requiring clarification remain (e.g., optimal number of lymph nodes to 

examine, consideration that the optimal number of lymph node depends to some extent on 

patient and tumor characteristics). Technical measures associated with surgery are 

underdeveloped. Chemoradiation therapy has a number of potential and well-justified technical 

measures; here, the challenge is to identify and focus on those measures having the most impact 

on patient outcomes. Technical measures for pathology reporting are well developed, reflecting 

among other things the areas of emphasis among the relevant professional societies. As noted 

above, no such technical process measures were identified for operative reports or chemotherapy 

reports, although it can quite reasonably presumed that with sufficient attention from 

professional societies and other stakeholders such measures could be developed. 
8

Future Research 

Perhaps the most important areas for future development of technical measures are: (a) 

developing such measures in those areas for which they are lacking; and (b) in areas where such 

measures exist, identifying and focusing on those measures that have the greatest impact on 

outcome. Clarifying and otherwise fine-tuning the measures is a process that should be ongoing, 

both for technical and general measures. 
EVIDENCE REPORT 

11

Chapter 1. Introduction 

General Background 

The National Cancer Policy Board (NCBP) of the Institute of Medicine (IOM) has concluded 

that “for many Americans with cancer, there is a wide gulf between what could be construed as 

the ideal and the reality of their experience with cancer care.”1

 Concerns about the nation’s 

system of cancer care prompted the NCPB to undertake a comprehensive review of the evidence 

on the effectiveness of cancer services and delivery systems in the U.S., the adequacy of quality 

assurance mechanisms, and barriers that impede access to cancer care. As a part of these efforts, 

the IOM released three reports entitled Ensuring Quality Cancer Care, Enhancing Data Systems 

to Improve the Quality of Cancer Care, and Improving Palliative Care for Cancer, which 

summarized the state of knowledge in quality cancer care and identified efforts to improve it.1-3 

A key recommendation was for further development and monitoring of measures of the quality 

of cancer-related health care. The focus of this evidence report is measures of the quality of care 

provided to patients with colorectal cancer (CRC). 

The present report was sponsored by the Agency for Healthcare Research and Quality 

(AHRQ), the National Cancer Institute (NCI), the Centers for Medicare & Medicaid Services 

(CMS), and the Centers for Disease Control and Prevention (CDC). Information from the report 

will be used by technical panels convened by the National Quality Forum (NQF). One mission 

of the NQF is to encourage the development and application of validated measures of the quality 

of care for patients with various medical conditions. This report will provide part of the 

evidence base from which the NQF will make recommendations about quality measures for 

patients with CRC. For this report, the NQF, in conjunction with representatives of AHRQ, NCI, 

CMS, and CDC, provided a set of research questions of interest, a list of review criteria that have 

been successfully applied to quality measures in other fields, and general expertise in quality 

measurement, which was made available to us on an as-needed basis. 

The focus of this evidence report combines two basic ideas: (a) care provided to patients 

with CRC; and (b) quality measures. The remainder of this Chapter provides non-technical 

descriptions of each. That is, it describes in basic terms what care should be expected for 

patients with CRC and provides an operational definition of a quality measure. More detailed 

discussion of these issues is provided in subsequent sections of this report. 

Care of Patients With Colorectal Cancer 

Burden of Disease 

The burden of illness from CRC is substantial. In 2003, it is estimated that there were 

145,290 new cases and 56,290 deaths attributable to CRC in the U.S. CRC is the second leading 

cause of cancer death in the U.S.4

 
12

Conceptual Model 

Our conceptual model of the process of care for colon and rectal cancer patients in the U.S. is 

depicted in Figures 1 and 2. The next sections discuss the various elements of care represented 

within these figures. 
13

Figure 1. Diagnosis and management of colon cancer 

Biopsy/Pathology review 

Colonoscopy Signs/ Symptoms

Abdominal/pelvic CT; Baseline CEA 

Surgery Pathology 

review 

Chemotherapy 

Surve

Screening: Flexible sigmoidoscopy, barium enema, or CT colography 

Stage III, some stage II 

Stage I, some stage II 

Chemotherapy 

Stage IV 
14

Figure 2. Diagnosis and management of rectal cancer 

 T1/T2

Any node + or T3/T4 

Biopsy/ Pathology review 

Proctoscopy, EUS, or MRI 

Surgery Pathology 

review 

Chemotherapy+ Radiotherapy 

Imaging Colonoscopy Other modes, e.g., CEA 

Surveillance 

Colonoscopy Signs/ Symptoms 

Screening 

Chemoradiotherapy for T3, T4, or node + 
15

Diagnosis of CRC 

The preferred method of CRC detection is through screening of asymptomatic patients. The 

clinical rationale for screening is that CRC is a cancer that is usually slow growing, and if the 

cancer is caught at an early stage (ideally, at the polyp stage), then it is more likely to be cured. 

However, in spite of demonstrated benefit and published clinical guidelines, CRC screening tests 

are underutilized.5 In part due to this underutilization, some patients will be diagnosed with 

CRC because of symptoms. 

Quality measures related to CRC screening have been evaluated elsewhere. At the request of 

the NQF, we restricted our analysis to: (a) the additional pattern of tests used to diagnose CRC 

in patients for whom either a screening test result or the presence of symptoms suggests an 

increased risk of CRC; and (b) screening for recurrence of disease following surgery for CRC, 

this latter application being termed “surveillance.” The preferred test in both applications is a 

colonoscopy. 

Colonoscopy is the most sensitive test for detection of colorectal neoplasia and the only 

method that allows both examination of the entire large bowel and the opportunity to biopsy or 

even remove suspected lesions.6

 Various screening tests such as double contrast barium enema, 

flexible sigmoidoscopy, and fecal occult blood test may have preceded the colonoscopy. 

However, these modalities are no longer recommended as followup tests.6 Computed 

tomographic (CT) colonography (“virtual colonoscopy”) is an emerging technology with the 

potential to screen for CRC, but it is not currently endorsed for screening or diagnostic 

evaluation.6-10 

Staging 

Surgical resection remains the most effective therapy for colorectal carcinoma. Clinical 

staging involves evaluating the primary tumor and the presence of nodal and distant metastases; 

clinical staging begins with evaluation of signs and symptoms, radiographic, and laboratory tests, 

and includes findings from direct visualization via endoscopy or surgery. For rectal cancer, 

clinical staging also includes endoscopic ultrasound or Phased Array MRI preoperatively to 

determine the depth of penetration and the possibility of regional nodal disease, which cannot be 

accurately assessed by pelvic CT scanning. 

Pathologic staging is performed after surgical resection of the primary tumor, and accurate 

staging is critical for appropriate patient management and also for comparability of data across 

research studies.11 

The prognosis of patients with colon cancer is clearly related to the degree of penetration of 

the tumor through the bowel wall, the presence or absence of nodal involvement, and the 

presence or absence of distant metastases. These three characteristics form the basis for all 

staging systems developed for this disease. Bowel obstruction and bowel perforation are 

additional indicators of poor prognosis.12

Although the microscopic appearance of adenocarcinoma of the colon and rectum tends to be 

straightforward, a detailed pathology review is essential for diagnosis, characterization, and 

staging. Pathologic stage combines the clinical staging information with surgical findings, 

incorporating data from the pathologic examination of resected primary and regional lymph 

nodes.13 The pathology report is a critical step in the process of pathologic staging. Information 

on pathology reports should be sufficient to stage the patient using the Tumor, Node, Metastasis 
16

Staging System (TNM).14 Other key factors include the grade of the tumor and the surgical 

margin status. 

The assessment of nodal metastases at the time of surgical treatment is particularly crucial for 

patients with apparently localized carcinoma of the colon and rectum, since the presence of nodal 

metastases is currently the most important factor in determining whether an individual is a 

candidate for adjuvant therapy.15 Moreover, stage of disease is the single most important factor 

in predicting survival.16-18 The small size of many mesenteric nodes, including some with 

micrometastatic carcinoma, increases the risk of missing metastases. Recent studies have 

demonstrated that 10 to 15 lymph nodes are needed for reliable staging.19-21

Management: Surgery 

Standard treatment for patients with stages I-III colon cancer has been open surgical 

resection of the primary tumor and regional lymph nodes for localized disease. Polypectomy for 

malignant polyps is adequate for polyps that have not invaded the submucosa, because these 

tumors do not metastasize. For polyps that reveal submucosal, lymphovascular, or neural 

invasion, a bowel resection should be performed. Although surgery is usually reserved for 

patients with non-metastatic disease, palliative surgery or diverting colostomy is sometimes 

performed in patients with metastatic disease and bowel obstruction. Some patients who have 

developed metastases in the liver are also candidates for surgery. Improved surgical techniques 

and advances in preoperative imaging have allowed for better patient selection for resection. 

Localized rectal cancer is treated by surgical resection of the primary tumor and regional 

lymph nodes. Although previously patients underwent surgery and then postoperative 

chemoradiotherapy, currently patients with clinical stage T1,2 disease proceed to surgical 

resection, while most patients with clinical stage T3 or T4 or N+ tumors now receive 

preoperative chemotherapy and radiotherapy. Most rectal cancers that are located 5 cm or more 

from the anal verge can be removed while preserving the anal sphincter. For small rectal cancers 

with favorable pathologic features, local full-thickness excisions may be adequate. The resective 

technique of total mesorectal excision has led to reduction in local recurrences and improvement 

in overall survival.22 

Management: Chemotherapy 

Because their cancers have spread beyond the colon and rectum, patients with stage III colon 

cancer without medical or psychosocial contraindications to treatment are recommended to have 

systemic adjuvant chemotherapy following surgical resection.15 Patients with stage II and III 

rectal cancer also benefit from chemotherapy.15,23 Palliative multiagent chemotherapy also 

increases survival for those with stage IV disease, although it is not curative.24,25

The steps in the process of receiving appropriate adjuvant chemotherapy are that patients 

with CRC must be staged appropriately, be referred to an oncologist, receive an adequate 

performance status evaluation, and be ascertained to be free of major postoperative 

complications. Chemotherapy must then be offered and administered at an appropriate dosage 

for an appropriate length of time.26 Finally, adequate reporting must be provided, this 

requirement applying equally to surgical therapy, chemotherapy, and radiation therapy. 
17

Management: Radiation Therapy 

Because the pattern of recurrence following surgical excision is more often local in patients 

with rectal cancer than in those with colon cancer, the potential impact of radiation therapy is 

greater in patients with rectal cancer than in patients with colon cancer.27 Both preoperative and 

postoperative radiation therapy decrease local failure.28-31 Patients with stage II or III rectal 

cancer are at high risk for local and systemic relapse and are thus the best candidates for adjuvant 

therapy including chemotherapy and/or radiation therapy. However, it is important to follow 

precise procedures for radiation therapy in order to achieve optimum results. 

The role of adjuvant radiation therapy for patients with colon cancer (above the peritoneal 

reflection) is not well defined and has no current standard role in the management of patients 

with colon cancer following curative resection.32

Surveillance 

Surveillance refers to the ongoing followup of CRC patients after treatment. Colonoscopic 

surveillance after curative resection of CRC is recommended, but the optimal timing of the 

surveillance coloscopies after a perioperative “clearing” colonoscopy is yet to be determined. If 

a preoperative colonoscopy is not possible (e.g., because of an obstructing lesion), it should be 

performed 6 months after surgery. 

Non-endoscopic or combined modality surveillance, such as following tumor markers 

(carcinoembryonic antigen [CEA]) or abdominal imaging (CT), is controversial. More 

generally, while there is an overall survival benefit for intensifying the followup of patients after 

curative surgery for CRC, the optimal combination of clinic visits, laboratory tests and 

procedures is not currently known.33-35

Quality Measures for Patients With Colorectal Cancer 

At its most fundamental level, a “quality measure” is something that can be used to measure 

the quality of care provided to patients. The scope of this definition can be illustrated by 

example. Consider the following sequence of steps: 

Type of measure Step 

Structural Sufficient numbers of physicians given formal endoscopy training 

 ↓

General process Greater proportion of colonoscopies 

 ↓

Intermediate or Reduced rate of colonoscopy complications 

proximal outcome 

(technical process) 

 ↓

Distal outcome Better 5-year patient survival because of early diagnosis of cancer 
18

In this formulation, structural measures relate to the structural characteristics of providers 

and hospitals.36 Process measures describe the components of the encounter between a physician 

or another health professional and a patient.36,37 Within the category of process measures, 

technical measures pertain to specific details of a particular test or procedure, whereas general 

process measures are defined on a broader scale. Outcome measures reflect the net effect of 

healthcare delivery; that is, they describe what actually happens to the patient.36,37

When assessing quality of care, each of the above elements is of potential interest. For 

example, “volume-outcome” studies describe the relationship between structural characteristics 

and distal outcomes, recognizing that the more steps that intervene between structure and 

outcome the less definitive will be the inference. While our scope of work includes both 

structural characteristics and distal outcomes, our primary interest is in the process measures 

comprising the middle portion of the above pathway. 

To define a process measure, the initial requirement is a statement of what constitutes highquality clinical care; for example, “adequate bowel preparation” might be operationally defined 

as “no fecal material obscured a lesion 5 mm in diameter.” This statement might be derived 

from a clinical guideline that is circulated by a professional society. (Our scope of work does not 

include assessing the quality of evidence used in generating guidelines, but is limited to 

recording which quality measures are based on guidelines and which are not). Alternatively, this 

statement might be based on clinical first principles supported to a greater or lesser degree by 

formal research findings. All else being equal, guideline-based measures are preferred, one 

reason being that the guideline provides a standard of care that the user of the quality measure 

might plausibly anticipate should be followed. 

Once this statement of what constitutes high-quality clinical care is generated, the next step is 

to define the quality measure using a denominator denoting the number of eligible patients and a 

numerator denoting the number of eligible patients who satisfied the criteria for high-quality 

clinical care. For example, for the quality measure rate of colonoscopy complications, the 

denominator is the number of patients undergoing colonoscopy and the numerator is the number 

of patients undergoing colonoscopy who have at least one complication. 

Process measures have been used in both the formal quality improvement context and in 

more general research settings, the primary difference being that in a quality improvement 

context various details about the measure tend to be more explicitly specified. For example, 

some of these details could include the goals of the quality improvement program, the users and 

uses of the measure, a list of potential data sources along with their advantages and 

disadvantages, thresholds for action (e.g., take action if the rate of colonoscopy complications 

exceeds two percent), and so forth. We have defined our scope of work to include both measures 

that have been used in a formal quality improvement context and measures that have been used 

in other contexts, the rationale being that many of these latter measures are excellent and could 

readily be adapted to quality improvement applications. As noted in the next chapter, our 

methods are general enough to apply not only to process measures that have been used in a 

quality improvement context, but also to other process measures, structural measures, and 

outcome measures as well. 

 
19

Chapter 2. Methods 

Topic Assessment and Refinement 

Five key questions are addressed by this report. The Duke research team clarified and 

refined the overall research objectives and key questions by first consulting with the study 

sponsors, and then convening a panel of national experts to serve as advisors to the project. 

These experts were selected to represent relevant specialties including colorectal surgery, 

medical and radiation oncology, internal medicine, clinical pathology, and gastroenterology, as 

well as representatives of the National Cancer Institute (NCI) and the National Quality Forum 

(NQF). The eight members of the technical expert panel were: 

John Z. Ayanian, M.D., M.P.P., Harvard Medical School 

Steven Boyd Clauser, Ph.D., National Cancer Institute 

Rodger Jeffrey Winn, M.D., National Quality Forum 

Richard M. Goldberg, M.D., University of North Carolina at Chapel Hill 

Marcia R. Gottfried, M.D., Duke University Medical Center 

Michael P. Pignone, M.D., University of North Carolina at Chapel Hill 

Dawn T. Provenzale, M.D., Duke University Medical Center 

Kirk A. Ludwig, M.D., Duke University Medical Center 

Six key questions were originally proposed by the report’s sponsors: 

Question 1: What quality-of-care measures are available and what evidence is available for 

these measures to assess the quality of diagnosis of colon cancer, including: (a) appropriate use 

of colon imaging, endoscopic visualization, and biopsy; and (b) availability and accuracy of 

pathologic staging? 

Question 2: As appropriate to specific stages of colorectal cancer, what quality-of-care 

measures are available and what evidence is available for measures of quality of care of 

treatment of colorectal cancer, including: (a) polypectomy for malignant polyps, including 

evaluation of surgical margins; (b) surgical therapy specifically for rectal cancers; (c) appropriate 

use of adjuvant chemotherapy; and (d) appropriate use of radiation therapy for either curative or 

palliative therapy, specifically for rectal cancers? 

Question 3: What quality-of-care measures are available and what evidence is available for 

measures of colonoscopic surveillance for colorectal cancer? 

Question 4: What measures are available and what evidence is available for measures to 

assess the adequacy and completeness of documentation of pathology, operative, and 

chemotherapy reports? 

Question 5: For questions 1-4 above: (a) in what patient populations and for what purposes 

have these quality-of-care measures been used; and (b) does evidence support the use of any of 
20

these measures to assess differences in quality of care across patients’ age, race/ethnicity, and/or 

socioeconomic status? 

Question 6: For questions 1-4 above, what gaps in our knowledge of measurement of quality 

of care are evident from the currently available evidence, either for the population of colon 

cancer patients as a whole, or for specific subpopulations? 

As a result of conference calls with the technical experts and the report’s sponsors, and in 

response to comments on the peer review draft, the Duke research team modified these 

questions. Question 6 was eliminated as a separate question, and information on gaps in our 

knowledge was incorporated into the responses to Questions 1-4 in Chapter 3 and into Chapter 4. 

The refined key questions were as follows (changes from the initial versions are italicized): 

Question 1: What quality-of-care measures are available and what evidence is available for 

these measures to assess the quality of diagnosis of colorectal cancer, including: (a) appropriate 

use of colon imaging, endoscopic visualization, and biopsy; and (b) availability and accuracy of 

pathologic staging? 

Question 2: As appropriate to specific stages of colorectal cancer, what quality-of-care 

measures are available and what evidence is available for measures of quality of care of 

treatment of colorectal cancer, including: (a) polypectomy for malignant polyps, including 

evaluation of surgical margins; (b) surgical therapy for colon and rectal cancers; (c) appropriate 

use of adjuvant chemotherapy and adjuvant radiation therapy, including for patients with 

metastatic but potentially curable (hepatic/pulmonary-resectable) disease; and (d) appropriate 

use of radiation therapy for either curative or palliative therapy, specifically for rectal cancers? 

Question 3: What quality-of-care measures are available and what evidence is available for 

measures of colonoscopic surveillance for colorectal cancer? 

Question 4: What measures are available and what evidence is available for measures to 

assess the adequacy and completeness of documentation of pathology, operative, and 

chemotherapy reports? 

Question 5: For questions 1-4 above: (a) in what patient populations and for what purposes 

have these quality-of-care measures been used; and (b) does evidence support the use of any of 

these measures to assess differences in quality of care across patients’ age, race/ethnicity, and/or 

socioeconomic status? 

Analytic Framework 

Based on the original proposal and discussions with our technical expert panel, we developed 

the analytical framework represented by Figures 1 and 2 (in Chapter 1). Briefly, these figures 

describe the various steps in the diagnosis of colorectal cancer (CRC), the treatment of CRC, and 

the post-treatment surveillance for recurrence of CRC. Each step describes a potential leverage 

point for improving the quality of care and thus represents the opportunity for defining a process 

measure. The ultimate goal is to improve patient outcomes, in particular both survival and 

quality of life. 
21

As noted in the introduction, the quality of CRC-related care has been assessed at both the 

level of process and of outcome. Outcome measures most directly reflect what is ultimately of 

interest, but the connection with process of care might be weak (e.g., a good process of care 

might nevertheless result in poor outcomes), especially if (a) there are a number of links between 

the process of care and outcome; and/or (b) the outcome might be caused by many factors other 

than the process of care. With this in mind, we have separated the presentation of outcome 

measures from those pertaining to process. In particular, in the literature we observed outcome 

measures of two types: survival and quality of life. Survival measures (e.g., percentage of 

patients surviving 5 years) are discussed within the main body of the text (e.g., under Question 5, 

we describe differences in survival by age). Quality-of-life measures, which were relatively 

uncommon in the literature, are discussed in Appendix A.∗

While recognizing the ultimate importance of outcomes, the main focus of the text is on 

measures of process. These process measures have been applied in a number of different 

contexts. When applied within the context of clinical epidemiology, a process measure might 

simply be defined as, for example, “the number of patients with stage III colon cancer receiving 

adjuvant chemotherapy.” On the other hand, when applied within a quality improvement 

context, process measures generally require not just this basic descriptive component, but also 

additional information such as a list of data elements that are necessary to construct and/or report 

the measure, detailed specifications regarding the population on whom the measure is 

constructed, the source of the data, how the data elements are to be collected, the timing of data 

collection and reporting, the analytic models used to construct the measure, and the format in 

which the results will be presented. Measures may also include thresholds, standards, or other 

benchmarks of performance. 

In summary, the application of a process measure in a quality improvement setting includes 

not just the measure itself, but also various critical details as illustrated above. Such details need 

not necessarily be specified when applying the same process measure in a more general research 

setting. Nevertheless, some process measures that have been applied outside the setting of 

quality improvement might be excellent candidates for the quality improvement setting, 

(conditional on the specification of the various additional details as noted above); accordingly, 

we interpreted our task as finding both the former (actual quality improvement process 

measures) and the latter (potential quality improvement process measures). 

One of the implications of the above was the need to extend the criteria for assessing quality 

measures. In particular, the assessment criteria provided by the NQF are most naturally intended 

for the circumstance where a ratio-based process measure is applied within a quality 

improvement application. For such measures we applied the NQF assessment criteria as 

provided. For other measures, we developed criteria intended to be consistent with the principles 

underlying the NQF’s assessment criteria, as applied to this somewhat different circumstance. 

Additional details are provided in the section on “Quality Assessment Criteria,” below. 

 

∗ Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
22

Literature Search and Review 

Sources 

The primary sources of literature were MEDLINE® (1966-January 2005) and the Cochrane 

Database of Systematic Reviews. Searches of these databases were supplemented by reviews of 

reference lists contained in seed articles, all included articles, relevant review articles, and metaanalyses. We made special attempts to retrieve grey literature, loosely defined as institutional or 

government reports, professional society documents, research papers, fact sheets, and other 

literature, in print or electronic format, that is not controlled by commercial publishing interests. 

Sources for grey literature included professional organization websites and the Internet. 

Search Strategies 

The basic search strategy used the National Library of Medicine’s Medical Subject Headings 

(MeSH) key word nomenclature developed for MEDLINE® and was adapted for use in the other 

databases. The searches were limited to the English language. The text of the major search 

strategy is given in Appendix B∗, as well as a history of search terms considered. 

The final search, conducted in January 2005, yielded 5450 citations. When limited to 

publication date since 1990 and English language, this totaled 3,771 citations. These records 

were maintained in a ProCite (Thompson ISI ResearchSoft, Berkeley, CA) database. 

Abstract and Full-Text Screening 

Paired content experts from the Duke research team independently reviewed a set of abstracts 

and classified each as “include” or “exclude.” An abstract was retained at this stage if at least 

one of the paired reviewers recommended that it be included, and abstractors were instructed to 

apply the inclusion criteria liberally. Inter-rater reliability for include/exclude decisions was 

tested by having five pairs of readers review a subset of abstracts (n = 765). Agreement ranged 

from a kappa of 0.23 to 0.46. Samples of abstracts for which abstractors disagreed were 

reviewed, and it was noted that most disagreements occurred for marginal articles and reflected 

the abstractors’ attempts to be as inclusive as possible during this preliminary screening stage. A 

total of 874 abstracts were included for the further “full-text review” stage. 

At the full-text review stage, the paired researchers independently reviewed a set of the 

articles, and indicated a decision to “include” or “exclude” the article for the data abstraction 

stage. Detailed inclusion and exclusion screening criteria were developed for this purpose (see 

“Full-text Screening Criteria,” below). When a pair of reviewers arrived at a different opinion 

about whether to include an article, they were asked to reconcile the difference. Detailed 

inclusion and exclusion screening criteria were developed and are listed below. 

 

∗ Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
23

Full-Text Screening Criteria 

The full-text screening criteria were as follows: 

Include if: 

• The study population pertains to adults undergoing diagnostic evaluation for, or in 

treatment for, colon or rectal cancer, and at least one of the following apply: 

o Asymptomatic patients who have been screened AND have a positive finding 

suspicious for colon or rectal cancer (e.g., polyps). 

o Patients being diagnosed through the use of colon imaging, endoscopic 

visualization, or biopsy (Question 1a). 

o Diagnosed patients undergoing pathology staging (Question 1b). 

o Patients undergoing polypectomy for malignant colonic polyps (Question 2a). 

o Patients treated with surgical therapy for either rectal cancer or colon cancer 

(Question 2b). 

o Patients with metastatic, but potentially curable (hepatic/pulmonary-resectable) 

disease, when a majority of the study population has CRC (Question 2c). 

o Patients treated with adjuvant chemotherapy or radiation therapy (Questions 2c 

and 2d). 

o Patients treated with palliative chemotherapy or radiotherapy (Questions 2c and 

2d). 

o Patients undergoing colonoscopic surveillance (no other form of surveillance) for 

CRC (Question 3). 

AND 

• The study refers to at least one of the following: 

o An explicit quality measure (process, structure, or outcome). 

o A measure that assesses the adequacy and completeness of documentation of 

pathology, operative, or chemotherapy reports (operative procedures will include 

reports for colonoscopy and radiation therapy) (Question 4). 

o The testing of a quality measure’s validity or reliability. 

o A data source (e.g., registry) used for measuring quality, but may not specify the 

quality measure itself. 

Exclude if one or more of the following apply: 

• The study uses non-U.S. data. 

• The study was published prior to 1990. 

• The report is in a non-English language. 

• The study population involves any of the following: 

o The screening of asymptomatic patients. 

o Patients with familial polyposis syndrome. 

o Patients undergoing polypectomy for hereditary polyposis. 

o Patients with hereditary non-polyposis colorectal cancer. 
24

o Patients with multiple metastases, BUT the majority of the patients ARE NOT 

CRC patients. 

Summaries of the results of the abstract screening and full-text review are provided in Tables 

1 and 2. A list of excluded articles by reason for exclusion is found in Appendix C*

. 

Table 1. Results of abstract screening and full-text review

Articles identified 3,771 

Abstracts reviewed 3,771 

 Included 948 

 Excluded 2,823 

Full-text articles reviewed 947* 

 Included 74 

 Excluded 873 

*We were unable to obtain in full-text form 1 article included at the abstract screening stage. 

Table 2. Included full-text articles by research question 

 

Question 1a: 15 

Question 1b: 7 

Question 2a: 0 

Question 2b: 33 

Question 2c: 31 

Question 2d: 22 

Question 3: 8 

Question 4: 3 

Question 5: 30 

Total number of included articles 74* 

*Total of table does not equal total number of included articles because some articles were included for more than one question. 

Data Abstraction and Development of Evidence Tables 

The Duke research team developed and piloted evidence table formats for abstracting the 

quality measures and related data to answer each of the six research questions (see Appendix 

D*

). A pair of researchers was assigned to each research question based on clinical expertise and 

was instructed to abstract the quality measures and related data from the eligible articles. One of 

the paired researchers abstracted the quality measures and related data into the evidence tables, 

 

* Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
25

and the second researcher over-read the article and accompanying evidence table to check for 

accuracy and completeness. The completed evidence tables are provided in Appendix E. 

Quality Assessment Criteria 

At the data abstraction stage, the abstractor was asked to evaluate each article for factors 

affecting external and internal validity. External validity was primarily addressed by the items 

on the data abstraction form pertaining to the study population. 

As an assessment of internal validity, our original charge recommended that we assess the 

quality measures retrieved during our literature search using a set of criteria that had previously 

been applied by the NQF with success. These criteria are listed below. 

Importance: A measure is considered important if one or more audiences find the 

information produced from a measure useful for some purpose. Measures can be 

important because: 

 a. There is considerable variation in the quality of care provided. 

 b. Quality is substandard. 

 c. They relate to one of the established national goals. 

 d. They represent a significant leverage point for achieving the goal. 

 e. The information produced is usable by a stakeholder in the system. 

It is not necessary for all conditions to apply. However, criteria a or b must be present 

and be associated with c, d, or e for the measure to be considered important. 

Usability: An intended audience must be able to understand the results of the measure 

and find them useful for decisionmaking. For a measure to be useful: 

a. It must contain information that is compelling within the decisionmaking 

framework of the user. 

b. It must assess differences that are subjected to statistical testing, e.g., between 

groups, over time. 

 c. Differences must be practically meaningful. 

 d. Analysis should be done appropriately. 

 e. It must present results that are effective and consistent with intended use. 

Scientific Acceptability: A measure is scientifically acceptable if it produces consistent 

and credible results when implemented. There are five criteria on which to evaluate each 

quality measure (in a given study) to determine if it is scientifically acceptable: 

Precise specifications – a specification of a measure should include its format, and a 

standard reference source for defining key terms (clinical conditions, procedures, etc.) 

that is consistent with current requirements. 

Reliability – a measure is reliable when it produces the same result a high proportion 

of the time when assessed on the same population using the same data source. 

Validity – the validity of a measure is the extent to which it accurately represents the 

concept being evaluated. A measure is considered valid if (1) the scores that are 
26

produced from the measure distinguish between good and bad quality, and (2) the 

construction of the measure adequately represents the concept of interest. 

Adaptability – a measure is adaptable if it is appropriate for use in a variety of 

contexts and settings. Three dimensions of adaptability important to quality 

measurement are: (1) the ability to take into account patient preferences, (2) the 

flexibility to account for different clinical scenarios and (3) the applicability of the 

measure in different settings. 

Adequacy of risk-adjustment – when there is a clear clinical rationale, a quality 

measure should be adjusted for risk differences when making comparisons among 

health plans or physicians. It is less important to risk-adjust a quality measure when 

using the measure to track progress toward a health goal, for example, or when 

measuring changes over time within a single health care setting. 

 

Each of the above elements was scored on a 1-5 scale, with 1 denoting “poor” and 5 denoting 

“ideal.” The evidence tables report assessments for measures as reported in individual studies. 

In order to encourage reliability between abstractors, we held a “calibration” meeting where 

each investigator scored the same five quality measures (picked randomly, one for each task 

order question), discussed the scores and the differences in individual scoring, and re-scored 

them. The abstractors agreed on the scores 83 percent of the time. Subsequent to this calibration 

meeting, at least three investigators scored every measure, and a mean score was determined and 

finally allocated to each criterion. 

Note that at this stage there could be several scores attached to the same quality measure, and 

that a single study could produce more than one measure. The evidence tables include scores for 

every study assessing a given measure in that study. 

Another purpose of the calibration meeting was to identify and resolve any questions that the 

abstractors might have about how to fill in the abstraction form. During this meeting it was 

noted that, while the above scoring system tended to perform excellently for those “actual” 

quality measures that were used in quality improvement applications, it performed less well for 

those “potential” quality measures that were reported from other research studies. For these 

latter measures, abstractors were instructed to fill out the abstraction form as originally designed 

and (a) for those elements that were not applicable, either code the measure as “poor” or “not 

applicable”, as indicated; and (b) when feasible, to envision an application where the measure is 

used in a quality improvement context, and then to score the measure according to that 

application. For example, for applications in which the quality measure in question was not riskadjusted, the abstractor might nevertheless be able to express an informed opinion about whether 

such risk-adjustment was possible. As another example, the review criteria for usability assume 

that a decisionmaker and decisionmaking context has already been specified (as would be the 

case in a quality improvement application); in this case as well, the abstractor might nevertheless 

be able to envision a typical user and proceed. 

Recognizing that the ultimate use of the quality measures included in our evidence report was 

to inform the sponsors regarding the extent to which they were ready to be applied in the field, at 

this final stage we rated each quality measure across studies using three criteria: importance and 

usability, scientific acceptability, and extent of testing. For assigning these rates, we considered 
27

all studies that had utilized the particular measure. Details of the method we used to assign these 

ratings are provided in Appendix F*. Each measure was given an “I-S-T” rating, where: 

“I” describes the extent to which a measure is important and usable. This 

rating was determined by examining all the studies that utilized the measure and 

considering the best application. 

 

“S” describes the extent to which the measure is scientifically acceptable, that 

is, the extent to which the measure is precisely defined, valid, reliable, adaptable, 

and risk-adjusted. This rating was also determined by examining all the studies 

that utilized the measure and considering the best application. For example, if a 

measure was successfully risk-adjusted in one study but not in another, we rated 

the measure as one in which risk-adjustment had successfully been applied. 

 

“T” describes the extent to which the measure is tested. This rating was 

determined by examining all the studies that utilized the measure and considered 

two aspects of the measure: (a) the number of studies that utilized the measure, 

and (b) the extent to which these studies were scientifically acceptable (i.e., the 

“S” rating). 

Each of the above criteria was rated on a scale of 1-5, with 1 denoting “poor” and 5 denoting 

“ideal.” For example, if a quality measure has been assigned a rating of I5S3T1, the measure has 

high importance and moderate scientific acceptability, but has not been tested. 

This final scoring and rate assignment were done exclusively by the writer of each section 

(the writer was an investigator trained in that content area of the report). Although we had 

provided objective criteria for rating the quality measures, in order to encourage maximum 

flexibility (especially for “potential” quality measures), the writer was instructed to adapt the 

basic scoring and rating rules to the quality measure using their subjective judgment. Most of 

our recommendations follow directly from this assignment of ratings. 

For some measures that we retrieved from professional society websites or from personal 

communications (i.e., not from peer-reviewed literature) we did not attempt to retrieve any 

information regarding their testing. Therefore those measures may not have a “T” rating, 

although it is possible that they have undergone testing. 

At the end of each section in Chapter 3, a table with the structure illustrated below (Table 3) 

summarizes the I-S-T ratings assigned to the quality measures considered. The text in the 

“Recommendations” column of each table synthesizes the information in the corresponding 

“Future Research” section of the text. One use of this report will be to assist the NQF technical 

panel in identifying gaps in measurement that could be the basis for a future research agenda. 

Such research might include pilot testing or assessment of the measure’s appropriateness for 

populations other than that for which it was designed; evaluation of the appropriateness of the 

measure for use in routine clinical practice, quality improvement, or accountability; or data 

collection about the care of specific populations. 

 

* Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
28

Table 3. Sample quality measure ratings table 

Quality measure rating 

(range 1-5, where 1 = poor and 5 = ideal)

Quality measure

Important 

and usable 

(I) 

Scientifically 

acceptable 

(S) 

Well 

tested 

(T) 

Recommendations for the 

measure

 

Note: Investigators used their judgment to determine the quality measure rating for each measure across all studies 

that utilized the measure. See Appendix F*

 for details. 

To comment, inclusion of potential in addition to actual quality measures necessitated the 

above relatively modest modifications of the quality assessment process. However, since many 

of the potential quality measures that we identified were excellent (and since our initial 

consultation with NQF suggested that our charge in identifying quality measures should be 

defined broadly rather than narrowly), we nevertheless felt that the benefits of including these 

potential measures outweighed their limitations. For any particular measure, the distinction 

between actual and potential quality measures should be clear from the context, so those readers 

who wish to limit their focus to measures that have been developed and tested in quality 

improvement applications should be able to do so. 

Additional Analyses 

There were no additional analyses beyond those described here. 

Peer Review Process 

We employed internal and external quality-monitoring checks through every phase of the 

study to reduce bias, enhance consistency, and verify accuracy. Examples of internal monitoring 

procedures include: three progressively stricter screening opportunities for each article (abstract 

screening, full-text article review, data abstraction review); involvement of three individuals 

(two clinicians and copy editor) in each data abstraction; agreement of at least two clinicians on 

all included studies. 

 

* Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
29

Our principal external quality-monitoring device was the peer review process. Nominations 

for peer reviewers were solicited from several sources, including our technical expert panel and 

interested federal agencies. The list of nominees was forwarded to AHRQ for vetting and 

approval. 
 
31 

Chapter 3. Results 

Question 1a: Colon Imaging, Endoscopic 

Visualization, and Biopsy 

Question 1a was: What quality-of-care measures are available and what evidence is 

available for these measures to assess the quality of diagnosis of colorectal cancer, including 

appropriate use of colon imaging, endoscopic visualization, and biopsy? 

Background 

The preferred method of colorectal cancer (CRC) detection is through screening of 

asymptomatic patients. However, in spite of demonstrated benefit and published clinical 

guidelines, CRC screening tests are underutilized.5 In part due to underutilized screening, some 

patients will be diagnosed with CRC because of symptoms. The American Society for 

Gastrointestinal Endoscopy (ASGE) strongly endorses colonoscopy in the diagnosis of CRC and 

further states that “multiple biopsy specimens should be obtained from all suspicious lesions, and 

polypoid lesions should be removed.”38 In addition, patients who have a positive CRC screening 

test by any modality other than colonoscopy (i.e., fecal occult blood test [FOBT], flexible 

sigmoidoscopy, or double contrast barium enema [DCBE]) require a full colonoscopy for further 

evaluation.6

 DCBE was previously considered acceptable for the followup of a positive FOBT, 

but the latest guidelines emphasize that colonoscopy is the preferred followup test based on the 

sensitivity of colonoscopy.6

 Virtual colonoscopy is another promising and emerging imaging 

technique that has some advantages over traditional colonoscopy.6-10

Studies of colonoscopy miss rates for adenomas underscore that even this “gold standard” is 

not perfect.39 In 2002 a Multi-Society Task Force published its recommendations for assessment 

of quality in the technical performance of colonoscopy.40 Quality targets included: use of 

recommended screening and surveillance (e.g., postcancer resection) guidelines, cecal intubation 

rates of ≥ 90 percent overall and ≥ 95 percent of screening cases, documentation of cecal 

intubation by landmarks, mean colonic examination time (generally on withdrawal of 

colonoscope) of at least 6 to 10 minutes, incidence of perforation < 1/1,000 (all) and < 1/2,000 

(screening exams), and incidence of postpolypectomy bleeding (< 1/100). Recommendations for 

the colonoscopy procedure report included documentation of procedure time; quality of bowel 

preparation; cecal landmarks; and polyps identified, removed (technique), and recovered. 

Results 

Our literature search revealed several quality measures for outcomes of interest related to this 

question: 

• Percentage of patients with positive FOBT who underwent an appropriate evaluation.41-45 

• Time from patient presentation with symptoms to cancer diagnosis.46 
32 

• Proportion of colonoscopies that were completed in a timely fashion.47 

• Percentage of patients with colon or rectal cancer undergoing colonoscopy as part of their 

evaluation.48

• Colonoscopic miss rate for significant colonic neoplasia.49,50

• Complication rate of colonoscopy.51

• Serious postendoscopic procedure complication rates.47,52

• Rate of unplanned reversal of sedation medication.47

• Intraprocedure colonoscopy complication rate.53 

• Colonoscopy completion rate.54

• Cecal intubation rate.47,52

• Adenoma removal rate for patients over 50 years old.47 

• Percentage of patients with adequate bowel preparation prior to colonoscopy.47,52,55

• Proportion of colonoscopies performed by physicians with specialized training.47 

• Proportion of patients who adequately understood the colonoscopy procedure.53

General Process Measures. The first, and most widely used, measure we consider is the 

percentage of patients who underwent appropriate evaluation for a positive FOBT. This is a 

general process measure that depends critically on the specification of what constitutes an 

“appropriate” evaluation. The measure is based on evidence-based guidelines that have evolved 

over time as additional data on what constitutes an appropriate evaluation have become 

available. The older American Cancer Society (ACS) guidelines56 recommended the 

combination of barium enema and flexible sigmoidoscopy if a colonoscopy was not performed. 

The 1997 guidelines57 allowed for either colonoscopy or barium enema for followup of a positive 

FOBT. The updated 2003 guidelines6

 emphasized that colonoscopy was the preferred method of 

followup for a positive FOBT because of newer data supporting the increased sensitivity of 

colonoscopy to detect adenomatous polyps. Therefore, depending on the guideline cited, age of 

the data, and the timing of the study publication, “adequate evaluation” could be defensibly 

defined with some variation. Ideally, “appropriate evaluation” should also include consideration 

of the maximum acceptable time lag between the positive FOBT and a more definitive diagnostic 

workup, although the above guidelines do not explicitly specify such a time lag. 

In applying this measure, Shields et al.41 used data from 1986-1988 and defined “adequate 

evaluation” as colonoscopy or DCBE with flexible sigmoidoscopy per the cited 1980 ACS 

guidelines.58 The measure was applied in a community-based mass screening program, and the 
33 

source included patient self-report up to 3 years after the evaluation. Such an extensive delay 

might call the reliability of self-report data into question. 

Levin et al.42 used data from 1993 and defined “adequate evaluation” as colonoscopy or 

DCBE per the 1992 ACS guidelines.56 The data sources were patient and physician self-report, 

the accuracy of which were not tested. Although formal validation was not performed, it was 

found that consultation of a gastroenterologist significantly increased the chance of adequate 

evaluation, thus supporting the discriminant validity of the measure. 

Baig et al.43 defined “appropriate evaluation” to include colonoscopy or DCBE (without 

flexible sigmoidoscopy), which is consistent with the 1997 guidelines,57 although these 

guidelines were not explicitly cited. The followup period was restricted to 60 days, which may 

limit the usability and adaptability of this measure in systems where 2-month followup may not 

be possible (e.g., the Veterans Administration [VA] system). The data sources were provider 

survey and health maintenance organization (HMO) claims data. 

Myers et al.44 defined “adequate evaluation” as colonoscopy or DCBE with flexible 

sigmoidoscopy within 180 days, which may be a more realistic followup period. This study was 

in the setting of an intervention to improve followup rates for positive FOBT. Data sources were 

physician surveys and HMO administration claims data. 

Mandel et al.45 is another variation on this same measure and defined “appropriate 

evaluation” as followup colonoscopy or DCBE with flexible sigmoidoscopy. What is most 

noteworthy about this version of the measure is the study population: an 18-year followup from 

one of the original clinical trials that demonstrated the efficacy of FOBT. These data may be 

difficult to generalize, since patients that are enrolled in a clinical trial could be expected to have 

higher rates of appropriate followup than general practice. 

Two measures focus on timeliness. One is the time from patient presentation with symptoms 

to physician diagnosis. Marble et al.46 applied this measure to tumor registry data from a single 

medical center. Details of how time to diagnosis was determined are lacking. Moreover, there 

was no explicit demonstration that delays in diagnosis are related to survival (or any other 

outcome of interest). This measure is not well developed. 

A related measure, proposed by Minnesota Gastroenterology PA as part of their internal 

quality improvement initiative,47 is the proportion of colonoscopies that were completed in a 

timely fashion, where timeliness is operationally defined as “acceptable to patient and referring 

physician according to internal practice standards.” Ideally, this internal practice standard should 

reflect the indication for the procedure. While potentially important, this measure is not clearly 

linked to outcomes of interest or validated. 

Another general process measure was applied by Beart et al.:48 percentage of patients with 

colon or rectal cancer undergoing colonoscopy as part of their evaluation. The major flaw in 

this measure is the failure to distinguish between those colonoscopies used for the purpose of 

diagnosis and those used for preoperative clearance. This measure is not considered further. 

Yet another general process measure is the colonoscopy miss rate (i.e., the miss rate for 

detection of significant colonic neoplasia). Haseman et al.50 defined this measure as “percentage 

of colon cancers not detected by colonoscopy in symptomatic patients.” He performed a 

retrospective review of colon cancer cases that were not detected during a colonoscopy 

performed up to 3 years prior to cancer diagnosis. The numerator and denominator were well 

defined; however, the assumption that lesions should have been detected up to 3 years prior is 

not validated. Further limitations to the study were that the hospitals had volunteered to 

participate and that this was a retrospective review. 
34 

 Shehadeh et al.49 estimated the miss rate for “big” (≥ 10 mm) polyps by retrospectively 

reviewing the endoscopy database and pathology records of patients with an initial colonoscopy 

including polypectomy and at least one subsequent colonoscopy at a single, academically 

affiliated VA medical center. “Big” polyps seen on the subsequent colonoscopy were considered 

“missed” by the first colonoscopy. While the rate of missed advanced adenomatous polyps is a 

potentially important quality indicator, the method of measuring this value is more appropriately 

obtained with a prospective design and consideration of tandem colonic examinations. 

Technical Process Measures. Some of the most fundamental technical measures concern 

procedural complication rates. Various specific measures have been proposed. Ure et al.51 use 

the measure complication rate of colonoscopy. The numerator included the following 

complications: bowel perforation, intraprocedural complications (hypoxia [O2 saturation < 90 

percent], hypotension, bradycardia, hemorrhage), postpolypectomy bleeding, cardiopulmonary 

arrest, and death. The presence or absence of any complications was noted and the rates of 

individual complications were separately described, as was the total number of complications. 

Some of the complications (i.e., the numerator) were more precisely described than others. 

Overall this measure could be better developed and needs to be risk-adjusted. Furthermore, the 

importance of the individual complications is not uniform. For example, in terms of patient 

outcomes, perforation, postpolypectomy bleeding and death have established importance, 

whereas complications such as intraprocedural hypoxia treated with supplemental oxygen may 

not be clinically important. 

Minnesota Gastroenterology PA47 defined the complication rate measure as serious 

postendoscopy complication rate, where the serious complications considered included death 

within 30 days, perforation, bleeding with transfusion, cardiopulmonary arrest, and unplanned 

surgery. Focusing on these severe and well-described complications adds to the development, 

importance, and face validity of this measure. Similarly, a large VA study52 was conducted in 

order to study major complication rates following screening colonoscopy. The complications 

were defined accurately and evaluated rigorously for up to 30 days after the procedure. 

Potentially less serious, but clinically important, complications of sedation are captured by a 

second complication measure, proposed by the same group, the rate of unplanned reversal of 

sedation medication.

47

The National Quality Measures Clearinghouse (NQMC) publishes a similar measure, 

intraprocedure colonoscopy complication rate (proposed by the Accreditation Association for 

Ambulatory Health Care [AAHC] Institute for Quality Improvement), which includes 

arrhythmia, bleeding requiring treatment, extended recovery, hospital transfer, hypotension, 

hypoxia, perforations, and respiratory arrest.53 The specific definitions of these complications 

are lacking, although guidelines by the Accreditation Association for Ambulatory Health Care 

(AAAHC) Institute for Quality Improvement are cited.59

 Another technical measure is the colonoscopy completion rate, i.e., whether the colonoscopy 

was adequately completed. The Colon Cancer Workgroup54 applied the definition “colonoscopy 

to the ileocecal valve performed prior to surgical resection of CRC.” This measure is well 

developed and is based on National Comprehensive Cancer Network (NCCN) guidelines.24 

Cases of obstructing or perforated carcinoma should be excluded from the denominator of this 

measure. Ure et al.51 similarly defined colonoscopy completion rate as “visualization of the 

cecum or ileocolic anastomosis.” The rationale is similar to the above. 
35 

Cecal intubation rates and adenoma removal rates for patients over 50 years old were quality 

measures proposed by Minnesota Gastroenterology PA.47 Both are well defined, linked to 

outcomes of interest, and based on guidelines.40 Cecal intubation was defined as examination of 

the entire colon to the cecum or to a surgical anastomosis (when applicable). Adenoma removal 

rates were assessed in patients over 50 years of age. The rationale for this latter measure appears 

to be that if the adenoma detection rate is less than “population norms,” then polyps are probably 

being missed. “Population norms” were well defined. 

 Another technical measure pertains to the quality of bowel preparation preceding the 

colonoscopy. Harewood et al.55 examined the relationship between adequate bowel preparation 

and the detection of colonic polyps or cancer. The data source was a national gastrointestinal 

endoscopic database (Clinical Outcomes Research Initiative [CORI]), and the findings supported 

significantly greater detection of colonic neoplasia (although not necessarily cancer) in patients 

with adequate bowel preparation compared to without adequate preparation. This measure is 

important, but a more appropriate quality measure for health care might be the documentation of 

the bowel preparation quality at the time of the procedure, as this may affect the interval of the 

next examination and the level of certainty that neoplasia or malignancy has been ruled out. 

Weaknesses of the study itself included the lack of validation of the CORI data, lack of 

standardized definitions for the quality of bowel preparation, and inconsistent histology data for 

colonic lesions. 

 Minnesota Gastroenterology PA47 also utilizes a measure of the “proportion of colonoscopies 

with adequate preparation.” This version of the measure is more specific: “no fecal material 

obscured a lesion 5 mm or more in diameter and a high quality examination takes place.” 

All measures proposed by Minnesota Gastroenterology PA have been implemented within 

their system and will require further testing for broader application. 

 

Other Measures. Minnesota Gastroenterology PA47 propose a structural measure of the 

proportion of colonoscopies performed by physicians with specialized training 

(gastroenterology, colorectal surgery, general surgery, pediatric surgery). This follows ASGE 

guidelines.60 The measure is well defined, but needs validation and testing. 

The NQMC53 publishes the measure proportion of patients who respond “yes” to whether 

they have an adequate understanding of their procedure as asked in a post procedure telephone 

interview (proposed by the AAHC Institute for Quality Improvement). This is guideline-based,59

but not validated (e.g., patients may want to give socially acceptable answers in a telephone 

interview), and measures only perception of understanding and not true understanding. 

A final quality measure is an indirect one: incidence of metastatic diagnosis among health 

plan members aged 50 or older.

61 Reduction of metastatic disease is supported by NCCN 

guidelines24 as a measure of the long-term effectiveness of cancer screening programs, one 

component of which is quality of diagnosis. Because the primary determinant of this outcome is 

likely to be the proportion of eligible patients undergoing screening, this measure is considered 

to be outside the scope of interest here. 

Conclusions 

We retrieved several general, technical, and structural measures that have been utilized in the 

United States for assessing the quality of care related to diagnosis of CRC. Ratings for these 

measures are given in Table 4; further details are provided in Evidence Tables 1 and 2 in 
36 

Appendix E*

. Perhaps the most important, well-developed, and widely tested41-44 measure 

identified was percentage of patients who underwent appropriate evaluation for a positive FOBT

(I5S4T4), which is linked to an outcome of interest (reduced mortality from CRC) and is based on 

an evidence-based recommendation. The leverage points are awareness of the positive FOBT 

and referral for colonoscopy. 

Although the benefits of expeditious action are intuitive, time from presentation to diagnosis

has not been explicitly linked to an outcome of interest, is not based on any evidence-based 

guidelines, and can therefore be considered to be relatively less well developed and untested 

(I3S1T1). The same can be said for the technical measure proportion of colonoscopies that were 

completed in a timely fashion (I5S4T-). 

Any measure that relates to miss rates (I5S3T3) is open to debate because no matter how the 

miss rate is estimated it is unlikely to be useful to compare quality of care, since the 

methodological gold standard for determining a miss rate is tandem or closely timed 

colonoscopies, and such a gold standard procedure is not realistic within general clinical 

practice. In other words, it is not realistically possible, in practice, to determine whether a case 

has in fact been “missed.” 

Also important is the technical process measure of colonoscopy: cecal intubation rate 

(I5S4T4). This technical measure is endorsed by guidelines, but is inconsistently linked to 

outcomes of interest; it tends to be fairly well developed, but poorly tested. While cecal 

intubation rate is an important measure, a precise definition of intubation is required for 

appropriate evaluation. Toward this end, colonoscopy completion rate (I5S4T-), specified in the 

studies we examined as visualization “to the ileocecal valve” or “of the entire cecum or ileocolic 

anastomosis,” may be a better measure. 

Intraprocedural complication rate (I5S4T-), postprocedural complication rate (I5S4T4), and 

complication rate of colonoscopy (I5S4T4) are other important technical measures; however, an 

accurate definition of the complication rate is required. Complications should be divided into 

those of a severe nature with links to outcomes of interest (e.g., bleeding requiring transfusion or 

hospitalization) and those of a more minor nature (e.g., intraprocedural transient hypoxia with 

oxygen saturation < 90 percent reversed with oxygen supplementation via nasal canula and 

resolution during the recovery period). For purposes of data collection (particularly since 

colonoscopy is primarily an outpatient procedure) it may also be useful to divide complications 

into intraprocedural complications, immediately postprocedural complications (e.g., within 2 

hours), and 30-day postprocedural complications. This final category would require not only 

examination of medical records, but potentially patient survey in order to determine whether the 

patient was treated at another facility. 

Patient understanding of the colonoscopy procedure was fairly well developed, but its 

relationship to outcomes has not been definitively demonstrated (I5S4T-). 

The Multi-Society Task Force guidelines40 suggest an additional potentially useful quality 

measure, colonoscopic withdrawal time (which has been linked to polyp detection rates), but we 

found no data on development or validation of this potential measure. 

The recommendations made herein are based on a review of currently available data. 

Obviously, future data or clinical realities (e.g., costs, changes in societal goals) or the 

development of new technologies may alter the recommendations in the future. Nonetheless, for 

the major recommendations, we do not expect their basic core importance to change. 

 

* Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
37 

Table 4. Quality measure ratings – Question 1a 

Quality measure rating 

(range 1-5, where 1 = poor and 5 = ideal) 

Quality measure

Important 

and usable 

(I) 

Scientifically 

acceptable 

(S) 

Well 

tested 

(T) 

Recommendations for the 

measure

General measures 

Percentage of patients with positive 

FOBT who underwent an 

appropriate evaluation41-45

5 4 4 Further testing required 

with reliable data sources. 

Time from patient presentation with 

symptoms to cancer diagnosis46 3 1 1 Additional testing required 

with larger data set. 

Measure should be clearly 

linked to the outcome of 

interest. 

Proportion of colonoscopies that 

were completed in a timely 

fashion47

5 4 - Additional testing required 

with larger data set. 

Measure should be clearly 

linked to the outcome of 

interest. 

Percentage of patients with colon 

or rectal cancer undergoing 

colonoscopy as part of their 

evaluation48

4 3 2 Measure should be clearly 

linked to the outcome of 

interest, such as change in 

patient management, more 

accurate staging. 

Colonoscopy miss rate for 

significant colonic neoplasia49,50

5 3 3 No matter how the miss 

rate is estimated, this is 

likely not a usable measure 

because tandem or closely 

timed colonoscopies are 

not performed in general 

clinical practice. 

Technical measures 

Complication rate of colonoscopy51 5 4 4 Complications to be 

identified need to be those 

that are clinically relevant, 

and precisely defined. The 

period during which they 

are identified needs to be 

accurately specified. 

Serious postendoscopic procedure 

complication rate47,52

5 4 4 Studies have been 

performed for screening 

colonoscopies only. 

Rate of unplanned reversal of 

sedation medication47 5 4 - Testing required. 

Intraprocedure colonoscopy 

complication rate53 5 4 - Testing required. 
38 

Table 4. Quality measure ratings – Question 1a – continued

Quality measure rating 

(range 1-5, where 1 = poor and 5 = ideal)

Quality measure

Important 

and usable 

(I)

Scientifically 

acceptable 

(S)

Well 

tested 

(T)

Recommendations for the 

measure

Colonoscopy completion rate54 5 4 - Must explicitly exclude from 

the numerator and 

denominator patients who 

are unable to undergo 

complete colonoscopy prior 

to surgery (because of 

obstruction, need for 

emergent operation, etc.). 

Cecal intubation rate47,52 5 4 4 Testing required. “Cecal 

intubation” should be 

appropriately defined 

including method of cecal 

identification by landmarks 

(appendical orifice and 

ileocecal valve). 

Adenoma removal rate for patients 

over 50 years old47 5 4 - Testing required, including 

validation of the “benchmark” adenoma rate for 

different patient populations 

and establishment of both 

clinically and statistically 

relevant deviations from 

these benchmarks. 

Percentage of patients with 

adequate bowel preparation prior to 

colonoscopy47,52,55

5 4 4 Additional testing of the 

measure required. Needs 

to be linked to a provideror facility-level process for, 

e.g., adequate instruction 

for bowel preparation. 

Otherwise, this is a 

measure of patient 

adherence and not a 

measure of the quality of 

care. 

Proportion of colonoscopies 

performed by physicians with 

specialized training47

5 4 - Needs to be developed and 

further tested on a 

representative group of 

physicians. Also, the 

definition of specialized 

training needs to be 

evaluated, specifically is 

there a threshold number of 

procedures required? 

Percentage of patients with an 

adequate understanding of the 

colonoscopy procedure53

5 4 - Testing required. The link 

to an outcome of interest 

needs to be established 

(e.g., patient satisfaction, 

patient compliance with 

next scheduled procedure). 
39 

Note: Investigators used their judgment to determine the quality measure rating for each measure across all studies 

that utilized the measure. See Appendix F*

 for details. 

Future Research 

Since several potentially useful quality measures for diagnosis of CRC were retrieved, future 

work should focus on correcting any deficiencies of these measures. For the measure percentage 

of patients who underwent appropriate evaluation for a positive FOBT, the numerator should 

include only those patients who underwent colonoscopy, as this reflects the most recent 

guidelines.6

 The denominator should include all patients who underwent FOBT by the 

appropriate technique (stool collection at home, any applicable dietary or medication restriction 

[depends on the specific FOBT used], results evaluated by a certified laboratory) and had a 

positive test (defined as any positive window). The following patients should be excluded from 

the numerator and denominator: those with positive occult blood test based on a digital rectal 

exam only; individuals for whom screening is inappropriate either because of life-limiting 

comorbidity, because they are younger than screening guidelines recommend or because they are 

not yet due for screening (e.g., had a colonoscopy within the past 5 to 10 years); and individuals 

who were offered colonoscopy, but refused. The first and second exclusions are based on the 

premise that one cannot measure the quality of screening followup if the screening itself was 

incorrectly performed or inappropriately applied. The third exclusion reflects that patient 

preference, if based on an informed decision, should not penalize the provider or system being 

evaluated. 

 One additional point to resolve in the definition of “adequate evaluation” is the maximum 

acceptable time lag between the positive FOBT and the subsequent colonoscopy. At present, this 

specification cannot rely on recourse to a guideline, but might instead be derived from basic first 

principles and the structural characteristics of the health systems being studied. 

 Finally, it should be noted that this quality measure needs to undergo additional testing. The 

link between followup of positive FOBT and CRC-specific survival is well supported. The 

reliability and accuracy of various data sources, however, require additional investigation. Data 

sources could include chart review, administrative claims data, patient survey, and provider 

survey. 

 Other technical process measures of colonoscopy also have potential for improvement to 

become useful quality measures. These measures should address the accuracy and safety of the 

procedure. Complication rates require precise descriptions including the specific complication 

(e.g., bleeding), the definition (e.g., bleeding that results in hospitalization or transfusion), and 

the time frame (e.g., within 30 days of the procedure). It would be best to separately collect the 

data for individual complications within prespecified time frames (e.g., intraprocedural, within 2 

hours of the procedure, within 30 days) even if the complications might be reasonably reported 

in combination. Potential data sources are procedural flow sheets, chart review, and patient 

survey. These sources require testing for reliability and accuracy. Cecal intubation rate requires 

a precise definition and should include those patients postcecal resection by the inclusion of “to 

the surgical anastomosis.” The numerator should include all patients who meet this definition. 

The denominator should include all colonoscopies regardless of test indication. Patients with 

obstructing lesions should be excluded from the numerator and denominator. Patients with other 

 

* Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
40 

reasons for failed complete colonoscopy, such as poorly tolerated examination, poor bowel 

preparation, or intraprocedural complications, should still be included in the denominator. 

Success and accuracy of this quality measure depends on accurate documentation of cecal 

landmarks or surgical anastomoses in the colonoscopy report. The first step is to have 

documentation of the extent of the colonoscopy as a required part of the colonoscopy report 

template. The next and more difficult step is validation of the report itself and of collective 

endoscopic databases, such as CORI. Corroboration by another health provider or inclusion of a 

still picture or video documentation are possible methods to compare the report to another source 

of information. Results may be biased if those performing the colonoscopy know that they are 

being observed. 

Question 1b: Pathologic Staging 

Question 1b was: What quality-of-care measures are available and what evidence is 

available for these measures to assess the quality of diagnosis of colorectal cancer, including the 

availability and accuracy of pathologic staging?

Background 

Although the microscopic appearance of adenocarcinoma of the colon and rectum tends to be 

straightforward, a detailed pathology review is essential for diagnosis, characterization, and 

staging. Stage is the most accurate predictor of survival for patients with CRC, and accurate 

staging is critical for appropriate patient management and meaningful clinical research.62 

Pathologic stage combines the clinical staging information with surgical findings, incorporating 

data from the pathologic examination of resected primary and regional lymph nodes.13

Historically, numerous different staging systems for CRC have been used, but a single 

internationally recognized system is required to ensure a common language for cancer that is 

understood by clinicians in all specialties. The Tumor, Node, Metastasis (TNM) Staging System 

of the American Joint Committee on Cancer (AJCC) and the International Union Against Cancer 

(UICC) fulfills this mandate and has the added advantages of being data-driven, updated on a 

regular basis, multidisciplinary in design, and applicable to all current techniques of stage 

evaluation.14 Uniquely, the TNM Staging System is governed by rules of application, both 

general and site-specific.63 

The single most important determinant of prognosis in patients with apparently localized 

carcinoma of the colon and rectum is the presence or absence of nodal metastases at the time of

surgical treatment. Patients with stage I or II cancer of the colon or rectum have an anticipated 5-

year survival rate in excess of 75 percent. In contrast, individuals with N1 disease have a 5-year 

survival rate of only 45 to 60 percent.64 The presence of nodal metastases is currently the most 

important factor in determining whether an individual is a candidate for adjuvant therapy15 and is

arguably the single most important factor in predicting survival.16-18 

Nodal staging accuracy for colorectal carcinoma is currently under scrutiny. Several factors 

contribute to variation in nodal staging accuracy, including the amount of mesentery resected, 

fixative, diligence of search for nodes, and number of histologic levels examined. The small size 

of many mesenteric nodes, including some with micrometastatic carcinoma, increases the risk of 

missing metastases. Recent studies demonstrated that 10 to 15 lymph nodes are needed for 
41 

reliable staging. Although the exact number of lymph nodes required for adequate lymph node 

evaluation in patients has been debated, the 1999 consensus statement by the College of 

American Pathologists (CAP) recommended evaluating 12 to 15 nodes in node-negative 

patients.19 

There is evidence to indicate that the quantity of lymph node harvest has a direct effect on 

staging.20,21,65 The principles of surgical resection of colon and rectal cancer dictate en bloc

removal of the cancer and sufficient proximal and distal bowel to encompass potential 

submucosal lymphatic tumor spread and include the regional mesenteric draining lymphatics. 

Standard resections of the colon and rectum are founded on anatomic structures that provide not 

only for adequate radial and longitudinal margins but for adequate regional lymph node clearance 

as well. Despite these well-recognized guidelines, there may be substantial variability in the type 

of resection performed for CRC, which could lead to variability in the number of nodesremoved. 

Additionally, the number of nodes in a surgical specimen may vary from one patient to another

and may perhaps depend on the extent and diligence of the pathologic examination,66 since many 

pathologists are unfamiliar with these rules or unsure how to interpret them in specific 

situations.67 

Results 

Our literature search retrieved two process measures that have been utilized to assess the 

quality of staging of CRC: 

• Adequate lymph node retrieval and evaluation.13,54,61,68,69

• Percentage of newly diagnosed CRC cases who were staged using the AJCC system.70,71

Applications have ranged from clinical epidemiology to quality improvement. 

Baxter et al.68 applied the measure adequate lymph node retrieval and evaluation. Adequate 

sampling was defined as an examination of at least 12 lymph nodes, based on the UICC and the 

AJCC definitions. Population-based data from the National Cancer Institute’s (NCI’s) 

Surveillance, Epidemiology, and End Results (SEER) program was used for this purpose. 

Exclusion criteria were clearly mentioned (in situ or metastatic disease, patients with prior 

malignancies, non-adenocarcinomas, patients with appendiceal malignancies who underwent 

radical surgery and did not receive neoadjuvant radiation). The study evaluated the mean and the 

median number of lymph nodes examined. More traditional ratio-based measures of the 

proportion of patients who had no lymph nodes examined and the proportion of patients who had 

at least 12 nodes examined were also calculated. Trends over time were evaluated and tested for 

any association with patient age, race/ethnicity, sex, or geographic location; year of diagnosis; or 

tumor stage, grade, and anatomic site. The study was limited by its use of population-based data 

with partial information on patient factors, tumor factors, and specimen adequacy. The measure 

does not account for the fact that lymph node evaluation depends on the surgical quality, which 

could lead to variability in the number of nodesremoved. Additionally, the number of nodes in a 

surgical specimen may vary from one patient to another. 

An Institute of Medicine (IOM) report13 proposed a similar measure to assess the quality of 

cancer care in Georgia. In this case, appropriate histological assessment of CRC was defined as 
42 

an assessment of 12 or more lymph nodes; this measure is thus essentially equivalent to one of 

the variations of the measure described by Baxter et al.,68 above. 

Two other measures used an essentially identical structure, but prescribed retrieval of 

different numbers of lymph nodes for their thresholds. The National Committee for Quality 

Assurance (NCQA)61 proposed 14 lymph nodes, based on NCCN guidelines.24 The Colon 

Cancer Workgroup,54 on the other hand, proposed eight lymph nodes and based this decision on 

the SEER median of eight. 

Galvis et al.69 utilized a similar measure to compare the pathology specimen gross 

examination performance of pathology residents and pathologists’ assistants. Investigators 

retrospectively reviewed surgical pathology reports for 176 colorectal specimens (submitted 

from 1997-1999) from one anatomic pathology laboratory serving a teaching center. For all 

specimens, the total number of lymph nodes retrieved, the number of positive nodes retrieved, 

and the length of colorectal specimens were recorded. The measure is not explicitly validated; 

however it is known that the number of lymph nodes sampled is an important predictor of CRC 

outcomes.20,21 The measure was applied at a single center and needs further testing. 

It should be noted that this measure (adequate lymph node retrieval and evaluation) requires 

documentation of the examination on operation notes and pathology gross specimen examination 

reports. The choice of the number of lymph nodes could be an institutional decision. A trigger 

that was utilized by the Colon Cancer Workgroup54 may be insightful: this institution evaluates 

any situation where a hospital or a surgeon-pathologist combination regularly obtains fewer than 

eight lymph nodes. The group also recognizes that rectal cancer resection specimens may have 

fewer lymph nodes. 

Steele71 analyzed the National Cancer Data Base (NCDB) to examine the trends in stage at 

diagnosis. Although not explicitly defined as a quality measure, the percentage of newly 

diagnosed CRC cases that were staged using the AJCC system was reported and used as a 

marker of appropriate cancer diagnosis and treatment over the time period studied. The study 

evaluated trends over time, census region, and hospital type. This measure can be generalized, 

but is limited by its inability to differentiate between those cases that were staged using 

pathology criteria versus those that utilized clinical criteria for staging.

Chiaverini et al.70 utilized essentially the same measure to assess the progress in the control 

of CRC from 1987-2000 in Rhode Island using data from the Rhode Island Cancer Registry. 

Conclusions

We retrieved two major quality measures that have been utilized in the United States for 

assessing the quality of pathology staging of patients with CRC. Ratings for these measures are 

given below in Table 5; further details are provided in Evidence Tables 3 and 4 in Appendix E*. 

Adequate lymph node retrieval and evaluation is linked to an outcome of interest (better 

staging, leading to appropriate management and better survival) and based on evidence.20,21,65,72-

77 It is also well developed and tested (I5S4T5) and could be utilized for internal quality 

assessment of personnel. 

It may be important to note that the number of lymph nodes examined reflects not only the 

quality of pathology care, but also reflects an interaction between tumor factors (right-sided 

 

* Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
43 

colon resections may often contain a larger mesentry and more lymph nodes) and the quality of 

the surgical procedure (removal of an adequate number of lymph nodes, an adequate length of 

specimen, and surgical clearance).78 

Percentage of newly diagnosed CRC cases that were staged using the AJCC system is 

another measure that is well developed, has face validity, is related to an outcome of interest, and 

represents an important leverage point for improving the quality of CRC care.71 However, it 

does not distinguish between pathology and clinical staging (I4S3T3). 

The recommendations made herein are based on a review of currently available data. 

Obviously, future data or clinical realities (e.g., costs, changes in societal goals) or the 

development of new technologies may alter the recommendations in the future. Nonetheless, for 

the major recommendations, we do not expect their basic core importance to change. 

Table 5. Quality measure ratings – Question 1b 

Quality measure ratings 

(range 1-5, where 1 = poor and 5 = ideal) 

Quality measure

Important 

and usable 

(I) 

Scientifically 

acceptable 

(S) 

Well 

tested 

(T) 

Recommendations for the 

measure

Technical measure 

Adequate lymph node retrieval and 

evaluation13,54,61,68,69 5 4 5 Lymph node evaluation 

also depends on the quality 

of surgery and tumorrelated factors (these 

factors are true for all 

pathology measures). 

Could be utilized for internal 

quality assessment of 

personnel. 

General measure 

Percentage of newly diagnosed 

CRC cases who were staged using 

the AJCC system70,71

4 3 3 Needs further testing. In 

order to evaluate quality of 

pathology staging a 

distinction must be made 

between clinical and 

pathology staging. 

Note: Investigators used their judgment to determine the quality measure rating for each measure across all studies 

that utilized the measure. See Appendix F*

 for details. 

Future Research 

For newly diagnosed CRC cases to be staged according to the AJCC (or another) system is a 

necessary, but not sufficient, condition for accurate pathologic staging. Recognizing this, future 

research should focus on clarifying precisely what constitutes an adequate lymph node 

evaluation. In particular, first principles suggest that not all patients should have exactly the 

same number of lymph nodes evaluated, and expanding the evidence base on this subject would 

 

* Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
44 

be of help. Also, in order to better establish the points of leverage, components of these 

measures that are attributable to the surgeon should be separated from those that are attributable 

to the pathologist. 

Question 2a: Polypectomy for Malignant Polyps 

Question 2, in its entirety, was: As appropriate to specific stages of colorectal cancer, what 

quality-of-care measures are available and what evidence is available for measures of quality of 

care of treatment of colorectal cancer, including: (a) polypectomy for malignant polyps, 

including evaluation of surgical margins; (b) surgical therapy for colon and rectal cancers; (c) 

appropriate use of adjuvant chemotherapy and adjuvant radiation therapy, including for patients 

with metastatic but potentially curable (hepatic/pulmonary-resectable) disease; and (d) 

appropriate use of radiation therapy for either curative or palliative therapy, specifically for rectal 

cancers? We will discuss each of the subquestions (a, b, c, and d) in separate sections, beginning 

with Question 2a. 

Question 2a was: As appropriate to specific stages of colorectal cancer, what quality-of-care 

measures are available and what evidence is available for measures of quality of care of 

treatment of colorectal cancer, including polypectomy for malignant polyps, including evaluation 

of surgical margins? 

Background 

Adenomas of the gastrointestinal tract may undergo malignant transformation in the 

adenoma-carcinoma sequence and are regarded as precursors of most colorectal carcinomas. 

Colonoscopy is considered a safe and effective method of screening for colorectal adenomas, and 

endoscopic polypectomy has been demonstrated to prevent progression to adenocarcinoma. The 

term “malignant polyp” refers to a macroscopically benign-appearing adenoma in which invasive 

carcinoma is detected histologically in the resected specimen. With tumor infiltration through 

the lamina propria and possible invasion of venous and lymphatic vessels, malignant polyps have 

the potential for lymph node metastases, and endoscopic polypectomy may not be curative. The 

decision on subsequent management of malignant polyps may be difficult because of 

controversial studies in the literature. Several studies show adequacy of endoscopic 

polypectomy alone for selected cases, but some authors advocate colectomy for any invasive 

carcinoma arising in an adenoma. Many endoscopists tend to favor subsequent surgery as the 

strategy of choice for all these patients.79 Some elements of this decision include level of 

invasion (probably the most important factor), the type of polyp, the presence of lymphatic or 

blood vessel invasion, poor differentiation of the tumor, and positive margins of resection in the 

polypectomy section. 

Even though subsequent management decisions are difficult and must be individualized to 

the circumstances of individual patients, one point of general agreement is that a crucial piece of 

information is provided by the pathology report, the successful generation of which requires the 

surgeon to collect the specimen according to sound clinical principles and the pathologist to 

provide a comprehensive report. In some cases, the result of this report will suggest a clear plan 

of action: for example, for a level 0 invasion of a pedunculated adenoma a polypectomy should 

suffice, while a level 4 invasion of a sessile adenoma into subserosal connective tissue or 

microscopic cancer at the resection margin80 suggests surgery. However, there are many 
45 

intermediate cases between these two extremes for which guidelines are either not present or not 

helpful. 

Results 

No quality measures on this topic were found. 

Conclusions 

In the absence of any current quality measures, our recommendations are derived from basic 

first principles and are provided immediately below, under “Future Research.” 

Future Research 

The clinical decision for the treatment of a malignant polyp (e.g., endoscopic polypectomy 

alone, endoscopic polypectomy followed by segmental colectomy, colonic resection alone, 

primary colotomy and polypectomy) is not straightforward, and the evidence base is relatively 

modest (e.g., including the lack of guidelines). Accordingly, potential measures of the form 

percentage of patients with malignant polyps receiving a certain form of treatment do not seem 

to be a fruitful approach at present, although as the evidence base improves this kind of measure 

might be specified for those subgroups of patients for which the clinical decision is clear. On the 

other hand, a measure such as percentage of patients with malignant polyps receiving a 

sufficiently comprehensive evaluation (however defined), and treatment selected from a menu of 

appropriate alternatives (this menu perhaps specified using expert consensus) might be helpful. 

Recognizing the importance of the collection and analysis of the pathologic specimen, 

various technical measures could be generated. The portion of Question 4 that pertains to 

pathology reports illustrates how such reports might be formatted. 

In summary, for future research we recommend extending the pathology reports described in 

Question 4 to the analysis of polyps, developing more general measures addressing the overall 

adequacy of the evaluation of possibly malignant polyps, and developing measures that specify 

whether a patient received one out of a set of clinically reasonable treatments. Notably, the CAP 

has a checklist available for polypectomy specimens. 

Question 2b: Surgical Therapy 

Question 2b was: As appropriate to specific stages of colorectal cancer, what quality-of-care 

measures are available and what evidence is available for measures of quality of care of 

treatment of colorectal cancer, including surgical therapy for colon and rectal cancers? 

Background 

In responding to this question, we extended the patient population to include patients with 

colon cancer, as well as patients with rectal cancer. 
46 

Surgery is part of the standard management of patients with colon and rectal cancer stages I, 

II, and III, and for selected patients with stage IV disease. In addition, chemotherapy is the 

standard adjuvant therapy for stage III colon cancer, and chemoradiotherapy is the standard 

adjuvant therapy for stage II and III rectal cancer.81 However, for stage IV colon and rectal 

cancer, chemotherapy alone may be appropriate,81 with surgery and/or radiation reserved for 

palliation of symptoms due to bulky primary tumors. 

Although adjuvant therapies have improved local recurrence rates and survival, it remains 

clear that proper surgical resection is critical in determining the overall outcome of a patient with 

CRC. In regards to colon cancer, it is important to remove the tumor with adequate margins and 

provide the pathologist proper mesenteric tissue to allow for proper lymph node sampling. 

Precise surgical technique is even more critical for rectal cancer. 

One of the most significant advances in the treatment of rectal cancer has been the concept of 

performing rectal cancer resection according to the principle of total mesorectal excision (TME). 

Previously, blunt dissection irrespective of the anatomic planes of the rectum was widely utilized 

and taught as the proper mechanism to excise rectal tumors. However, 15 to 50 percent local 

recurrence rates were reported using these blunt techniques. In 1982, Heald et al.82 recognized 

that most local recurrences after rectal cancer resection were a result of inadequate and imprecise 

surgical excision of the rectal lymphovascular pedicle that may contain micrometastases. 

Moreover, they recognized that by using meticulous, sharp dissection under direct vision and not 

violating the anatomical planes of the visceral and parietal pelvic fascia down to the level of the 

levator ani muscles, the rectum and its mesentery could be removed as an intact unit. The 

quality of the excision can be examined by careful macroscopic as well as microscopic 

assessment of the resection specimen.83 These techniques dramatically reduced the incidence of 

positive lateral margins and led to astonishingly low local failure rates while still performing a 

sphincter-preserving operation. Recently, several studies have concluded that using combined 

modalities and TME, local recurrence remains under 10 percent, with cancer-specific survival of 

over 70 percent22 compared with conventional blunt surgical techniques, where local failure rates 

range from 15 to 45 percent.84,85 In these studies, precise anatomic dissection also prevented 

injury to the sympathetic and parasympathetic nerves in the pelvis, which reduced the previously 

high incidence of bladder and sexual dysfunction following rectal cancer surgery. TME has 

rapidly been adopted throughout the world. 

The appropriate approach to surgery for rectal cancer depends to a large extent on tumor 

characteristics as measured with such tools as endoscopic ultrasound (EUS), computed 

tomography (CT) scans, magnetic resonance imaging (MRI), and history and physical 

examination. NCI guidelines currently recommend rigid proctoscopy, EUS, or MRI and an 

abdomino-pelvic CT to stage the cancer preoperatively. By implication, measures that focus on 

sphincter preservation rates should take the characteristics of the tumor into consideration. 

Regardless of the location of the tumor, one of the critical tasks of the surgeon is to generate 

a high-quality specimen for the pathologist to analyze. Classically, this has been perceived as 

removing the major vasculature to the section of colon being removed or a total mesorectal 

excision for rectal cancers. This technique assures adequate lymph node harvesting for the 

pathologist, although the precise number of nodes cannot be described by the surgeon. These 

measures are difficult to ascertain and are again a function of the surgeon’s technical abilities. 

These should be considered valid measures of the quality of surgery since they determine the 

subsequent pathology staging of the tumor and management decisions based on the stage. 
47 

Survival of CRC increases with the number of lymph nodes recovered, regardless of their 

positivity.65

An important point to note in any outcome measure related to surgery is the need to examine 

all management processes beyond surgical resection (as appropriate to stage and location of 

tumor): was adjuvant therapy offered subsequent to surgery?; did the patient refuse it?; was 

radiotherapy offered?; was surveillance performed? 

Results 

We identified a number of measures that have been utilized to assess the quality of surgery in 

colon and rectal cancer. These are either general measures that assess whether surgery was 

performed, or technical measures that assess the quality of the surgical technique employed. 

Each measure takes the form of a numerator describing the number of eligible patients receiving 

some form of surgery for colon cancer, rectal cancer, or for both types of cancers, divided by a 

denominator describing the number of eligible patients. 

Regarding the denominator, there is considerable variation: some measures focus on colon 

cancer only, some focus on rectal cancer only, and some focus on both. The best approach to 

quality measures is to focus on a single location and stage (e.g., stages I and II colon cancer, or 

stage III colon cancer) or to present the data disaggregated by location and stage. 

Newcomb et al.86 and Demissie et al.87 have studied rates of refusal in patients and have 

determined that a significant percentage of patients refuse to undergo surgery (from 9 to 33 

percent, with refusal rates being higher among African-Americans) and chemotherapy (from 16 

to 33 percent, with refusal rates being higher among the elderly). With these observations in 

mind, all measures should incorporate patient preference within them, or else adopt a measure 

that assesses “intent-to-treat rates” instead of rates of “treatment received.” 

Measures That do not Account for Stage. Several general process measures have been utilized 

to assess the surgery-related quality of care in all available colon and rectal cancer patients, 

without accounting for stage and/or location. For example, such a strategy would be necessitated 

when available administrative databases do not contain this information. A common and very 

serious problem with all these measures is that not performing surgery may be appropriate for 

some stages of colon and rectal cancer (e.g., stage IV colon and rectal cancer). Accordingly, 

these measures are simply listed here and are not considered further. 

The aforementioned measures include: 

• Rate of surgery for CRC.88,89

• Percentage of patients with CRC who received surgery.86,90,91

• Number of CRC patients who received surgery as part of primary treatment.92 

Measures That Assess Appropriate Receipt (or Non-Receipt) of Surgery for Colon or 

Rectal Cancer for Patients Having Cancers of Various Location and Stage. Demissie et al.87

analyzed the SEER database (which contains specific information regarding physician 

recommendations regarding surgical treatment, the actual receipt of the treatment, and reasons 

for not receiving treatment) to determine non-receipt of surgery. The numerator and 
48 

denominator were well defined. Results were presented by stage of tumor and histologic type. 

The study considered the intention of the surgeon to treat as an important factor in categorizing 

patients as “having received standard surgical treatment;” a sensitivity analysis was performed 

using the actual treatment received to observe if the results were significantly different. 

Cooper et al.93 utilized Medicare provider analysis and review files from 1984 to 1987 to 

study whether race was associated with differences in curative surgical resection rates. The 

numerator and denominator were moderately well defined. The results were presented by race, 

by differentiating black and white patients in the population studied. Differences in race-related 

patient preferences may have accounted for the observed difference in receipt of surgical 

resection among black and white patients. 

Merrill et al.91 also analyzed the rate of surgical resection by using two tumor registries, part 

of the SEER databases, Medicare inpatient claims data, two HMO databases, and 1990 U.S. 

census data. An appropriate definition was used to specify the numerator and denominator, for 

all stages of tumor 0-IV. The aim of the study was to compare treatment utilization in HMO and 

fee-for-service (FFS) settings for Medicare colorectal cancer cases. 

Steele71 used the NCDB to assess trends in colon and rectal cancer diagnosis and 

management in order to provide benchmarks for clinicians and policymakers. An appropriate 

rate was used, and the numerator and denominator were specified: percentage of stage 0-III 

colon cancer patients who underwent surgery, and stage III patients who received chemotherapy 

in addition to surgery. AJCC staging was used, and the measure was based on AJCC guidelines. 

The NCDB may overstate performance because of its voluntary nature. 

Roetzheim et al.94 studied the impact of health insurance and race on colorectal cancer 

surgery using the measure percentage of patients with CRC who underwent cancer-directed 

surgery. Cancer-directed surgery rates were identified from the Florida Cancer Database 

System, supplemented with information from discharge abstracts using Current Procedural 

Terminology (CPT) codes. The measure was precisely defined, and confounders examined were 

age, sex, marital status, household income, and educational level; site of cancer; stage at 

diagnosis (information was not detailed); residence; and comorbid scores. The study evaluated 

colon and rectal cancer patients separately and concluded that treatments received by patients 

varied considerably according to insurance payer: those with HMO insurance were more likely 

to receive definitive surgical treatment, and this difference increased with advancing cancer 

stage. Among non-Medicare patients, those having Medicaid and the uninsured were less likely 

to receive surgical treatment than private FFS patients. There were no differences between races 

in receipt of definitive surgical treatment. 

Temple et al.95 assessed two measures for stage IV rectal cancer: cancer-directed surgery 

rates and metastatectomy rates. The general approach regarding the measure was well 

structured, numerators and denominators were defined, and the SEER-Medicare registry was 

used as the data source. The main difficulty with this measure is that the evidence for a health 

benefit from removing the primary cancer in patients with stage IV cancer is uncertain and likely 

to be highly patient-dependent. In many cases, such patients may derive greater benefit from 

more aggressive systemic chemotherapy than from surgical intervention. 

O’Connell et al.96 compared cancer-directed surgery rates by age using data from the 1991-

1999 SEER database. The measure was precisely specified, AJCC staging was used for defining 

the tumor stage, and results were appropriately broken out by stage. Patient preference was not 

accounted for. There was no validation of the receipt of surgery beyond the usual policies of the 

SEER registry, which are quite stringent: accuracy is ensured by checking the accuracy of 
49 

sample cases by re-abstracting data from medical records every year. The completeness has 

been recorded as 98 percent.97 The study is limited by the age and geographic constraints of the 

linked SEER-Medicare files. 

O’Connell et al.98 later assessed cancer-directed surgery rates in order to compare rectal 

cancer patient outcomes in younger and older patients using the SEER registry. 

Hyman et al.99 also measured the curative resection rate by utilizing a voluntary registry 

created by the Vermont chapter of ACS with the Vermont Program for Quality Health Care. The 

study did not specifically aim to compare quality of care among participating surgeons, but 

rather to compare quality of care with national benchmarks. 

A better defined measure was percentage of patients with colon or rectal cancer who had 

curative resection.100 Curative surgery in this 10-year retrospective study using a tumor registry 

was defined as surgery performed in the absence of documented spread. The well-defined 

measure was utilized to compare surgery rates and outcomes in black and white patients with 

CRC with some risk-adjustment. There were no significant differences in treatments received 

between black and white patients; but significantly worse survival was noted among black 

patients in spite of similar rates of chemotherapy and radiation treatments received by patients of 

the two races. 

Tropman et al.101 assessed the rates of appropriate primary therapy (wide surgical resection 

and anastomosis) for CRC as defined by the NCI guidelines. The denominator included patients 

with stages I-III colon cancer, all of whom should receive surgery in the absence of 

contraindications or refusal. Medical records and physician office records were used to collect 

data. The only addition that may be suggested is the documentation of patient preference for or 

against surgical treatment. 

Govindrajan et al.102 studied the impact of race on rate of surgery in stage II and III colon 

cancer patients in a small group of CRC patients over the period 1984-1994 in a single 

institution through a retrospective analysis of a hospital tumor registry. The numerator and the 

denominator were appropriately defined, and results were presented by location of the tumor and 

stage of the disease. The degree to which these results can be generalized is unknown, as is the 

quality of the data obtained from this registry. 

Sphincter-Saving Surgery for Rectal Cancer. Schrag et al.103 used the SEER-Medicare 

database to study ostomy rates after adjusting for various patient characteristics. The measure is 

well defined, risk-adjusted, and relevant. However, details regarding the tumor (e.g., size of 

tumor, distance from the anal verge) were not considered. 

Hodgson et al.104 utilized a similar measure, the rate of colostomy for patients with rectal and 

rectosigmoid cancers. The adequately defined measure was studied by examining the California 

Cancer Registry. The validity and risk-adjustment for this study was fair. 

Morris et al.105 used the SEER database to examine the rate of sphincter-preserving 

operations. However, this study did not allow the authors to examine the anatomical location of 

the tumor, which would influence the type of operation performed. The measure used precise 

specifications for including patients, and the data sources were fairly reliable. Risk-adjustment 

was not adequately performed. 

A similar measure was used by Simons et al.,106 who analyzed a regional cancer surveillance 

program registry (the quality of the registry is unknown) to study the sphincter-saving procedure 

rate for patients with different stages of rectal cancer. (The main predictor of interest was 
50 

hospital volume, and the data set was matched by stage.) The numerator and the denominator 

were well defined. 

In a recent study, Purves et al.107 compared rates of sphincter-saving procedures and 

abdomino-pelvic resections among surgeons with varying caseload volumes. They utilized the 

1997 Nationwide Inpatient Sample (NIS) and the Hospital Cost and Utilization Project (HCUP) 

databases; the measure was well defined, but the data sources had no information regarding 

tumor staging. 

The NCQA61 proposed the quality measure percentage of rectal cancer cases that received a 

sphincter preservation procedure at time of surgery. It was based on the NCCN guideline,25 was 

well defined, and suggested that the data sources could be cancer registries and claims data. 

Schrag et al.108 followed 2,815 patients to assess the abdominal perineal resection (APR) 

rates. The denominator was all patients identified from the SEER-Medicare linked registry for 

1992-1996, aged 65 years and older, with stage I-IV rectal cancer undergoing surgery and 

receiving FFS care. APR rates were compared by providers with different case volumes. A 

deficiency of this measure is that while APR is not the ideal procedure for most patients, it is 

clearly indicated for some. As currently constructed, the measure does not differentiate between 

those patients for whom APR is indicated. 

Measures That Assess the Quality of Surgery for Colon and/or Rectal Cancer. Controlling 

for tumor stage, Read et al.109 evaluated the impact of surgeon specialty on the extent to which 

patients experienced local tumor recurrence, or local control rate. Patients undergoing curative 

treatment for primary rectal adenocarcinoma with neoadjuvant radiotherapy followed by 

proctectomy were followed for 5 years using medical records, cancer registry, or 

physician/patient data. The measure was well specified and adjusted for tumor size and location. 

One of the measures most often utilized to measure the technical quality of CRC surgery is 

whether the surgical specimen provided was sufficient to accurately stage the cancer. Such 

measures13,54,61 are discussed under Question 1b (pathologic staging). This section will focus on 

measures that assess the complications and outcomes of surgery. 

Measures That Assess the Complication Rates of Colon and/or Rectal Cancer Surgery. 

Patients greater than 80 years old with colon or rectal cancer during 1961-1987 from a single 

institution were studied by Coburn et al.,110 who calculated the postsurgical complication rate

(i.e., any complication), including anastomotic leak rates. Inpatient, outpatient, and office 

records were utilized as data sources. The measures were not clearly specified (e.g., the period 

within which postsurgical complications or leak rate would be assessed was not defined), and the 

ability to generalize to other data sources is uncertain. 

Measures That Assess Intermediate Outcomes of Surgery for Colon and/or Rectal Cancer. 

Thirty-day mortality and in-hospital mortality are two intermediate outcomes that can assess the 

quality of surgery. However, important considerations for these measures are the comorbidity 

score and the stage of cancer. Unless these two factors are accounted for, these measures are 

unlikely to be valid. Another important consideration is the nature of the lesions: whether the 

lesions operated upon were obstructed or perforated is important to consider because these 

factors are associated with a higher risk of postoperative mortality. 
51 

Rabeneck et al.111 assessed 30-day operative mortality rates using the national VA patient 

treatment file. The measure is well-defined and compares mortality in elderly versus younger 

patients following surgical resection for colorectal cancer. 

Coburn et al.110 also assesses 30-day mortality for patients who received operative treatment 

for colon and rectal cancer between 1961-1987. This measure is well defined using inpatient, 

outpatient, and office records as data sources. The study adjusted for obstructed and perforated 

lesions, but not for comorbidities. 

Cooper et al.93 performed a similar analysis using Medicare provider analysis and review 

files. Comobidities and nature of lesions were not considered. 

Schrag et al.103,108,112 compared 30-day mortality rates for low- and high-volume hospitals. 

In several studies, they assessed the rates for colon cancer, rectal cancer, or both cancers. They 

adjusted for most confounders: stage and hospital volume, comorbid illnesses, age, sex, race, 

socioeconomic status, and acuity of hospitalization. 

Hodgson et al.104also examined the impact of hospital volume on postoperative (30-day) 

mortality rates of patients diagnosed with rectal cancer stages I-III using state registries. 

 Temple et al.95 restricted the measure to 30-day mortality for stage IV colon cancer. Using 

the SEER database and Medicare claims data, they defined the measure precisely and adjusted 

for comorbidities. Whittle et al.113 also used perioperative mortality rate for colon cancer 

patients enrolled in Medicare. The measure was not adjusted for by disease stage, comorbid 

factors, or emergency presentation, all which can affect peri- and postoperative survival rates. 

A well-defined measure was used by Agarwal et al.114 to assess 30-day operative mortality 

rate, by analyzing a tumor registry from 1975 to 1980. Five-year survival distributions, as well 

as analyses by stage, were done to determine what factors influence operative mortality. 

Dimick et al.,115 Harmon et al.,116 and Ko et al.117 assessed in-hospital mortality rates. All of 

them defined the numerator and denominator appropriately. All of them adjusted for comorbid 

conditions, nature of admission (urgent versus emergent), age, and sex of patients. Dimick and 

colleagues115 used the NIS for colon cancer resections to compare mortality by hospital volume. 

Harmon et al.116 utilized Health Services Cost Review Commission (HSCRC) data, and Ko and 

colleagues117 used the NIS to examine both hospital and surgeon volume impact on mortality 

rates. 

Conclusions 

Ratings for the measures described here are given in Table 6; further details are provided in 

Evidence Tables 5 and 6 in Appendix E*

. The most basic measure of the quality of surgical 

management is whether surgical therapy is provided as appropriate to the location and stage of 

the cancer. Results could potentially be broken out by stage or presented as a single summary 

measure, rates of appropriate primary therapy for CRC as defined by the NCI guidelines 

(I4S4T4). Regarding the denominator of this measure, patients for whom the therapy is 

inappropriate should be excluded. This point is particularly problematic for patients with stage 

IV colon or rectal cancer, for whom the intent of the surgery is palliation rather than cure. 

Ideally, the measure should also distinguish between patients who were offered “appropriate” 

surgery and refused, and those patients who were not offered such surgery. Making these 

 

* Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
52 

distinctions increases the data collection demands (e.g., although sometimes included within 

registries, such information is not included in the typical administrative database). 

Surgical complication rates are another very natural measure. Most variations on this 

measure focus on mortality rates (e.g., 30-day mortality [I5S4T5], in-hospital mortality [I5S3T3]), 

although a delineation of the actual complications would be helpful as well, this latter type of 

measure being relatively underdeveloped. Presumably, such a measure should be both stratified 

by stage and location, and also risk-adjusted to take into account other clinical characteristics of 

the patient. 

For patients with rectal cancer, various measures have been proposed regarding the rate of 

sphincter-saving surgery (I5S4T4). This is an outcome of importance to patients. However, 

quality measures should consider detailed information about the anatomical location and other 

characteristics of the tumor, which is not yet the case in practice. 

Technical measures associated with surgery are underdeveloped. 

The recommendations made herein are based on a review of currently available data. 

Obviously, future data or clinical realities (e.g., costs, changes in societal goals) or the 

development of new technologies may alter the recommendations in the future. Nonetheless, for 

the major recommendations, we do not expect their basic core importance to change. For 

example, no matter how surgery changes technically, it will not be able to cure all stage III 

patients. 

Table 6. Quality measure ratings – Question 2b 

Quality measure rating 

(range 1-5, where 1 = poor and 5 = ideal)

Quality measure

Important 

and usable 

(I) 

Scientifically 

acceptable 

(S) 

Well 

tested 

(T) 

Recommendations for the 

measure

General measures 

Non-receipt of surgery87 5 4 4 Data sources should 

include stage and location 

of tumor. 

Surgical resection rates91,93 5 4 4 Data sources should 

include stage and location 

of tumor. 

Percentage of stage III colon 

cancer patients receiving surgery 

and chemotherapy71

4 3 2 Needs testing. 

Percentage of CRC patients who 

underwent cancer-directed 

surgery94-96,98

4 3 3 “Cancer-directed surgery” 

needs to be defined with 

reference to a guideline. 

Metastastectomy rate for rectal 

cancer95 4 3 2 Needs testing. 

Curative resection rate99,100 5 3 4 “Curative resection” needs 

to be accurately defined. 

Rate of appropriate primary therapy 

for CRC as defined by the NCI 

guidelines101

4 4 4 - 

 
53 

Table 6. Quality measure ratings – Question 2b – continued

Quality measure rating 

(range 1-5, where 1 = poor and 5 = ideal)

Quality measure

Important 

and usable 

(I)

Scientifically 

acceptable 

(S)

Well 

tested 

(T)

Recommendations for the 

measure

Percentage of colon cancer 

patients (stages specified as 0-III or 

I-II or II & III) who underwent 

surgery102

5 2 1 Needs to be precisely 

defined. 

Ostomy rates103,104 5 4 5 Details need to include the 

precise preoperative 

measurement of the tumor 

from the anal verge, the 

assessment of the tumor’s 

involvement of the anal 

sphincter complex, the 

“fixed” nature of the tumor, 

preoperative sphincter 

function of the patient, the 

body habitus of the patient, 

and finally the patient’s 

preference for kind of 

surgery performed. 

Percentage of rectal cancer cases 

receiving a sphincter preservation 

procedure at time of surgery61,105-

107 

5 4 4 Details need to include the 

precise preoperative 

measurement of the tumor 

from the anal verge, the 

assessment of the tumor’s 

involvement of the anal 

sphincter complex, the 

“fixed” nature of the tumor, 

preoperative sphincter 

function of the patient, the 

body habitus of the patient, 

and finally the patient’s 

preference for kind of 

surgery performed. 

Abdominoperineal resection (APR) 

rate108 5 4 4 Details need to include the 

precise preoperative 

measurement of the tumor 

from the anal verge, the 

assessment of the tumor’s 

involvement of the anal 

sphincter complex, the 

“fixed” nature of the tumor, 

preoperative sphincter 

function of the patient, the 

body habitus of the patient, 

and finally the patient’s 

preference for kind of 

surgery performed. 

Local control rate109 4 4 4 Depends to some degree 

on management 

subsequent to surgery. 

 
54 

Table 6. Quality measure ratings – Question 2b – continued

Quality measure rating 

(range 1-5, where 1 = poor and 5 = ideal)

Quality measure

Important 

and usable 

(I)

Scientifically 

acceptable 

(S)

Well 

tested 

(T)

Recommendations for the 

measure

Complication rate110 5 3 2 Complications include 

anastomatic leak rate. 

30-day mortality 

rate93,95,103,104,108,110-114

5 4 5 Comorbid illnesses, tumor 

biology, tumor location and 

stage, perioperative care 

may all affect 30-day 

mortality rate. 

In-hospital mortality rate115-117 5 3 3 Time frame under 

consideration needs to be 

precisely defined. Several 

factors apart from the 

quality of surgery could 

contribute to mortality. 

Note: Investigators used their judgment to determine the quality measure rating for each measure across all studies 

that utilized the measure. See Appendix F*

 for details.

Future Research 

Staging Workup. Considering the relevance of an appropriate staging workup and the 

availability of NCI guidelines defining it, quality measures that assess this workup would be 

appropriate. 

Appropriate Primary Therapy for Stage IV CRC. Although the NCI standards have been 

used effectively for stage I-III colorectal cancers, there does not seem to be a consensus on the 

effectiveness, timing, or type of surgery that should be performed on a patient with metastatic 

disease. The reason for this may be multi-factorial, including physicians’ historic pessimism, the 

heterogeneous nature of stage IV disease (e.g., ranging from a single liver metastasis to 

carcinomatosis), the risk of obstruction, and the use of new highly potent chemotherapy agents. 

At present, this heterogeneity greatly complicates the use of quality measures based on anything 

but data sources that describe in detail both the patient’s clinical condition and preferences. The 

question of metastasectomy for patients with stage IV CRC remains controversial. Future 

research should focus on determining appropriate management modalities for stage IV CRC. 

Quality measures can only be developed subsequent to this research. 

Improvement in Sphincter-Preserving Data. Following complete oncologic resection, 

sphincter-preservation remains an important aspect of rectal surgery for the patient. However, 

the current data comparing sphincter-preserving operations to APRs reveal very high APR rates. 

However, these data often lack critical details, including the precise preoperative measurement of 

the tumor from the anal verge, the assessment of the tumor’s involvement of the anal sphincter 

 

* Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
55 

complex, the “fixed” nature of the tumor, preoperative sphincter function of the patient, the body 

habitus of the patient, and finally the patient’s preference for kind of surgery performed. Ideal 

measures should account for all these aspects that lead to decisions regarding APR. 

Anastomotic Leak Rates. Quality measures need to be developed to determine what factors 

influence this potentially life-threatening risk following colon or rectal excision. In regards to 

rectal cancer, numerous factors may affect the leak rates, including comorbid illnesses, obesity, 

and use of preoperative chemoradiation. Finally, surgical experience and potential 

volume/outcome measures could be examined for relationship to this complication rate. 

Recurrence Rates Following Resection of CRC. For colon cancer, recurrence rates may be 

influenced by the number of lymph nodes harvested during the colectomy. With rectal cancers, 

the use of TME has clearly decreased the local recurrence rates; however, the margin that should 

be considered adequate is controversial. Twenty years ago, the surgical dictum was to obtain a 5 

cm margin; however, within the last 10 years, this has been reduced to a 2 cm margin, and now 

even this distance is being questioned. There clearly is a need for a well-constructed trial to 

examine this critical surgical issue, the results of which could lead to specifying the optimal 

surgical margins as part of a quality measure. 

Improvement in Quality of Rectal Surgery. Quality measures need to be developed to 

ascertain that the operation was performed according to the principles of TME and the specimen 

retrieved is ideal. 

Complications of Rectal Cancer Surgery Due to the Use of Neoadjuvant Therapy. Although 

several recent studies demonstrate no apparent increase in surgical complications with the use of 

preoperative chemoradiation, newer and potentially more toxic approaches may further 

complicate this already difficult procedure. If so, this should be accounted for by quality 

measures. Bladder and sexual dysfunction also need to be assessed, although recent evidence 

indicates that these complications may be multifactorial, with radiation playing a much larger 

role.118-122

Appropriate Treatment for T1 Rectal Cancer. This is very controversial. Recommendations 

vary from transanal excision with or without chemoradiation to proctectomy. Unless there is 

adequate research leading to consensus regarding treatment, no quality measure can be 

developed in this area. 

Question 2c: Adjuvant Chemotherapy 

Question 2c was: As appropriate to specific stages of colorectal cancer, what quality-of-care 

measures are available and what evidence is available for measures of quality care of treatment 

of colorectal cancer, including appropriate use of adjuvant chemotherapy and adjuvant radiation 

therapy, including for patients with metastatic but potentially curable (hepatic/pulmonaryresectable) disease? 
56 

Background 

In responding to this question, we extended the original patient population to include patients 

with metastatic but potentially curable (hepatic/pulmonary-resectable) disease. We also 

extended the definition of adjuvant therapy to include chemotherapy, radiation therapy, and their 

combination. 

We identified a number of measures that have been utilized to assess the appropriate use of 

adjuvant chemotherapy. Each measure takes the form of a numerator describing the number of 

eligible patients receiving some form of adjuvant therapy (e.g., adjuvant chemotherapy, adjuvant 

chemotherapy plus radiation therapy) divided by a denominator describing the number of 

patients in whom the treatment is indicated. The 1990 National Institutes of Health (NIH) 

Consensus Guidelines state that patients with stage II and III rectal cancer should receive 

adjuvant chemotherapy plus radiation therapy, whereas patients with stage III colon cancer 

should receive chemotherapy alone.15 Although most of the quality measures regarding 

appropriate use of adjuvant chemotherapy and radiotherapy for patients with colorectal cancer 

used the 1990 NIH consensus guidelines as a benchmark, it is important to note that these 

guidelines may no longer represent state-of-the-art treatment for all patients. For example, for 

carefully selected patients with T1-2N1 or T3N0 rectal cancer, some authorities have suggested 

that omission of adjuvant radiation therapy be evaluated in clinical trials.123 More recent 

recommendations from the American Society of Clinical Oncology say that adjuvant 

chemotherapy might also be clinically reasonable, on a case-by-case basis, for patients with stage 

II colon cancer.23 The NCCN guidelines state that both chemotherapy and radiation therapy 

might be considered for palliation of patients with stage IV rectal cancer,25 and chemotherapy 

plus monoclonal antibodies (cetuximab and bevacizumab) as palliation for patients with stage IV 

colon cancer,24 at least among those patients that can tolerate these therapies. Regarding resected 

stage IV colorectal cancer, the NCCN guidelines also state that patients who have completely 

resected liver metastases should be offered 6 months of adjuvant chemotherapy.

Regarding the denominator, there is considerable variation in stage of cancer: this includes 

reporting the results of stages I, II, III, and IV separately; limiting consideration to a single stage 

(e.g., stage III); grouping stages II and III; and grouping stages into local (stages I and II), 

regional (stage III), and distant (stage IV). Some measures focus on colon cancer only, some 

focus on rectal cancer only, and some focus on colorectal cancer without differentiating between 

the two. The best approaches are to focus on a single location and stage (e.g., stage III colon 

cancer) or to present the data disaggregated by location and stage (e.g., stage I rectal cancer, 

stage I colon cancer, and stage IV colon cancer). It is also reasonable to combine those 

circumstances where the guideline recommendation is the same (e.g., stage II rectal cancer and 

stage III rectal cancer). The approach to be avoided is the grouping of circumstances for which 

the guideline recommendation differs (e.g., the recommendation for stage III rectal cancer is 

chemotherapy plus radiation therapy, while the recommendation for stage III colon cancer is 

chemotherapy only, so the grouping of stage III CRC should be avoided). 

It should be noted that there is substantial overlap in the measures considered under 

Questions 2c and 2d. As an organizing principle, all general measures that described 

chemotherapy and/or radiation therapy are included under Question 2c. The primary focus under 

Question 2d is technical measures pertaining to radiation therapy (regardless of whether this 

radiation therapy is given with chemotherapy). 
57 

Results 

Measures That do not Specify Stage and/or Location. A few versions of the basic measures 

do not specify stage of disease. Because the clinical issues are so different among patients with 

different stages of disease, measures that do not explicitly include stage cannot be recommended. 

Examples include Dominitz et al.,88 which used an administrative database that did not include 

information on stage, the National Patient Treatment File from the Department of Veterans 

Affairs. This was also attempted by Wudel et al.100 in a circumstance where the sample size was 

too small to subdivide patients by stage. Coia et al.124 used the Patterns of Care Study database, 

a nationally based sample of patients receiving radiation therapy during 1988, to inquire how 

many patients receiving radiation for cancer of the colon or the rectum also received surgery. 

Beginning with the subset of patients receiving radiation, rather than all patients with disease of a 

certain stage, makes the measure impossible to map back to an inception cohort of all patients 

with a certain stage of disease, and thus difficult to interpret. Coburn et al.110 analyzed the effect 

of age on the provision of adjuvant therapy, but did not break out the analysis by stage (or 

provide a precise description of the database used). 

Measures That Use Suboptimal Combinations of Stage and/or Location. The measure by 

Newcomb and Carbone86 makes an inadequate differentiation between types of cancer, using the 

categories of local, regional, and distant colorectal cancers, with colon and rectal cancers 

grouped (i.e., the measure does not recognize that the guideline recommendations within the 

category “regional” differ according to location and stage). Stage was based on a state tumor 

registry, and adjuvant therapy was based on patient report (which is unvalidated). This version 

of the basic adjuvant therapy measure is not recommended. 

The measure by Govindarajan et al.102 also makes an inadequate differentiation between 

types of cancers. Medical records of patients with stage II or III colon or rectal cancer treated at 

a hospital in Arkansas between 1984 and 1997 were used to report the numbers receiving 

surgery, radiation, and chemotherapy. Not only were the data for stages and locations with 

different guideline recommendations combined, it was not possible to tell how many patients 

had, for example, both surgery and radiation (e.g., the number of patients with surgery, 

regardless of radiation status, is presented). This approach is not recommended. 

Rogers et al.90 identified patients with CRC from Tennessee Medicare and Medicaid files 

over the period 1984-1994. The primary intention was the elucidation of racial differences in 

processes and outcomes of care, including the use of chemotherapy, in stage III colorectal 

cancer. Moreover, there was no validation of the very low use of adjuvant therapy against 

medical records, suggesting that these administrative data files might not have been appropriate 

for this purpose. Combining stage III colon cancer and stage III rectal cancer is suboptimal, and 

this measure is not recommended. 

NCQA61 described the measure percent of late stage (stage ≥ III) colon cancer patients that 

received one or more courses of adjuvant chemotherapy within 1 year of initial cancer surgery

and specified the window for receiving adjuvant therapy (to allow inclusion of therapy up to 1 

year from surgery), the stages of colon cancer when adjuvant therapy is administered (stage III 

and greater), and the number of courses of chemotherapy. A potential problem with this measure 

is that adjuvant therapy for resected stage IV patients is not a standard recommendation because 

there is not universal agreement on its benefits, so application of this measure could lead to the 

impression that too few patients are receiving adjuvant therapy. 
58 

Measures for Stage IV Colorectal Cancer. Temple et al.95 followed patients identified from 

the SEER-Medicare linked registry for 1991-1996, aged 65 years and older, with stage IV colon 

or rectal cancer, enrolled in a FFS health plan. Patients were cross-classified according to the 

use of primary-cancer-directed surgery, chemotherapy, and radiation therapy. The use of 

adjuvant chemotherapy, with or without radiation therapy, might be considered to be a potential 

quality measure, although it is not supported by a guideline that suggests that this strategy is 

likely to be beneficial for all patients. In particular, this is a decision that should be made on a 

case-by-case basis in consideration of the characteristics of the tumor, comorbidities (e.g., the 

poor performance status of many patients would preclude the use of chemotherapy), and patient 

preferences. The most reasonable application of such a measure would be in a database that 

includes a record of those cases where a physician considered the potential application of 

adjuvant therapy, but in consultation with the patient decided against it. In the absence of such 

information, this is not a version of the basic quality measure that can be strongly recommended. 

Measures for Stage III Colon Cancer. Potosky et al.125 followed patients in the SEER registry 

(1990-1991 and 1995), aged 21 and above, who had completed surgery. For approximately 81 

percent of sampled cases, the offer or administration of adjuvant therapy was verified by 

physician contacts. The authors note that “doctor verification significantly improves the 

completeness of chemotherapy ascertainment because adjuvant chemotherapy is frequently given 

outside the hospital setting, and a note of any planned chemotherapy is typically not reported in 

the hospital record.” The numerator was the number of patients “offered, recommended or 

administered adjuvant 5-Fluorouracil (5-FU) plus levamisole or leucovorin.” This version of the 

measure reflects physician intention rather than actual receipt of therapy, since patients that are 

offered but refuse the therapy are nevertheless included in its numerator. Adjuvant 

chemotherapy use was broken out by various sociodemographic characteristics. The study does 

not explicitly validate the quality measure. 

Hyman et al.99 reported data from a surgeon-initiated registry in Vermont during 1999-2001. 

The measure is the number of stage III colon cancers that were offered referral for adjuvant 

therapy. Because this is a prospectively collected registry, this information is likely to be 

collected with particularly high accuracy. The degree to which this database can be generalized 

is uncertain. 

Mahoney et al.126 followed 69 patients identified from a New Jersey tumor registry for 1989-

1996 with stage III colon cancer and focused on a chart-based examination of the reason that 

adjuvant chemotherapy might not be provided. Of 35 patients not receiving chemotherapy 

therapy, 12 were not offered such therapy, 11 refused, 7 were too old, and 5 had significant 

concomitant disease. Although a small study, this provides support for the notion that 

administrative databases will understate provider compliance with guidelines by omitting 

information that might document that this therapy was either offered but declined, or else might 

not have been clinically appropriate. 

 Ayanian et al.127 followed patients identified from three regional registries within the 

California Cancer Registry during 1996-1997 who were 18 or older, with stage III 

adenocarcinoma of the colon, surviving at least 30 days postsurgery. Information about adjuvant 

therapy was obtained from physician surveys and office records. The numerator of the quality 

measure was the number of patients receiving adjuvant chemotherapy. The study does not 

validate the quality measure, but does include some validation of the data sources. The 

sensitivity of routinely collected registry on receipt of chemotherapy (i.e., using medical record 
59 

review as the gold standard) was 87 percent. Various sociodemographic characteristics were 

considered. Results from a single state registry might not reflect national patterns of care. 

Schrag et al.103,112,128 followed patients identified from the SEER-Medicare linked registry for 

1991-1996, aged 65 years and older, with stage III colon cancer, enrolled in a FFS health plan 

and surviving at least 3 months. The numerator was the number of these patients who received 

chemotherapy within 3 months of surgery. Among others, the impacts of age, race, and hospital 

volume were considered. There was no validation of the receipt of chemotherapy beyond the 

usual policies of the SEER registry, although an unpublished reference asserting that the SEER 

data has 90 percent sensitivity for identifying the use of chemotherapy was cited. The study is 

limited by the age and geographic constraints of the linked SEER-Medicare files. 

 Sundararajan et al.129 followed patients identified from the SEER-Medicare linked registry 

for 1992-1996, aged 65 years and older, with stage III colon cancer, enrolled in a FFS health plan 

and surviving at least 4 months. The numerator was the number of patients receiving 5-FU 

within 4 months. Strengths and weaknesses of the measure are essentially identical to the 

discussion of Schrag et al. above. 

 Keating et al.130 utilized SEER-Medicare linked data for 1993-1999 and followed patients 

aged 66 years and above, with stage III colon cancer, who were enrolled in FFS Medicare Parts 

A and B. The numerator was the number of patients who received adjuvant chemotherapy. 

County-specific utilization rates were calculated and compared according to market share of FFS 

plans versus HMOs. The study does not validate the quality measure and is limited by the age 

and geographic constraints of the SEER-Medicare linked files. Use of adjuvant chemotherapy is 

not validated against medical records (with the exception of the usual SEER Registry 

procedures) and might be underestimated. 

Beart et al.48 presented a similar analysis, subdivided by both location and stage, for patients 

treated in 1983 and 1988 in the national survey of the Commission on Cancer, a nationally based 

survey including up to 25 cases per hospital per year. The numerator was the number of patients 

receiving chemotherapy, of all stage III colon cancer patients. 

Tropman et al.101 followed patients identified from tumor registries in North and South 

Carolina during 1991 and 1996 and supplemented by review of medical records. The numerator 

was the number of patients that were treated with the consensus-based guidelines at the time of 

the cancer. This application was otherwise unremarkable.131 

The Colon Cancer Workgroup54 attempted to refine the numerator of the basic quality 

measure further by specifying “lymph node positive colon cancer” instead of using the 

terminology “stage III.” Specifically, the measure is defined by the declarative statement 

“adjuvant chemotherapy is administered to patients with lymph node positive colon cancer.” It 

is noted that older patients and those patients with many comorbidities might be poor candidates 

for chemotherapy. This refinement of the basic measure was not explicitly validated beyond the 

argument for its face validity. 

 The IOM, in Assessing the Quality of Cancer Care,

13 described the measure adjuvant 

chemotherapy after surgery for stage III colon cancer. It is well defined: “number of patients 

with stage III colon cancer who receive a full course of adjuvant chemotherapy after surgery, 

divided by the number of node-positive stage III patients who undergo surgery.” The rationale is 

as before, the primary distinction being the basis of the measure in the more recent 2005 

guidelines from the NCCN.24 
60 

Measures for Stages II and III Rectal Cancer. Ayanian et al.127 followed patients identified 

from three regional registries of the California Cancer Registry during 1996-1997 who were 18 

or older, with stage III adenocarcinoma of the colon, surviving at least 30 days postsurgery. 

Information about adjuvant chemotherapy and radiation therapy was obtained from physician 

surveys and office records. The numerator of the measure was the number of patients receiving 

both adjuvant chemotherapy and radiation therapy. The study does not validate the quality 

measure, but validates the data sources. The sensitivity of routinely collected registry data for 

whether patients actually received chemotherapy was 87 percent, while the sensitivity for 

radiation therapy was 93 percent. Various sociodemographic characteristics were considered. 

Results from a single state registry might not reflect national patterns of care. 

Keating et al.130 utilized SEER-Medicare linked data for 1993-1999 and followed patients 

aged 66 years and above, with stage II or III rectal cancer, who were enrolled in FFS Medicare 

Parts A and B. The numerator was whether patients received adjuvant chemotherapy and 

radiation therapy. County-specific utilization rates were calculated and compared according to 

market share of FFS versus HMOs. The study does not validate the quality measure and is 

limited to the age and geographic constraints of the SEER-Medicare linked files. Neither 

adjuvant chemotherapy nor radiation therapy was validated against medical records (with the 

exception of the usual SEER Registry procedures), and might be underestimated. 

 Schrag et al.132 followed patients identified from the SEER-Medicare linked registry for 

1992-1996, aged 65 years and older, with stage II or stage III rectal cancer, enrolled in a FFS 

health plan and surviving at least 4 months. Two versions of the numerator were given: the 

number of these patients who received adjuvant chemotherapy (with 5-FU) and radiation therapy 

within 4 months of surgery, and the number of these patients that received radiation alone; the 

number of patients receiving adjuvant chemotherapy alone was also noted. Among others, the 

impacts of age and race were considered. There was no validation of the receipt of 

chemotherapy and radiation therapy beyond the usual policies of the SEER registry, so it is 

possible that these are underreported. Results are also presented separately by stage. The 

analysis recognizes that radiation might be given preoperatively, thus making the distinction 

between preoperative and postoperative use of radiation therapy. The study is limited by the age 

and geographic constraints of the linked SEER-Medicare files. 

Hyman et al.,99 mentioned above, reported data from a surgeon-initiated registry in Vermont 

during 1999-2001. The measure is the number of stage II or stage III rectal cancers that were 

offered referral for adjuvant therapy. Presumably, this would be the stage at which radiation 

therapy would be considered as well, although it was not precisely specified whether this 

adjuvant therapy included chemotherapy, radiation therapy, or both. Because this is a 

prospectively collected registry, this information is likely to be collected accurately. The degree 

to which this database can be generalized is uncertain. 

Potosky et al.125 followed patients in the SEER registry (1990-1991 and 1995), aged 21 and 

above, who had completed surgery. The numerator was the number of patients “offered, 

recommended or administered adjuvant 5-FU plus levamisole or leucovorin.” For approximately 

81 percent of sampled cases, the offer or administration of adjuvant therapy was verified by 

physician contacts. While the data were presented separately according to whether the 

numerator was the number of patients receiving chemotherapy only, the number of patients 

receiving radiation therapy alone, or the number of patients receiving both, the authors 

considered the primary quality measure to be the receipt of chemotherapy (i.e., regardless of 
61 

radiation). Adjuvant chemotherapy use was broken out by various sociodemographic 

characteristics. The study does not explicitly validate the quality measure. 

Neugut et al.133 followed patients identified from the SEER-Medicare linked registry for 

1992 and 1996, with stage II or stage III rectal cancer, enrolled in a FFS health plan and 

surviving at least 4 months. The numerators of the potential measures were the number of 

patients receiving chemotherapy only, the number of patients receiving radiation therapy alone, 

and the number of patients receiving both. Among others, the impacts of age, sex, and race were 

considered. There was no validation of the receipt of chemotherapy and radiation therapy 

beyond the usual policies of the SEER registry, so it is possible that these are underreported. 

Results are also presented separately by stage. The study is limited by the age and geographic 

constraints of the linked SEER-Medicare files. 

Reports That are Disaggregated by Stage and Location (or Stage Only). Any report that 

disaggregates results by location and stage has the potential to create a quality measure for each 

available subgroup. For example, Steele et al.131 uses data on colon cancer cases from the 

NCDB during 1985, 1988, and 1990; breaks this out into stage 0, stage I, stage II, stage III, and 

stage IV; then classifies treatment as surgery only, surgery and chemotherapy, surgery radiation 

and chemotherapy, other, and none. An example of a possible quality measure is the number of 

patients with stage III cancer receiving surgery and chemotherapy. Steele71 presents a similar 

analysis using both colon and rectal cancer patients (i.e., subdivided by both location and stage) 

from 1991, as does Steele134 using patients from 1992. Jessup et al.135 presents a similar analysis 

of this data base for the years 1986, 1987, 1988, 1990, 1991, 1992, and 1993. Participation in 

the NCDB is voluntary and thus has the potential to skew the results in the direction of 

overstating performance. 

 Beart et al.48 presents a similar analysis, subdivided by both location and stage, for patients 

treated in 1983 and 1988, in the national survey of the Commission on Cancer, a nationally based 

survey including up to 25 cases per hospital per year. 

Roetzheim et al.94 presents a similar analysis, subdivided by location and stage (but not 

both), for patients treated in 1994, from Florida’s statewide cancer registry. Because the 

guidelines recommendations for stage III colon cancer differ from those of stage III rectal 

cancer, it would have been preferable to disaggregate the analysis by both location and stage. 

Retchin and Brown136 collected data using medical record review by nurse abstractors (for a 

comparison of HMO versus FFS, after first identifying potential cases from administrative files), 

and disaggregated the data by stage but not location and stage. Inter-rater reliability of the 

abstraction was assessed, as was a comparison versus a sham chart. Because the guideline 

recommendations for stage III colon cancer differ from those of stage III rectal cancer, it would 

have been preferable to disaggregate the analysis by both location and stage. 

 

Other: Intermediate Process Measures. A quality measure developed by Oliveria and 

colleagues137 is referral to a medical oncologist for consideration of adjuvant chemotherapy. 

This intermediate process measure is indirectly based on the 1990 NIH Consensus Statement on 

chemotherapy utilization in stage III disease.15 Since most chemotherapy is administered by 

medical oncologists, “referral to a medical oncologist” is a first step in assuring compliance with 

this guideline. The denominator consists of members of a Massachusetts HMO newly diagnosed 

with CRC from 1997 to 1999. For purposes of assessing chemotherapy utilization, the analysis 

was performed by stage. The numerator is the subset of these patients referred to a medical 
62 

oncologist within 4 months of surgery. Data were obtained from the computerized database of 

the HMO. The study does not specifically validate the quality measure, but reliability of the data 

sources was ascertained by reviewing medical records to confirm data in the computerized 

database. The study is limited by the use of data from a single HMO. 

Conclusions 

With the exception of the intermediate process measure described by Oliveria et al.,137 all the 

measures we identified took the form of number of patients receiving one or more forms of 

adjuvant therapy divided by the number of patients with some form of CRC. Not all of these 

measures were applied in a formal quality improvement context; for example, for the purpose of 

documenting national patterns of care, Steele71,131 cross-classifies the universe of CRC patients 

into all possible combinations of location and stage, then for each of these possible combinations 

reports the number of patients receiving adjuvant chemotherapy, adjuvant radiation therapy, both 

chemotherapy and radiation therapy, and so forth. These potential quality measures more closely 

approximate actual quality measures when an additional analytical step is taken; that is, to select 

(e.g., based on guidelines) for its numerator the most appropriate item from the list of possible 

adjuvant therapies. (For example, the appropriate numerator for patients with stage III colon 

cancer is the number of patients receiving adjuvant chemotherapy). 

 Our presentation is organized according to the denominator of the quality measure (e.g., 

measures for patients with stage III colon cancer). Measures having certain denominators will 

not be formally rated. In particular, when an investigator uses as the denominator every possible 

combination of location and stage then (so long as the numerator is also reported at a high level 

of detail) the reader has complete freedom to define whatever quality measure is desired. We 

also do not formally rate those measures that fail to specify stage and/or location, or else specify 

suboptimal combinations of stage and/or location (here, the former is an example of the latter). 

The most typical such combination of a stage and location involves the grouping of 

circumstances for which the guideline recommendation differs (e.g., the recommendation for 

stage III rectal cancer is chemotherapy plus radiation therapy, while the recommendation for 

stage III colon cancer is chemotherapy only, so the grouping of stage III colorectal cancer should 

be avoided). 

With this in mind, we considered measures for stage III colon cancer, stages II and III rectal 

cancer, and stage IV colorectal cancer. Roughly speaking, these measures were defined as the 

proportion of patients with stage III colon cancer receiving adjuvant chemotherapy (I5S4T5), the 

proportion of patients with stages II or III rectal cancer receiving chemoradiotherapy (I5S3T2), 

and the proportion of patients with stage IV colon cancer or stage IV rectal cancer receiving 

palliative chemotherapy (I5S3T2). 

 Of these measures, perhaps the best developed and best-tested measure applies to patients 

with stage III colon cancer. It is derived from an evidence-based recommendation.15,138-140 

Various investigators have attempted to fine-tune the measure; for example, by further defining 

stage III as lymph node positive, by specifying the maximum time between surgery and the 

receipt of adjuvant therapy, and so forth. Such enhancements seem clinically plausible and 

follow the principle that quality measures should be specified as precisely as possible, but they 

do not change the basic nature of the measure. In particular, applications that use slightly 

different approaches to specifying the above details should be sufficiently comparable for most 

practical purposes. 
63 

 Perhaps more salient are other features of the measure and/or the databases from which this 

measure can be estimated. In particular, because chemotherapy is often administered on an 

outpatient basis, administrative inpatient databases tend to underestimate the number of patients 

that receive chemotherapy. Some investigators have supplemented the use of administrative 

databases by contacting the treating physician directly, although it must be acknowledged that 

such an approach is costly and time-consuming. At the very least, users of administrative 

databases should either find or generate some estimate of the likely number of patients for whom 

the administration of chemotherapy will be missed. 

 Another important distinction applies to the difference between the numbers of patients 

receiving adjuvant chemotherapy, the greater number of patients who are offered this therapy, 

and the (perhaps) still greater number of patients who should be offered this therapy. In some 

quality measurement applications the appropriate numerator is the number of patients for whom 

the recommended therapy is offered, for others it is the number of patients for whom this therapy 

is actually received. A significant number of patients refuse chemotherapy when offered. 

Moreover, failure to consider the number of patients who should be offered this therapy can 

potentially bias the comparison between groups; for example, one explanation for the lower rate 

of utilization of adjuvant chemotherapy among older patients is that poorer functional status is 

more likely to preclude its use. Comorbidities, particularly including the presence of postoperative complications, should also be considered. Investigators that work with administrative 

databases can probably do no better than to use standard techniques of risk-adjustment, while at 

the same time recognizing that these techniques cannot capture all the nuances of clinical 

decisionmaking. When data are collected prospectively, the ideal is to document each 

component of the decision: whether and why each patient was considered by the physician to be 

eligible or ineligible for adjuvant chemotherapy, whether this therapy was offered, whether this 

therapy was accepted, and whether this therapy was completed. 

 With the above background in mind, it can be noted that the quality measure proportion of 

patients with stage III colon cancer receiving adjuvant chemotherapy (I5S4T5) is important, 

represents a significant leverage point in the care of patients that is strongly associated with 

outcomes, and has been used in various applications such as comparing quality of care provided 

over years of diagnosis, clinical variables of the condition, sex, age, race, marital status, median 

income of patients, geographic regions of the country, and across health care delivery systems. 

 Also evidence-based and well validated is the measure assessing the percentage of patients 

with stages II and III rectal cancer receiving chemoradiotherapy (I5S3T2). Most considerations 

are similar to those for stage III colon cancer and are not repeated here. For stages II and III 

rectal cancer, somewhat different versions of the measure have been created. As discussed under 

Question 2d, the ideal version of the measure is the one that most closely matches the most 

recent recommendations from guidelines, which specify that chemoradiotherapy is the preferred 

method of management for essentially all such patients. 

 The measure proportion of patients with stage IV colon cancer or stage IV rectal cancer 

receiving palliative chemotherapy (I5S3T2) is reasonable in principle, but suffers from the 

difficulty that the decision to provide this therapy should very much be made on a case-by-case 

basis, thus implying that prospective data collection is probably necessary. 

 The measure on the proportion of patients referred to an oncologist (I5S3T2) pertains to an 

important leverage point in practice, although it must be admitted that this is neither a necessary 

nor a sufficient condition for the provision of high-quality care. Thus, the relationship between 
64 

the quality measure and outcomes, while plausible, seems less direct that is the case for the other 

measures in this section. 

Final ratings for these measures are summarized below in Table 7; further details are 

provided in Evidence Tables 7 and 8 in Appendix E*

. 

The recommendations made herein are based on a review of currently available data. 

Obviously, future data or clinical realities (e.g., costs, changes in societal goals) or the 

development of new technologies may alter the recommendations in the future. Nonetheless, for 

the major recommendations, we do not expect their basic core importance to change. For 

example, the need for adjuvant therapy for stage III colon cancer is unlikely to change. 

Table 7. Quality measure ratings – Question 2c 

Quality measure rating 

(range 1-5, where 1 = poor and 5 = ideal)

Quality measure

Important 

and usable 

(I) 

Scientifically 

acceptable 

(S) 

Well 

tested 

(T) 

Recommendations for the 

measure

General measures 

Percentage of patients with stage 

III colon cancer receiving adjuvant 

chemotherapy13,54,71,103,112,126,128,129,

131,134,135

5 4 5 Quality of the data needs to 

be ensured. The 

importance of patient 

preferences is elucidated in 

the studies. 

Percentage of patients with stage II 

or III rectal cancer receiving 

chemoradiotherapy132

5 3 2 Guidelines for adjuvant 

therapy for colon and 

rectum cancers are distinct; 

measures need to be 

distinct, too. Needs to 

account for possible 

reasons for non-receipt of 

therapy (e.g., insurance 

coverage or patient 

preference). 

Percentage of patients with stage 

IV colon cancer or stage IV rectal 

cancer receiving palliative 

chemotherapy95,136

5 3 2 Additional testing of quality 

measure required; riskadjustment is especially 

important. Further 

development and testing 

needed. 

Percentage of patients referred to 

medical oncologist for 

consideration of adjuvant 

chemotherapy137

5 3 2 Testing needed with more 

studies. 

 

 

* Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
65 

Table 7. Quality measure ratings – Question 2c – continued

Quality measure rating 

(range 1-5, where 1 = poor and 5 = ideal)

Quality measure

Important 

and usable 

(I)

Scientifically 

acceptable 

(S)

Well 

tested 

(T)

Recommendations for the 

measure

Percentage of patients with stage 

III colon and stage II and III rectal 

cancer receiving adjuvant 

chemotherapy48,90,99,125,127,130

5 4 4 Quality of the data needs to 

be addressed. Studies 

address the need for 

validation of registry data 

with office records. Riskadjustment is important, 

and patient preference 

should be accounted for. 

Rate of adjuvant chemotherapy for 

CRC86,88,94,100-102

5 3 3 Measures for colon and 

rectal cancer need to be 

distinct. 

Percentage of patients receiving 

adjuvant radiotherapy who also 

received adjuvant chemotherapy 

for cancer of the sigmoid colon or 

rectum124

4 3 2 More extensive testing is 

required (in conjunction 

with measures for radiation 

therapy). 

Adjuvant therapy rates110 5 3 2 More testing to be 

conducted from more 

recent data. 

Use of chemotherapy in Stage II 

and III rectal patients131,133,134 5 3 3 Quality measure to be 

tested with more studies, 

with patient preference 

taken into account, and with 

risk-adjustment for 

comorbidities. 

Percentage of late stage rectal 

cancer (stage ≥ III) that received 

one or more courses of adjuvant 

chemotherapy within 1 year of 

initial cancer surgery61

5 4 - Testing needed on whether 

specifying “late stage 

(stage ≥ III)” produces 

similar results as specifying 

“stage III” because there is 

no agreement on use of 

adjuvant chemotherapy for 

stage IV. 

Note: Investigators used their judgment to determine the quality measure rating for each measure across all studies 

that utilized the measure. See Appendix F*

 for details. 

Future Research 

Our recommendations for future research involve better definitions of the measures discussed 

above and development of new measures. 

 Regarding the quality measure adjuvant chemotherapy administered to patients with lymph 

node positive colon cancer patients, it is important to determine if the same number of cases will 

 

* Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
66 

be found for the numerator as would be found if “stage III” or “regional disease” is used to 

define these patients. 

 Regarding the quality measure use of adjuvant chemotherapy for stage III colon cancer, the 

ideal denominator would be patients with stage III colon cancer, who have had a complete 

surgical resection, and who have no contraindications to chemotherapy or severe comorbidities. 

Two possible ideal numerators arise. The first is the number of individuals offered 

chemotherapy, and the second is the number of individuals who received chemotherapy. More 

research would need to focus on determinants of actually receiving chemotherapy among 

patients offered chemotherapy. Also, research should center on which of the two numerators 

yields a better indicator of quality care. Furthermore, for the measure to be useful, adequate 

documentation must be available. Most databases discussed above do not take into account the 

fact that some patients will decline the offer of chemotherapy. More research is necessary on 

how to best derive data for whether chemotherapy was offered and received. 

 The receipt of chemotherapy is only a surrogate marker for whether patients received the 

correct agents, dose, and schedule. Data in breast cancer and lymphoma therapy demonstrate 

that inadequate dose intensity leads to inferior outcomes. While these data are not available for 

CRC, it is possible that similar observations would be made. Research on the ideal measure of 

“proper chemotherapy administration” is necessary. Further study is necessary on whether a 

global measure of chemotherapy adequacy (dose intensity) or individual measures (adequate 

dose, adequate frequency, adequate duration) is preferred. 

 The toxicities of chemotherapy can be substantial, and mortality is reported. Measures of the 

quality of chemotherapy could include the rate of toxicities, but more study on whether this is a 

true indication of quality is necessary. 

 Regarding the quality measure referral to medical oncologist for consideration of adjuvant 

chemotherapy, it is necessary to determine how this measure correlates with a measure of 

whether patients were offered chemotherapy. Also, this measure is entirely untested. 

 Although perhaps outside the current scope of interest, as a point of context it might be noted 

that the quality of the facility in which chemotherapy is administered is also quite important. 

Such issues as ensuring sterility, correct dose and concentration, correct diluent, correct vessel 

and tubing for holding the chemotherapy, and care in how the drug is actually administered to 

the patient are crucial not only for colon cancer, but for chemotherapy in general. 

Question 2d: Radiation Therapy 

Question 2d was: As appropriate to specific stages of colorectal cancer, what quality-of-care 

measures are available and what evidence is available for measures of quality of care of 

treatment of colorectal cancer, including appropriate use of radiation therapy for either curative 

or palliative therapy, specifically for rectal cancers? 

Background 

In patients with locally advanced rectal cancer, multiple randomized controlled trials have 

demonstrated that adjuvant chemoradiotherapy decreases local failure and improves overall 

survival.141,142 Based on these results, the NIH issued a Consensus Statement in 1990 

recommending that all patients with TNM stage II (i.e., a tumor penetrating the rectal wall, 
67 

without regional lymph node involvement) or stage III (i.e., any tumor with regional lymph node 

involvement) rectal cancer receive postoperative adjuvant chemoradiotherapy.15 Since 1990, a 

number of important developments have occurred which have led to significant changes in the 

management of patients with rectal cancer. Preoperative chemoradiotherapy has been compared 

to postoperative chemoradiotherapy, and has been found to improve local control and reduce 

toxicity while being equally effective with regard to survival.143 The use of TME has reduced 

rates of locoregional recurrence compared with older surgical techniques. (Local recurrence may 

further reduced with the use of adjuvant radiotherapy in combination with TME, although the 

absolute benefit of radiotherapy is lower in patients undergoing TME as compared to older, less 

comprehensive resections.144) Finally, refinements in risk stratification for the heterogeneous 

group of patients with stage II and III rectal cancer have enabled tailoring of treatment 

approaches based on the estimated risks of locoregional and distant failure associated with each 

Tumor-Node stage combination.123

Given the importance of adjuvant chemoradiotherapy for rectal cancer, delivered either 

preoperatively or postoperatively, efforts to ensure that it is offered to patients who can tolerate it 

(likely the vast majority of patients who are medically fit for surgery), and that it is delivered in 

accordance with accepted standards are crucial components of quality medical care for rectal 

cancer patients. The outcomes of interest regarding adjuvant chemoradiotherapy are 

locoregional recurrence, overall survival, sphincter preservation, toxicity, and quality of life. 

The major leverage points are as follows: 

1) Adequate preoperative staging. 

2) Referral to medical and radiation oncology if tumor stage T3 or T4, or if nodal 

involvement suspected. 

3) Delivery of chemotherapy and radiotherapy in accordance with accepted professional 

standards. 

The first leverage point is addressed under Question 1b. Our literature search did not reveal 

any measures that addressed the second leverage point. The available measures with regard to 

the third leverage point are discussed below. 

It should be noted that there is substantial overlap in the measures considered under 

Questions 2c and 2d. As an organizing principle, all general measures that described 

chemotherapy and/or radiation therapy are included under Question 2c. The primary focus under 

Question 2d is technical measures pertaining to radiation therapy (regardless of whether this 

radiation therapy is given with chemotherapy). 

Results 

We identified two types of quality measures. The first type, a process measure, addresses the 

question of whether patients with rectal cancer receive radiation therapy as appropriate to their 

stage of disease. The second type of measure, a technical measure, addresses the question of 

whether the technique used to deliver radiation therapy conformed to accepted standards of 

quality. 

Measures of the first type are discussed as part of the response to Question 2c, above. In 

particular, we first eliminated from further discussion those measures that did not specify stage 

and/or location,88,100,110,124 and also those measures that used suboptimal combinations of stage 

and/or location.86,90,102 Then we noted that investigators who report the data according to the 

finest possible disaggregation of stage and location, and use as their numerators a cross-
68 

classification of chemotherapy versus radiation therapy, implicitly allow the users to create a 

guideline-based measure for each stage of disease;48,71,94,134-136 these measures are not discussed 

further. In other words, those investigators who used the most extensive possible treatment of 

their data by stage and location could, by implication, group the data in whatever way seems 

appropriate; in particular, the group could map to the guidelines when discussing grouping by 

stage and location. This report does not consider these further and instead only considers those 

studies that grouped the data before presenting results. Finally, Temple et al.95 measures the 

proportion of patients with stage IV colon or rectal cancer receiving palliative chemotherapy, 

radiation therapy, or both. This measure has been included, although there are no guidelines or 

evidence regarding the use of palliative radiotherapy. 

The main focus of the potential process measures was patients with stages II or III rectal 

cancer. Measures by Ayanian et al.,127 Keating et al.,130 Schrag et al.,132 Hyman et al.,99 Potosky 

et al.,125 O’Connell et al.,98 Baxter et al.,145 and Neugut et al.133 were discussed; additional and 

similar applications by Morris et al.105 and Demissie et al.87 were identified as part of the 

literature review for Question 2d. This set of measures primarily differs according to whether 

their potential numerators were broken out according to the presence of chemotherapy or 

radiation therapy, or instead were broken out according to the presence of chemoradiotherapy 

(i.e., adjuvant chemotherapy plus radiation therapy). Implicitly, the former set of measures is 

agnostic about the most appropriate treatment for this class of patients and simply strives to 

describe utilization among the two clinically plausible approaches of chemotherapy and radiation 

therapy. On the other hand, the latter set of measures, which are based on the NCI and NCCN 

guidelines, explicitly assume that chemoradiotherapy is the approach of choice. The issues will 

be discussed more fully in the “Conclusions” section of this Chapter. 

Since the above process measures have already been described under Question 2c, this 

section is limited to a description of the various technical measures that have been applied to 

radiation therapy for patients with rectal cancer. 

The report by Kline et al.146 serves as the basis for the potential technical quality measure 

adherence to radiotherapy management treatment guidelines for patients with adenocarcinoma 

of the rectum or sigmoid colon. This measure is based on consensus guidelines from the Patterns 

of Care Study (PCS) Treatment Planning Committee. It is well developed. The numerator and 

denominator are well specified. The data source was the 1989-1990 PCS survey. The study 

compares quality indicators across three different types of radiotherapy facilities including 

academic centers, hospital-based practices, and freestanding facilities. The following indicators 

of quality of radiation therapy were assessed: placement of clips by surgeon when tumor 

adhered to other pelvic structures; isodose distribution generated for dose prescription; beam 

energy used should be ≥ 4 MeV; use of at least three radiation therapy fields; fields should be 

shaped with custom blocks; wedges or compensators should be used as needed; port films should 

be taken; patients should be simulated with contrast in the rectum; a small bowel series should be 

done if the total dose will exceed 50 Gy; and prone setup should be used. The PCS Treatment 

Planning Committee guidelines also suggested that information from a pelvis CT should be used 

for radiation therapy planning. The fact that a random sampling of health care facilities in the 

U.S. was used suggests that the results are a valid representation of radiotherapy practice in the 

U.S. However, the study is limited by the fact that the PCS survey was conducted during the 

1989-90 time period, and radiotherapy practices have changed considerably since then. 

The report by Minsky et al.147 formed the basis for the potential technical quality measure 

rate of use of modern radiation therapy techniques and adherence to recommendations of NCI 
69 

sponsored randomized controlled trials in rectal cancer patients. This measure is based on 

recommendations by a consensus of experts (the PCS Rectal Cancer Committee) and on 

adherence to recommendations from randomized control trials. It is well developed. The 

numerator and denominator were clearly specified. The data source was the PCS national survey 

in radiation therapy for the years 1992-1994.148 Many factors considered indicative of highquality radiation therapy were measured in the survey, including the field arrangement; the 

prescription point; the beam energy; the patient position; treatment of all fields each day; 

treatment with a full bladder; use of a belly board; attempt to exclude the small bowel in patients 

treated postoperatively; use of small bowel contrast; placement of surgical clips; and inclusion of 

the scar in all treatment fields in patients treated with APR. Information on the radiotherapy 

dose to the whole pelvis and to the boost field, the radiation fraction size, and the total treatment 

time were obtained by the survey, although the authors do not make any assessment of what they 

view as the appropriate radiation dose, fraction size, or treatment duration. Therefore, adherence 

to guidelines with regard to those radiotherapy factors cannot be assessed from this report. The 

data source was not validated. The fact that a random sampling of health care facilities in the 

U.S. was used suggests that the results are a valid representation of radiotherapy practice in the 

U.S. However, the study is limited by the fact that the PCS survey was conducted during the 

1992-1994 time period, and radiotherapy practices have changed considerably since then. 

Conclusions 

Perhaps the most important process measure for appropriate use of radiation therapy for 

rectal cancer identified by our literature search is the percentage of patients with stage II or III 

rectal cancer who received adjuvant chemoradiotherapy (I5S3T2). In particular, this measure is 

linked to an outcome of interest (locoregional control and survival) and is based on an evidencebased recommendation.15,25 It is well developed, fairly well tested, and a significant leverage 

point. An ideal version of this measure would explicitly consider possible contraindications to 

therapy (e.g., serious medical comorbidity, previous pelvic radiotherapy, inflammatory bowel 

disease, or connective tissue disorder), the distinction between being offered and receiving 

therapy, and the reasons for non-receipt of adjuvant therapy. Indeed, while this measure can be 

used for comparing the quality of care for patients of different demographic characteristics, 

treated in different health care settings, and with different insurance coverage, unless the reasons 

for non-receipt of therapy are analyzed, then the potential remains that the application of this 

measure could be biased (e.g., older patients are more likely to have contraindications to 

chemoradiotherapy than younger patients). 

The other version of this process measure that might reasonably be considered is to break out 

the numerator into patients receiving chemotherapy alone, radiation therapy alone, both 

chemotherapy and radiation therapy, and neither chemotherapy nor radiation therapy. Although 

concurrent administration of chemotherapy and radiotherapy is the preferred approach to 

treatment, there may be circumstances when clinicians recommend one adjuvant modality but 

not the other. For example, it might be reasonable to offer adjuvant chemotherapy but omit 

adjuvant radiotherapy in a patient with a significant connective tissue disease (e.g., systemic 

lupus erythematosus or scleroderma), or in a patient who has received prior pelvic irradiation for 

prostate cancer or cervical cancer. 

While it is vitally important to know whether a patient received chemoradiotherapy, it is also 

critical to assess the technical quality of its administration. Obtaining information on the details 
70 

of chemotherapy and radiotherapy obviously presents a significantly greater challenge than 

determining simply whether chemotherapy or radiotherapy was used at all. The two potential 

quality measures for the technical administration of radiotherapy identified in this report 

demonstrate that this type of measure is feasible. Final ratings for these measures are described 

in Table 8; further details are provided in Evidence Table 9 in Appendix E*

. In addition to 

demonstrating feasibility, these studies also illustrate some of the hazards of attempting to 

ascertain this type of data. In particular, a significant challenge is obtaining data in a timely 

fashion. If there is excessive delay between data collection and reporting of the results, there is a 

significant risk that the results will not be applicable to modern practice. Because of this time 

lag, neither of the two technical radiotherapy quality measures reviewed above is ready for 

immediate use. 

The recommendations made herein are based on a review of currently available data. 

Obviously, future data or clinical realities (e.g., costs, changes in societal goals) or the 

development of new technologies may alter the recommendations in the future. Nonetheless, for 

the major recommendations, we do not expect their basic core importance to change. For 

example, the need for adjuvant therapy for stage III colon cancer is unlikely to change. 

Table 8. Quality measure ratings – Question 2d 

Quality measure rating 

(range 1-5, where 1 = poor and 5 = ideal)

Quality measure

Important 

and usable 

(I) 

Scientifically 

acceptable 

(S) 

Well 

tested 

(T) 

Recommendations for the 

measure

General measures 

Percentage of patients with stage II 

or III rectal cancer receiving 

chemoradiotherapy132

5 3 2 Guidelines for adjuvant 

therapy for colon and 

rectum cancers are distinct; 

measures need to be 

distinct, too. Needs to 

account for possible 

reasons for non-receipt of 

therapy (e.g., insurance 

coverage or patient 

preference). 

Percentage of patients with stage II 

and III rectal cancer receiving 

radiation therapy48,91,98,105,132,133

5 4 4 Quality of the data needs to 

be addressed. Studies 

address the need for 

validation of registry data 

with office records. Riskadjustment is important, 

and patient preference 

should be accounted for. 

Non-receipt of standard radiation 

therapy87 5 4 4 - 

 

 

* Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
71 

Table 8. Quality measure ratings – Question 2d – continued

Quality measure rating 

(range 1-5, where 1 = poor and 5 = ideal)

Quality measure

Important 

and usable 

(I)

Scientifically 

acceptable 

(S)

Well 

tested 

(T)

Recommendations for the 

measure

Rate of adjuvant radiation therapy 

for patients with stage II or III rectal 

cancer71,130,131,134,145

5 3 3 - 

Percentage of patients with stage 

IV colon or rectal cancer receiving 

palliative chemotherapy, radiation 

therapy, or both95

4 3 2 - 

Technical measures 

Adherence of radiotherapy 

management treatment guidelines 

for patients with adenocarcinoma of 

the rectum or sigmoid colon146

5 4 4 Needs to be updated by a 

panel of radiotherapy 

experts to reflect current 

standards of practice. No 

data exist linking any of 

these measures to 

outcomes such as reduced 

locoregional recurrence 

rates, improved diseasefree or overall survival, or 

reduced complication rates. 

Rate of use of modern radiation 

therapy techniques and adherence 

to recommendations of NCIsponsored randomized controlled 

trials in rectal cancer patients147

3 3 2 Needs to be updated by a 

panel of radiotherapy 

experts to reflect current 

standards of practice. No 

data exist linking any of 

these measures to 

outcomes such as reduced 

locoregional recurrence 

rates, improved diseasefree or overall survival, or 

reduced complication rates. 

Note: Investigators used their judgment to determine the quality measure rating for each measure across all studies 

that utilized the measure. See Appendix F*

 for details.

Future Research 

The limitations of the process measures for the stage-appropriate administration of radiation 

therapy for colorectal cancer and the avenues for future research on these process measures are 

discussed above in the “Future Research” section of Question 2c. Herein we provide suggestions 

for future research on technical measures of the quality of radiation therapy for patients with 

colorectal cancer. 

One major drawback to both of the technical radiotherapy measures described above 

(adherence to radiotherapy management treatment guidelines for patients with adenocarcinoma 

 

* Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
72 

of the rectum or sigmoid colon and rate of use of modern RT techniques and adherence to 

recommendations of NCI-sponsored randomized controlled trials in rectal cancer patients) is 

obsolescence due to the time lag between the collection of the data and the reporting of the 

results. 

The feasibility and cost-effectiveness of real-time technical measures of the quality of 

radiation therapy for CRC need to be investigated. With modern telecommunications, the delay 

between collection of data on radiotherapy treatment parameters and reporting could potentially 

be greatly reduced and meaningful, timely quality measures obtained. A panel of radiotherapy 

experts could determine the components of high quality radiotherapy in modern day practice and 

a survey could potentially be conducted via telephone or the internet so that the data could be 

rapidly obtained and used to ensure uniform quality across radiotherapy facilities. 

The principle of radiotherapy quality assurance in “real time” has been demonstrated by 

several ongoing radiotherapy trials administered by the Radiation Therapy Oncology Group 

(RTOG). The RTOG has established mechanisms for (1) credentialing of facilities participating 

in RTOG trials, and (2) rapid review and approval of radiotherapy plans for specific patients 

enrolled on these trials. The costs of reviewing every colorectal cancer patient’s radiotherapy 

treatment plan would likely be prohibitively large. However, facilities might be 

encouraged/required to undergo random sampling of cases as part of a quality assurance 

mechanism. 

The major shortcoming of both of the technical measures of the quality of radiation therapy 

for CRC that are reviewed in this report is that neither have been linked to outcomes such as 

reduced locoregional recurrence rates, improved disease-free or overall survival, or reduced 

complication rates. An implicit assumption has been made that the technical quality of radiation 

therapy is linked to at least some of these outcomes, but there is very little if any data 

demonstrating that such a link exists. Establishing links between specific technical measures of 

the quality of radiotherapy and these outcomes should be a priority of future research on this type 

of quality measure. 

Finally, in creating measures of the technical quality of radiation therapy, one significant 

issue for future research is whether the quality of radiation therapy should be reported with 

regard to specific technical parameters or whether it can be summarized by one global measure. 

In other words, should the results of a quality measure’s individual items should be combined 

into a total score (in effect, creating a single measure), or should they be assessed as individual 

items (in effect, creating a large number of measures)? If one argues that any deficiency in the 

provision of radiation therapy can potentially lead to a worsening of patient outcomes, and 

furthermore that the points of leverage are each individual element of radiation therapy practice, 

then developers of quality measures should perform at least some (although not necessarily all) 

analyses at the level of the individual element. On the other hand, if one argues that there will 

inevitably be some heterogeneity in radiotherapy practices, even among high quality radiation 

therapy centers, then developers should employ a global measure of the quality of radiation 

therapy. Research is needed to determine which of these types of measures is more important, 

usable, and scientifically acceptable. 

 Question 3: Colonoscopic Surveillance 

Question 3 was: What quality-of-care measures are available and what evidence is available 

for measures of colonoscopic surveillance for colorectal cancer? 
73 

Background 

Surveillance refers to the ongoing followup of those at increased risk for disease after 

treatment. Surveillance leads to decreased incidence of CRC and improved survival;33,149,150

however, heterogeneity of followup strategies has precluded assessment of the optimal 

combination of clinic visits, laboratory tests, and procedures.33 The effectiveness of nonendoscopic or combined modality surveillance, such as following tumor markers 

(carcinoembryonic antigen [CEA]) or abdominal CT imaging, is controversial. 

Several professional societies have endorsed guidelines for colonoscopic surveillance after 

curative resection of CRC (Table 9). These guidelines are similar, but differ on the timing of the 

first colonoscopy after a perioperative “clearing” colonoscopy. “Clearing the colon” (1) refers to 

a sufficiently comprehensive examination of the colon to exclude other significant neoplasia, 

such as adenomatous polyps or synchronous cancers, and (2) signifies that all polyps were 

removed. 

All of the most recent guidelines specifically endorse colonoscopy for bowel surveillance. 

Several older articles described measures that included sigmoidoscopy with or without barium 

enema as part of a more generic “bowel surveillance.” Since these other two modalities are no 

longer endorsed for surveillance, we have excluded articles where the colonoscopy rate was not 

reported separately. 

Table 9. Guidelines for colonoscopic surveillance after curative resection of CRC 

Guideline Clearing 

colonoscopy 

1st surveillance 

colonoscopy, time 

postresection 

2nd surveillance 

colonoscopy, if first 

was normal 

American Gastroenterological 

Association (AGA)6

 

At diagnosis or within 

6 months of resection 

3 years 5 years 

ACS151 Not discussed 1 year 3 years 

American Society of Colon and 

Rectal Surgeons (ASCRS)152 At diagnosis or within 

6 months of resection 

3 years 3 years 

American Society of Clinical 

Oncology (ASCO)153 Pre- or perioperative

colonoscopy 

3 to 5 years 3 to 5 years 

ASGE154 At or around the time 

of resection 

3 years 3 to 6 years 

NCCN24 At diagnosis or within 

3 to 6 months of 

resection 

1 year 2 to 3 years 

Sigmoidoscopy and barium enema are not currently recommended for surveillance in CRC 

patients and are beyond the scope of the present question. Several of the studies we considered 

combined colonoscopy with other bowel imaging in the numerator,155-157 which made these 

measures less useable and important. In cases where receipt of colonoscopy could not be 

separated from other bowel imaging modalities, we excluded the study.155,156
74 

Results 

Postoperative (Surveillance) Colonoscopy. Several studies, using slightly different definitions, 

examined the measure percentage of patients receiving postoperative surveillance colonoscopy. 

Cooper and colleagues published two studies evaluating the same measure in the same study 

population.158,159 These studies defined the quality measure the “percentage of patients with nonmetastatic CRC who had colonoscopy up to 3 years after diagnosis.” This process (general) 

measure was based on earlier versions of current guidelines,57,160,161 which recommended 

postoperative surveillance, but the timing of the surveillance (relative to the cancer resection) 

was not specifically included in the guideline. The numerator and denominator for the measure 

were ill defined. The two stages of CRC were “local” and “regional.” The data source was a 

merged SEER and Medicare administrative database. Neither the measure nor the data source 

was validated in this study, although the measure has face validity, being guideline-based. The 

results are limited due to the inclusion of patients aged 65 and older, restriction to the nine SEER 

regions, and the lack of clinical details from an administrative database to determine if these 

were truly “surveillance” colonoscopies or diagnostic procedures to evaluate symptoms. The 

data are relatively dated (diagnosis in 1991 with followup through 1994), and although subjects 

had to have survived at least 6 months for inclusion, other attempts to risk-adjust this measure 

(e.g., it is inappropriate to perform surveillance colonoscopy in patients with poor anticipated 

survival) were not attempted. 

Lafata et al.162 used administrative data and tumor registry data and defined the measure as 

the “percentage of patients with CRC who received a followup colonoscopy.” “Followup” 

meant 2 months to 8 years postresection with curative intent. The measure is fairly well 

developed. The timing of the colonoscopy was not included in the measure. Cited guidelines 

included ASCRS 1992,161 ASCO 1999,163 NCCN 1996,164 and AGA 1997.57 The setting was a 

large multispecialty group and affiliated HMO in Michigan. The restricted geographic location 

and exclusive HMO setting are limitations. 

Keating et al.130 used the SEER-Medicare database to measure the “percentage of patients 

who underwent colonoscopy.” They specifically examined whether or not the colonoscopy was 

performed 7 to 18 months after diagnosis. A comorbidity score was measured and used in an 

adjusted analysis that was primarily aimed at examining the relationship between managed care 

market share and the quality measure, but the measure did not explicitly exclude any patients 

with high comorbidity burden who may not have been surveillance candidates. 

Retchin and Brown136 used HMO clinical data and Medicare claims data to examine the 

“proportion of patients with CRC who had undergone surgery who had had a colonoscopy by 6 

months postresection.” This definition of the measure was based on a physician advisory panel 

and not on published guidelines. It is possible that some of these colonoscopies were in fact 

“clearing colonoscopies” and not the first surveillance examination. This measure was well 

described, but not well developed or well tested. 

An IOM report13 described a similar measure with the suggested use of the Medicare-SEER 

merged database or databases derived from medical records. In this case, the measure was 

refined as number of stage I to stage III CRC cases with a colonoscopy within 1 year of surgery, 

which was based on the NCCN guideline. This was a better developed measure because of the 

explicit inclusion of surveillance colonoscopy timing congruent with at least one published 

guideline. The usability of the specific timing, however, is unclear since conflicting guidelines 
75 

exist. This measure, as described in this report,13 was not risk-adjusted although it was noted 

that “local practice patterns and patient-related factors affect the use of endoscopic procedures.” 

Rulyak et al.157 used HMO claims data, HMO pathology databases, and SEER data from 

Seattle to measure the percentage of patients with local or regional CRC who had colonoscopy 

or flexible sigmoidoscopy with barium enema. They examined two time points: 18 months and 

5 years after diagnosis. The weaknesses were the inclusion of a small number of cases of noncolonoscopic surveillance and the use of a 5-year time point (which is not related to any 

published guidelines). In this regard, the 18-month time frame is reasonable and arguably close 

to the 1-year recommendation of some guidelines (NCCN, ACS). The measure was not 

explicitly risk-adjusted, but patients who did not live at least 6 months after diagnosis were 

excluded. 

NCQA61 proposed the measure percentage of colon cancer cases who receive followup 

colonoscopy within 36 months of surgical treatment. This version of the quality measure is well 

developed in that it includes only colonoscopy and specifies a guideline-endorsed surveillance 

interval. Its weakness is a lack of adjustment for patients too sick to benefit from surveillance 

either because of advanced cancer (i.e., in the absence of specifying disease stage or curative 

intent, patients with palliative operations could have been included in the denominator), or from 

comorbid illnesses. 

NCQA61 also proposed another measure: percentage of rectal cancer cases that received 

endoscopic examination within 12 months per NCCN guidelines. The caveats listed above apply 

to the importance, validity, and usability of this measure as well. 

Perioperative (Clearing) Colonoscopy. Keating et al.130 used the SEER-Medicare database to 

measure the percentage of patients who underwent colonoscopy pre- or postoperatively (defined 

as in the period of 45 days prior to diagnosis through 30 days after surgery). While the measure 

is referred to as “complete colonoscopy,” this can be difficult to determine from procedure codes 

alone, as any endoscopy past the splenic flexure may be coded as a “colonoscopy” even if it was 

not “complete” by the criteria of documented cecal intubation. Furthermore the guidelines (see 

Table 9, above) allow for up to 6 months postresection to perform the “clearing” colonoscopy in 

cases where preoperative complete colonoscopy was not possible due to obstruction by the 

tumor. 

Conclusions 

Ratings for the quality measures discussed in this section are given below in Table 10; 

further details are provided in Evidence Tables 10 and 11 in Appendix E*. 

The measures describing the performance of postoperative surveillance colonoscopies were 

fairly well developed and well motivated: their ratings ranged from I5S4T5 for the measure 

percentage of patients with CRC receiving postoperative (surveillance) colonoscopy to I4S3T2 for 

percentage of patients with local or regional CRC who had colonoscopy or flexible 

sigmoidoscopy with barium enema. The most problematic point was the lack of consensus 

regarding the optimal scheduling of this procedure. (Even in the absence of such consensus, an 

ideal measure would be based on at least one guideline and roughly concordant with the others.) 

 

* Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
76 

The numerator should include colonoscopies only, and not other tests such as barium enema or 

flexible sigmoidoscopy. The measure could also benefit by explicitly specifying whether 

patients without clearing colonoscopy should be included. Technical criteria should be 

considered as well; for example, whether the colonoscopy was complete according to the 

standard of documented cecal intubation. 

The measure describing the performance of perioperative colonoscopy was similarly well 

motivated, but suffered from a lack of precision in distinguishing clearing colonoscopies from 

similar procedures pre- and postsurgery (I5S4T4). Including this additional detail would be 

substantively helpful, although potentially increasing the data collection requirements as not all 

databases (particularly administrative databases) will contain sufficient information to make this 

distinction. 

An additional methodological issue involves the linkage between clearing and surveillance 

colonoscopies. One approach is to measure the rates of clearing colonoscopy and surveillance 

colonoscopy, separately evaluating both the performance and timing of a complete perioperative 

colonoscopy and complete surveillance colonoscopy. A second approach is to define the 

colonoscopic surveillance measure to include only those patients who underwent a clearing 

colonoscopy. The limitation of the latter approach is that a patient who failed to undergo 

perioperative clearing colonoscopy, but then had a first postoperative colonoscopy at 2 years, 

would be excluded from any measure of colonoscopic surveillance quality of care. As a result, 

an opportunity to improve CRC care will be missed. A limitation of only evaluating presurgical 

colonoscopy (rather than perioperative colonoscopy) is that some patients will be unable to have 

a complete colonoscopy at that time due to obstruction or need for emergent surgery (e.g., 

perforation). 

Various technical measures examining critical details of the colonoscopy are examined under 

Question 1a. 

The recommendations made herein are based on a review of currently available data. 

Obviously, future data or clinical realities (e.g., costs, changes in societal goals) or the 

development of new technologies may alter the recommendations in the future. Nonetheless, for 

the major recommendations, we do not expect their basic core importance to change. 
77 

Table 10. Quality measure ratings – Question 3 

Quality measure rating 

(range 1-5, where 1 = poor and 5 = ideal)

Quality measure

Important 

and usable 

(I) 

Scientifically 

acceptable 

(S) 

Well 

tested 

(T) 

Recommendations for the 

measure

General measures 

Percentage of patients with CRC 

receiving postoperative 

(surveillance) 

colonoscopy130,136,158,159,162 

5 4 5 Needs a guidelineconcordant time frame 

(e.g., 1 year or 3 years 

postresection). Testing 

needed, particularly of the 

data source (was staging 

accurate?; was 

documentation of 

colonoscopy accurate?). 

Number of stage I to stage III CRC 

cases with a colonoscopy within 1 

year of surgery13

5 4 - Need to specify that a 

clearing colonoscopy was 

performed. Testing 

needed, particularly of the 

data source (was staging 

accurate?; was 

documentation of 

colonoscopy accurate?). 

Percentage of patients with local or 

regional CRC who had 

colonoscopy or flexible 

sigmoidoscopy with barium 

enema157

4 3 2 Needs to include 

colonoscopy only. 

Percentage of colon cancer cases 

who receive followup colonoscopy 

within 36 months of surgical 

treatment61

5 4 - Testing needed, particularly 

of the data source (was 

staging accurate?; was 

documentation of 

colonoscopy accurate?). 

Percentage of rectal cancer cases 

that received a post surgical 

endoscopic examination within 12 

months postsurgery61

5 4 - Need to clarify if this is 

clearing or surveillance 

colonoscopy, and if the 

latter, need to specify that 

clearing colonoscopy was 

performed. Timing should 

be guideline-concordant, 

i.e., at 1 year not within 1 

year. 

Percentage of patients who 

underwent colonoscopy pre- or 

postoperatively130

5 4 4 Need to distinguish clearing 

colonoscopies from similar 

procedures pre- and 

postsurgery. Further 

testing needed, particularly 

of the data source (was 

staging accurate?; was 

documentation of 

colonoscopy accurate?). 
78 

Note: Investigators used their judgment to determine the quality measure rating for each measure across all studies 

that utilized the measure. See Appendix F*

 for details.

Future Research 

Several potential quality measures related to postoperative colonoscopy were retrieved. They 

relate to two important clinical issues that should be the focus of future research: (1) 

performance of postoperative surveillance colonoscopy, and (2) the performance of perioperative 

clearing colonoscopy. The actual surveillance colonoscopy should be defined as the first 

followup colonoscopy after both resection of the cancer and after “clearing of the colon” for 

other significant neoplasia (adenomatous polyps or synchronous cancers). Therefore, the 

clearing colonoscopy is a leverage point for the surveillance colonoscopy and is arguably worth 

measuring separately. It should be noted at this point that patients with unresectable metastases 

should be excluded from the denominator for measures regarding surveillance, since they do not 

benefit from it. 

While several professional societies endorse guidelines for colonoscopic surveillance after 

curative resection of CRC (Table 9), the evidence upon which these guidelines are based is 

somewhat indirect. The mortality benefit of colonoscopic surveillance has not been tested in 

randomized controlled trials. The general logic is that individuals with a personal history of 

cancer are at increased risk for colon or rectal cancer and therefore should have at least as much 

and likely more followup as those individuals at average risk for CRC. The first step to improve 

the quality measure is to strengthen the link of the quality measure to outcomes of interest. The 

ideal outcome would be survival, but intermediate outcomes such as changes in patient 

management, or detection of metachronous or recurrent cancers at a treatable stage, may also be 

of interest. Without outcomes data it is not surprising that the guidelines differ on the timing of 

the first surveillance colonoscopy. The lack of consensus regarding the timing is one of the 

limiting factors for developing an appropriate measure. Outcomes data would encourage 

consistency across the guidelines, but even without the data consensus could be achieved. Once 

a standard interval for surveillance colonoscopy is established, the quality measure should reflect 

this interval (e.g., percentage of patients who receive the first surveillance colonoscopy at 3 

years). 

Any measure of colonoscopic surveillance should exclude all patients with greater than stage 

III disease at diagnosis, patients who died before the first surveillance colonoscopy was due, and 

patients who did not undergo resection of the cancer (with curative intent). It is reasonable to 

assume that patients in sufficient health to undergo and survive the cancer resection would be in 

sufficient health to undergo at least the first surveillance colonoscopy. This is particularly true if 

the ACS and NCCN recommendations of a 1-year interval are followed. On the other hand, it is 

conceivable that within a 3-year time frame the health status of an individual could change 

considerably and therefore need to reevaluated for the appropriateness (and safety) of followup 

colonoscopy. 

The clearing colonoscopy, while not itself postoperative surveillance, also requires 

evaluation to make the measure of the colonoscopic surveillance precise and guidelineconcordant. The clearing colonoscopy, which is an important opportunity to improve CRC care, 

should be defined as “complete” (to cecal landmarks or surgical anastomosis), with “adequate 

 

* Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
79 

bowel preparation” (“no fecal material obscured a lesion 5 mm or more in diameter and a high 

quality examination takes place”), and with all polyps removed. The denominator is “the 

number of patients diagnosed with colon or rectal cancer.” The numerator is “the number of 

patients who undergo the clearing colonoscopy either prior to surgical resection or within 6 

months of resection.” 

Finally, the data sources need evaluation for reliability and accuracy. While administrative 

or claims data can provide cancer diagnosis and documentation of procedures (surgical resection, 

colonoscopy) they will need supplementation with other data for accurate staging (e.g. registry 

data, chart review) and for details of the colonoscopy itself. The latter will most likely require 

examination of colonoscopy reports as preparation quality, extent of procedure, and 

documentation of polyps or masses seen and removed are not documented in other data sources. 

A standard colonoscopy report template, entered into an electronic database, would facilitate use 

of the quality measurement. In addition, chart review may be necessary to determine changes in 

the health status of the patient if the first surveillance interval follows the 3-year rather than the 

1-year recommendation. 

Question 4: Pathology, Operative, 

and Chemotherapy Reports 

Question 4 was: What measures are available and what evidence is available for measures to 

assess the adequacy and completeness of documentation of pathology, operative, and 

chemotherapy reports? 

Background 

Our literature search did not reveal any measures that related to the documentation of 

surgical and chemotherapy reports. Accordingly, this section will focus on pathology reports. 

The pathology report is the product of several steps, which include surgical dissection and 

histologic block selection, communication between surgeon and pathologist, pathologic 

dissection with macroscopic and microscopic analysis, and finally dictation and transcription of 

the completed report. It is important for making decisions regarding adequacy of surgical 

resection, need for adjuvant therapy, and surveillance. Therefore, a poor quality pathology 

report can adversely affect clinical outcomes.165 Additional value of pathology reports includes 

their ability to provide information for clinical audits, for assessing accuracy of new diagnostic 

and preoperative staging techniques, and for comparing patient groups in clinical trials. 

In recent years, the Association of Directors of Anatomic and Surgical Pathology (ADASP) 

and CAP have both undertaken to publish guidelines for the reporting of common cancers.166,167 

While the formats of the two guidelines are somewhat different, their contents are essentially the 

same. 

The American College of Surgery Commission on Cancer (ACS-COC) accredits cancer 

centers in the United States. Recently, the ACS-COC decided to require elements deemed as 

essential by the CAP to be described in all pathology reports in their accredited cancer centers as 

of January 2004. However, they do not require that the specific CAP protocols or synoptic 

reports be utilized. ADASP has also updated all of its protocols and checklists to comply with 

the ACS-COC requirements. The different elements in the revised ADASP Diagnostic 
80 

Checklists have been divided into the categories Required and Optional. The term Required in 

this context signifies only compliance with the ACS-COC guidelines. ADASP realizes that 

specimens and practices vary, and it will not be possible to report these elements in every case. 

CAP has developed and updated guidelines for reporting cancers of the colon and rectum and 

offers protocols to assist pathologists in providing clinically useful and relevant information in 

the reporting of the results of their examination of surgical specimens.168 CAP regards the 

reporting elements in the Surgical Pathology Cancer Case Summary portion of its protocols as 

essential elements of the pathology report. However, the manner in which these elements are 

reported is at the discretion of each pathologist – taking into account clinician preferences, 

institutional policies, and individual practice. CAP also recognizes that the ACS-COC mandated 

the use of the checklist elements of the protocols as part of the Cancer Program Standards for 

Approved Cancer Programs. However, not all these elements stipulated in the CAP checklist are 

required for accreditation (for approved cancer programs) purposes, since not all of them are 

validated for use in patient management (e.g., venous invasion). 

Results 

The IOM report Assessing the Quality of Cancer Care13 proposed the quality measure 

proportion of CRC cases in which pathologic staging preceded chemotherapy and radiation 

treatment. The numerator is the number of new CRC cases with a medical chart documentation 

of pathologic stage before chemotherapy or radiation is initiated; the denominator is the number 

of new CRC cases with chemotherapy or radiation treatment. The measure recommends that 

medical records be used as the data source. 

The Colon Cancer Workgroup54 recommended the measure pathology report in concordance 

with CAP guidelines. They further define the measure as pathology reporting that includes 

lymph nodes resected, number of nodes positive, tumor characteristics (grade, depth of invasion), 

and mucosal and radial margins. They stipulate the data required for application of the measure 

(pathology and surgical reports); it is based on the CAP and NCCN guidelines, clinically 

relevant, is amenable to change by providers and the system, and is free from confounders. The 

measure was utilized for assessing and improving quality of care in a single health system. The 

measure is well developed, and is linked to an outcome of interest. No information was available 

regarding testing. 

The IOM13 recommended the measure adequacy of pathology reports on CRC to assess the 

quality of cancer care in Georgia. Adequacy is defined as those reports that meet the CAP data 

elements as required by the ACS-COC. The measure is important, usable, precisely defined, 

valid, recommends fairly reliable data sources, and the conditions for use of the measure are 

stated. Risk-adjustment need not be applied in this context. 

 Wei et al.165 evaluated variations in colon carcinoma reporting by laboratory type and 

hospital volume, as related to the ADASP recommendations, in order to identify areas for 

improvement. Data were collected from pathology reports from patients participating in a 

population-based cancer study (North Carolina Cancer Study [NCCS] from 1997 to 2000). The 

denominator consisted of patients who had surgically resected T2-4 pathologically confirmed 

invasive adenocarcinoma and for whom surgical pathology reports were available and who 

resided in the 33-county area used for NCCS. Patients with T0s and T1 were excluded because 

several items of the pathology report were not applicable to localized malignancies. The 

following items were included in the evaluation: percentage of reports that mentioned (a) how 
81 

specimen was received; (b) how it was identified; (c) part of intestine included; (d) the tumor 

site; (e) proximity of tumor to the nearest margin; (f) macroscopic subtype; (g) tumor 

dimensions; (h) macroscopic depth of penetration; (i) appearance of serosa adjacent to the tumor; 

(j) appearance of residual bowel; (k) histological features including histologic type and grade; (l) 

depth of infiltration; (m) lymph node metastases; and (n) involvement of margins. Overall 

percentage of each reported characteristic was noted. Pathology laboratories were categorized as 

contract, teaching hospital, or community hospital laboratories in one analysis, and categorized 

by hospital case volume in another. The study did not assess the accuracy of the reported 

findings, and did not assess the impact of pathology reports on patient outcome. Each of these 

items can be considered to be a separate quality measure. A global score was not calculated. 

 We did not identify any manuscripts dealing with the adequacy of chemotherapy reports. 

Although currently the most important measure is whether chemotherapy is administered to stage 

III colon cancer patients, there may be future interest in determining whether an adequate dose 

intensity was given. Giving an adequate amount of chemotherapy is important for the outcome 

of patients with breast cancer, and if similar data are reported for colon cancer in the future, then 

it will be important to be able to measure whether adequate chemotherapy has been given. In 

this case, the documentation contained in chemotherapy reports will need to be of high quality 

(e.g., reporting the agents given and their dosage). 

Conclusions 

Pathology reports play a very important role in providing clinically useful and relevant 

information from the examination of surgically resected specimens. 

 Ratings for the quality measures discussed in this section are given below in Table 11; 

further details are provided in Evidence Tables 12 and 13 in Appendix E*. The quality measure 

proportion of CRC cases in which pathologic staging preceded chemotherapy and radiation 

treatment,

13 is well developed, free from confounders, and is an important leverage point, but 

needs testing (I5S4T-). 

The quality measure pathology report in concordance with CAP guidelines54 is well 

developed, free from confounders, and is an important leverage point, but needs testing (I5S4T-). 

Not all items included in the CAP guidelines are related to outcomes of interest, and it may be 

useful to select those that address a leverage point. This aspect is considered by the measure 

adequacy of pathology reports on CRC, which limits measurement to scientifically validated 

elements of the CAP guidelines, and is therefore a potentially ideal measure; however, it needs 

field-testing (I5S4T-).13 

The other measures (each being an item of the ADASP guideline) take more of a micro-level 

perspective, addressing individual elements of the pathology report. These are well developed, 

and address some important leverage points for improving quality of colon and rectum cancer 

pathology reporting standards.165 The most important ones are: local extent of tumor, regional 

lymph node metastases, and residual tumor at surgical resection margin. If those elements are 

selected, the process measures will address a leverage point, and be related to an outcome of 

interest (I5S5T4). 

 

* Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
82 

The advantage of using individual items as measures is that they allow a comparison between 

laboratories that utilize different guidelines (e.g., ADASP guidelines instead of CAP 

recommendations), or permit a comparison of only those items that have a proven link to 

survival and therefore are leverage points, or allow one to perform a comparison over time, as 

the continued availability of evidence may alter the relevance of some elements of the 

checklists/guidelines. 

When utilizing any of these measures to assess and/or compare quality of pathology 

reporting the following considerations are relevant: (a) the measures evaluate the presence or 

absence of certain items in a report, not the accuracy of those items; and (b) some omissions in a 

pathology report may not represent poor quality of care: some microscopic reporting (e.g. depth 

of penetration) may obviate the need for reporting on macroscopic features (e.g. appearance of 

tumor); and (c) the measures may not distinguish between quality of the surgical procedure and 

quality of pathology services. 

The recommendations made herein are based on a review of currently available data. 

Obviously, future data or clinical realities (e.g., costs, changes in societal goals) or the 

development of new technologies may alter the recommendations in the future. Nonetheless, for 

the major recommendations, we do not expect their basic core importance to change. 

Table 11. Quality measure ratings – Question 4 

Quality measurerating 

(range 1-5, where 1 = poor and 5 = ideal)

Quality measure

Important 

and usable 

(I) 

Scientifically 

acceptable 

(S) 

Well 

tested 

(T) 

Recommendations for the 

measure

General measures 

Proportion of CRC cases in which 

pathologic staging preceded 

chemotherapy and radiation 

treatment13

5 4 - Needs testing. 

Pathology report in concordance 

with CAP guidelines13 5 4 - Validity needs to be tested. 

Reliability will depend on 

the quality of data available 

in laboratories. Not all 

items in the CAP guidelines 

are related to outcomes of 

interest. 

Adequacy of pathology reports on 

CRC13 5 4 - The inclusion of validated 

elements makes this 

measure particularly useful. 

Needs testing. 

Technical measures 

Percentage of reports mentioning 

how specimen was received165 5 5 4 Reliability will depend on 

the quality of data available 

in laboratories. 

Percentage of reports mentioning 

how specimen was identified165 5 5 4 Validity needs to be tested. 

Reliability will depend on 

the quality of data available 

in laboratories. 

 
83 

Table 11. Quality measure ratings – Question 4 – continued

Quality measurerating 

(range 1-5, where 1 = poor and 5 = ideal)

Quality measure

Important 

and usable 

(I)

Scientifically 

acceptable 

(S)

Well 

tested 

(T)

Recommendations for the 

measure

Percentage of reports mentioning 

part of intestine included165 5 5 4 Validity needs to be tested. 

Reliability will depend on 

the quality of data available 

in laboratories. 

Percentage of reports mentioning 

the tumor site165 5 5 4 Validity needs to be tested. 

Percentage of reports mentioning 

proximity of tumor to the nearest 

margin165

5 5 4 Validity needs to be tested. 

Reliability will depend on 

the quality of data available 

in laboratories. 

Percentage of reports mentioning 

macroscopic subtype165 5 5 4 Validity needs to be tested. 

Reliability will depend on 

the quality of data available 

in laboratories. 

Percentage of reports mentioning 

tumor dimensions165 5 5 4 Validity needs to be tested. 

Percentage of reports mentioning 

macroscopic depth of 

penetration165

5 5 4 Validity needs to be tested. 

Reliability will depend on 

the quality of data available 

in laboratories. 

Percentage of reports mentioning 

appearance of serosa adjacent to 

the tumor165

5 5 4 Validity needs to be tested. 

Reliability will depend on 

the quality of data available 

in laboratories. 

Percentage of reports mentioning 

appearance of residual bowel165 5 5 4 Validity needs to be tested. 

Reliability will depend on 

the quality of data available 

in laboratories. 

Percentage of reports mentioning 

histological features including 

histologic type and grade165

5 5 4 Reliability will depend on 

the quality of data available 

in laboratories. 

Percentage of reports mentioning 

depth of infiltration165 5 5 4 Reliability will depend on 

the quality of data available 

in laboratories. 

Percentage of reports mentioning 

lymph node metastases165 5 5 4 Reliability will depend on 

the quality of data available 

in laboratories. 

Percentage of reports mentioning 

involvement of margins165 5 5 4 Reliability will depend on 

the quality of data available 

in laboratories. 

Note: Investigators used their judgment to determine the quality measure rating for each measure across all studies 

that utilized the measure. See Appendix F*

 for details.

 

* Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
84 

Future Research 

The basic and sound premise behind the currently available measures of the quality of 

pathology reports is that standardization of reports is a crucial step toward the goal of ensuring 

their quality. Additional and related areas for development include the development of summary 

scoring systems (i.e., for those measures that include multiple items), assessing the relative 

importance of various items, and extending the evidence base that links elements of the 

pathology report to patient outcomes. 

This same premise can be applied with equal force to operative and chemotherapy reports, 

and generating analogous measures should be a high priority for future research. With special 

reference to rectal cancers, TME of rectal tumors has clearly demonstrated a reduction in local 

recurrence and an improvement in long-term survival. However, it is difficult to discern if in 

fact a TME has been performed, since operative notes tend to be extemporaneous. Perhaps the 

inclusion of specific anatomic landmarks in the operative note, as well as the pathologist 

comment on the gross integrity of the rectal facial planes, would help confirm the fact that a 

TME of the rectum was performed. Then surgical results of rectal cancer procedures could be 

directly compared without the confounding factor of surgical technique. 

Standardized chemotherapy notes could also be developed to include information on patient 

preferences regarding chemotherapy. In addition, since “receipt” of chemotherapy is only a 

surrogate marker for whether patients receive the correct agents, dose, and schedule, reports 

should include the latter sort of information. In order for chemotherapy reports to be of use, 

more research will be needed on whether information that would be contained in such reports 

(agents used, dose used, over what period of time) will need to be established for colon cancer. 

Subsequently, a more global measure, such as chemotherapy dose intensity, will need to be 

evaluated for whether it is related to outcome. 

As illustrated in the development of quality measures for pathology reports, the role of the 

professional society can be key. 

Question 5a: Patient Populations and Purposes 

Question 5a was: For questions 1-4 above, in what patient populations and for what purposes 

have these quality of care measures been used? 

The variety of patient populations for which the various quality of care measures have been 

used is described in detail primarily in the evidence tables (see Appendix E*), secondarily in this 

Chapter under Questions 1-4, and also in Tables 12-15 under Question 5b. This section is 

intended as a summary and synthesis of these presentations. 

In brief, patients have been studied at every stage of the sequence from diagnosis to treatment 

to posttreatment surveillance. Patient populations (data sources) include population-based 

registries (in particular, the SEER registries, but also the NCDB, statewide registries, and 

others); administrative files of primarily FFS patients (in particular, Medicare databases); 

administrative files of HMOs; direct patient interview (either alone or in conjunction with 

another data source such as a registry); hospital records; individual facility records; and (less 

commonly) randomized trials. 

 

* Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
85 

Comments about individual data sources are included in the responses to Questions 1-4. As a 

rule, technical measures require access to one or more elements that will not be available in 

administrative or similar kinds of databases and require a special effort to collect. Such technical 

measures are usually applied in the context of formal quality improvement (or quality 

measurement), and their inputs are obtained either from medical record audit or the 

establishment of data collection protocols that include the appropriate elements. The latter 

approach is preferable, as the accompanying systemization both encourages the collection of 

accurate data and serves as a prompt toward improving the technical quality of care. 

For example, consider the pathology reporting system that focuses on the 14 submeasures of 

(a) how the specimen was received; (b) how it was identified; (c) the part of the intestine 

included; (d) the tumor site; (e) proximity of tumor to the nearest margin; (f) macroscopic 

subtype; (g) tumor dimensions; (h) macroscopic depth of penetration; (i) appearance of serosa 

adjacent to the tumor; (j) appearance of residual bowel; (k) histological features including 

histologic type and grade; (l) depth of infiltration; (m) lymph node metastases; and (n) 

involvement of margins. At present, these measures all focus on the percentage of reports for 

which these various elements are mentioned. However, if at an individual facility the system of 

reporting was changed to include, for example, a spreadsheet including each of the 14 elements 

for each patient, then such a system would serve as a prompt for the pathologist and surgeon to 

cooperatively work to ensure that all these elements were in fact collected as desired. 

 As noted elsewhere, in contrast to technical process measures, general process and outcome 

measures have been used not only in the quality improvement setting but also in more general 

research settings as well. Many general process and outcome measures can be obtained from 

extant databases (e.g., 5-year survival can be obtained by linking with the National Death Index, 

while the presence of serious complications can be obtained from administrative claims, as can 

the presence of adjuvant chemotherapy), although the link between the quality measure and the 

data source must be examined on a case-by-case basis in order to assure that the information in 

question is collected with sufficient reliability. 

Question 5b: Age, Race/Ethnicity, and Socioeconomic Status 

Question 5b was: For questions 1-4 above, does evidence support the use of any of these 

measures to assess differences in quality of care across patients’ age, race/ethnicity, and/or 

socioeconomic status? 

Background 

We identified several types of reports that were relevant to the question of the usefulness of 

quality measures to assess differences in the quality of care across various patient-level 

characteristics. A relatively small number of studies were specifically intended to assess the 

impact of patient, provider, or health system characteristics on measures of either process or 

outcome. There were a far greater number of studies that evaluated and reported, to a limited 

extent, data on the association of such patient, provider, or system characteristics with process or 

outcome measures. 

This section of the report is intended to provide a comprehensive list and description of those 

studies that were designed to measure and examine the impact of patient characteristics on 
86 

processes and outcomes. The report identifies a representative, but not necessarily complete, list 

of studies for which evaluation of patient level characteristics was not a stated goal. Reasons for 

making this distinction include: (1) studies designed to assess patient factors will be more 

reliable in their measurement and analysis; (2) studies for which patient characteristics were not 

the major focus of the analysis are more difficult to identify (and, thus, were more subject to 

biased ascertainment); and (3) possible reporting bias (studies that mention age, race, or 

socioeconomic status associations in passing are more likely to do so if a statistically significant 

result is found). 

Results 

Tables 12-15 summarize the use of various CRC quality-of-care measures by various patient 

characteristics including age, sex, race/ethnicity, and socioeconomic status. The investigators 

reported 29 instances in which age was related to either outcomes (n = 6) or process measures 

(n = 23; Table 12). Many studies described lower rates of use of chemotherapy or radiation 

therapy for older compared with younger patients using age cutoffs ranging from 65 to 85; 

however, few studies controlled for relevant covariates (e.g., performance status or comorbid 

illnesses) that might explain this difference. 

Similarly, in five cases (Table 13) sex was related to either outcomes (n = 1) or process 

measures (n = 4). Four of the five studies described lower rates of use of adjuvant chemotherapy 

or radiation therapy for women compared with men; however, none controlled for relevant 

covariates (e.g., performance status or comorbid illnesses) that might explain this difference. 

Race was related to process measures, most commonly receipt of certain types of treatment 

in 30 instances (Table 14). With the exception of several measures in the VA system, which did 

not show racial differences,88 each study found that whites were more likely than blacks to 

receive certain types of treatment including surgery, chemotherapy, and radiotherapy. 

Finally, in 16 instances (Table 15) socioeconomic status was related to process measures, 

most commonly receipt of certain types of treatment. Socioeconomic status was most commonly 

measured as income or health insurance status. More than half of the studies found that patients 

of higher socioeconomic status were more likely than patients of lower socioeconomic status to 

receive certain types of treatment; however, several studies showed no statistically significant 

differences in quality measures across socioeconomic status. 

Table 12. Quality measures used to assess differences in quality of care by patients’ age 

Question Quality measure Study Results – age 

1a Colonoscopy complication rate Ure et al.51 Compared to patients 50-70 yr (1%), 

patients ≥ 70 yr (17%) were significantly 

more likely (p < 0.05) to require termination 

of the procedure because of inadequate 

bowel preparation. 

Time from patient presentation 

to physician diagnosis 

Marble et al.46 No statistically significant difference in time 

from patient presentation to physician 

diagnosis for patients < 40 yr compared to > 

40 yr. 

 
87 

Table 12. Quality measures used to assess differences in quality of care by patients’ age – continued

Question Quality measure Study Results – age

2b Postsurgical complication rate Coburn et al.110 Controlling for demographic and clinical risk 

factors, complication rate was 55% for 

patients > 80 yr compared to 35% for 

patients < 80 yr. 

 Anastomotic leak rate Coburn et al.110 Controlling for demographic and clinical risk 

factors, postanastomotic leak rate was 6% 

for patients > 80 yr compared to 3% for 

patients < 80 yr. 

 30-day mortality Coburn et al.110 Controlling for demographic and clinical risk 

factors, mortality rate was 6.2% for patients 

over 80 yr compared to 2.4% for patients 

under 80 yr. 

 30-day operative mortality rate Agarwal et al.114 Higher operative mortality rate in > 80 yr vs. 

< 80 yr (25% vs. 6%), p < 0.01. 

 Perioperative mortality rate Whittle et al.113 Age was significantly associated with 

adjusted rates for perioperative mortality, 

and 1- and 2-yr postoperative mortality. The 

odds ratio for 75-84 yrs of age was 1.69 

(reference group 66-69 yrs), controlling for 

sex, and 3.23 for 85 and older. 

 Surgery rate for CRC patients Newcomb and 

Carbone86 Adjusting for disease stage, no statistically 

significant difference in patients’ rates of 

surgery by age under and over 65 yr. 

2c Adjuvant chemotherapy rate in 

patients with stage III colon or 

stages II or III rectal cancer 

Ayanian et al.127 Patient age inversely related to adjusted 

adjuvant chemotherapy rates adjusting for 

socioeconomic status, clinical and other 

demographic risk factors. 88% of patients > 

55 yr received treatment, compared to 11% 

of patients between 85-89 yr, p = 0.001. 

 Adjuvant chemotherapy rate in 

patients with stage III colon 

cancer 

Schrag et al.128 Adjusting for demographics, socioeconomic 

status, comorbidity and other clinical risk 

factors, age at diagnosis was the strongest 

determinant for receipt of chemotherapy, p < 

0.001. 

 Adjuvant chemotherapy rate in 

patients with stage III colon 

cancer 

Sundararajan et 

al.129 Controlling for demographic, socioeconomic 

status, and clinical risk factors, therapy use 

declined in older vs. younger patients, p = 

0.04. 

 Rate of adjuvant 

chemotherapy for CRC 

Newcomb and 

Carbone86 41% of patients < 65 yr compared to 26% ≥

65 yr received therapy, adjusting for disease 

stage, p = 0.05. 

 Adjuvant chemotherapy rate in 

patients with stage III colon or 

stage II, III rectal cancer 

Potosky et al.125 Controlling for demographics, 

socioeconomic status and clinical risk 

factors, higher rates were observed among 

younger compared to older patients, p = 

0.003. 

 
88 

Table 12. Quality measures used to assess differences in quality of care by patients’ age – continued

Question Quality measure Study Results – age 

 Adjuvant chemoradiotherapy 

rate 

Coburn et al.110 Controlling for demographic and clinical risk 

factors, adjuvant therapy rate was 5.1% for 

patients over 80 yr compared to 20% for 

patients under 80 yr, p = 0.00001. 

 Chemotherapy rate for stage II 

or III rectal cancer patients 

Neugut et al.133 Controlling for sociodemographic and 

clinical factors, younger patients had a 

higher probability of receiving adjuvant 

chemotherapy and radiation therapy, p < 

0.01. 

 Combined chemoradiotherapy 

rate for stage II, III rectal 

cancer patients 

Neugut et al.133 Controlling for sociodemographic and 

clinical factors, age was inversely 

associated with receipt of combined 

chemoradiation therapy. 

 Chemoradiotherapy rate stage 

II or III rectal cancer 

Schrag et al.132 Adjusting for demographic, socioeconomic 

status, comorbidity and other clinical risk 

factors, higher chemoradiation therapy use 

was observed in younger vs. older patients, 

p < 0.0001. 

 Rate of patients referred to a 

medical oncologist for adjuvant 

chemotherapy 

Oliveria et al.137 Controlling for sex, stage and year at 

diagnosis, patient age was a significant 

predictor of referral. Younger patients (< 70 

yr) were more likely to be referred to an 

oncologist, p < 0.0001. 

 Chemotherapy rate within 120 

days of diagnosis 

Rogers et al.90 Controlling for sex, nursing home status, 

socioeconomic status, tumor stage, 

comorbidity and other potential risk factors, 

rates of use declined with increasing age. 

Among those 65 to 74 yr, 75 to 84 yr, 85 and 

older, rates were 11.3%, 8.4%, and 4.5%, 

respectively. 

 Adjuvant chemotherapy rate of 

patients with stage III colon 

cancer 

Mahoney et al.126 Older patients were less likely than younger 

patients to receive adjuvant chemotherapy 

after adjusting for confounding variables, p = 

0.003. 

2d Radiotherapy rate in stage II or 

III rectal cancer 

Neugut et al.133 Controlling for sociodemographic and 

clinical factors, the use of radiation therapy 

declined as age increased, p < 0.01. 

 rate of radiation therapy for 

rectal cancer 

Wudel et al.100 Older patients may be less likely to receive 

radiation therapy (unadjusted) compared to 

younger patients. 

 Rate of adjuvant radiotherapy 

for CRC patients 

Newcomb and 

Carbone86 No statistically significant difference found in 

patients under 65 (14%) and over 65 yr 

(18%) in therapy, adjusting for disease 

stage. 

 Adjuvant radiation therapy rate 

for patients with stage II or III 

rectal cancer 

Ayanian et al.127 Adjusting for clinical, socioeconomic status 

and demographic factors, older patients 

were less likely to receive adjuvant therapy 

compared to younger patients, p = 0.001. 
89 

Table 12. Quality measures used to assess differences in quality of care by patients’ age – continued 

Question Quality measure Study Results – age 

 Percentage of patients 

diagnosed with colon or rectal 

cancer who received radiation 

therapy 

Roetzheim et al.94 Younger age was a predictor of receipt of 

radiation therapy, after controlling for 

demographic socioeconomic status factors, 

clinical risk factors. p < 0.001. 

3 Percentage of patients with 

non-metastatic CRC who 

received follow up 

colonoscopy 

Cooper et al.158,159 Older patients were less likely than younger 

patients to receive followup testing after 

controlling for age, comorbidity, location of 

tumor and other risk factors, p = 0.001. 

 Rate of colonoscopy or flexible 

sigmoidoscopy with barium 

enema after diagnosis 

Rulyak et al.157 Controlling for demographic, clinical, 

socioeconomic status and site of service 

covariates, patients > 80 yr of age were less 

likely to undergo surveillance, p < 0.05. 

 Percentage of CRC patients 

receiving a posttreatment 

colonoscopy 

LaFata et al.162 Adjusting for demographic, socioeconomic 

status, and clinical risk factors, age was 

inversely related to surveillance, p = 0.01. 

Table 13. Quality measures used to assess differences in quality of care by patients’ sex 

Question Quality measure Study Results – sex 

2b Perioperative mortality rate Whittle et al.113 After adjusting for age, males had a 31% 

higher perioperative mortality rate compared 

to female patients, p < 0.05. 

2c Adjuvant chemotherapy rate of 

patients with stage III colon 

cancer 

Mahoney et al.126 After adjusting for age and other potential 

confounding variables, males had a 5.8 

times greater chance of receiving 

chemotherapy than female patients, p = 

0.002. 

 Percent of patients diagnosed 

with stage III colon cancer 

receiving adjuvant 

chemotherapy 

Schrag et al.128 Adjusting for demographic, socioeconomic 

status, comorbidity and other clinical risk 

factors, males were slightly more likely to 

receive treatment compared to female 

patients, p = 0.06. 

 Percent of patients diagnosed 

with stage III colon cancer 

receiving adjuvant 

chemotherapy 

Sundararajan et 

al.129 Controlling for demographic, socioeconomic 

status, and clinical risk factors, males were 

more likely to receive treatment, p = 0.002. 

 Percent of patients diagnosed 

with stage II, III rectal cancer 

receiving adjuvant therapy 

Potosky et al.125 Controlling for demographic, socioeconomic 

status, and clinical risk factors, females were 

more likely to receive treatment than males, 

p = 0.06. 
90 

Table 14. Quality measures used to assess differences in quality of care by patients’ race/ethnicity 

Question Quality measure Study Results – race/ethnicity 

2b Rate of non-receipt of standard 

surgical treatment of patients 

diagnosed with colon cancer 

Demissie et al.87 The disparity between black and white 

patients with stages I and IV colon cancer 

was small, but persisted after controlling for 

risk factors including age, location of cancer, 

histologic type, and tumor grade. No 

statistically significant difference was seen 

for patients with stages II and III. A higher 

adjusted odds ratio for non-receipt of 

surgical treatment was observed for stage I, 

II rectal cancer in black females. An 

adjusted odds ratio for non-receipt of 

surgical treatment was higher for black 

males compared to white males for stages I, 

III, and IV rectal cancer. For stage I rectal 

cancer, black males had a lower likelihood 

of non-receipt of radiation therapy. The 

black-white disparity in non-receipt was 

more prominent when actual treatment 

received vs. intent to treat was considered. 

Refusal rate was 32.8% among black males 

compared to 9.2% for white males. 

 Surgical resection rate Cooper et al.93 Surgical resection was performed less often 

in black compared to white patients (68% vs. 

78%), after controlling for age, sex, 

comorbidity, location of tumor, and other 

potential risk factors. 

 30-day mortality rate Cooper et al.93 Black patients had a higher mortality rate 

(6.1%) than white patients (4.6%), after 

controlling for age, sex, comorbidity, location 

of tumor, and other potential risk factors. 

Differences held at 1 and 2 years 

postsurgery. 

 Rate of sphincter-preserving 

operation 

Morris et al.105 Controlling for age, sex, year of disease, 

geographic location, stage, and anatomic 

location, lower procedure rate was observed 

in black (57%) compared to white (63%) 

population with an adjusted odds ratio of 

1.42. 

 Surgery rate Dominitz et al.88 Adjusting for demographics, comorbidity, 

and other clinical factors, no statistically 

significant difference was found across 

racial lines in receipt of surgery in the VA 

population. 

 Surgery rate for patients with 

stages II and III CRC 

Govindarajan et 

al.102 Controlling for sex, site, socioeconomic 

status, and therapeutic modalities, 

compared to blacks, white patients were 

more likely to undergo surgery (p = 0.067) at 

all stages. 

 
91 

Table 14. Quality measures used to assess differences in quality of care by patients’ race/ethnicity – 

continued 

Question Quality measure Study Results – race/ethnicity 

 Curative resection rate Wudel et al.100 No statistically significant difference in 

unadjusted rates for curative resection 

treatment between ethnic groups was 

observed. Black patients were significantly 

worse in overall and disease-specific 

survival in both institutions. 

 Surgery rate of CRC patients Rogers et al.90 Controlling for sex, nursing home status, 

socioeconomic status, tumor stage, 

comorbidity, and other potential risk factors, 

91% of whites compared to 86% of blacks 

received surgery, p = 0.02. Higher adjusted 

odds ratio. 

2c Adjuvant chemotherapy rate in 

patients with stage III colon or 

stages II or III rectal cancer 

Ayanian et al.127 Adjusting for demographic, clinical, and 

socioeconomic status risk factors, treatment 

was used less in black vs. white population, 

p = 0.001. 

 Chemotherapy rate within 120 

days of diagnosis 

Rogers et al.90 Controlling for sex, nursing home status, 

socioeconomic status, tumor stage, 

comorbidity, and other potential risk factors, 

7.8% of whites compared to 9.4% of blacks 

received chemotherapy within 120 days of 

diagnosis. 

 Adjuvant chemotherapy rate in 

patients with stage III colon 

cancer 

Schrag et al.128 Adjusting for demographic, socioeconomic 

status, comorbidity, and other clinical risk 

factors, blacks were less likely to receive 

therapy, p < 0.004. 

 Chemotherapy rate Dominitz et al.88 Adjusting for demographic, comorbidity, and 

other clinical factors, no statistically 

significant difference was found across 

racial lines in receipt of chemotherapy in the 

VA population. 

 Chemotherapy rate of patients 

with stages II and II CRC 

Govindarajan et 

al.102 Controlling for sex, site, socioeconomic 

status, and therapeutic modalities, higher 

rates of treatment were found among white 

patients compared to blacks at stages II and 

III, p = 0.007. 

 Adjuvant chemotherapy rate in 

patients with stage III colon or 

stage II or III rectal cancer 

Potosky et al.125 Controlling for demographic, socioeconomic 

status, and clinical risk factors, higher rates 

of treatment were observed among whites 

compared to blacks, p = 0.02. 

 Percentage of patients with 

stage III colon cancer receiving 

adjuvant chemotherapy 

Sundararajan et 

al.129 Controlling for demographic, socioeconomic 

status, and clinical risk factors, therapy use 

declined in non-Hispanic black vs. nonHispanic white patients, p = 0.0001. 

 Chemotherapy rate for patients 

with colon cancer 

Wudel et al.100 There was no statistically significant 

difference in unadjusted rates of treatment 

received along racial lines. 

 
92 

Table 14. Quality measures used to assess differences in quality of care by patients’ race/ethnicity – 

continued 

Question Quality measure Study Results – race/ethnicity 

 Chemotherapy rate for stage II 

or III rectal cancer patients 

Neugut et al.133 Controlling for sociodemographic and 

clinical factors, non-black patients had a 

higher probability of receiving adjuvant 

chemotherapy and radiation therapy. 

 Chemoradiotherapy rate stage 

II or III rectal cancer 

Schrag et al.132 Adjusting for demographic, socioeconomic 

status, comorbidity, and other clinical risk 

factors, higher chemoradiation therapy use 

in white vs. black patients, p = 0.003. 

2d Non-receipt of standard 

radiation treatment 

Demissie et al.87 Stages I and III rectal cancer black patients, 

when compared to white patients, 

experienced a significantly greater rate of 

non-receipt of standard radiation treatment 

after controlling for age, location of cancer, 

histological type, and tumor grade. There 

was no statistically significant difference for 

patients with stages II and IV rectal cancer. 

 Radiation therapy rate Dominitz et al.88 Adjusting for demographic, comorbidity, and 

other clinical factors, no statistically 

significant difference across racial lines was 

found in the receipt of radiation therapy in 

the VA population. 

 Radiation therapy rate for 

patients with stages II and III 

CRC 

Govindarajan102 Controlling for sex, site, socioeconomic 

status, and therapeutic modalities, higher 

percent of whites (43%) compared to blacks 

(26%) received therapy, p = 0.008. 

 Rate of radiation therapy use 

for stage II, III rectal cancer 

patients 

Morris et al.105 Controlling for age, sex, year of disease, 

geographic location, stage, and anatomic 

location, an adjusted odds ratio for nonreceipt of treatment was higher for black 

(1.47) vs. white (1.15) patients. 

 Adjuvant radiotherapy rate for 

rectal cancer patients 

Rogers et al.90 Controlling for sex, nursing home status, 

socioeconomic status, tumor stage, 

comorbidity, and other potential risk factors, 

4% of white patients received therapy 

compared to 13% of black patients, p = 

0.001, adjusting for other factors. 

 Rate of radiation therapy Roetzheim et al.94 After adjusting for demographic, 

socioeconomic status, and clinical risk 

factors, no statistically significant difference 

was found in an adjusted odds ratio for 

receipt of treatment across racial lines. 

 Rate of radiation therapy for 

rectal cancer 

Wudel et al.100 No statistically significant difference was 

found in unadjusted rate of receipt of 

treatment across racial lines. 

 Adjuvant radiation therapy rate 

for patients with stage II or III 

rectal cancer 

Ayanian et al.127 Adjusting for demographic, socioeconomic 

status, and clinical factors, black patients 

were less likely to receive adjuvant therapy 

compared to white patients. 

 
93 

Table 14. Quality measures used to assess differences in quality of care by patients’ race/ethnicity – 

continued 

Question Quality measure Study Results – race/ethnicity 

3 Rate of colonoscopy or flexible 

sigmoidoscopy with barium 

enema after diagnosis 

Rulyak et al.157 Controlling for demographic, clinical, 

socioeconomic status, and site of service 

covariates, lower use was observed among 

black compared to white patients. 

 Percentage of patients who 

received posttreatment 

colonoscopy 

LaFata et al.162 Adjusting for demographic, clinical, and 

socioeconomic status characteristics, whites 

were slightly more likely than non-white 

patients to receive followup colonoscopy (p 

= 0.09). 

Table 15. Quality measures used to assess differences in quality of care by patients’ socioeconomic status 

Question Quality measure Study Results – socioeconomic status 

(income/insurance status) 

2b Surgery rate for colon or rectal 

cancer patients 

Roetzheim et al.94 Controlling for demographic, socioeconomic 

status factors, and clinical risk factors, the 

uninsured and Medicaid patients were less 

likely to undergo surgical treatment 

compared to FFS patients, p < 0.01. 

 Surgery rate Vernon et al.92 No statistically significant difference in 

treatment use of HMO vs. FFS patients 

when controlling for demographic, clinical, 

and socioeconomic status factors. 

 Surgical resection rate of colon Merrill et al.91 Controlling for demographic, socioeconomic 

status, and clinical factors, no difference 

observed in rectal cancer treatment of HMO 

vs. FFS patients undergoing surgery. HMO 

patients with rectal cancer were more likely 

to receive postsurgical radiation treatment 

than FFS patients. 

2c Rate of adjuvant 

chemotherapy use for stage III 

colon or stage II, III rectal 

cancer 

Ayanian et al.127 Controlling for demographic, other 

socioeconomic status, and clinical risk 

factors, median household income was a 

predictor of rate of treatment; the higher the 

income, the more likely receipt of treatment, 

p = 0.006. 

Percentage of patients with 

stage II or III rectal cancer 

receiving chemoradiotherapy 

Schrag et al.132 A slight, positive relationship (p = 0.09) was 

found between median income and odds of 

receiving adjuvant chemoradiation therapy, 

adjusting for demographic, socioeconomic 

status, comorbidity, and other clinical risk 

factors. 

 Percentage of patients 

diagnosed with stage III colon 

cancer who received adjuvant 

chemotherapy 

Schrag et al.128 Adjusting for demographic, comorbidity, and 

other clinical risk factors, a positive 

relationship was observed between median 

household income and receipt of treatment, 

p = 0.06. 

 
94 

Table 15. Quality measures used to assess differences in quality of care by patients’ socioeconomic status 

– continued 

Question Quality measure Study Results – socioeconomic status 

(income/insurance status) 

 Percent of patients diagnosed 

with stage II, III rectal cancer 

receiving adjuvant therapy 

Potosky et al.125 Controlling for demographic, socioeconomic 

status, and clinical risk factors, income was 

found to be inversely related to standard 

adjuvant therapy (chemotherapy for stage III 

colon and chemotherapy and radiation 

therapy for stage II, III rectal cancer) use, p 

< 0.001. 

 Adjuvant chemotherapy rate 

for patients with stage III colon 

cancer 

Keating et al.130 After controlling for age, race, tumor stage 

and grade, comorbidity, socioeconomic 

status, and other risk factors, researchers 

found no statistically significant difference in 

increased market share penetration 

associated with lower rates of therapy use. 

 Adjuvant chemotherapy rate 

for patients with stage II or III 

rectal cancer 

Keating et al.130 After controlling for age, race, tumor stage 

and grade, comorbidity, socioeconomic 

status, and other risk factors, researchers 

found increased market share of managed 

care resulted in significant decrease in 

percent of patients with stage II or III rectal 

cancer received adjuvant chemotherapy and 

radiotherapy. 

 Chemotherapy rate for patients 

with colon or rectal cancer 

Roetzheim et al.94 Controlling for demographic, socioeconomic 

status factors, and clinical risk factors, 

patients who are members of a community 

HMO were less likely to receive treatment 

compared to those participating in a FFS 

program. 

 Palliative chemotherapy rate Retchin and 

Brown136 No statistically significant difference 

between HMO and FFS patients undergoing 

palliative chemotherapy. 

 Rate of adjuvant radiation 

therapy use for stage II, III 

rectal cancer 

Ayanian et al.127 Controlling for demographic, other 

socioeconomic status, and clinical risk 

factors, median household income was 

positively related to receipt of treatment, p = 

0.006. 

2d Rate of radiation therapy Roetzheim et al.94 Controlling for demographic and clinical risk 

factors, lower income level was a predictor 

of receipt of radiation therapy. Medicare 

patients enrolled in an HMO with colon 

cancer were less likely to receive radiation 

therapy. Among non-Medicare HMO 

enrollees, no insurance-related differences 

were observed. 
95 

Table 15. Quality measures used to assess differences in quality of care by patients’ socioeconomic status 

– continued

Question Quality measure Study Results – socioeconomic status 

(income/insurance status) 

3 Rate of complete colonoscopy 

pre- or perioperatively 

Keating et al.130 After controlling for age, race, tumor stage 

and grade, comorbidity, socioeconomic 

status, and other risk factors, researchers 

found that increased market share of 

managed care in a given county resulted in 

significant increase in the percent of patients 

with complete colonoscopy pre- or 

perioperatively, p = 0.001. 

 Rate of colonoscopy during 7 

to 18 mo after diagnosis 

Keating et al.130 After controlling for age, race, tumor stage 

and grade, comorbidity, socioeconomic 

status, and other risk factors, researchers 

found no statistically significant difference in 

colonoscopy rates and market share 

increase of managed care. 

 Rate of colonoscopy or flexible 

sigmoidoscopy with barium 

enema after diagnosis 

Rulyak et al.157 Controlling for demographic, clinical, 

socioeconomic status, and site of service 

covariates, high median household annual 

income (≥ $40,000) was positively 

associated with receipt of first complete 

exam of the colon. 

 Percentage of patients who 

received posttreatment 

colonoscopy 

LaFata et al.162 Adjusting for demographic, socioeconomic 

status, and clinical risk factors, income was 

inversely related to receipt of followup 

colonoscopy (p = 0.03). 

 Percentage of CRC patients 

having a surveillance 

colonoscopy after surgery 

Retchin and 

Brown136 No statistically significant difference found 

between receipt of surveillance colonoscopy 

and health plan coverage (HMO vs. FFS). 

Abbreviations: FFS = fee-for-service; HMO = health maintenance organization 

Conclusions 

The question of whether the evidence supports the use of any of the available quality 

measures ultimately devolves into asking whether the databases in question exhibit differential 

bias according to the patient subgroups being studied. For example, suppose that patients with 

stage III colon cancer are followed, and the process measure of interest is the percentage of 

patients receiving adjuvant chemotherapy. As discussed elsewhere, most but not all of such 

patients are good candidates for adjuvant chemotherapy, and not all patients that are offered such 

therapy will accept. If the data source is prospectively collected and designed to be sensitive to 

such issues (e.g., a form in the patient’s medical record that documents whether or not the patient 

was a good candidate for chemotherapy and, if so, whether the therapy was accepted), then 

sufficient information is present to make an accurate comparison of, for example, rates of 

adjuvant chemotherapy use by age. On the other hand, if such information is absent, then the 

possibility exists that older patients may have more contraindications and/or a greater propensity 

to refuse therapy, thus biasing the conclusions. The answers to the above question about 
96 

differential bias depend entirely on the quality measure, the groups to be compared, and the 

available database, but clearly this is the question that should be of fundamental importance to 

the analyst.

 
97

Chapter 4. Future Research 

In brief, we have identified the following gaps in our knowledge, or equivalently, areas of 

emphasis for future research: 

• Developing the two general process measures pertaining to the treatment of polyps 

(particularly, those polyps with evidence of malignancy) identified in this report. 

• Expanding technical process measures for those topics with too few or no such measures; 

these include operative reports, chemotherapy reports, the treatment of polyps, and 

surgery. 

• For those areas in which technical measures are well developed, identifying and focusing 

on those measures having the greatest impact on patient outcomes. 

• Improving the evidence base on which all quality measures for colorectal cancer (CRC) 

are based. 

• Clarifying and otherwise fine-tuning both the numerator and denominator of various 

measures (while at the same time addressing the trade-off between increased information 

requirements and data sources). 

• Providing more explicit directions regarding risk-adjustment. 

• Capturing data regarding patient preferences. 

• Addressing quality of life (which is of immense importance to patients) within the 

context of quality improvement for CRC. 

More detailed discussion regarding these gaps and future research required to fill them is 

provided under Questions 1 to 4, above. 

The progress to date, as reflected by the existing quality measures, lends confidence that such 

future research will be fruitful. 

99

Chapter 5. Conclusions 

This section summarizes the overall conclusions about the quality measures identified in our 

literature review. More detailed conclusions about individual measures are included in the text 

of Chapter 3, under Questions 1-4. 

Our approach to the classification of measures used as its organizing principle the distinction 

between structure, process, and outcome. Structural measures tended to be linked with the 

outcomes of complication rates and survival. An example of a typical application is the 

comparison of hospitals with different volumes of surgery according to the outcome of major 

complications associated with colorectal cancer (CRC)-related surgery. In these applications the 

structural variables tended to be generic (and not specific to CRC as such); for example, hospital 

volume and hospital type (e.g., urban vs. rural, academic vs. other). One application that used a 

structural measure that was more specific to CRC was the proportion of colonoscopies 

performed by physicians with formal endoscopy training. 

 Most applications were quite typical of the literature on structural variables and 

unremarkable for the purpose of identifying quality measures specific to CRC. The usual 

caveats to interpretation apply; most particularly, that crucial elements of the pathway between 

structure and outcome might be missing or incompletely specified. These elements include 

patient characteristics (e.g., comorbidities, preference for type of treatment, and sometimes even 

more critical information about stage of disease) and process of care. 

 Whether part of applications that linked structure to outcome, process to outcome, or focused 

on outcome alone, outcome variables were of three types: (a) complication rates; (b) survival; 

and (c) quality of life. Complication rates (e.g., complications of CRC surgery, complications of 

diagnostic colonoscopy) were straightforwardly defined in the same fashion as the process 

measures, i.e., as a ratio using a denominator counting the number of eligible patients and a 

numerator counting the number of such patients with the event of interest; here, a complication 

of treatment. The points of emphasis were the precise specification of the population at risk in 

the denominator and the precise specification of the events of interest in the numerator. For 

example, some outcome measures would most reasonably focus on any complications, whereas 

others would limit their focus to major complications only. 

 Survival-related outcome measures were of two types, proportion surviving at a specific 

point in time (e.g., 30-day mortality rates), and time until event. The former measures can be 

treated as traditional ratio measures, the main challenge being to select the time period that is 

most appropriate to the question at hand. For example, mortality rates associated with CRC 

surgery might reasonably be defined as either in-hospital mortality or 30-day mortality, 30-day 

mortality being a bit more pertinent but also inducing somewhat more stringent data 

requirements, as a mechanism for following patients that are discharged before 30 days postsurgery would be required. Time-to-event measures can be, and were, analyzed using traditional 

techniques of survival analysis such as Kaplan-Meier curves, and Cox models with hazard ratio 

estimators. Neither version of a survival-related outcome measure involves methodological 

issues that are unique to CRC. 

 The final type of outcome measure pertains to quality of life. While quality of life is of 

immense importance to patients with CRC, it was seldom used in the context of quality 

improvement, perhaps because of the large number of steps between the leverage points on 

which an intervention can be performed and the outcome of quality of life (and also because of 
100

the large number of other things, besides quality of care, that can affect CRC-related quality of 

life). Quality-of-life measures are also unique in that they are usually most naturally presented 

as continuous variables rather than on a ratio scale. A brief discussion of quality-of-life 

measures is provided in an Appendix A*

. 

 The primary focus of this report was on measures of CRC-related processes of care. Here, 

the main distinction is between those measures that are general versus technical. 

 The formula for creating a general process measure is to take the sequence of steps 

describing the optimal pattern of care for patients with CRC (i.e., as graphically summarized in 

Figures 1 and 2 in Chapter 1, or as described in more detail throughout the document), identify 

each of these steps as both a leverage point and a potential quality measure, and then define such 

a measure. For example, one of the earliest steps in the process involves the proper diagnosis of 

patients with a suspicion of CRC. A well-developed and well-tested measure associated with 

this step is percentage of patients who underwent appropriate evaluation for a positive fecal 

occult blood test (FOBT). As with all ratio-based measures, it is critical to carefully define both 

the numerator and denominator of the measure. Here, the numerator can be based on a guideline 

and involves colonoscopy with complete visualization of the colon, although the literature did 

discuss various nuances, such as the precise definition of what complete visualization entails, 

how soon the colonoscopy should be performed after the positive FOBT, and so forth. The 

numerators of some general process measures will have a stronger evidence base than others, 

with those that are based on guidelines (especially those guidelines that are consistent across 

professional societies, when multiple such guidelines have been proffered) having the strongest 

evidence base, and those that are derived from basic first principles supported by some research 

findings being relatively weaker, but often quite sufficient for the task at hand. 

 Regarding the denominator, “patients with positive FOBT” comprise only a subset of those 

patients of ultimate interest, namely, “all patients with sufficiently high clinical suspicion of 

CRC to require followup testing.” These patients would include those with positive tests 

according to other modalities (e.g., barium enema, flexible sigmoidoscopy), and also those who 

come to the provider’s attention because of symptoms such as bleeding. Often, a clinically 

precise statement of the denominator involved deleting various patients; for example, those that 

would be ineligible because of comorbid conditions or other clinical issues, patients that refuse a 

form of therapy such as chemotherapy, and so forth. 

 A consistent source of tension is the distinction between the clinically derived fine-tuning of 

the definition of the denominator of a quality measure and the limitations of available data 

sources (which often do not contain sufficient information to act on such distinctions). Our 

approach here has been to encourage the definition of denominators that are as precise as 

possible, even if the available data sources might not meet all their requirements. In practice, it 

is crucially important to assess both the strengths and weaknesses of available data sources. 

Formal validation efforts are encouraged, but are not always strictly necessary. The assessment 

of data sources should include the differential impact of data quality on patient subgroups. For 

example, older patients tend to have more comorbidity and might thus be less likely to tolerate 

adjuvant chemoradiation therapy; an analysis that does not take this tendency into account could 

incorrectly lead to the conclusion that an age-related bias exists in the provision of such therapy. 

A similar caveat applies to analyses comparing patients by race and ethnicity, hospitals 

according to type, and so forth. 

 

* Appendixes cited in this report are provided electronically at 

http://www.ahrq.gov/downloads/pub/evidence/pdf/colocanqm/colocanqm.pdf
101

 Overall, the coverage of general process measures (i.e., across the various leverage points in 

the sequence of care) is extensive. At least some process measures are available for the steps 

involving diagnostic imaging, staging, surgical therapy, adjuvant chemotherapy, adjuvant 

radiation therapy, and colonoscopic surveillance. Additional general process measures might be 

developed for polyp assessment and removal, although it must be recognized that the proper 

clinical course of action when presented with a malignant polyp is not always clear. While the 

above formula could be followed to generate even more general process measures within the 

above categories, perhaps the task of greater importance is to continue to refine those measures 

that already exist. The continued expansion of the evidence base, and the continued updating of 

guidelines, should assist this process. Moreover, the basic structure of general process measures 

facilitates such updating. For example, if future research suggests that the optimal interval for 

colonoscopic surveillance should be changed, then the structure of the existing quality measures 

related to surveillance need not be changed; all that would require change is the replacement of 

one recommended interval with another. 

 General process measures were used in both formal quality improvement applications as well 

as more general research applications, the distinction being that these latter applications did not 

tend to specify information such as the users and uses of the measure (i.e., the quality 

improvement context), as well as various other details about databases and other procedures that 

reflect the specificity required for quality improvement. Chapter 3 did not always explicitly 

maintain the distinction between these types of application, the main and quite encouraging 

reason being that it appeared that many of those measures that were used outside the quality 

improvement context could be rather straightforwardly modified to fit into this context. Overall, 

the general state of the science regarding general process measures was encouraging. 

 In contrast to general process measures, as might be anticipated, technical measures tended to 

be used in a quality improvement context. Although some excellent technical measures were 

found, the overall development of technical measures seems somewhat less advanced than that of 

the general process measures. In particular, for some elements of the care pathway, such as 

operative reports and chemotherapy reports, no technical measures were found. Various 

technical measures for quality of colonoscopy were found (e.g., cecal intubation rate). These 

were guideline-based, well developed, but less well tested, and the linkage between the technical 

measures and patient outcomes, although intuitive, was not always explicitly provided. To the 

extent that complication rates of colonoscopy can be considered a technical measure, this area of 

inquiry is relatively well developed. Adequate lymph node retrieval and evaluation is an 

excellent technical measure related to disease staging, although residual areas requiring 

clarification remain (e.g., optimal number of lymph nodes to examine, consideration that the 

optimal number of lymph node depends to some extent on patient characteristics). Technical 

measures associated with surgery are underdeveloped. Chemoradiation therapy has a number of 

potential and well-justified technical measures; here, the challenge is to identify and focus on 

those measures having the most impact on patient outcomes. Regarding reporting, technical 

measures for pathology reporting are well developed, reflecting among other things the areas of 

emphasis among the relevant professional societies. As noted above, no such technical process 

measures were identified for operative reports or chemotherapy reports, although it can quite 

reasonably presumed that with sufficient attention from professional societies and other 

stakeholders such measures could be developed. 

 
102

Perhaps the most important areas for future development of technical measures are (a) 

developing such measures in those areas for which they are lacking; and (b) in areas where such 

measures exist, identifying and focusing on those measures having the greatest impact on 

outcome. As with the general measures, clarifying and otherwise fine-tuning the technical 

measures is a process that should be ongoing. 
103

References Cited in the Evidence Report 

1. Hewitt M, Simone V. Ensuring quality cancer care 

[electronic resource]. National Cancer Policy 

Board, Institute of Medicine and Commission on 

Life Sciences, National Research Council. [An 

electronic book accessible through the World Wide 

Web at http://www.netlibrary.com.] Washington, 

DC: National Academy Press; 1999. 

2. Foley KM, Gelband H. Improving palliative care 

for cancer. National Cancer Policy Board. 

Washington, DC: National Academy Press; 2001. 

3. Hewitt M, Simone V. Enhancing data systems to 

improve the quality of cancer care. Institute of 

Medicine. Washington, DC: National Academy 

Press; 2000. 

4. Jemal A, Murray T, Ward E, et al. Cancer statistics, 

2005. CA Cancer J Clin 2005;55(1):10-30. 

5. Anonymous. Colorectal cancer test use among 

persons aged > or = 50 years--United States, 2001. 

MMWR Morb Mortal Wkly Rep. 2003;52(10):193-

6. 

6. Winawer S, Fletcher R, Rex D, et al. Colorectal 

cancer screening and surveillance: clinical 

guidelines and rationale-Update based on new 

evidence. Gastroenterology 2003;124(2):544-60. 

7. Rockey D, Paulson E, Niedzwiecki D, et al. 

Analysis of air contrast barium enema, computed 

tomographic colonography and colonscopy: 

prospective comparison. Lancet 2005;365:305-11. 

8. Mulhall B, Veerappan G, Jackson J. Meta-analysis: 

computed tomographic colonography. Ann Intern 

Med 2005;142(8):635-50. 

9. Pickhardt P, Nugent P, Mysliwiec P, et al. Location 

of adenomas missed by optical colonoscopy. Ann 

Intern Med 2004;141(5):352-9. 

10. Gluecker TM, Johnson CD, Harmsen WS, et al. 

Colorectal cancer screening with CT colonography, 

colonoscopy, and double-contrast barium enema 

examination: prospective assessment of patient 

perceptions and preferences. Radiology 

2003;227(2):378-84. 

11. Hobday TJ, Erlichman C. Colorectal cancer. In: 

Gospodarowick MK, Henson DE, Hutter RV, et al., 

editors. Prognostic factors in cancer. New York: 

Wiley-Liss; 2001. p. 333-47. 

12. American Joint Committee on Cancer. Colon and 

rectum. In: AJCC cancer staging manual. 6th 

edition. New York: Springer; 2002. p. 113-24. 

13. Institute of Medicine. Asessing the Quality of 

Cancer Care: An Approach to Measurement in 

Georgia. Committee on Assessing Improvements in 

Cancer Care in Georgia. National Cancer Policy 

Board. The National Academies Press: Washington, 

DC; 2005. 

14. Greene FL, Page DL, Fleming ID, et al. (eds). 

AJCC Cancer Staging Manual. 6th edition. New 

York, NY: Springer; 2002. 

15. National Institutes of Health (NIH). NIH consensus 

conference. Adjuvant therapy for patients with 

colon and rectal cancer. JAMA 1990;264(11):1444-

50. 

16. Duke CE, Bussey HJ. The spread of rectal cancer 

and its effect on prognosis. Br J Surg 1958;12:309-

20. 

17. Chapuis PH, Dent OF, Fisher R, et al. A 

multivariate analysis of clinical and pathological 

variables in prognosis after resection of large bowel 

cancer. Br J Surg 1985;72(9):698-702. 

18. Newland RC, Chapuis PH, Pheils MT, et al. The 

relationship of survival to staging and grading of 

colorectal carcinoma: a prospective study of 503 

cases. Cancer 1981;47(6):1424-9. 

19. Compton CC, Fielding LP, Burgart LJ, et al. 

Prognostic factors in colorectal cancer. College of 

American Pathologists Consensus Statement 1999. 

Arch Pathol Lab Med 2000;124(7):979-94. 

20. Swanson RS, Compton CC, Stewart AK, et al. The 

prognosis of T3N0 colon cancer is dependent on the 

number of lymph nodes examined. Ann Surg Oncol 

2003;10(1):65-71. 

21. Tepper JE, O'Connell MJ, Niedzwiecki D, et al. 

Impact of number of nodes retrieved on outcome in 

patients with rectal cancer. J Clin Oncol 

2001;19(1):157-63. 

22. Heald RJ, Moran BJ, Ryall RD, et al. Rectal cancer: 

the Basingstoke experience of total mesorectal 

excision, 1978-1997. Arch Surg 1998;133:894-9. 
104

23. Benson AB 3rd, Schrag D, Somerfield MR, et al. 

American Society of Clinical Oncology 

recommendations on adjuvant chemotherapy for 

stage II colon cancer. J Clin Oncol 

2004;22(16):3408-19. 

24. National Comprehensive Cancer Network (NCCN). 

Clinical practice guidelines in oncology: colon 

cancer. Version 3, 2005. Available at: 

http://www.nccn.org/professionals/physician_gls/P

DF/colon.pdf. Accessed July 7, 2005. 

25. National Comprehensive Cancer Network (NCCN). 

Clinical practice guidelines in oncology: rectal 

cancer. Version 3, 2005. Available at: 

http://www.nccn.org/professionals/physician_gls/P

DF/rectal.pdf. Accessed July 7, 2005. 

26. Sun W, Haller DG. Adjuvant therapy of colon 

cancer. Semin Oncol 2005;32(1):95-102. 

27. Gunderson LL, Sosin H. Areas of failure found at 

reoperation (second or symptomatic look) following 

"curative surgery" for adenocarcinoma of the 

rectum. Clinicopathologic correlation and 

implications for adjuvant therapy. Cancer 

1974;34(4):1278-92. 

28. Dahlberg M, Glimelius B, Pahlman L. Improved 

survival and reduction in local failure rates after 

preoperative radiotherapy: evidence for the 

generalizability of the results of Swedish Rectal 

Cancer Trial. Ann Surg 1999;229(4):493-7. 

29. Martling A, Holm T, Johansson H, et al. The 

Stockholm II trial on preoperative radiotherapy in 

rectal carcinoma: long-term follow-up of a 

population-based study. Cancer 2001;92(4):896-

902. 

30. Medical Research Council Rectal Cancer Working 

Party. Randomised trial of surgery alone versus 

radiotherapy followed by surgery for potentially 

operable locally advanced rectal cancer. Lancet 

1996;348(9042):1605-10. 

31. Medical Research Council Rectal Cancer Working 

Party. Randomised trial of surgery alone versus 

surgery followed by radiotherapy for mobile cancer 

of the rectum. Lancet 1996;348(9042):1610-4. 

32. National Cancer Institute. Colorectal Cancer 

(PDQ®): Treatment. Available at 

http://www.cancer.gov/cancertopics/pdq/treatment/c

olon/HealthProfessional. Accessed July 1, 2005. 

33. Jeffery GM, Hickey BE, Hider P. Follow-up 

strategies for patients treated for non-metastatic 

colorectal cancer. The Cochrane Database of 

Systematic Reviews 2002, Issue 1. Art. No.: 

CD002200. DOI: 10.1002/14651858.CD002200. 

34. Renehan A, Egger M, Saunders M, et al. Impact on 

survival of intensive follow-up after curative 

resection for colorectal cancer: a systematic review 

and meta-analysis of randomised trials. BMJ 

2002;324:813. 

35. Pfister DG, Benson AB 3rd, Somerfield MR. 

Clinical practice. Surveillance strategies after 

curative treatment of colorectal cancer. N Engl J 

Med 2004;350(23):2375-82. 

36. Earle CC, Park ER, Lai B, et al. Identifying 

potential indicators of the quality of end-of-life 

cancer care from administrative data. J Clin Oncol 

2003;21(6):1133-8. 

37. Brook RH, McGlynn EA, Cleary PD. Quality of 

health care. Part 2: measuring quality of care. N 

Engl J Med 1996;335(13):966-70. 

38. Davila RE, Rajan E, Adler D, et al. ASGE 

guideline: the role of endoscopy in the diagnosis, 

staging, and management of colorectal cancer. 

Gastrointest Endosc 2005;61(1):1-7. 

39. Rex DK, Cutler CS, Lemmel GT, et al. 

Colonoscopic miss rates of adenomas determined 

by back-to-back colonoscopies. Gastroenterology 

1997;112(1):24-8. 

40. Rex D, Bond J, Winawer S, et al. Quality in the 

technical performance of colonoscopy and the 

continuous quality improvement process for 

colonoscopy: recommendations of the U.S. MultiSociety Task Force on Colorectal Cancer. Am J 

Gastroenterol 2002;97(6):1296-308. 

41. Shields HM, Weiner MS, Henry DR, et al. Factors 

that influence the decision to do an adequate 

evaluation of a patient with a positive stool for 

occult blood. Am J Gastroenterol 2001;96(1):196-

203. 

42. Levin B, Hess K, Johnson C. Screening for 

colorectal cancer. A comparison of 3 fecal occult 

blood tests. Arch Intern Med. 1997;157(9):970-6. 

43. Baig N, Myers RE, Turner BJ, et al. Physicianreported reasons for limited follow-up of patients 

with a positive fecal occult blood test screening 

result. Am J Gastroenterol 2003;98(9):2078-81. 

44. Myers RE, Turner B, Weinberg D, et al. Impact of a 

physician-oriented intervention on follow-up in 

colorectal cancer screening. Prev Med 

2004;38(4):375-81. 
105

45. Mandel JS, Church TR, Bond JH, et al. The effect 

of fecal occult-blood screening on the incidence of 

colorectal cancer. N Engl J Med 

2000;343(22):1603-7. 

46. Marble K, Banerjee S, Greenwald L. Colorectal 

carcinoma in young patients. J Surg Oncol 

1992;51(3):179-82. 

47. Minnesota Gastroenterology PA. 2004 Colonoscopy 

Quality Assurance Initiative. Electronically 

received: January 26, 2005. 

48. Beart RW, Steele GD Jr, Menck HR, et al. 

Management and survival of patients with 

adenocarcinoma of the colon and rectum: a national 

survey of the Commission on Cancer. J Am Coll 

Surg 1995;181(3):225-36. 

49. Shehadeh I, Rebala S, Kumar R, et al. Retrospective 

analysis of missed advanced adenomas on 

surveillance colonoscopy. Am J Gastroenterol 

2002;97(5):1143-7. 

50. Haseman JH, Lemmel GT, Rahmani EY, et al. 

Failure of colonoscopy to detect colorectal cancer: 

evaluation of 47 cases in 20 hospitals. Gastrointest 

Endosc 1997;45(6):451-5. 

51. Ure T, Dehghan K, Vernava AM 3rd, et al. 

Colonoscopy in the elderly. Low risk, high yield. 

Surg Endosc 1995;9(5):505-8. 

52. Nelson DB, McQuaid KR, Bond JH, et al. 

Procedural success and complications of large-scale 

screening colonoscopy. Gastrointest Endosc 

2002;55(3):307-14. 

53. National Quality Measures Clearinghouse (NQMC). 

Intra-procedure colonosopy complication rate: 

percentage of patients who developed one or more 

intrao-procedure complications. Available at: 

http://www.qualitymeasures.ahrq.gov/summary/su

mmary.aspx?doc_id=4868. Accessed: March 21, 

2005. 

54. Colon Cancer Workgroup. Quality of Cancer Care 

Evaluation and Improvement: Commission on 

Cancer, American College of Surgeons, American 

Cancer Society and Tenet Health Care System, 

December, 2003. Electronically received Febraru 7, 

2005. 

55. Harewood GC, Sharma VK, de Garmo P. Impact of 

colonoscopy preparation quality on detection of 

suspected colonic neoplasia. Gastrointest Endosc 

2003;58(1):76-9. 

56. Levin B, Murphy GP. Revision in American Cancer 

Society recommendations for the early detection of 

colorectal cancer. CA Cancer J Clin 

1992;42(5):296-9. 

57. Winawer SJ, Fletcher RH, Miller L, et al. Colorectal 

cancer screening: clinical guidelines and 

rationale.[see comment][erratum appears in 

Gastroenterology 1997 Mar;112(3):1060]. 

Gastroenterology 1997;112(2):594-642. 

58. Eddy D. ACS report on the cancer-related health 

checkup. CA Cancer J Clin 1980;30(4):193-240. 

59. Accreditation Association for Ambulatory Health 

Care (AAAHC) Institute for Quality Improvement. 

Procedure specific colonoscopy survey 

[Colonoscopy CPT-45378-45385]. Wilmette (IL): 

AAAHC Institute for Quality Improvement; 2003. 

60. Wexner SD, Litwin D, Cohen J, et al. Principles of 

privileging and credentialing for endoscopy and 

colonoscopy. Gastrointest Endosc 2002;55(2):145-

8. 

61. National Committee for Quality Assurance 

(NCQA). Final List of Colon Cancer Measures 

Based on Domain Expert Review. Measures 

developed by Kaiser Permanente, Internountain 

Health Care and NCQA. Electronically delivered 

February 8, 2005. 

62. O'Connell JB, Maggard MA, Ko CY. Colon cancer 

survival rates with the new American Joint 

Committee on Cancer sixth edition staging. J Natl 

Cancer Inst 2004;96(19):1420-5. 

63. Bast RC. Cancer medicine. 5th ed. Hamilton, 

Ontario: B. C. Decker, Inc.; 2000. Section 29, 

Neoplasms of the alimentary canal. 

64. Erlichman C, Fine S, Wong A, et al. A randomized 

trial of fluorouracil and folinic acid in patients with 

metastatic colorectal carcinoma. J Clin Oncol 

1988;6(3):469-75. 

65. Prandi M, Lionetto R, Bini A, et al. Prognostic 

evaluation of stage B colon cancer patients is 

improved by an adequate lymphadenectomy: results 

of a secondary analysis of a large scale adjuvant 

trial. Ann Surg 2002;235(4):458-63. 

66. Wong JH, Severino R, Honnebier MB, et al. 

Number of nodes examined and staging accuracy in 

colorectal carcinoma. J Clin Oncol 

1999;17(9):2896-900. 

67. Compton CC. Pathologic staging of colorectal 

cancer: an advanced users' guide. Pathology Case 

Reviews 2004;9(4):150-62. 
106

68. Baxter N, Virnig D, Rothenberger D, et al. Lymph 

Node Evaluation in Colorectal Cancer Patients: A 

Population Based Study. Jl Natl Cancer Inst 

2005;97(3):219-25. 

69. Galvis CO, Raab SS, D'Amico F, et al. Pathologists' 

assistants practice: a measurement of performance. 

Am J Clin Pathol 2001;116(6):816-22. 

70. Chiaverini L, Fulton JP, Darcy DM. Progress in the 

control of colorectal cancer in Rhode Island, 1987-

2000. Med Health R I 2002;85(7):226-8. 

71. Steele G. National Cancer Database: Annual review 

of patient care, 1994. Atlanta, Ga. American Cancer 

Society. 1994;24-42. 

72. Goldstein N. Lymph node recoveries from 2427 

pT3 colorectal resection specimen spanning 45 

years: recommendations for a minimum number of 

recovered lymph nodes based on predictive 

probabilities. Am J SurgPathol 26(2):179-89. 

73. Caplin S, Cerottini JP, Bosman FT, et al. For 

patients with Dukes' B (TNM Stage II) colorectal 

carcinoma, examination of six or fewer lymph 

nodes is related to poor prognosis. Cancer 

1998;83(4):666-72. 

74. Cianchi F, Palomba A, Boddi V, et al. Lymph node 

recovery from colorectal tumor specimens: 

recommendation for a minimum number of lymph 

nodes to be examined. World J Surg 

2002;26(3):384-9. 

75. Goldstein NS, Sanford W, Coffey M, et al. Lymph 

node recovery from colorectal resection specimens 

removed for adenocarcinoma. Trends over time and 

a recommendation for a minimum number of lymph 

nodes to be recovered. Am J Clin Pathol 

1996;106(2):209-16. 

76. Cserni G, Vinh-Hung V, Burzykowski T. Is there a 

minimum number of lymph nodes that should be 

histologically assessed for a reliable nodal staging 

of T3N0M0 colorectal carcinomas? J Surg Oncol 

2002;81(2):63-9. 

77. Pocard M, Panis Y, Malassagne B, et al. Assessing 

the effectiveness of mesorectal excision in rectal 

cancer: prognostic value of the number of lymph 

nodes found in resected specimens. Dis Colon 

Rectum 1998;41(7):839-45. 

78. Martling A, Singnomklao T, Holm T, et al. 

Prognostic significance of both surgical and 

pathological assessment of curative resection for 

rectal cancer. Br J Surg 2004;91(8):1040-5. 

79. Seitz U, Bohnacker S, Seewald S, et al. Is 

endoscopic polypectomy an adequate therapy for 

malignant colorectal adenomas? Presentation of 114 

patients and review of the literature. Dis Colon 

Rectum 2004;47(11):1789-96; discussion 1796-7. 

80. Rothenberger DA, Akbari R, Baxter NN. Are we 

overtreating some patients with rectal cancer?. 

Oncology (Huntingt) 1799;18(14):1789-96; 

discussion 1796. 

81. National Cancer Institute. Rectal Cancer (PDQ®): 

Treatment. Available at 

http://www.nci.nih.gov/cancertopics/pdq/treatment/

rectal/healthprofessional. Accessed July 1, 2005. 

82. Heald RJ, Husband DM, Ryall RD. The 

mesorectum in rectal cancer surgery - the clue to 

pelvic recurrence? Br J Surg 1982;69(10):613-6. 

83. Autschbach F. The pathological assessment of total 

mesorectal excision: what are the relevant resection 

margins? Recent Results Cancer Res 2005;165:30-

9. 

84. Phillips RK, Hittinger R, Blesovsky L, et al. Local 

recurrence following 'curative' surgery for large 

bowel cancer: I. The overall picture. Br J Surg 

1984;71(1):12-6. 

85. World Health Organization, International Agency 

for Research on Cancer, European Commission. 

Survival of cancer patients in Europe: the Euro 

Study. Lyon, France: IARC Scientific Publications; 

1995. 

86. Newcomb PA, Carbone PP. Cancer treatment and 

age: patient perspectives. J Natl Cancer Inst 

1993;85(19):1580-4. 

87. Demissie K, Oluwole OO, Balasubramanian BA, et 

al. Racial differences in the treatment of colorectal 

cancer: a comparison of surgical and radiation 

therapy between Whites and Blacks. Ann Epidemiol 

2004;14(3):215-21. 

88. Dominitz JA, Samsa GP, Landsman P, et al. Race, 

treatment, and survival among colorectal carcinoma 

patients in an equal-access medical system. Cancer. 

1998;82(12):2312-20. 

89. Weaver P, Harrison B, Eskander G, et al. Colon 

cancer in blacks: a disease with a worsening 

prognosis. J Natl Med Assoc 1991;83(2):133-6. 

90. Rogers SO, Ray WA, Smalley WE. A populationbased study of survival among elderly persons 

diagnosed with colorectal cancer: does race matter 

if all are insured? (United States). Cancer Causes 

Control 2004;15(2):193-9. 
107

91. Merrill RM, Brown ML, Potosky AL, et al. Survival 

and treatment for colorectal cancer Medicare 

patients in two group/staff health maintenance 

organizations and the fee-for-service setting. Med 

Care Res Rev 1999;56(2):177-96. 

92. Vernon SW, Hughes JI, Heckel VM, et al. Quality 

of care for colorectal cancer in a fee-for-service and 

health maintenance organization practice.[erratum 

appears in Cancer 1994 Dec 15;74(12):3249]. 

Cancer 1992;69(10):2418-25. 

93. Cooper GS, Yuan Z, Landefeld CS, et al. Surgery 

for colorectal cancer: Race-related differences in 

rates and survival among Medicare beneficiaries. 

Am J Public Health 1996;86(4):582-6. 

94. Roetzheim RG, Pal N, Gonzalez EC, et al. Effects 

of health insurance and race on colorectal cancer 

treatments and outcomes. Am J Public Health 

2000;90(11):1746-54. 

95. Temple LK, Hsieh L, Wong WD, et al. Use of 

surgery among elderly patients with stage IV 

colorectal cancer. J Clin Oncol 2004;22(17):3475-

84. 

96. O'Connell JB, Maggard MA, Liu JH, et al. Do 

young colon cancer patients have worse outcomes? 

World J Surg 2004;28(6):558-62. 

97. Virnig BA, Warren JL, Cooper GS, et al. Studying 

radiation therapy using SEER-Medicare-linked 

data. Med Care 2002;40(8 Suppl):IV-49-54. 

98. O'Connell JB, Maggard MA, Liu JH, et al. Are 

survival rates different for young and older patients 

with rectal cancer? Dis Colon Rectum 

2004;47(12):2064-9. 

99. Hyman N, Labow SB, Vermont Chapter of the 

American College of Surgeons. The Vermont 

colorectal cancer project: self-portrait. Arch Surg 

2002;137(4):413-6. 

100. Wudel LJ Jr, Chapman WC, Shyr Y, et al. Disparate 

outcomes in patients with colorectal cancer: effect 

of race on long-term survival. Arch Surg 

2002;137(5):550-4; discussion 554-6. 

101. Tropman S, Hatzell T, Paskett E, et al. Colon cancer 

treatment in rural North and South Carolina. Cancer 

Detect Prev 1999;23(5):428-34. 

102. Govindarajan R, Shah RV, Erkman LG, et al. Racial 

differences in the outcome of patients with 

colorectal carcinoma. Cancer 2003;97(2):493-8. 

103. Schrag D, Panageas KS, Riedel E, et al. Surgeon 

volume compared to hospital volume as a predictor 

of outcome following primary colon cancer 

resection. J Surg Oncol 2003;83(2):68-78; 

discussion 78-9. 

104. Hodgson DC, Zhang W, Zaslavsky AM, et al. 

Relation of hospital volume to colostomy rates and 

survival for patients with rectal cancer. J Natl 

Cancer Inst 2003;95(10):708-16. 

105. Morris AM, Billingsley KG, Baxter NN, et al. 

Racial disparities in rectal cancer treatment: a 

population-based analysis. Arch Surg 

2004;139(2):151-5; discussion 156. 

106. Simons AJ, Ker R, Groshen S, et al. Variations in 

treatment of rectal cancer: the influence of hospital 

type and caseload. Dis Colon Rectum 

1997;40(6):641-6. 

107. Purves H, Pietrobon R, Hervey S, et al. Relationship 

between surgeon caseload and sphincter 

preservation in patients with rectal cancer. Dis 

Colon Rectum 2005;48(2):195-202; discussion 202-

4. 

108. Schrag D, Panageas KS, Riedel E, et al. Hospital 

and surgeon procedure volume as predictors of 

outcome following rectal cancer resection. Ann 

Surg 2002;236(5):583-92. 

109. Read TE, Myerson RJ, Fleshman JW, et al. Surgeon 

specialty is associated with outcome in rectal cancer 

treatment. Dis Colon Rectum 2002;45(7):904-14. 

110. Coburn MC, Pricolo VE, Soderberg CH. Factors 

affecting prognosis and management of carcinoma 

of the colon and rectum in patients more than eighty 

years of age. J Am Coll Surg 1994;179(1):65-9. 

111. Rabeneck L, Davila JA, Thompson M, et al. 

Outcomes in elderly patients following surgery for 

colorectal cancer in the veterans affairs health care 

system. Aliment Pharmacol Ther 

2004;20(10):1115-24. 

112. Schrag D, Cramer LD, Bach PB, et al. Influence of 

hospital procedure volume on outcomes following 

surgery for colon cancer. JAMA 

2000;284(23):3028-35. 

113. Whittle J, Steinberg EP, Anderson GF, et al. Results 

of colectomy in elderly patients with colon cancer, 

based on Medicare claims data. Am J Surg 

1992;163(6):572-6. 
108

114. Agarwal N, Leighton L, Mandile MA, et al. 

Outcomes of surgery for colorectal cancer in 

patients age 80 years and older. Am J Gastroenterol 

1990;85(9):1096-101. 

115. Dimick JB, Cowan JA Jr, Upchurch GR Jr, et al. 

Hospital volume and surgical outcomes for elderly 

patients with colorectal cancer in the United States. 

J Surg Res 2003;114(1):50-6. 

116. Harmon J, Tang D, Gordon T, et al. Hospital 

Volume Can Serve as a Surrogate for Surgeon 

Volume for Achieving Excellent Outcomes in 

Colorectal Resection. Ann Surg 1999;230(3):404-

13. 

117. Ko CY, Chang JT, Chaudhry S, et al. Are highvolume surgeons and hospitals the most important 

predictors of in-hospital outcome for colon cancer 

resection? Surgery 2002;132(2):268-73. 

118. Marijnen CA , van de Velde CJ , Putter H , et al. 

Impact of short term preoperative radiotherapy on 

health related quality of life and sexual functioning 

in primary rectal cancer: report of a multicenter 

randomized trial. J Clin Oncol 2005;23(9):1847-58. 

119. Havenga K , Maas CP , DeRuiter MC , et al. 

Avoiding long term distrubance to bladder and 

sexual function in pelvic surgery, particularly with 

rectal cancer. Sem in Surg Oncol 2000;18(3):235-

43. 

120. Lindsey I , George B , Kettlewell M. Randomized 

double blind placebo controlled trial of sildenafil 

(Viagra) for erectile dysfunction after rectal 

excision for cancer and inflammatory bowel 

disease. Dis Colon Rectum 2002;45:727-32. 

121. Temple LK , Wonng WD , Minsky B. The impact 

of radiation on functional outcomes in patients with 

rectal cancer and sphincter preservation. Semin 

Radiat Oncol 2003;13(469-77). 

122. Hendren SK , O'Connor BI , Liu M , et al. 

Prvalence of male and female sexual dysfunction is 

high following surgery for rectal cancer. Ann of 

Surg 2005;242(2):212-23. 

123. Gunderson LL, Sargent DJ, Tepper JE, et al. Impact 

of T and N stage and treatment on survival and 

relapse in adjuvant rectal cancer: a pooled analysis. 

J Clin Oncol 2004;22(10):1785-96. 

124. Coia L, Wizenberg M, Hanlon A, et al. Evaluation 

and treatment of patients receiving radiation for 

cancer of the rectum or sigmoid colon in the United 

States: results of the 1988-1989 Patterns of Care 

Study process survey. J Clin Oncol 1994;12(5):954-

9. 

125. Potosky AL, Harlan LC, Kaplan RS, et al. Age, sex, 

and racial differences in the use of standard 

adjuvant therapy for colorectal cancer. J Clin Oncol 

2002;20(5):1192-202. 

126. Mahoney T, Kuo YH, Topilow A, et al. Stage III 

colon cancers: why adjuvant chemotherapy is not 

offered to elderly patients. Arch Surg 

2000;135(2):182-5. 

127. Ayanian JZ, Zaslavsky AM, Fuchs CS, et al. Use of 

adjuvant chemotherapy and radiation therapy for 

colorectal cancer in a population-based cohort. J 

Clin Oncol 2003;21(7):1293-300. 

128. Schrag D, Cramer LD, Bach PB, et al. Age and 

adjuvant chemotherapy use after surgery for stage 

III colon cancer. J Natl Cancer Inst 

2001;93(11):850-7. 

129. Sundararajan V, Grann VR, Jacobson JS, et al. 

Variations in the use of adjuvant chemotherapy for 

node-positive colon cancer in the elderly: a 

population-based study. Cancer J 2001;7(3):213-8. 

130. Keating N, Landrum M, Meara E, et al. Do 

Increases in the Market Share of Managed Care 

Influence Quality of Care in the Fee-For-Service 

Sector? J Natl Cancer Inst 2005;97(4):257-64. 

131. Steele GD Jr. The National Cancer Data Base report 

on colorectal cancer. Cancer 1994;74(7):1979-89. 

132. Schrag D, Gelfand SE, Bach PB, et al. Who gets 

adjuvant treatment for stage II and III rectal cancer? 

Insight from surveillance, epidemiology, and end 

results--Medicare. J Clin Oncol 2001;19(17):3712-

8. 

133. Neugut AI, Fleischauer AT, Sundararajan V, et al. 

Use of adjuvant chemotherapy and radiation therapy 

for rectal cancer among the elderly: a populationbased study. J Clin Oncol 2002;20(11):2643-50. 

134. Steele G, Jessup J. National Cancer Database: 

Annual review of patient care, 1995. Atlanta, Ga. 

American Cancer Society. 1995;66-83. 

135. Jessup JM, McGinnis LS, Steele GD Jr, et al. The 

National Cancer Data Base. Report on colon cancer. 

Cancer 1996;78(4):918-26. 

136. Retchin SM, Brown B. Management of colorectal 

cancer in Medicare health maintenance 

organizations. J Gen Intern Med. 1990;5(2):110-4. 

137. Oliveria SA, Yood MU, Campbell UB, et al. 

Treatment and referral patterns for colorectal 

cancer. Med Care 2004;42(9):901-6. 
109

138. Wolmark N, Fisher B, Rockette H, et al. 

Postoperative adjuvant chemotherapy or BCG for 

colon cancer: results from NSABP protocol C-01. J 

Natl Cancer Inst 1988;80(1):30-6. 

139. Laurie JA, Moertel CG, Fleming TR, et al. Surgical 

adjuvant therapy of large-bowel carcinoma: an 

evaluation of levamisole and the combination of 

levamisole and fluorouracil. The North Central 

Cancer Treatment Group and the Mayo Clinic. J 

Clin Oncol 1989;7(10):1447-56. 

140. Moertel CG, Fleming TR, Macdonald JS, et al. 

Levamisole and fluorouracil for adjuvant therapy of 

resected colon carcinoma. N Engl J Med 

1990;322(6):352-8. 

141. Gastrointestinal Tumor Study Group. Prolongation 

of the disease-free interval in surgically treated 

rectal carcinoma. N Engl J Med 

1985;312(23):1465-72. 

142. Krook JE, Moertel CG, Gunderson LL, et al. 

Effective surgical adjuvant therapy for high-risk 

rectal carcinoma. N Engl J Med 1991;324(11):709-

15. 

143. Sauer R, Becker H, Hohenberger W, et al. 

Preoperative versus postoperative 

chemoradiotherapy for rectal cancer. N Engl J Med 

2004;351(17):1731-40. 

144. Kapiteijn E, Marijnen CA, Nagtegaal ID, et al. 

Preoperative radiotherapy combined with total 

mesorectal excision for resectable rectal cancer. N 

Engl J Med 2001;345(9):638-46. 

145. Baxter NN, Rothenberger DA, Morris AM, et al. 

Adjuvant radiation for rectal cancer: do we measure 

up to the standard of care? An epidemiologic 

analysis of trends over 25 years in the United States. 

Dis Colon Rectum 2005;48(1):9-15. 

146. Kline RW, Smith AR, Coia LR, et al. Treatment 

planning for adenocarcinoma of the rectum and 

sigmoid: a patterns of care study. PCS Committee. 

Int J Radiat Oncol Biol Phys 1997;37(2):305-11. 

147. Minsky BD, Coia L, Haller DG, et al. Radiation 

therapy for rectosigmoid and rectal cancer: results 

of the 1992-1994 Patterns of Care process survey. J 

Clin Oncol. 1998;16(7):2542-7. 

148. Patterns of Care Study (PCS) Consensus 

Committee. PCS Consensus Committee: 1996 

decision trees and management guidelines. Sem 

Radiat Oncol 1997;7:163-81. 

149. Citarda F, Tomaselli G, Capocaccia R, et al. 

Efficacy in standard clinical practice of 

colonoscopic polypectomy in reducing colorectal 

cancer incidence.[see comment]. Gut 

2001;48(6):812-5. 

150. Winawer SJ, Zauber AG, Ho MN, et al. Prevention 

of colorectal cancer by colonoscopic polypectomy. 

The National Polyp Study Workgroup. N Engl J 

Med 1993;329(27):1977-81. 

151. American Cancer Society. Cancer reference 

information. Detailed guide: colon and rectum 

cancer. Can colorectal polyps and cancer be found 

early? 2005. Available at: 

http://www.cancer.org/docroot/CRI/content/CRI_2_

4_3x_Can_Colon_and_rectum_cancer_be_found_e

arly.asp. Accessed July 8, 2005. 

152. Anthony T, Simmang C, Hyman N, et al. Practice 

parameters for the surveillance and follow-up of 

patients with colon and rectal cancer. Dis Colon 

Rectum 2004;47(6):807-17. 

153. Benson A, Desch C, Flynn P, et al. 2000 Update of 

American Cancer Society of Clinical Oncology 

Colorectal Cancer Surveillance Guidelines. J Clin 

Oncol 2000;18(20):3586-8. 

154. Anonymous. Guidelines for colorectal cancer 

screening and surveillance. Gastrointest Endosc 

2000;51(6):777-82. 

155. Knopf KB, Warren JL, Feuer EJ, et al. Bowel 

surveillance patterns after a diagnosis of colorectal 

cancer in Medicare beneficiaries. Gastrointest 

Endosc 2001;54(5):563-71. 

156. Ellison GL, Warren JL, Knopf KB, et al. Racial 

differences in the receipt of bowel surveillance 

following potentially curative colorectal cancer 

surgery. Health Serv Res 2003;38(6 Pt 2):1885-903. 

157. Rulyak SJ, Mandelson MT, Brentnall TA, et al. 

Clinical and sociodemographic factors associated 

with colon surveillance among patients with a 

history of colorectal cancer. Gastrointest Endosc 

2004;59(2):239-47. 

158. Cooper GS, Yuan Z, Chak A, et al. Patterns of 

endoscopic follow-up after surgery for 

nonmetastatic colorectal cancer. Gastrointest 

Endosc. 2000;52(1):33-8. 

159. Cooper GS, Yuan Z, Chak A, et al. Geographic and 

patient variation among Medicare beneficiaries in 

the use of follow-up testing after surgery for 

nonmetastatic colorectal carcinoma. Cancer 

1999;85(10):2124-31. 
110

160. Byers T, Levin B, Rothenberger D, et al. American 

Cancer Society guidelines for screening and 

surveillance for early detection of colorectal polyps 

and cancer: update 1997. American Cancer Society 

Detection and Treatment Advisory Group on 

Colorectal Cancer. CA Cancer J Clin 

1997;47(3):154-60. 

161. American Society of Colon and Rectal Surgeons. 

Practice parameters for the detection of colorectal 

neoplasms. Dis Colon Rectum 1992;35(4):389-90. 

162. LaFata EJ, Johnson C, Ben-Menachem T, et al. 

Sociodemographic differences in the receipt of 

colorectal cancer surveillance care following 

treatment with curative intent. Med Care 

2001;39(4):361-72. 

163. Desch C, Benson A, Smith T, et al. Recommended 

Colorectal Cancer Surveillance Guidelines by the 

American Society of Clinical Oncology. J Clin 

Oncol 1999;17(4):1312-21. 

164. Engstrom PF, Benson AB 3rd, Cohen A, et al. 

NCCN Colorectal Cancer Practice Guidelines. The 

National Comprehensive Cancer Network. 

Oncology (Huntingt). 1996;10(11 Suppl):140-75. 

165. Wei JT, Miller EA, Woosley JT, et al. Quality of 

colon carcinoma pathology reporting: a process of 

care study. Cancer 2004;100(6):1262-7. 

166. Association of Directors of Anatomic and Surgical 

Pathology. Final diagnosis checklist: colon 

carcinoma. Ver. 1.1, November 2003. Available at: 

http://www.adasp.org. Accessed July 8, 2005. 

167. Compton CC, College of American Pathologists 

Cancer Committee. Cancer protocol: colon and 

rectum. January 2005. Available at: 

http://www.cap.org/apps/docs/cancer_protocols/prot

ocols_index.html. Accessed July 8, 2005. 

168. Compton CC, Henson DE, Hutter RV, et al. 

Updated protocol for the examination of specimens 

removed from patients with colorectal carcinoma. A 

basis for checklists. Arch Pathol Lab Med 

1997;121(12):1247-54. 
111

Listing of Included Studies (alphabetical)

Agarwal N, Leighton L, Mandile MA, et al. Outcomes of 

surgery for colorectal cancer in patients age 80 years and 

older. Am J Gastroenterol 1990;85(9):1096-101. 

Ayanian JZ, Zaslavsky AM, Fuchs CS, et al. Use of 

adjuvant chemotherapy and radiation therapy for colorectal 

cancer in a population-based cohort. J Clin Oncol 

2003;21(7):1293-300. 

Baig N, Myers RE, Turner BJ, et al. Physician-reported 

reasons for limited follow-up of patients with a positive 

fecal occult blood test screening result. Am J Gastroenterol 

2003;98(9):2078-81. 

Baxter N, Virnig D, Rothenberger D, et al. Lymph Node 

Evaluation in Colorectal Cancer Patients: A Population 

Based Study. J Natl Cancer Inst 2005;97(3):219-25. 

Baxter NN, Rothenberger DA, Morris AM, et al. Adjuvant 

radiation for rectal cancer: do we measure up to the 

standard of care? An epidemiologic analysis of trends over 

25 years in the United States. Dis Colon Rectum 

2005;48(1):9-15. 

Beart RW, Steele GD Jr, Menck HR, et al. Management 

and survival of patients with adenocarcinoma of the colon 

and rectum: a national survey of the Commission on 

Cancer. J Am Coll Surg 1995;181(3):225-36. 

Chiaverini L, Fulton JP, Darcy DM. Progress in the control 

of colorectal cancer in Rhode Island, 1987-2000. Med 

Health R I 2002;85(7):226-8. 

Coburn MC, Pricolo VE, Soderberg CH. Factors affecting 

prognosis and management of carcinoma of the colon and 

rectum in patients more than eighty years of age. J Am Coll 

Surg 1994;179(1):65-9. 

Coia L, Wizenberg M, Hanlon A, et al. Evaluation and 

treatment of patients receiving radiation for cancer of the 

rectum or sigmoid colon in the United States: results of the 

1988-1989 Patterns of Care Study process survey. J Clin 

Oncol 1994;12(5):954-9. 

Colon Cancer Workgroup. Quality of Cancer Care 

Evaluation and Improvement: Commission on Cancer, 

American College of Surgeons, American Cancer Society 

and Tenet Health Care System, December, 2003. 

Electronically received February 7, 2005. 

Cooper GS, Yuan Z, Chak A, et al. Geographic and patient 

variation among Medicare beneficiaries in the use of 

follow-up testing after surgery for nonmetastatic colorectal 

carcinoma. Cancer 1999;85(10):2124-31. 

Cooper GS, Yuan Z, Chak A, et al. Patterns of endoscopic 

follow-up after surgery for nonmetastatic colorectal cancer. 

Gastrointest Endosc 2000;52(1):33-8. 

Cooper GS, Yuan Z, Landefeld CS, et al. Surgery for 

colorectal cancer: Race-related differences in rates and 

survival among Medicare beneficiaries. Am J Public Health 

1996;86(4):582-6. 

Demissie K, Oluwole OO, Balasubramanian BA, et al. 

Racial differences in the treatment of colorectal cancer: a 

comparison of surgical and radiation therapy between 

Whites and Blacks. Ann Epidemiol 2004;14(3):215-21. 

Dimick JB, Cowan JA Jr, Upchurch GR Jr, et al. Hospital 

volume and surgical outcomes for elderly patients with 

colorectal cancer in the United States. J Surg Res 

2003;114(1):50-6. 

Dominitz JA, Samsa GP, Landsman P, et al. Race, 

treatment, and survival among colorectal carcinoma 

patients in an equal-access medical system. Cancer 

1998;82(12):2312-20. 

Galvis CO, Raab SS, D'Amico F, et al. Pathologists' 

assistants practice: a measurement of performance. Am J 

Clin Pathol 2001;116(6):816-22. 

Govindarajan R, Shah RV, Erkman LG, et al. Racial 

differences in the outcome of patients with colorectal 

carcinoma. Cancer 2003;97(2):493-8. 

Harewood GC, Sharma VK, de Garmo P. Impact of 

colonoscopy preparation quality on detection of suspected 

colonic neoplasia. Gastrointest Endosc 2003;58(1):76-9. 

Harmon J, Tang D, Gordon T, et al. Hospital Volume Can 

Serve as a Surrogate for Surgeon Volume for Achieving 

Excellent Outcomes in Colorectal Resection. Ann Surg 

1999;230(3):404-13. 

Haseman JH, Lemmel GT, Rahmani EY, et al. Failure of 

colonoscopy to detect colorectal cancer: evaluation of 47 

cases in 20 hospitals. Gastrointest Endosc 1997;45(6):451-

5. 

Hodgson DC, Zhang W, Zaslavsky AM, et al. Relation of 

hospital volume to colostomy rates and survival for patients 

with rectal cancer. J Natl Cancer Inst 2003;95(10):708-16. 

Hyman N, Labow SB, Vermont Chapter of the American 

College of Surgeons. The Vermont colorectal cancer 

project: self-portrait. Arch Surg 2002;137(4):413-6. 
112

Institute of Medicine. Asessing the Quality of Cancer Care: 

An Approach to Measurement in Georgia. Committee on 

Assessing Improvements in Cancer Care in Georgia. 

National Cancer Policy Board. The National Academies 

Press: Washington, DC; 2005. 

Jessup JM, McGinnis LS, Steele GD Jr, et al. The National 

Cancer Data Base. Report on colon cancer. Cancer. 

1996;78(4):918-26. 

Keating N, Landrum M, Meara E, et al. Do Increases in the 

Market Share of Managed Care Influence Quality of Care 

in the Fee-For-Service Sector? J Natl Cancer Inst 

2005;97(4):257-64. 

Kline RW, Smith AR, Coia LR, et al. Treatment planning 

for adenocarcinoma of the rectum and sigmoid: a patterns 

of care study. PCS Committee. Int J Radiat Oncol Biol 

Phys 1997;37(2):305-11. 

Ko CY, Chang JT, Chaudhry S, et al. Are high-volume 

surgeons and hospitals the most important predictors of inhospital outcome for colon cancer resection? Surgery 

2002;132(2):268-73. 

LaFata EJ, Johnson C, Ben-Menachem T, et al. 

Sociodemographic differences in the receipt of colorectal 

cancer surveillance care following treatment with curative 

intent. Med Care 2001;39(4):361-72. 

Levin B, Hess K, Johnson C. Screening for colorectal 

cancer. A comparison of 3 fecal occult blood tests. Arch 

Intern Med 1997;157(9):970-6. 

Mahoney T, Kuo YH, Topilow A, et al. Stage III colon 

cancers: why adjuvant chemotherapy is not offered to 

elderly patients. Arch Surg 2000;135(2):182-5. 

Mandel JS, Church TR, Bond JH, et al. The effect of fecal 

occult-blood screening on the incidence of colorectal 

cancer. N Engl J Med 2000;343(22):1603-7. 

Marble K, Banerjee S, Greenwald L. Colorectal carcinoma 

in young patients. J Surg Oncol 1992;51(3):179-82. 

Merrill RM, Brown ML, Potosky AL, et al. Survival and 

treatment for colorectal cancer Medicare patients in two 

group/staff health maintenance organizations and the feefor-service setting. Med Care Res Rev 1999;56(2):177-96. 

Minnesota Gastroenterology PA. 2004 Colonoscopy 

Quality Assurance Initiative. Electronically received: 

January 26, 2005. 

Minsky BD, Coia L, Haller DG, et al. Radiation therapy for 

rectosigmoid and rectal cancer: results of the 1992-1994 

Patterns of Care process survey. J Clin Oncol 

1998;16(7):2542-7. 

Morris AM, Billingsley KG, Baxter NN, et al. Racial 

disparities in rectal cancer treatment: a population-based 

analysis. Arch Surg 2004;139(2):151-5; discussion 156. 

Myers RE, Turner B, Weinberg D, et al. Impact of a 

physician-oriented intervention on follow-up in colorectal 

cancer screening. Prev Med 2004;38(4):375-81. 

National Committee for Quality Assurance (NCQA). Final 

List of Colon Cancer Measures Based on Domain Expert 

Review. Measures developed by Kaiser Permanente, 

Internountain Health Care and NCQA. Electronically 

delivered February 8, 2005. 

National Quality Measures Clearinghouse (NQMC). Intraprocedure colonosopy complication rate: percentage of 

patients who developed one or more intrao-procedure 

complications. Available at: 

http://www.qualitymeasures.ahrq.gov/summary/summary.a

spx?doc_id=4868. Accessed: March 21, 2005. 

Nelson DB, McQuaid KR, Bond JH, et al. Procedural 

success and complications of large-scale screening 

colonoscopy. Gastrointest Endosc 2002;55(3):307-14. 

Neugut AI, Fleischauer AT, Sundararajan V, et al. Use of 

adjuvant chemotherapy and radiation therapy for rectal 

cancer among the elderly: a population-based study. J Clin 

Oncol 2002;20(11):2643-50. 

Newcomb PA, Carbone PP. Cancer treatment and age: 

patient perspectives. J Natl Cancer Inst 1993;85(19):1580-

4. 

O'Connell JB, Maggard MA, Liu JH, et al. Are survival 

rates different for young and older patients with rectal 

cancer? Dis Colon Rectum 2004;47(12):2064-9. 

O'Connell JB, Maggard MA, Liu JH, et al. Do young colon 

cancer patients have worse outcomes? World J Surg 

2004;28(6):558-62. 

Oliveria SA, Yood MU, Campbell UB, et al. Treatment and 

referral patterns for colorectal cancer. Med Care. 

2004;42(9):901-6. 

Potosky AL, Harlan LC, Kaplan RS, et al. Age, sex, and 

racial differences in the use of standard adjuvant therapy 

for colorectal cancer. J Clin Oncol 2002;20(5):1192-202. 

Purves H, Pietrobon R, Hervey S, et al. Relationship 

between surgeon caseload and sphincter preservation in 

patients with rectal cancer. Dis Colon Rectum 

2005;48(2):195-202; discussion 202-4. 

Rabeneck L, Davila JA, Thompson M, et al. Outcomes in 

elderly patients following surgery for colorectal cancer in 

the veterans affairs health care system. Aliment Pharmacol 

Ther 2004;20(10):1115-24. 
113

Read TE, Myerson RJ, Fleshman JW, et al. Surgeon 

specialty is associated with outcome in rectal cancer 

treatment. Dis Colon Rectum 2002;45(7):904-14. 

Retchin SM, Brown B. Management of colorectal cancer in 

Medicare health maintenance organizations. J Gen Intern 

Med 1990;5(2):110-4. 

Roetzheim RG, Pal N, Gonzalez EC, et al. Effects of health 

insurance and race on colorectal cancer treatments and 

outcomes. Am J Public Health 2000;90(11):1746-54. 

Rogers SO, Ray WA, Smalley WE. A population-based 

study of survival among elderly persons diagnosed with 

colorectal cancer: does race matter if all are insured? 

(United States). Cancer Causes Control 2004;15(2):193-9. 

Rulyak SJ, Mandelson MT, Brentnall TA, et al. Clinical 

and sociodemographic factors associated with colon 

surveillance among patients with a history of colorectal 

cancer. Gastrointest Endosc 2004;59(2):239-47. 

Schrag D, Cramer LD, Bach PB, et al. Age and adjuvant 

chemotherapy use after surgery for stage III colon cancer. J 

Natl Cancer Inst 2001;93(11):850-7. 

Schrag D, Cramer LD, Bach PB, et al. Influence of hospital 

procedure volume on outcomes following surgery for colon 

cancer. JAMA 2000;284(23):3028-35. 

Schrag D, Gelfand SE, Bach PB, et al. Who gets adjuvant 

treatment for stage II and III rectal cancer? Insight from 

surveillance, epidemiology, and end results--Medicare. J 

Clin Oncol 2001;19(17):3712-8. 

Schrag D, Panageas KS, Riedel E, et al. Hospital and 

surgeon procedure volume as predictors of outcome 

following rectal cancer resection. Ann Surg 

2002;236(5):583-92. 

Schrag D, Panageas KS, Riedel E, et al. Surgeon volume 

compared to hospital volume as a predictor of outcome 

following primary colon cancer resection. J Surg Oncol 

2003;83(2):68-78; discussion 78-9. 

Shehadeh I, Rebala S, Kumar R, et al. Retrospective 

analysis of missed advanced adenomas on surveillance 

colonoscopy. Am J Gastroenterol 2002;97(5):1143-7. 

Shields HM, Weiner MS, Henry DR, et al. Factors that 

influence the decision to do an adequate evaluation of a 

patient with a positive stool for occult blood. Am J 

Gastroenterol 2001;96(1):196-203. 

Simons AJ, Ker R, Groshen S, et al. Variations in treatment 

of rectal cancer: the influence of hospital type and caseload. 

Dis Colon Rectum 1997;40(6):641-6. 

Steele GD Jr. The National Cancer Data Base report on 

colorectal cancer. Cancer 1994;74(7):1979-89. 

Steele G. National Cancer Database: Annual review of 

patient care, 1994. Atlanta, Ga. American Cancer Society. 

1994;24-42. 

Steele G, Jessup J. National Cancer Database: Annual 

review of patient care, 1995. Atlanta, Ga. American Cancer 

Society. 1995;66-83. 

Sundararajan V, Grann VR, Jacobson JS, et al. Variations 

in the use of adjuvant chemotherapy for node-positive 

colon cancer in the elderly: a population-based study. 

Cancer J 2001;7(3):213-8. 

Temple LK, Hsieh L, Wong WD, et al. Use of surgery 

among elderly patients with stage IV colorectal cancer. J 

Clin Oncol 2004;22(17):3475-84. 

Tropman S, Hatzell T, Paskett E, et al. Colon cancer 

treatment in rural North and South Carolina. Cancer Detect 

Prev 1999;23(5):428-34. 

Ure T, Dehghan K, Vernava AM 3rd, et al. Colonoscopy in 

the elderly. Low risk, high yield. Surg Endosc 

1995;9(5):505-8. 

Vernon SW, Hughes JI, Heckel VM, et al. Quality of care 

for colorectal cancer in a fee-for-service and health 

maintenance organization practice.[erratum appears in 

Cancer 1994 Dec 15;74(12):3249]. Cancer 

1992;69(10):2418-25. 

Weaver P, Harrison B, Eskander G, et al. Colon cancer in 

blacks: a disease with a worsening prognosis. J Natl Med 

Assoc 1991;83(2):133-6. 

Wei JT, Miller EA, Woosley JT, et al. Quality of colon 

carcinoma pathology reporting: a process of care study. 

Cancer 2004;100(6):1262-7. 

Whittle J, Steinberg EP, Anderson GF, et al. Results of 

colectomy in elderly patients with colon cancer, based on 

Medicare claims data. Am J Surg 1992;163(6):572-6. 

Wudel LJ Jr, Chapman WC, Shyr Y, et al. Disparate 

outcomes in patients with colorectal cancer: effect of race 

on long-term survival. Arch Surg 2002;137(5):550-4; 

discussion 554-6.

115

Acronyms and Abbreviations 

5-FU 5-Fluorouracil 

AAAHC Accreditation Association for Ambulatory Health Care 

ACS American Cancer Society 

ACS-COC American College of Surgery Commission on Cancer 

ADASP Association of Directors of Anatomic and Surgical Pathology 

AGA American Gastroenterological Association 

AHRQ Agency for Healthcare Research and Quality 

AJCC American Joint Committee on Cancer 

APR Abdominal perineal resection 

ASCO American Society of Clinical Oncology 

ASCRS American Society of Colon and Rectal Surgeons 

ASGE American Society for Gastrointestinal Endoscopy 

BE Barium enema 

CAP College of American Pathologists 

CDC Centers for Disease Control and Prevention 

CEA Carcinoembryonic antigen 

CMS Centers for Medicare & Medicaid Services 

CORI Clinical Outcomes Research Initiative 

CPT Current Procedural Terminology

CRC Colorectal cancer 

CT Computed tomography/tomographic 

DCBE Double contrast barium enema 

EUS Endoscopic ultrasound 

FFS Fee-for-service 

FOBT Fecal occult blood test 

Gy Gray 

HCUP Hospital Cost and Utilization Project 

HMO Health maintenance organization 

HSCRC Health Services Cost Review Commission 

IOM Institute of Medicine 

MeV Million electron volts 

MRI Magnetic resonance imaging 

NCCS North Carolina Cancer Study 

NCDB National Cancer Data Base 

NCCN National Comprehensive Cancer Network 

NCI National Cancer Institute 

NCPB National Cancer Policy Board 

NCQA National Committee for Quality Assurance 

NIH National Institutes of Health 

NIS Nationwide Inpatient Sample 

NQF National Quality Forum 

NQMC National Quality Measures Clearinghouse 

PCS Patterns of Care Study 
116

RTOG Radiation Therapy Oncology Group 

SEER Surveillance, Epidemiology, and End Results 

SLN Sentinel lymph node 

TME Total mesorectal excision 

TNM Tumor, Node, Metastasis Staging System 

UICC International Union Against Cancer 

VA Veterans Administration 
APPENDIXES 

to 

“Cancer Care Quality Measures: 

Diagnosis and Treatment of Colorectal Cancer” 

Prepared by the Duke Evidence-based Practice Center 

(Contract #290-02-0025) 
A-1

Appendix A: Quality of Life 

Methods of Assessing Quality of Life for Patients 

With Colorectal Cancer 

When measuring quality of life for persons with colorectal cancer (CRC), most authors 

recommend the use of several instruments including a general instrument (e.g., SF-36, EQ-5D) 

that allows benchmarking against other conditions and a cancer-specific instrument that focuses 

on those aspects that are most salient to cancer. Ideally, the cancer-specific instrument would 

appropriately account for those aspects that are unique to CRC. Sprangers1 and Provenzale and 

Gray2 provide excellent reviews of the overall topic of how to assess quality of life in patients 

with CRC. 

 We describe various instruments that have been used in patients with CRC, including general 

health measures, cancer-specific measures, and CRC-specific measures. The general health 

measures and cancer-specific measures are listed and/or given only a very brief description. The 

CRC-specific measures are described, any validation efforts are noted, and the measures are 

rated. 

General Health Status Measures 

Various American researchers (e.g., Anthony et al.3) have tended to characterize the general 

health status of CRC patients using the Medical Outcome Study Short Form Health Survey (SF36).4

 European researchers have also used the EQ-5D (e.g., Johnson et al.5). Other general 

health status measures, such as the Sickness Impact Profile and the Nottingham Health Profile, 

have also been used from time to time.1

 Various studies (e.g., Weeks et al.6) also use a singleitem global measure of quality of life (e.g., “On a scale of 0-100, how do you rate your quality of 

life today?”), this latter measurement having the advantages of minimal respondent burden and 

being easily translated into a utility value usable within a cost-effectiveness analysis. 

 By design, it can reasonably be presumed that the validation efforts of these general health 

status instruments apply equally well to patients with CRC. For example, the eight dimensions 

of the SF-36 are physical functioning, physical role, pain, general health, vitality, social 

functioning, emotional role, and mental health, all of which are relevant to patients with CRC. 

Although it is helpful for many reasons to include a generic assessment of health-related quality 

of life, for the purposes of quality measurement its connection with the process of care is likely 

to be weaker than would be the corresponding connection with a measure that is more specific to 

CRC. 

Cancer-Related Measures 

Sprangers1 notes that among the cancer-specific measures most commonly applied to patients 

with CRC are the Functional Living Index-Cancer (FLIC),7

 the Rotterdam Symptom Checklist 

(RSCL),8

 and the European Organization for Research and Treatment of Cancer (EORTC) Core 

Quality of Life Questionnaire (QLQ-C30),9

 these latter two measures tending to be used in 
A-2

Europe and, as documented by Provenzale and Gray,2 the most commonly used in practice. 

Additional cancer-specific measures include the Symptom Distress Scale,6

 the Quality of Life 

Index,6 and the Functional Assessment of Cancer Therapy-General (FACT-G).10

 As an illustration of the nature of typical cancer-specific instruments, the 13-item Symptom 

Distress Scale6 measures symptom frequency and distress in the domains of nausea, appetite, 

insomnia, pain, fatigue, bowel function, concentration, appearance, breathing, outlook, and 

cough, all with five response categories and a summary score using all items. The 5-item 

Quality of Life Index6 measures quality of life in the domains of activity, daily living, health, 

support and outlook, all with three response categories, and a summary score using all items. 

The 22-item FLIC instrument7 addresses physical well-being, social well-being, hardship due to 

cancer and nausea (all items are coded on a 1-7 scale). The FACT-G10 is subdivided into seven 

items on physical well-being, seven items on social well-being, six items on emotional wellbeing, and seven items on functional well-being. 

 Chapman et al.11 measured quality of life for 2,202 patients with ileal pouch-anal 

anastomosis (not all from cancer-related surgery) and applied a reasonable but not formally 

validated instrument to measure quality of life. The instrument measures constructs (sexual life, 

social, sports and work activities, family relationships) not on an absolute scale, but relative to 

the patient’s status before surgery. 

 Ayanian et al.12 conducted phone interviews of CRC patients obtained from a cancer registry 

in northern California using a 34-item instrument addressing emotional, social, physical and 

functional well-being. The instrument, with the exception of one question pertaining to bowel 

function, appears to be equally applicable to other cancer patients. The instrument successfully 

measures differences in satisfaction with care according to age, race, and sex, among other 

factors. With the possible exception of the instrument from which the candidate items were 

originally derived, the current version of this instrument has not been validated outside the 

context of this study. 

 As above, it can reasonably be presumed that the validation efforts of these cancer-specific 

quality-of-life instruments apply equally well to patients with CRC. It can also be presumed that 

the connection between quality of life as measured by these instruments and process of care is 

stronger than would be the case for general health instruments, but weaker than would be the 

case for CRC-specific instruments. 

CRC-Related Measures 

Most investigators have adopted what Spangers1 terms the “modular approach” to CRCspecific measures; that is, beginning with a general cancer-specific scale and adding items 

relevant to CRC. The most noteworthy examples are the FACT-C and the QLQ-CR38. 

The FACT-C begins with the 27-item FACT-G scale (described in brief above), then adds a 

9-item Colorectal Cancer Subscale (CCS). Two of the nine CCS items pertain to ostomy 

appliances and thus are not relevant to most CRC patients. In many applications, only the 

remaining seven items are used. Thus, the usual version of the FACT-C has 34 items (27 from 

the FACT-G and 7 from the CCS). Ward et al.10 describe the validation of the FACT-C in 60 

patients with advanced CRC and 156 patients with CRC but without distant metastases, these 

patients being sub-divided into 63 English-speakers and 93 Spanish-speakers. Separate analyses 

were also performed on the 29 patients in this latter sample. Standard techniques of 

psychometric analysis were applied, including an examination of internal consistency, 
A-3

convergent validity, divergent validity, the ability to distinguish among known groups, and 

sensitivity to change. Overall, the results strongly support the validity of the FACT-C. 

 The QLQ-CR38 uses the QLQ-C30 (discussed above), and adds 38 CRC-specific questions. 

Nineteen of these questions are answered by all respondents, and 19 depend on circumstances. 

Items are combined into sub-scales of micturition problems, gastrointestinal symptoms, side 

effects of chemotherapy, bowel problems, stoma-related problems, sexual problems, body image, 

and future-looking orientation. Validation was performed on 117 Dutch patients with CRC, and 

an English version is available. Overall, the results strongly support the validity of the 

instrument. 

 Schwenk et al.13 compared results from the EORTC QLQ-C30 and the Gastrointestinal 

Quality of Life Index (GIQLI). Although the research was performed outside the United States 

(116 German patients undergoing surgery for CRC), to our knowledge this is the first formal 

comparison between the scales. The level of agreement between the instruments was low; the 

EORTC QLQ-C30 was found to be more sensitive in the detection of impairment of quality of 

life in the early post-operative period, causing the authors to recommend the EORTC QLQ-C30 

as the preferred instrument. 

 Gupta et al.14 validated the use of the subjective global assessment (SGA), a clinical 

technique that combines data from subjective and objective aspects of medical history and 

physical examination, in stage III and IV colorectal cancer patients, finding those with poorer 

SGA also had poorer survival. 

 Temple et al.15 report on the development and initial validation of an instrument to evaluate 

bowel function after sphincter-preserving surgery for rectal cancer. Elements of validation, as 

applied to 184 patients undergoing sphincter-preserving surgery, included internal consistency, 

discriminant and construct validity, test-retest, and reliability. This 18-item, 5-response-category 

scale shows promising psychometric properties. 

 In addition to these formal CRC-related measures, various investigators have applied the 

above module-based thinking on an ad hoc basis. For example, Caffo et al.16 created a selfadministered diary card, recorded daily by 32 patients with rectal cancer, consisting of 10 

clinically derived questions, nine of which (e.g., “Did you feel any nausea today?”) are coded on 

a scale of 1 to 4; the final question (“How many times have you had bowel movements today?”) 

is coded as an integer. Results were analyzed at the item level, and there was no total score. No 

formal validation of the measure was provided. 

 In a study comparing the quality-of-life experience of 44 patients undergoing surgery for 

benign colonic disease, 61 patients with CRC undergoing surgery alone, and 53 CRC patients 

undergoing both surgery and adjuvant therapy, Anthony et al.3

 added five CRC-specific 

questions to the SF-36, asking about appetite, weight, bowel function, urinary function, and 

whether the patient would repeat the same therapy as before. No formal validation was 

provided. 

Ratings 

Rating only those measures that were formally developed in order assess CRC-related quality 

of life, both the FACT-C and the QLQ-CR38 can be rated as Importance 4, Usability 4, and 

Validity 4. Regarding importance, the salience of the items to quality of life with CRC is clear, 

the only negative for the purpose of quality measurement being the possibly weak link between 

process of care and quality of life. All measures are usable, except for very sick patients for 
A-4

whom respondent burden is a major issue, in which case a single-item general health scale might 

be preferred. In both cases, the validation of the parent measure has been extensive, and the 

validation of the CRC-related subscale less comprehensive but nevertheless quite encouraging. 

References

1. Sprangers MA. Quality-of-life assessment in 

colorectal cancer patients: evaluation of cancer 

therapies. Semin Oncol 1999;26(6):691-6. 

2. Provenzale D, Gray RN. Colorectal cancer screening 

and treatment: review of outcomes research. J Natl 

Cancer Inst Monogr 2004;(33):45-55. 

3. Anthony T, Jones C, Antoine J, et al. The effect of 

treatment for colorectal cancer on long-term healthrelated quality of life. Ann Surg Oncol 2001;8(1):44-

9. 

4. Ware JE, Sherbourne CD. The MOS 36-item shortform health survey (SF-36). I. Conceptual framework 

and item selection. Med Care 1992;30(6):473-83. 

5. Johnson J, Luo N, Shar J, et al. Valuations of EQ-5D 

Health States: Are the United States and United 

Kingdon Different? Med Care 2005;43(3):221-8. 

6. Weeks J, Nelson H, Gelber S, et al. Short-term quality 

of life outcomes following laparoscopic-assisted 

colectomy vs open colectomy for colon cancer: a 

randomized trial. JAMA 2002;287(3):321-8. 

7. Schipper H, Clinch J, McMurray A, et al. Measuring 

the quality of life of cancer patients: The Functional 

Living Index-Cancer: development and validation. J 

Clin Oncol 1984;2(5):472-83. 

8. de Haes JC, van Knippenberg FC, Neijt JP. Measuring 

psychological and physical distress in cancer patients: 

structure and application of the Rotterdam Symptom 

Checklist. Br J Cancer 1990;62(6):1034-8. 

9. Aaronson NK, Ahmedzai S, Bergman B, et al. The 

EORTC QLQ-C30: a quality-of-life instrument for use 

in international clinical trials in oncology. J Natl 

Cancer Inst 1993;85(5):365-76. 

10. Ward WL, Hahn EA, Mo F, et al. Reliability and 

validity of the Functional Assessment of Cancer 

Therapy-Colorectal (FACT-C) quality of life 

instrument. Qual Life Res 1999;8(3):181-95. 

11. Chapman JR, Larson DW, Wolff BG, et al. Ileal 

pouch-anal anastomosis: does age at the time of 

surgery affect outcome? Arch Surg 2005;140(6):534-

9; discussion 539-40. 

12. Ayanian JZ , Zaslavsky AM , Guadagnoli E , et al. 

Patient perceptions of quality of care for colorectal 

cancer by race, ethnicity, and language. J Clin Oncol 

2005;23(27):TBD. 

13. Schwenk W, Neudecker J, Haase O, et al. Comparison 

of EORTC quality of life core questionnaire (EORTCQLQ-C30) and gastrointestinal quality of life index 

(GIQLI) in patients undergoing elective colorectal 

cancer resection. Int J Colorectal Dis 2004;19(6):554-

60. 

14. Gupta D, Lammersfeld CA, Vashi PG, et al. 

Prognostic significance of Subjective Global 

Assessment (SGA) in advanced colorectal cancer. Eur 

J Clin Nutr 2005;59(1):35-40. 

15. Temple LK, Bacik J, Savatta SG, et al. The 

development of a validated instrument to evaluate 

bowel function after sphincter-preserving surgery for 

rectal cancer. Dis Colon Rectum 2005;48(7):1353-65. 

16. Caffo O, Amichetti M, Romano M, et al. Evaluation 

of toxicity and quality of life using a diary card during 

postoperative radiotherapy for rectal cancer. Dis 

Colon Rectum 2002;45(4):459-65; discussion 465-7. 
B-1

Appendix B: Exact Search String 

Database: Ovid MEDLINE(R) <1966 to January Week 1 2005> 

Search Strategy: 

-------------------------------------------------------------------------------- 

1 exp colorectal neoplasms/ or colonic neoplasms/ (83,325) 

2 exp *"Quality of Health Care"/ (286,107) 

3 exp *"Patient care management"/ (162,883) 

4 exp *"Organization and Administration"/ (326,572) 

5 exp *"health care quality, access, and evaluation"/ (521,870) 

6 or/72,82,85,87 (810,834) 

7 95 or age factors/ or african americans/ or specialties, surgical/ (1,067,364) 

8 99 and 68 (5742) 

9 limit 100 to (human and english language and yr=1990-2005) (3644) 
C-1

Appendix C: List of Excluded Studies 

All excluded studies list below were reviewed in their full-text version. Following each 

reference, in italics, is the reason for exclusion. Reasons for exclusion signify only the 

usefulness of the articles for this study and are not intended as criticisms of the articles. 

Aaltonen LA, Sankila R, Mecklin JP, et al. A novel 

approach to estimate the proportion of hereditary 

nonpolyposis colorectal cancer of total colorectal cancer 

burden. Cancer Detect Prev 1994;18(1):57-63. Exclude 

non-U.S. data.

Abel U, Wollermann C. Methodological aspects of the 

evaluation of postoperative cancer surveillance. Part II: 

Efficacy. Clin Lab 2003;49(7-8):379-98. Exclude non-U.S. 

data. 

Abulafi AM, Williams NS. Local recurrence of colorectal 

cancer: the problem, mechanisms, management and 

adjuvant therapy. Br J Surg 1994;81(1):7-19. Exclude 

review. 

Achkar E. Colorectal cancer screening in primary care: the 

long and short of it. Am J Gastroenterol 2004;99(5):837-8. 

Exclude falls outside Task Order scope. 

Adair A, Bennis M, Clifton MA. Referral guidelines for 

colorectal cancer--do they work? Ann R Coll Surg Engl 

2004;86(2):144. Exclude non-U.S. data. 

Adam IJ, Mohamdee MO. Role of Circumferential Margin 

Involvement in the Local Recurrence of Rectal Cancer. 

Lancet 1994;344(8924). Exclude non-U.S. data. 

Adams J, White M, Barker G, et al. Are there socioeconomic inequalities in age of resection of colorectal 

cancer in people with HNPCC? Fam Cancer 2003;2(3-

4):169-73. Exclude non-U.S. data.

Adams T, Dufton R, Lamb C, et al. Hospital secondments 

of community nurses to improve stoma care. Br J 

Community Nurs 2003;8(12):539-43. Exclude non-U.S. 

data.

Adler GS, Shatto A. Screening for osteoporosis and colon 

cancer under Medicare. Health Care Financ Rev 

2002;23(4):189-200. Exclude falls outside Task Order 

scope.

Afridi SA, Jafri SF, Marshall JB. Do gastroenterologists 

themselves follow the American Cancer Society 

recommendations for colorectal cancer screening?. Am J 

Gastroenterol 1994;89(12):2184-7. Exclude falls outside 

Task Order scope. 

Agrez MV, Coory M, Cockburn J. Population screening for 

colorectal carcinoma with fecal-occult blood testing: are we 

sufficiently informed? Cancer 1998;82(10):1803-7. 

Exclude falls outside Task Order scope. 

Ahlquist DA. Fecal occult blood testing for colorectal 

cancer. Can we afford to do this? Gastroenterol Clin North 

Am 1997;26(1):41-55. Exclude review. 

Ahlquist DA, Shuber AP. Stool screening for colorectal 

cancer: evolution from occult blood to molecular markers. 

Clin Chim Acta 2002;315(1-2):157-68. Exclude falls 

outside Task Order scope. 

Ahmad NA, Hoops TC. The role of colonoscopy for 

screening of colorectal cancer. Semin Roentgenol 

2000;35(4):404-8. Exclude review. 

Ahnen DJ, Lynch KL. Colorectal cancer screening in 

average - and high-risk groups. Adv Intern Med 

2001;46:77-106. Exclude review. 

Aihara H, Kawamura YJ, Konishi F. Reduced medical 

costs achieved after elective oncological colorectal surgery 

by early feeding and fewer scheduled examinations. J 

Gastroenterol 2003;38(8):747-50. Exclude non-U.S. data. 

Akerley WL 3rd, Moritz TE, Ryan LS, et al. Racial 

comparison of outcomes of male Department of Veterans 

Affairs patients with lung and colon cancer. Arch Intern 

Med 1993;153(14):1681-8. Exclude no explicit QM. 

Alexander D, Chatla C, Funkhouser E, et al. Postsurgical 

disparity in survival between African Americans and 

Caucasians with colonic adenocarcinoma. Cancer 

2004;101(1):66-76. Exclude survival only. 

Alexander D, Jhala N, Chatla C, et al. High-grade tumor 

differentiation is an indicator of poor prognosis in African 

Americans with colonic adenocarcinomas. Cancer 

2005;103(10):2163-70. Exclude review. 

Alici S, Aykan NF, Sakar B, et al. Colorectal cancer in 

young patients: characteristics and outcome. Tohoku J Exp 

Med 2003;199(2):85-93. Exclude non-U.S. data. 

Allal AS, Mermillod B, Roth AD, et al. The impact of 

treatment factors on local control in T2-T3 anal carcinomas 

treated by radiotherapy with or without chemotherapy. 

Cancer 1997;79(12):2329-35. Exclude no explicit QM. 
C-2

Allenmersh TG, Earlam S. Quality of Life and Survival 

with Continuous Heaptic-Artery Floxuridine Infusion for 

Colorectal Liver Metastases. Lancet 1994;344(8932). 

Exclude no explicit QM.

American Society of Clinical Oncology. Outcomes of 

cancer treatment for technology assessment and cancer 

treatment guidelines: outcomes for technology assessments 

and guideline development. Available at: 

http:www.asco.org/ac/. Accessed: April 6, 2005. 1995. 

Exclude review.

Andreoni B, Biffi R, Torri V. Follow-up in colorectal 

cancer. Tumori 1997;83(1 Suppl):S19-20. Exclude nonU.S. data.

Ang YS, Macaleenan N, Mahmud N, et al. The yield of 

colonoscopy in average-risk patients with non-specific 

colonic symptoms. Eur J Gastroenterol Hepatol 

2002;14(10):1073-7. Exclude non-U.S. data

Angelo N, Dreyer L. Colorectal carcinoma--a new threat to 

black patients? A retrospective analysis of colorectal 

carcinoma received by the Institute for Pathology, 

University of Pretoria. S Afr Med J 2001;91(8):689-93. 

Exclude non-U.S. data 

Anonymous. Recommendations for the reporting of 

resected large intestinal carcinomas. Association of 

Directors of Anatomic and Surgical Pathology. Mod Pathol 

1996;9(1):73-6. Exclude review.

Anonymous. Screening for colorectal cancer--United 

States, 1992-1993, and new guidelines. MMWR Morb 

Mortal Wkly Rep 1996;45(5):107-10. Exclude falls outside 

Task Order scope.

Anonymous. AHCPR releases evidence report on 

colorectal cancer screening. Oncology (Huntingt). 

1997;11(3):342-4. Exclude review.

Anonymous. Clinical pathways for general surgeons: 

elective colectomy for cancer. Am Surg 1998;64(5):481-4. 

Exclude no explicit QM.

Anonymous. Colonoscopy in the screening and 

surveillance of individuals at increased risk for colorectal 

cancer. American Society for Gastrointestinal Endoscopy. 

Gastrointest Endosc 1998;48(6):676-8. Exclude review.

Anonymous. Guidelines for colorectal cancer screening and 

surveillance. Gastrointest Endosc 2000;51(6):777-82. 

Exclude review.

Anonymous. Surgery for colorectal cancer in elderly 

patients: a systematic review. Colorectal Cancer 

Collaborative Group. Lancet 2000;356(9234):968-74. 

Exclude review.

Anonymous. Colonoscopy benchmarks suggest less 

invasive tack not always most productive. Data Strateg 

Benchmarks 2001;5(8):116-9, 113. Exclude review.

Anonymous. Summaries for patients. Choice of statistical 

analysis can change the results of studies of the relationship 

between hospital and surgeon volume and outcomes of 

cancer surgery. Ann Intern Med 2003;139(8):I44. Exclude 

no explicit QM.

Anonymous. Summaries for patients. Relationships 

between the amount of colon cancer surgery a hospital 

performs and patient outcomes. Ann Intern Med 

2003;139(8):I28. Exclude no explicit QM.

Anonymous. Summaries for patients. Using risks to tailor 

screening for colorectal cancer. Ann Intern Med 

2003;139(12):I10. Exclude non-U.S. data.

Anonymous. Summaries for patients. What is the best way 

to test for colorectal cancer?[original report in Ann Intern 

Med. 2005 Jan 18;142(2):81-5; PMID: 15657155]. Ann 

Intern Med 2005;142(2):I23. Exclude wrong population

Anthony T, Fleming JB, Bieligk SC, et al. Postoperative 

colorectal cancer surveillance. J Am Coll Surg 

2000;190(6):737-49. Exclude review.

Anthony T, Jones C, Antoine J, et al. The effect of 

treatment for colorectal cancer on long-term health-related 

quality of life. Ann Surg Oncol 2001;8(1):44-9. Exclude 

review.

Anthony T, Simmang C, Hyman N, et al. Practice 

parameters for the surveillance and follow-up of patients 

with colon and rectal cancer. Dis Colon Rectum 

2004;47(6):807-17. Exclude review.

Aparicio T, Desrame J, Lecomte T, et al. Oxaliplatin- or 

irinotecan-based chemotherapy for metastatic colorectal 

cancer in the elderly. Br J Cancer 2003;89(8):1439-44. 

Exclude non-U.S. data.

Apolone G, Cavuto S, Torri V, et al. Effectiveness of 

adjuvant fluorouracil in elderly colon cancer patients: the 

internal and external validity of nonrandomized research 

design. J Clin Oncol 2003;21(9):1892; author reply 1893. 

Exclude non-U.S. data. 

Apolone G, Filiberti A, Cifani S, et al. Evaluation of the 

EORTC QLQ-C30 questionnaire: a comparison with SF-36 

Health Survey in a cohort of Italian long-survival cancer 

patients. Ann Oncol 1998;9(5):549-57. Exclude non-U.S. 

data. 

Apse KA, Biesecker BB, Giardiello FM, et al. Perceptions 

of genetic discrimination among at-risk relatives of 

colorectal cancer patients. Genet Med 2004;6(6):510-6. 

Exclude falls outside Task Order scope.
C-3

Archer SB, Burnett RJ, Flesch LV, et al. Implementation of 

a clinical pathway decreases length of stay and hospital 

charges for patients undergoing total colectomy and ileal 

pouch/anal anastomosis. Surgery 1997;122(4):699-703; 

discussion 703-5. Exclude falls outside Task Order scope. 

Arenal JJ, Benito C, Concejo MP, et al. Colorectal 

resection and primary anastomosis in patients aged 70 and 

older: prospective study. Eur J Surg 1999;165(6):593-7. 

Exclude non-U.S. data.

Armbrecht U, Manus B, Bragelmann R, et al. Acceptance 

and outcome of endoscopic screening for colonic neoplasia 

in patients undergoing clinical rehabilitation for 

gastrointestinal and metabolic diseases. NZ Gastroenterol 

1994;32(1):3-7. Exclude non-U.S. data.

Arnaud JP, Schloegel M, Ollier JC, et al. Colorectal cancer 

in patients over 80 years of age. Dis Colon Rectum 

1991;34(10):896-8. Exclude non-U.S. data. 

Arndt V, Merx H, Stegmaier C, et al. Quality of life in 

patients with colorectal cancer 1 year after diagnosis 

compared with the general population: a population-based 

study.[erratum appears in J Clin Oncol 2005 Jan 

1;23(1):248]. J Clin Oncol 2004;22(23):4829-36. Exclude 

non-U.S. data

Arora A, Singh P. Colonoscopy in patients 80 years of age 

and older is safe, with high success rate and diagnostic 

yield. Gastrointest Endosc 2004;60(3):408-13. Exclude 

non-U.S. data.

Ashley OS, Nadel M, Ransohoff DF. Achieving quality in 

flexible sigmoidoscopy screening for colorectal cancer. Am 

J Med 2001;111(8):643-53. Exclude review.

Atkin W, Rogers P, Cardwell C, et al. Wide variation in 

adenoma detection rates at screening flexible 

sigmoidoscopy. Gastroenterology 2004;126(5):1247-56. 

Exclude non-U.S. data.

Audisio RA, Filiberti A, Geraghty JG, et al. Personalized 

surgery for rectal tumours: the patient's opinion counts. 

Support Care Cancer 1997;5(1):17-21. Exclude no explicit 

QM. 

Audisio RA, Robertson C. Colorectal cancer follow-up: 

perspectives for future studies. Eur J Surg Oncol 

2000;26(4):329-37. Exclude review.

Austin D, Russell EM. Is there ageism in oncology? Scott 

Med J 2003;48(1):17-20. Exclude non-U.S. data

Ayanian JZ , Zaslavsky AM , Guadagnoli E , et al. Patient 

perceptions of quality of care for colorectal cancer by race, 

ethnicity, and language. Jl Clin Oncol 2005;23(27):TBD. 

Exclude QOL.

Baier M, Calonge N, Cutter G, et al. Validity of selfreported colorectal cancer screening behavior. Cancer 

Epidemiol Biomarkers Prev 2000;9(2):229-32. Exclude 

falls outside Task Order scope.

Baig MK, Marks CG. Referral guidelines for colorectal 

cancer: a threat or a challenge? Hosp Med 2000;61(7):452-

3. Exclude review.

Bailes J. Final report of the national initiative on cancer 

care quality. American Society of Clinical Oncology. Slide 

presentation. Available at: http://www.asco.org/ac/. 

Accessed: April 6, 2005. 2004. Exclude review.

Bailey C, Corner J, Addington-Hall J, et al. Older patients' 

experiences of treatment for colorectal cancer: an analysis 

of functional status and service use. Eur J Cancer Care 

(Engl) 2004;13(5):483-93. Exclude non-U.S. data.

Bailey C, Corner J, Addington-Hall J, et al. Treatment 

decisions in older patients with colorectal cancer: the role 

of age and multidimensional function. Eur J Cancer Care 

(Engl). 2003;12(3):257-62. Exclude non-U.S. data.

Baker D. Current surgical management of colorectal 

cancer. Nurs Clin North Am 2001;36(3):579-92, xi-xii. 

Exclude review.

Balducci L. Believing is seeing: a new parameter to study 

cancer prevention and cancer treatment in older individuals. 

Cancer 2004;100(5):881-2. Exclude no explicit QM.

Balducci L, Mowry K. Pharmacology and organ toxicity of 

chemotherapy in older patients. Oncology (Huntingt) 

1992;6(2 Suppl):62-8. Exclude review.

Ball JK, Elixhauser A. Treatment differences between 

blacks and whites with colorectal cancer. Med Care 

1996;34(9):970-84. Exclude no explicit QM. 

Bampton PA, Sandford JJ, Cole SR, et al. Interval faecal 

occult blood testing in a colonoscopy based screening 

programme detects additional pathology. Gut 

2005;54(6):803-6. Exclude non-U.S. data. 

Bampton PA, Sandford JJ, Young GP. Applying evidencebased guidelines improves use of colonoscopy resources in 

patients with a moderate risk of colorectal neoplasia. Med J 

Aust 2002;176(4):155-7. Exclude non-U.S. data.

Baquet CR, Commiskey P. Colorectal cancer epidemiology 

in minorities: a review. J Assoc Acad Minor Phys 

1999;10(3):51-8. Exclude falls outside Task Order scope.

Barkun AN, Jobin G, Cousineau G, et al. The Quebec 

Association of Gastroenterology position paper on 

colorectal cancer screening - 2003. Can J Gastroenterol 

2004;18(8):509-19. Exclude non-U.S. data.
C-4

Baron PL, Sigurdson ER. Local surgical treatment of rectal 

cancer. Cancer Invest 1995;13(6):612-6. Exclude review.

Barrier A, Ferro L, Houry S, et al. Rectal cancer surgery in 

patients more than 80 years of age. Am J Surg 

2003;185(1):54-7. Exclude non-U.S. data.

Barry MJ. Fecal occult blood testing for colorectal cancer: 

a perspective. Ann Oncol 2002;13(1):61-4. Exclude review. 

Barton J, Langdale L, Cummins J, et al. The utility of 

routine preoperative computed tomography scanning in the 

management of veterans with colon cancer. The American 

Journal of Surgery 2002;183:499-503. Exclude no explicit 

QM. 

Bartram C. National audit of the sensitivity of doublecontrast barium enema for colorectal carcinoma, using 

control charts. Clin Radiol 2005;60(5):555-7. Exclude nonU.S. data.

Basdanis G, Papadopoulos VN, Michalopoulos A, et al. 

Colorectal cancer in patients over 70 years of age: 

determinants of outcome. Tech Coloproctol 2004;8 (Suppl 

1): s112-5. Exclude non-U.S. data.

Bast R, Ravdin P, Hayes D, et al. 2000 Update of 

Recommendations for the Use of Tumor Markers in Breast 

and Colorectal Cancer: Clinical Practice Guidelines of the 

American Society of Clinical Oncology. J Clin Oncol 

2001;19(6):1865-78. Exclude no explicit QM.

Beart RW. National Cancer Database: Annual review of 

patient care, 1992. Atlanta, GA. American Cancer Society 

1992;40-6. Exclude falls outside Task Order scope.

Beattie GC, McAdam TK, Elliott S, et al. Improvement in 

quality of colorectal cancer pathology reporting with a 

standardized proforma--a comparative study. Colorectal 

Dis 2003;5(6):558-62. Exclude non-U.S. data.

Bech K, Kronborg O, Fenger C. Adenomas and 

hyperplastic polyps in screening studies. World J Surg 

1991;15(1):7-13. Exclude non-U.S. data.

Beck NE, Tomlinson IP, Homfray T, et al. Genetic testing 

is important in families with a history suggestive of 

hereditary non-polyposis colorectal cancer even if the 

Amsterdam criteria are not fulfilled. Br J Surg 

1997;84(2):233-7. Exclude falls outside Task Order scope.

Begg C, Cramer L, Hoskins W, et al. Impact of Hospital 

Volume on Operative Mortality for Major Cancer Surgery. 

JAMA 1998;280(20):1747-51. Exclude falls outside Task 

Order scope. 

Belkacemi Y, Berger C, Poortmans P, et al. Management of 

primary anal canal adenocarcinoma: a large retrospective 

study from the Rare Cancer Network. Int J Radiat Oncol 

Biol Phys 2003;56(5):1274-83. Exclude no explicit QM. 

Benson A, Desch C, Flynn P, et al. 2000 Update of 

American Cancer Society of Clinical Oncology Colorectal 

Cancer Surveillance Guidelines. J Clin Oncol 

2000;18(20):3586-8. Exclude review.

Benson AB 3rd, Schrag D, Somerfield MR, et al. American 

Society of Clinical Oncology recommendations on adjuvant 

chemotherapy for stage II colon cancer. J Clin Oncol 

2004;22(16):3408-19. Exclude no explicit QM.

Berberoglu U. Prognostic significance of total lymph node 

number in patients with T1-4N0M0 colorectal cancer. 

Hepatogastroenterology 2004;51(60):1689-93. Exclude 

falls outside Task Order scope.

Berg A. Promoting early detection tests for colorectal 

carcinoma and adenomatous polyps. Cancer 

2003;97(9):2348. Exclude non-U.S. data.

Bergamaschi R. Surgical strategies in the treatment of 

colorectal cancer. Eur J Surg Acta Chir Suppl 

1995;(575):1-22. Exclude non-U.S. data.

Berlin L. Failure to order. AJR Am J Roentgenol 

1997;168(1):19-21. Exclude no explicit QM.

Berry DP, Clarke P, Hardcastle JD, et al. Randomized trial 

of the addition of flexible sigmoidoscopy to faecal occult 

blood testing for colorectal neoplasia population screening. 

Br J Surg 1997;84(9):1274-6. Exclude non-U.S. data.

Bertagnolli M, Miedema B, Redston M, et al. Sentinel node 

staging of resectable colon cancer: results of a multicenter 

study. Ann Surg 2004;240(4):624-8; discussion 628-30. 

Exclude review.

Biganzoli L, Aapro M. Adjuvant chemotherapy in the 

elderly. Ann Oncol 2003;14 (Suppl 3):iii-3. Exclude nonU.S. data.

Biondo S, Marti-Rague J, Kreisler E, et al. A prospective 

study of outcomes of emergency and elective surgeries for 

complicated colonic cancer. Am J Surg 2005;189(4):377-

83. Exclude non-U.S. data.

Birbeck KF, Macklin CP, Tiffin NJ, et al. Rates of 

circumferential resection margin involvement vary between 

surgeons and predict outcomes in rectal cancer surgery. 

Ann Surg 2002;235(4):449-57. Exclude non-U.S. data.

Birkmeyer J, Siewers A, Finlayson E, et al. Hospital 

volume and surgical mortality in the United States. N Engl 

J Med 2002;346(15):1128-37. Exclude falls outside Task 

Order scope.

Birkmeyer JD, Dimick J, Birkmeyer N. Measuring the 

quality of surgical care: structure, process or outcomes. J 

Am Coll Surg 2004;198(4):626-32. Exclude review.
C-5

Bjerkeset T, Edna TH. Rectal cancer: the influence of type 

of operation on local recurrence and survival. Eur J Surg 

1996;162(8):643-8. Exclude non-U.S. data.

Bleiker EM, Menko FH, Taal BG, et al. Experience of 

discharge from colonoscopy of mutation negative HNPCC 

family members. J Med Genet 2003;40(5):e55. Exclude 

non-U.S. data.

Blijham GH, Labianca R, Bleiberg H. Should patients with 

advanced colorectal cancer be treated with chemotherapy? 

Eur J Cancer 1997;33(6):815-24. Exclude non-U.S. data. 

Blom J, Liden A, Jeppsson B, et al. Compliance and 

findings in a Swedish population screened for colorectal 

cancer with sigmoidoscopy. Eur J Surg Oncol 

2002;28(8):827-31. Exclude non-U.S. data.

Blumberg D, Paty PB, Picon AI, et al. Stage I rectal cancer: 

identification of high-risk patients. J Am Coll Surg 

1998;186(5):574-9; discussion 579-80. Exclude no explicit 

QM.

Bond JH. Colorectal cancer and polyps: clinical decisions 

for screening, early diagnosis, and surveillance of high-risk 

groups. Compr Ther 1996;22(2):100-6. Exclude no explicit 

QM. 

Bond JH. Colorectal surveillance for neoplasia: an 

overview. Gastrointest Endosc 1999;49(3 Pt 2):S35-40. 

Exclude review. 

Bond JH. Colorectal cancer update. Prevention, screening, 

treatment, and surveillance for high-risk groups. Med Clin 

North Am 2000;84(5):1163-82, viii. Exclude review.

Bond JH. Update on colorectal polyps: management and 

follow-up surveillance. Endocsopy 2003;35(8):S35-40. 

Exclude review.

Borie F, Combescure C, Daures JP, et al. Costeffectiveness of two follow-up strategies for curative 

resection of colorectal cancer: comparative study using a 

Markov model. World J Surg 2004;28(6):563-9. Exclude 

non-U.S. data.

Borner M, Scheithauer W, Twelves C, et al. Answering 

patients' needs: oral alternatives to intravenous therapy. 

Oncologist 2001;6 (Suppl 4):12-6. Exclude review.

Borras M, Sanchez-Hernandez A, Navarro M, et al. 

Compliance, satisfaction, and quality of life of patients wth 

colorectal cancer receiving home chemotherapy or 

outpatient treatment: a randomized controlled trial. Br Med 

J 2001;322(7):1-5. Exclude no explicit QM.

Bosset JF, Horiot JC. Adjuvant treatment in the curative 

management of rectal cancer: a critical review of the results 

of clinical randomised trials. Eur J Cancer 

1993;29A(5):770-4. Exclude review.

Bouchardy C, Queneau PE, Fioretta G, et al. Adjuvant 

chemotherapy for colon carcinoma with positive lymph 

nodes: use and benefit in routine health care practice. Br J 

Cancer 2001;85(9):1251-7. Exclude non-U.S. data.

Bouhier K, Maurel J, Lefevre H, et al. Changing practices 

for diagnosis and treatment of colorectal cancer in 

Calvados: 1990-1999. Gastroenterol Clin Biol 

2004;28(4):371-6. Exclude non-U.S. data.

Bowman KF, Deimling GT, Smerglia V, et al. Appraisal of 

the cancer experience by older long-term survivors. 

Psychooncology 2003;12(3):226-38. Exclude no explicit 

QM.

Bozzetti F. Operative risk of rectal cancer surgery in the 

elderly. Am J Surg 2004;187(1):150-1; author reply 151. 

Exclude no explicit QM.

Bradley CJ, Given CW, Roberts C. Late stage cancers in a 

Medicaid-insured population. Med Care 2003;41(6):722-8. 

Exclude data sources.

Brady AP, Stevenson GW, Stevenson I. Colorectal cancer 

overlooked at barium enema examination and colonoscopy: 

a continuing perceptual problem. Radiology 

1994;192(2):373-8. Exclude non-U.S. data. 

Branston LK, Greening S, Newcombe RG, et al. The 

implementation of guidelines and computerised forms 

improves the completeness of cancer pathology reporting. 

The CROPS project: a randomised controlled trial in 

pathology. Eur J Cancer 2002;38(6):764-72. Exclude nonU.S. data. 

Brawley OW, Freeman HP. Race and outcomes: is this the 

end of the beginning for minority health research? J Natl 

Cancer Inst 1999;91(22):1908-9. Exclude no explicit QM.

Bretthauer M, Skovlund E, Grotmol T, et al. Interendoscopist variation in polyp and neoplasia pick-up rates 

in flexible sigmoidoscopy screening for colorectal cancer. 

Scand J Gastroenterol 2003;38(12):1268-74. Exclude nonU.S. data.

Brewster D, Muir C, Crichton J. Registration of colorectal 

cancer in Scotland: an assessment of data accuracy based 

on review of medical records. Public Health 

1995;109(4):285-92. Exclude non-U.S. data.

Brown BW, Brauner C, Minnotte MC. Noncancer deaths in 

white adult cancer patients. J Natl Cancer Inst 

1993;85(12):979-87. Exclude falls outside Task Order 

scope.

Brown ML, Riley GF, Potosky AL, et al. Obtaining longterm disease specific costs of care: application to Medicare 

enrollees diagnosed with colorectal cancer. Med Care 

1999;37(12):1249-59. Exclude no explicit QM. 
C-6

Brown MT, Frelick RW, Topham A. Trends in colorectal 

cancer in Delaware. Del Med J 1992;64(1):33-5. Exclude 

no explicit QM. 

Brown SC, Abraham JS, Walsh S, et al. Risk factors and 

operative mortality in surgery for colorectal cancer. Ann R 

Coll Surg Engl 1991;73(5):269-72. Exclude non-U.S. data.

Bruinvels DJ, de Brauw LM, Kievit J, et al. Attitudes 

towards detection and management of hepatic metastases of 

colorectal origin: a second look. HPB Surg 1994;8(2):115-

22. Exclude non-U.S. data.

Bujko K, Nowacki MP, Nasierowska-Guttmejer A, et al. 

Sphincter preservation following preoperative radiotherapy 

for rectal cancer: report of a randomised trial comparing 

short-term radiotherapy vs. conventionally fractionated 

radiochemotherapy. Radiother Oncol 2004;72(1):15-24. 

Exclude non-U.S. data

Bull AD, Biffin AH, Mella J, et al. Colorectal cancer 

pathology reporting: a regional audit. J Clin Pathol 

1997;50(2):138-42. Exclude non-U.S. data.

Bulow S, Bulow C, Nielsen TF, et al. Centralized 

registration, prophylactic examination, and treatment 

results in improved prognosis in familial adenomatous 

polyposis. Results from the Danish Polyposis Register. 

Scand J Gastroenterol 1995;30(10):989-93. Exclude nonU.S. data.

Bulow S, Burn J, Neale K, et al. The establishment of a 

polyposis register. Int J Colorectal Dis 1993;8(1):34-8. 

Exclude non-U.S. data.

Bumpers HL, Williams WL Jr, Hassett JM Jr, et al. Colon 

cancer in blacks: age-related presentation and survival 

within a similar socioeconomic group. J Natl Med Assoc 

1994;86(3):216-8. Exclude survival only. 

Burke C. The Cleveland Clinic Disease Management 

Project 2002. Exclude no explicit QM.

Burn J, Chapman P, Delhanty J, et al. The UK Northern 

region genetic register for familial adenomatous polyposis 

coli: use of age of onset, congenital hypertrophy of the 

retinal pigment epithelium, and DNA markers in risk 

calculations. J Med Genet 1991;28(5):289-96. Exclude nonU.S. data

Burnand B, Vader JP, Froehlich F, et al. Reliability of 

panel-based guidelines for colonoscopy: an international 

comparison. Gastrointest Endosc 1998;47(2):162-6. 

Exclude no explicit QM.

Burton A. Will you survive surgery for colorectal cancer? 

Lancet Oncol 2004;5(1):7. Exclude non-U.S. data.

Butler Nattinger A, Schapira MM, Warren JL, et al. 

Methodological issues in the use of administrative claims 

data to study surveillance after cancer treatment. Med Care 

2002;40(8 Suppl):IV-69-74. Exclude data sources.

Byers T, Levin B, Rothenberger D, et al. American Cancer 

Society guidelines for screening and surveillance for early 

detection of colorectal polyps and cancer: update 1997. 

American Cancer Society Detection and Treatment 

Advisory Group on Colorectal Cancer. CA Cancer J Clin 

1997;47(3):154-60. Exclude falls outside Task Order scope.

Byles JE, Sanson-Fisher RW, Redman S, et al. Early 

detection of colorectal cancer: a profile of current practice. 

Cancer Detect Prev 1992;16(4):245-52. Exclude non-U.S. 

data.

Caffo O, Amichetti M, Romano M, et al. Evaluation of 

toxicity and quality of life using a diary card during 

postoperative radiotherapy for rectal cancer. Dis Colon 

Rectum 2002;45(4):459-65; discussion 465-7. Exclude 

QOL

Cairns S, Scholefield JH. Guidelines for colorectal cancer 

screening in high risk groups. Gut 2002;51 (Suppl 5):V1-2. 

Exclude non-U.S. data.

Campbell NC, Elliott AM, Sharp L, et al. Impact of 

deprivation and rural residence on treatment of colorectal 

and lung cancer. Br J Cancer 2002;87(6):585-90. Exclude 

non-U.S. data.

Campbell RJ, Ferrante JM, Gonzalez EC, et al. Predictors 

of advanced stage colorectal cancer diagnosis: results of a 

population-based study. Cancer Detect Prev 

2001;25(5):430-8. Exclude no explicit QM.

Canadian Task Force on Preventive Health Care. Colorectal 

cancer screening. Recommendation statement from the 

Canadian Task Force on Preventive Health Care. CMAJ 

2001;165(2):206-8. Exclude review. 

Cancer Quality Council of Ontario. Improving the 

management of colorectal cancer. Summary of the cancer 

quality council of ontario signature event on colorectal 

cancer. Available at: 

http://www.cancer.care.on.ca/pdf/CQCOColorectal 

Proceedings.pdf. Accessed: April 6, 2005. 2003;1-30. 

Exclude review.

Cancer Quality Council of Ontario. Strengthening the 

Quality of Cancer Services in Ontario:Synopsis. Cancer 

Quality Council of Ontario. Available at 

http://www.cancercare.on.ca/pdf/synopsis_CQCO_Report.

pdf. Accessed: May 9, 2005. 2003. Exclude review.

Canon CL, Smith JK, Morgan DE, et al. Double reading of 

barium enemas: is it necessary? AJR Am J Roentgenol 

2003;181(6):1607-10. Exclude wrong population. 
C-7

Caplin S, Cerottini JP, Bosman FT, et al. For patients with 

Dukes' B (TNM Stage II) colorectal carcinoma, 

examination of six or fewer lymph nodes is related to poor 

prognosis. Cancer 1998;83(4):666-72. Exclude non-U.S. 

data. 

Capocaccia R, Gatta G, Roazzi P, et al. The EUROCARE-3 

database: methodology of data collection, standardisation, 

quality control and statistical analysis. Ann Oncol 

2003;13(Suppl 5):v14-27. Exclude non-U.S. data.

Carmody BJ, Otchy DP. Learning curve of transrectal 

ultrasound. Dis Colon Rectum 2000;43(2):193-7. Exclude 

no explicit QM.

Carstensen B, Soll-Johanning H, Villadsen E, et al. 

Familial aggregation of colorectal cancer in the general 

population. Int J Cancer 1996;68(4):428-35. Exclude nonU.S. data.

Carter S, Winslet M. Delay in the presentation of colorectal 

carcinoma: a review of causation. Int J Colorectal Dis 

1998;13(1):27-31. Exclude non-U.S. data.

Carter SD. The surgeon as a risk factor. Adv Surg 

2002;36:141-65. Exclude non-U.S. data.

Cascinu S, Del Ferro E, Grianti C, et al. S-phase fraction 

and tumor aneuploidy in colorectal carcinoma of young 

patients. Cancer 1996;78(9):1857-60. Exclude no explicit 

QM.

Castells A, Bessa X, Daniels M, et al. Value of 

postoperative surveillance after radical surgery for 

colorectal cancer: results of a cohort study. Dis Colon 

Rectum 1998;41(6):714-23; discussion 723-4. Exclude nonU.S. data.

Castiglione G, Zappa M, Grazzini G, et al. Cost analysis in 

a population based screening programme for colorectal 

cancer: comparison of immunochemical and guaiac faecal 

occult blood testing. J Med Screen 1997;4(3):142-6. 

Exclude non-U.S. data.

Chak A, Cooper GS, Blades EW, et al. Prospective 

assessment of colonoscopic intubation skills in trainees. 

Gastrointest Endosc 1996;44(1):54-7. Exclude no explicit 

QM. 

Chapman JR, Larson DW, Wolff BG, et al. Ileal pouchanal anastomosis: does age at the time of surgery affect 

outcome? Arch Surg 2005;140(6):534-9; discussion 539-

40. Exclude QOL.

Charnley N, Choudhury A, Chesser P, et al. Effective 

treatment of anal cancer in the elderly with low-dose 

chemoradiotherapy. Br J Cancer 2005;92(7):1221-5. 

Exclude falls outside Task Order scope

Chen HS, Sheen-Chen SM. Obstruction and perforation in 

colorectal adenocarcinoma: an analysis of prognosis and 

current trends. Surgery 2000;127(4):370-6. Exclude nonU.S. data.

Chen VW, Wu XC, Ahmed MN, et al. Cancer of the colon 

and rectum in Louisiana. J La State Med Soc 

1999;151(4):189-97. Exclude survival only. 

Cheson BD. Clinical trials referral resource. Adjuvant 

therapy for colon cancer. Oncology (Huntingt) 

1990;4(4):67-9, 74, 79-84. Exclude no explicit QM.

Cheson BD. Clinical trials referral resource. Rectal cancer. 

Oncology (Huntingt) 1990;4(6):58-71. Exclude no explicit 

QM.

Cheson BD. Clinical trials referral resource. Oncology 

(Huntingt) 1990;4(11):72. Exclude no explicit QM

Cheson BD. Clinical trials referral resource. Treatment of 

advanced colorectal cancer. Oncology (Huntingt) 

1991;5(4):61-2, 65-6, 68 passim. Exclude no explicit QM. 

Cheson BD, Kaplan RS, Phillips PH. Clinical trials referral 

resource. Colon cancer. Oncology (Huntingt) 1994;8(8):43-

4. Exclude no explicit QM.

Cheson BD, Ungerleider RS. Clinical trials referral 

resource.[erratum appears in Oncology (Huntingt) 1993 

Apr;7(4):38]. Oncology (Huntingt) 1993;7(3):55-6, 59. 

Exclude no explicit QM.

Child PW, Yan TD, Perera DS, et al. Surveillance-detected 

hepatic metastases from colorectal cancer had a survival 

advantage in seven-year follow-up. Dis Colon Rectum 

2005;48(4):744-8. Exclude non-U.S. data.

Chin SM. Clinical pathway for colorectal procedures. Hosp 

Case Manag 2002;10(12):183-5. Exclude non-U.S. data.

Christensen M, Katballe N, Wikman F, et al. Antibodybased screening for hereditary nonpolyposis colorectal 

carcinoma compared with microsatellite analysis and 

sequencing. Cancer 2002;95(11):2422-30. Exclude falls 

outside Task Order scope 

Chu E. Lessons learned from the Saltz and Mayo Clinic 

regimens. Clin Colorectal Cancer 2002;2(1):13. Exclude no 

data source val. or rel. testing.

Chung YF, Eu KW, Machin D, et al. Young age is not a 

poor prognostic marker in colorectal cancer. Br J Surg 

1998;85(9):1255-9. Exclude non-U.S. data. 

Church J, Kiringoda R, LaGuardia L. Inherited colorectal 

cancer registries in the United States. Dis Colon Rectum 

2004;47(5):674-8. Exclude review.
C-8

Church JM. The ileal pouch-anal anastomosis in 

challenging patients: stretching the limits. Ann N Z J Surg 

1995;65(2):104-6. Exclude non-U.S. data. 

Church JM. Clinical significance of small colorectal 

polyps. Dis Colon Rectum 2004;47(4):481-5. Exclude no 

explicit QM. 

Church TR, Ederer F, Mandel JS. Fecal occult blood 

screening in the Minnesota study: sensitivity of the 

screening test. J Natl Cancer Inst 1997;89(19):1440-8. 

Exclude falls outside Task Order scope. 

Cianchi F, Palomba A, Boddi V, et al. Lymph node 

recovery from colorectal tumor specimens: 

recommendation for a minimum number of lymph nodes to 

be examined. World J Surg 2002;26(3):384-9. Exclude 

non-U.S. data. 

Ciferri E, Bondanza GS, Municino O, et al. Colorectal 

cancer metastases: surgical indications and multimodal 

approach. Hepatogastroenterology 2003;50(54):1836-46. 

Exclude non-U.S. data. 

Clemmesen T, Sprechler M. Recording of patients with 

colorectal cancer on a database: results and advantages. Eur 

J Surg 1994;160(3):175-8. Exclude non-U.S. data. 

Clipp EC, Carver EH, Pollak KI, et al. Age-related 

vulnerabilities of older adults with colon adenomas: 

evidence from Project Prevent. Cancer 2004;100(5):1085-

94. Exclude no explicit QM.

Cockburn J, Paul C, Tzelepis F, et al. Screening for bowel 

cancer among NSW adults with varying levels of risk: a 

community survey. Aust N Z J Public Health 

2002;26(3):236-41. Exclude non-U.S. data 

Cockburn J, Paul C, Tzelepis F, et al. Delay in seeking 

advice for symptoms that potentially indicate bowel cancer. 

Am J Health Behav 2003;27(4):401-7. Exclude non-U.S. 

data. 

Cole SR, Young GP, Byrne D, et al. Participation in 

screening for colorectal cancer based on a faecal occult 

blood test is improved by endorsement by the primary care 

practitioner. J Med Screen 2002;9(4):147-52. Exclude nonU.S. data. 

Collett JA, Olynyk JK, Platell CF. Flexible sigmoidoscopy 

screening for colorectal cancer in average-risk people: 

update of a community-based project. Med J Aust 

2000;173(9):463-6. Exclude non-U.S. data.

Collopy BT. Quantity, qualifications and quality in surgery. 

Med J Aust 1999;171(6):306-7. Exclude review.

Colombet I, Xu Y, Jaulent MC, et al. A generic 

computerized method for estimate of familial risks. Proc 

AMIA Symp 2002;175-9. Exclude non-U.S. data.

Compton CC, Fielding LP, Burgart LJ, et al. Prognostic 

factors in colorectal cancer. College of American 

Pathologists Consensus Statement 1999. Arch Pathol Lab 

Med 2000;124(7):979-94. Exclude no explicit QM. 

Compton CC, Henson DE, Hutter RV, et al. Updated 

protocol for the examination of specimens removed from 

patients with colorectal carcinoma. A basis for checklists. 

Arch Pathol Lab Med 1997;121(12):1247-54. Exclude no 

explicit QM. 

Conley BA, Kaplan RS, Arbuck SG. National Cancer 

Institute clinical trials program in colorectal cancer. Cancer 

Chemother Pharmacol 1998;42 Suppl:S75-9. Exclude no 

explicit QM.

Connell W. PRO: Endoscopic surveillance minimizes the 

risk of cancer. Am J Gastroenterol 2004;99(9):1631-3. 

Exclude no explicit QM.

Connor S, Frizelle FA, Bagshaw PF. Follow-up after 

attempted curative surgery for colorectal cancer; postal 

survey of New Zealand surgeons' practice. N Z Med J 

2001;114(1129):151-3. Exclude non-U.S. data.

Cooney A, Donnelly NJ, Gattellari M, et al. Surgeons' 

views about colorectal cancer screening before and after 

national guidelines. Med J Aust 2002;177(5):278-9. 

Exclude non-U.S. data. 

Cooper GS, Fortinsky RH, Hapke R, et al. Primary care 

physician recommendations for colorectal cancer screening. 

Patient and practitioner factors. Arch Intern Med 

1997;157(17):1946-50. Exclude falls outside Task Order 

scope. 

Cooper GS, Fortinsky RH, Hapke R, et al. Factors 

associated with the use of flexible sigmoidoscopy as a 

screening test for the detection of colorectal carcinoma by 

primary care physicians. Cancer 1998;82(8):1476-81. 

Exclude falls outside Task Order scope.

Cooper GS, Koroukian SM. Racial disparities in the use of 

and indications for colorectal procedures in Medicare 

beneficiaries. Cancer 2004;100(2):418-24. Exclude no 

explicit QM. 

Cooper GS, Yuan Z, Stange KC, et al. Use of Medicare 

claims data to measure county-level variations in the 

incidence of colorectal carcinoma. Cancer 1998;83(4):673-

8. Exclude no explicit QM. 

Cordice JW Jr, Johnson H Jr. Anatomic distribution of 

colonic cancers in middle-class black Americans. J Natl 

Med Assoc 1991;83(8):730-2. Exclude no explicit QM. 

Courtier R, Casamitjana M, Macia F, et al. Participation in 

a colorectal cancer screening programme: influence of the 

method of contacting the target population. Eur J Cancer 

Prev 2002;11(3):209-13. Exclude non-U.S. data.
C-9

Cravo ML, Fidalgo PO, Lage PA, et al. Validation and 

simplification of Bethesda guidelines for identifying 

apparently sporadic forms of colorectal carcinoma with 

microsatellite instability. Cancer 1999;85(4):779-85. 

Exclude non-U.S. data.

Crawley MT, Booth A. Reducing dose at barium enema: 

radiographers do it digitally. Br J Radiol 2002;75(896):652-

6. 

Exclude non-U.S. data.

Crespi M, Stigliano V, Assisi D. Current trends in 

screening and secondary prevention of colorectal cancer. 

Hepatogastroenterology 2001;48(42):1635-40. Exclude 

non-U.S. data.

Cress RD, Zaslavsky AM, West DW, et al. Completeness 

of information on adjuvant therapies for colorectal cancer 

in population-based cancer registries. Med Care 2003; 

41(9):1006-12. Exclude no explicit QM. 

Cripps NP, Heald RJ. Family based colorectal cancer 

screening in a district hospital. Gut 1996;38(3):421-5. 

Exclude non-U.S. data.

Cross SS, Bull AD. Is the informational content of 

histopathological reports increasing?. J Clin Pathol 

1992;45(2):179-80. Exclude non-U.S. data. 

Cross SS, Burton JL, Dube AK, et al. Offline telepathology 

diagnosis of colorectal polyps: a study of interobserver 

agreement and comparison with glass slide diagnoses. J 

Clin Pathol 2002;55(4):305-8. Exclude non-U.S. data.

Cross SS, Feeley KM, Angel CA. The effect of four 

interventions on the informational content of 

histopathology reports of resected colorectal carcinomas. J 

Clin Pathol 1998;51(6):481-2. Exclude non-U.S. data.

Culpan DG, Mitchell AJ, Hughes S, et al. Double contrast 

barium enema sensitivity: a comparison of studies by 

radiographers and radiologists. Clin Radiol 

2002;57(7):604-7. Exclude non-U.S. data.

Cumbo A, Agre P, Dougherty J, et al. Online cancer patient 

education: evaluating usability and content. Cancer Pract 

2002;10(3):155-61. Exclude no explicit QM.

Curb JD, McTiernan A, Heckbert SR, et al. Outcomes 

ascertainment and adjudication methods in the Women's 

Health Initiative. Ann Epidemiol 2003;13(9 Suppl):S122-8. 

Exclude no explicit QM.

Cusack JC, Giacco GG, Cleary K, et al. Survival factors in 

186 patients younger than 40 years old with colorectal 

adenocarcinoma. J Am Coll Surg 1996;183(2):105-12. 

Exclude no explicit QM.

Dachman AH, Zalis ME. Quality and consistency in CT 

colonography and research reporting. Radiology 

2004;230(2):319-23. Exclude falls outside Task Order 

scope. 

Daher M. Consensus conference. Colorectal cancer: 

prevention, screening and management. J Med Liban 

1998;46(5):268-79. Exclude no explicit QM.

Dahlberg M, Stenborg A, Pahlman L, et al. Costeffectiveness of preoperative radiotherapy in rectal cancer: 

results from the Swedish Rectal Cancer Trial. Int J Radiat 

Oncol Biol Phys 2002;54(3):654-60. Exclude non-U.S. 

data. 

Damhuis RA, Wereldsma JC, Wiggers T. The influence of 

age on resection rates and postoperative mortality in 6457 

patients with colorectal cancer. Int J Colorectal Dis 

1996;11(1):45-8. Exclude non-U.S. data.

Damhuis RA, Wiggers T, Wereldsma JC. Association 

between age and local recurrence of rectal cancer: results 

from a retrospective study of 902 patients. Int J Colorectal 

Dis 1997;12(4):235-9. Exclude non-U.S. data. 

Daneker GW Jr, Staley CA. Surgical resection of hepatic 

tumors-patient selection and technical considerations. 

Cancer Treat Res 2001;109(229-46). Exclude review.

Dangelica MI, Shoup MC, Nissan A. Randomized clinical 

trials in advanced and metastatic colorectal carcinoma. 

Surg Oncol Clin N Am 2002;11(1):173-91, ix. Exclude 

review.

Davidson-Homewood J, Norman A, Kuchler T, et al. 

Development of a disease specific questionnaire to 

supplement a generic tool for QoL in colorectal cancer. 

Psychooncology 2003;12(7):675-85. Exclude non-U.S. 

data.

Davidson P, Cockburn J, Daly J, et al. Patient-centered 

needs assessment: rationale for a psychometric measure for 

assessing needs in heart failure. J Cardiovasc Nurs 

2004;19(3):164-71. Exclude no explicit QM.

Davies RJ, Collins CD, Vickery CJ, et al. Reduction in the 

proportion of patients with colorectal cancer presenting as 

an emergency following the introduction of fast-track 

flexible sigmoidoscopy: a three-year prospective 

observational study. Colorectal Dis 2004;6(4):265-7. 

Exclude non-U.S. data. 

Davis TC, Dolan NC, Ferreira MR, et al. The role of 

inadequate health literacy skills in colorectal cancer 

screening. Cancer Invest 2001;19(2):193-200. Exclude falls 

outside Task Order scope.
C-10

Dawkins FW, Laing AE, Smoot DT, et al. The impact of 

health insurance on an African-American population with 

colorectal cancer. J Natl Med Assoc 1995;87(4):301-3. 

Exclude survival only. 

de Bosset V, Froehlich F, Rey JP, et al. Do explicit 

appropriateness criteria enhance the diagnostic yield of 

colonoscopy? Endocsopy 2002;34(5):360-8. Exclude nonU.S. data.

de Haes JC, van Knippenberg FC, Neijt JP. Measuring 

psychological and physical distress in cancer patients: 

structure and application of the Rotterdam Symptom 

Checklist. Br J Cancer 1990;62(6):1034-8. Exclude nonU.S. data. 

de Vos tot Nederveen Cappel WH, Nagengast FM, 

Griffioen G, et al. Surveillance for hereditary nonpolyposis 

colorectal cancer: a long-term study on 114 families. Dis 

Colon Rectum 2002;45(12):1588-94. Exclude non-U.S. 

data.

Debinski HS, Love S, Spigelman AD, et al. Colorectal 

polyp counts and cancer risk in familial adenomatous 

polyposis. Gastroenterology 1996;110(4):1028-30. Exclude 

falls outside Task Order scope.

Debnath D, Dielehner N, Gunning KA. Guidelines, 

compliance, and effectiveness: a 12 months' audit in an 

acute district general healthcare trust on the two week rule 

for suspected colorectal cancer. Postgrad Med J 

2002;78(926):748-51. Exclude non-U.S. data. 

DeCosse J, Cennerazzo W. Quality - of - life management 

of patients with colorectal cancer. CA Cancer J Clin 

1997;47:198-206. Exclude review.

Dehni N, Schlegel D, Tiret E, et al. Effects of aging on the 

functional outcome of coloanal anastomosis with colonic Jpouch. Am J Surg 1998;175(3):209-12. Exclude non-U.S. 

data. 

Delaney CP, Fazio VW, Remzi FH, et al. Prospective, agerelated analysis of surgical results, functional outcome, and 

quality of life after ileal pouch-anal anastomosis. Ann Surg 

2003;238(2):221-8. Exclude no explicit QM.

Delea TE, Vera-Llonch M, Edelsberg JS, et al. The 

incidence and cost of hospitalization for 5-FU toxicity 

among Medicare beneficiaries with metastatic colorectal 

cancer. Value Health 2002;5(1):35-43. Exclude falls 

outside Task Order scope. 

Delgado S, Lacy AM, Garcia Valdecasas JC, et al. Could 

age be an indication for laparoscopic colectomy in 

colorectal cancer? Surg Endosc 2000;14(1):22-6. Exclude 

no explicit QM.

Demartines N, von Flue MO, Harder FH. Transanal 

endoscopic microsurgical excision of rectal tumors: 

indications and results. World J Surg 2001;25(7):870-5. 

Exclude no explicit QM. 

Demers RY, Severson RK, Schottenfeld D, et al. Incidence 

of colorectal adenocarcinoma by anatomic subsite. An 

epidemiologic study of time trends and racial differences in 

the Detroit, Michigan area. Cancer 1997;79(3):441-7. 

Exclude no explicit QM.

Demetriades H, Kanellos I, Vasiliadis K, et al. Ageassociated prognosis following curative resection for 

colorectal cancer. Tech Coloproctol 2004;8 Suppl 1:s144-

6. Exclude non-U.S. data. 

Demols A, Van Laethem JL. Adjuvant chemotherapy for 

colorectal cancer. Curr Gastroenterol Rep 2002;4(5):420-6. 

Exclude review.

Denis B, Weiss AM, Peter A, et al. Quality assurance and 

gastrointestinal endoscopy: an audit of 500 colonoscopic 

procedures. Gastroenterol Clin Biol 2004;28(12):1245-55. 

Exclude non-U.S. data.

Dent OF, Goulston KJ, Tennant CC, et al. Rectal bleeding. 

Patient delay in presentation. Dis Colon Rectum 

1990;33(10):851-7. Exclude non-U.S. data. 

Desch C, Benson A, Smith T, et al. Recommended 

Colorectal Cancer Surveillance Guidelines by the American 

Society of Clinical Oncology. J Clin Oncol 

1999;17(4):1312-21. Exclude review. 

Dharma-Wardene MW, de Gara C, Au HJ, et al. Ageism in 

rectal carcinoma? Treatment and outcome variations. Int J 

Gastrointest Cancer 2002;32(2-3):129-38. Exclude nonU.S. data.

DiBetta E, D'Hoore A, Filez L, et al. Sphincter saving 

rectum resection is the standard procedure for low rectal 

cancer. Int. J Colorectal Dis 2003;18:463-9. Exclude nonU.S. data. 

Dietrich AJ, Sox CH, Tosteson TD, et al. Durability of 

improved physician early detection of cancer after 

conclusion of intervention support. Cancer Epidemiol 

Biomarkers Prev 1994;3(4):335-40. Exclude no explicit 

QM.

Dignam JJ, Colangelo L, Tian W, et al. Outcomes among 

African-Americans and Caucasians in colon cancer 

adjuvant therapy trials: findings from the National Surgical 

Adjuvant Breast and Bowel Project. J Natl Cancer Inst 

1999;91(22):1933-40. Exclude no explicit QM. 

DiPalma AM, Barnes SE, DiPalma JA. Patient participation 

in colon cancer screening programs. South Med J 

1998;91(4):342-4. Exclude falls outside Task Order scope.
C-11

Dittrich K, Armbruster C, Hoffer F, et al. Nd:YAG laser 

treatment of colorectal malignancies: an experience of 4 1/2 

years. Lasers Surg Med 1992;12(2):199-203. Exclude nonU.S. data. 

Dolan JG, Bordley DR. Using the analytic hierarchy 

process (AHP) to develop and disseminate guidelines. QRB 

Qual Rev Bull 1992;18(12):440-7. Exclude falls outside 

Task Order scope. 

Dominitz JA, Provenzale D. Patient preferences and quality 

of life associated with colorectal cancer screening. Am J 

Gastroenterol 1997;92(12):2171-8. Exclude falls outside 

Task Order scope

Donaldson GW, Moinpour CM. Individual differences in 

quality-of-life treatment response. Med Care 2002;40(6 

Suppl):III39-53. Exclude no explicit QM.

Dorrance HR, Docherty GM, O'Dwyer PJ. Effect of 

surgeon specialty interest on patient outcome after 

potentially curative colorectal cancer surgery. Dis Colon 

Rectum 2000;43(4):492-8. Exclude non-U.S. data. 

Dowdall JF, Maguire D, McAnena OJ. Adjuvant therapy 

for rectal cancer cannot be based on the results of other 

surgeons (Br J Surg 2002; 89: 946-947). Br J Surg 

2003;90(5):594. Exclude non-U.S. data

Duffy MJ, van Dalen A, Haglund C, et al. Clinical utility of 

biochemical markers in colorectal cancer: European Group 

on Tumour Markers (EGTM) guidelines. Eur J Cancer 

2003;39(6):718-27. Exclude non-U.S. data.

Durkalski VL, Palesch YY, Pineau BC, et al. The virtual 

colonoscopy study: a large multicenter clinical trial 

designed to compare two diagnostic screening procedures. 

Control Clin Trials 2002;23(5):570-83. Exclude no explicit 

QM.

Duxbury MS, Brodribb AJ, Oppong FC, et al. Management 

of colorectal cancer: variations in practice in one hospital. 

Eur J Surg Oncol 2003;29(4):400-2. Exclude non-U.S. 

data. 

Dwyer K. Breaks in care in the ambulatory care setting: the 

risks to patient safety.[erratum appears in Int J Qual Health 

Care 2003 Feb;15(1):99]. Int J Qual Health Care 

2002;14(3):259-60. Exclude review

Eaden JA, Ward BA, Mayberry JF. How 

gastroenterologists screen for colonic cancer in ulcerative 

colitis: an analysis of performance. Gastrointest Endosc 

2000;51(2):123-8. Exclude non-U.S. data. 

Earle CC, Grunfeld E, Coyle D, et al. Cancer physicians' 

attitudes toward colorectal cancer follow-up. Ann Oncol 

2003;14(3):400-5. Exclude non-U.S. data

Earle CC, Neville BA. Under use of necessary care among 

cancer survivors. Cancer 2004;101(8):1712-9. Exclude no 

explicit QM. 

Easson AM, Cotterchio M, Crosby JA, et al. A populationbased study of the extent of surgical resection of potentially 

curable colon cancer. Ann Surg Oncol 2002;9(4):380-7. 

Exclude non-U.S. data. 

Eccersley AJ, Wilson EM, Makris A, et al. Referral 

guidelines for colorectal cancer--do they work? Ann R Coll 

Surg Engl 2003;85(2):107-10. Exclude non-U.S. data. 

Eckardt VF, Schmitt T, Kanzler G, et al. Does scant 

hematochezia necessitate the performance of total 

colonoscopy? Endocsopy 2002;34(8):599-603. Exclude 

non-U.S. data

Eddy DM. Clinical decision making: from theory to 

practice. Designing a practice policy. Standards, guidelines, 

and options. JAMA 1990;263(22):3077, 3081, 3084. 

Exclude falls outside Task Order scope

Ederer F, Church TR, Mandel JS. Fecal occult blood 

screening in the Minnesota study: role of chance detection 

of lesions. J Natl Cancer Inst 1997;89(19):1423-8. Exclude 

falls outside Task Order scope.

Ederer F, Geisser MS, Mongin SJ, et al. Colorectal cancer 

deaths as determined by expert committee and from death 

certificate: a comparison. The Minnesota Study. J Clin 

Epidemiol 1999;52(5):447-52. Exclude data sources.

Edna TH, Bjerkeset T. Perioperative blood transfusions 

reduce long-term survival following surgery for colorectal 

cancer. Dis Colon Rectum 1998;41(4):451-9. Exclude nonU.S. data.

Edna TH, Bjerkeset T. Association between transfusion of 

stored blood and infective bacterial complications after 

resection for colorectal cancer. Eur J Surg 

1998;164(6):449-56. Exclude non-U.S. data.

Edwards JT, Wood CJ, Mendelson RM, et al. Extracolonic 

findings at virtual colonoscopy: implications for screening 

programs. Am J Gastroenterol 2001;96(10):3009-12. 

Exclude non-U.S. data.

Efficace F, Bottomley A, Blazeby JM. Randomized clinical 

trial comparing quality of life after straight and pouch 

coloanal reconstruction (Br J Surg 2003; 89: 1108-1117). 

Br J Surg 2003;90(5):594-5. Exclude non-U.S. data.

Ekelund G. The colorectal surgeon's scientific and 

economic responsibility. Dis Colon Rectum 

1993;36(8):728-30. Exclude non-U.S. data.
C-12

Elliott BA, Renier CM, Haller IV, et al. Health-related 

quality of life (HRQoL) in patients with cancer and other 

concurrent illnesses. Qual Life Res 2004;13(2):457-62. 

Exclude QOL. 

Ellison GL, Warren JL, Knopf KB, et al. Racial differences 

in the receipt of bowel surveillance following potentially 

curative colorectal cancer surgery. Health Serv Res 

2003;38(6 Pt 2):1885-903. Exclude results combined 

across modalities. 

Elsaleh H, Iacopetta B. Re: Pooled analysis of fluorouracilbased adjuvant therapy of stage II and III colon cancer: 

who benefits and by how much? J Clin Oncol 

2005;23(3):653-4; author reply 654-5. Exclude no explicit 

QM.

Enblad G, Enblad P, Adami HO, et al. Relationship 

between age and survival in cancer of the colon and rectum 

with special reference to patients less than 40 years of age. 

Br J Surg 1990;77(6):611-6. Exclude non-U.S. data 

Engel AF, Oomen JL, Eijsbouts QA, et al. Nationwide 

decline in annual numbers of abdomino-perineal resections: 

effect of a successful national trial?. Colorectal Dis 

2003;5(2):180-4. Exclude non-U.S. data 

Engel J, Kerr J, Schlesinger-Raab A, et al. Comparison of 

breast and rectal cancer patients' quality of life: results of a 

four year prospective field study. Eur J Cancer Care (Engl) 

2003;12(3):215-23. Exclude non-U.S. data.

Engstrom PF, Benson AB 3rd, Cohen A, et al. NCCN 

Colorectal Cancer Practice Guidelines. The National 

Comprehensive Cancer Network. Oncology (Huntingt) 

1996;10(11 Suppl):140-75. Exclude no explicit QM

Enker WE. Potency, cure, and local control in the operative 

treatment of rectal cancer. Arch Surg 1992;127(12):1396-

401; discussion 1402. Exclude no explicit QM

Erban S, Zapka J, Puleo E, et al. Colorectal cancer 

screening in Massachusetts: measuring compliance with 

current guidelines. Eff Clin Pract 2001;4(1):10-7. Exclude 

falls outside Task Order scope.

Essiet A, Iwatt AR. Surgical management of large bowel 

cancer 1983-1988, University of Calabar Teaching Hospital 

audit. Cent Afr J Med 1994;40(1):8-13. Exclude non-U.S. 

data.

Extermann M. Measuring comorbidity in older cancer 

patients. Eur J Cancer 2000;36:453-71. Exclude no explicit 

QM.

Fallahzadeh H, Mays ET. Preexisting disease as a predictor 

of the outcome of colectomy. Am J Surg 1991;162(5):497-

8. Exclude no explicit QM.

Faris I. Guidelines for the management of colonic cancer. 

Ann N Z J Surg 1999;69(6):410. Exclude non-U.S. data. 

Farmer KC, Penfold C, Millar JL, et al. Rectal cancer in 

Victoria in 1994: patterns of reported management. ANZ J 

Surg 2002;72(4):265-70. Exclude non-U.S. data.

Farraye FA, Horton K, Hersey H, et al. Screening flexible 

sigmoidoscopy using an upper endoscope is better tolerated 

by women. Am J Gastroenterol 2004;99(6):1074-80. 

Exclude falls outside Task Order scope

Farrington SM, Lin-Goerke J, Ling J, et al. Systematic 

analysis of hMSH2 and hMLH1 in young colon cancer 

patients and controls. Am J Hum Genet 1998;63(3):749-59. 

Exclude falls outside Task Order scope.

Fata F, Mirza A, Craig G, et al. Efficacy and toxicity of 

adjuvant chemotherapy in elderly patients with colon 

carcinoma: a 10-year experience of the Geisinger Medical 

Center. Cancer 2002;94(7):1931-8. Exclude no explicit 

QM.

Fazio VW, Tekkis PP, Remzi F, et al. Assessment of 

operative risk in colorectal cancer surgery: the Cleveland 

Clinic Foundation colorectal cancer model. Dis Colon 

Rectum 2004;47(12):2015-24. Exclude QOL

Feliu J, Escudero P, Llosa F, et al. Capecitabine as first-line 

treatment for patients older than 70 years with metastatic 

colorectal cancer: an oncopaz cooperative group study. J 

Clin Oncol 2005;23(13):3104-11. Exclude non-U.S. data.

Feliu J, Mel JR, Camps C, et al. Raltitrexed in the treatment 

of elderly patients with advanced colorectal cancer: an 

active and low toxicity regimen. Eur J Cancer 

2002;38(9):1204-11. Exclude non-U.S. data.

Fernandez E, La Vecchia C, D'Avanzo B, et al. Oral 

contraceptives, hormone replacement therapy and the risk 

of colorectal cancer. Br J Cancer 1996;73(11):1431-5. 

Exclude non-U.S. data.

Fernandez E, Linares A, Alonso JL, et al. Colonoscopic 

findings in patients with lower gastrointestinal bleeding 

send to a hospital for their study. Value of clinical data in 

predicting normal or pathological findings. Rev Esp 

Enferm Dig 1996;88(1):16-25. Exclude no explicit QM.

Ferraris R, Senore C, Fracchia M, et al. Predictive value of 

rectal bleeding for distal colonic neoplastic lesions in a 

screened population. Eur J Cancer 2004;40(2):245-52. 

Exclude non-U.S. data.

Fielding LP, Arsenault PA, Chapuis PH, et al. 

Clinicopathological staging for colorectal cancer: an 

International Documentation System (IDS) and an 

International Comprehensive Anatomical Terminology 

(ICAT). J Gastroenterol Hepatol 1991;6(4):325-44. 

Exclude non-U.S. data
C-13

Finn-Faivre C, Maurel J, Benhamiche AM, et al. Evidence 

of improving survival of patients with rectal cancer in 

france: a population based study. Gut 1999;44(3):377-81. 

Exclude non-U.S. data.

Fitzgerald SD, Longo WE, Daniel GL, et al. Advanced 

colorectal neoplasia in the high-risk elderly patient: is 

surgical resection justified?. Dis Colon Rectum 

1993;36(2):161-6. Exclude no explicit QM.

Fitzpatrick RJ. Closing the gap in trust: to refer out or not?. 

HMO Pract 1997;11(3):137-8. Exclude review.

Fleming BB, Petzel RA, Performance Measurement 

Workgroup. Pitfalls of converting practice guidelines into 

quality measures. JAMA 2004;292(11):1301-2; author 

reply 1302. Exclude review.

Fludger S, Turner AM, Harvey RF, et al. Controlled 

prospective study of faecal occult blood screening for 

colorectal cancer in Bury, black pudding capital of the 

world. BMJ 2002;325(7378):1444-5. Exclude falls outside 

Task Order scope

Foley EF, Schoetz DJ Jr, Roberts PL, et al. Rediversion 

after ileal pouch-anal anastomosis. Causes of failures and 

predictors of subsequent pouch salvage. Dis Colon Rectum 

1995;38(8):793-8. Exclude falls outside Task Order scope.

Folkesson J, Nilsson J, Pahlman L, et al. The circular 

stapling device as a risk factor for anastomotic leakage. 

Colorectal Dis 2004;6(4):275-9. Exclude non-U.S. data. 

Folprecht G, Cunningham D, Ross P, et al. Efficacy of 5-

fluorouracil-based chemotherapy in elderly patients with 

metastatic colorectal cancer: a pooled analysis of clinical 

trials. Ann Oncol 2004;15(9):1330-8. Exclude non-U.S. 

data

Fong Y, Blumgart LH, Fortner JG, et al. Pancreatic or liver 

resection for malignancy is safe and effective for the 

elderly. Ann Surg 1995;222(4):426-34; discussion 434-7. 

Exclude survival only.

Fong Y, Fortner J, Sun RL, et al. Clinical score for 

predicting recurrence after hepatic resection for metastatic 

colorectal cancer: analysis of 1001 consecutive cases. Ann 

Surg 1999;230(3):309-18; discussion 318-21. 

Exclude review.

Foutch PG, Mai H, Pardy K, et al. Flexible sigmoidoscopy 

may be ineffective for secondary prevention of colorectal 

cancer in asymptomatic, average-risk men. Dig Dis Sci 

1991;36(7):924-8. Exclude falls outside Task Order scope.

Freeman H. The Meaning of Race in ScienceConsiderations for Cancer Research. Cancer 82(1):219-25. 

Exclude no explicit QM.

Freeman HP, Alshafie TA. Colorectal carcinoma in poor 

blacks. Cancer 2002;94(9):2327-32. Exclude falls outside 

Task Order scope. 

Frontin K. Colon cancer: update on treatment. Are outcome 

disparities real? Ethn Dis 2003;13(3 Suppl 3):S3-42-5. 

Exclude QOL.

Fu CG, Tominaga O, Nagawa H, et al. Role of p53 and 

p21/WAF1 detection in patient selection for preoperative 

radiotherapy in rectal cancer patients. Dis Colon Rectum 

1998;41(1):68-74. Exclude non-U.S. data.

Fujita S, Akasu T, Moriya Y. Resection of synchronous 

liver metastases from colorectal cancer. Jpn J Clin Oncol 

2000;30(1):7-11. Exclude non-U.S. data.

Funkhouser E, Cole P. Declining mortality rates for cancer 

of the rectum in the United States: 1940-1985. Cancer 

1992;70(11):2597-601. Exclude no explicit QM.

Gafa L, Amendola P, Dardanoni G, et al. Survival of 

patients with cancer of the stomach, colon and rectum, lung 

and breast in Ragusa, Sicily. Tumori 1991;77(1):7-11. 

Exclude non-U.S. data

Galandiuk S, Wieand HS, Moertel CG, et al. Patterns of 

recurrence after curative resection of carcinoma of the 

colon and rectum. Surg Gynecol Obstet 1992;174(1):27-32. 

Exclude no explicit QM.

Garbus JE, Potenti F, Wexner SD. Current controversies in 

pouch surgery. South Med J 2003;96(1):32-6. Exclude 

review. 

Garcia-Granero E, Marti-Obiol R, Gomez-Barbadillo J, et 

al. Impact of surgeon organization and specialization in 

rectal cancer outcome. Colorectal Dis 2001;3:179-84. 

Exclude non-U.S. data. 

Gattellari M, Ward J, Solomon M. Implementing guidelines 

about colorectal cancer: a national survey of target groups. 

ANZ J Surg 2001;71(3):147-53. Exclude non-U.S. data.

Gayowski TJ, Iwatsuki S, Madariaga JR, et al. Experience 

in hepatic resection for metastatic colorectal cancer: 

analysis of clinical and pathologic risk factors. Surgery 

1994;116(4):703-10; discussion 710-1. Exclude falls 

outside Task Order scope.

Gazelle GS, Hunink MG, Kuntz KM, et al. Costeffectiveness of hepatic metastasectomy in patients with 

metastatic colorectal carcinoma: a state-transition Monte 

Carlo decision analysis. Ann Surg 2003;237(4):544-55. 

Exclude falls outside Task Order scope.

Gazelle GS, McMahon PM, Scholz FJ. Screening for 

colorectal cancer. Radiology 2000;215(2):327-35. 

Exclude review.
C-14

Gazzoli I, De Andreis C, Sirchia SM, et al. Molecular 

screening of families affected by familial adenomatous 

polyposis (FAP). J Med Screen 1996;3(4):195-9. Exclude 

falls outside Task Order scope,

Geddes M, Carli S, Ercolanelli M, et al. Colorectal, lung, 

and breast cancer care during the three years following the 

diagnosis: a population-based study. Tumori 

1996;82(3):210-4. Exclude non-U.S. data. 

Gelfand DW. Colorectal cancer. Screening strategies. 

Radiol Clin North Am 1997;35(2):431-8. Exclude review.

Giacomantonio CA, Temple WJ. Quality of cancer surgery: 

challenges and controversies. Surg Oncol Clin N Am 

2000;9(1):51-60, vii. Exclude review.

Giardiello FM, Brensinger JD, Petersen GM. AGA 

technical review on hereditary colorectal cancer and 

genetic testing. Gastroenterology 2001;121(1):198-213. 

Exclude falls outside Task Order scope

Giardiello FM, Brensinger JD, Petersen GM, et al. The use 

and interpretation of commercial APC gene testing for 

familial adenomatous polyposis. N Engl J Med 

1997;336(12):823-7. Exclude falls outside Task Order 

scope 

Gibson M, Byrd C, Pierce C, et al. Laparoscopic colon 

resections: a five-year retrospective review. Am Surg 

2000;66(3):245-8; discussion 248-9. Exclude falls outside 

Task Order scope 

Gimbel MI, Paty PB. A current perspective on local 

excision of rectal cancer. Clin Colorectal Cancer 

2004;4(1):26-35; discussion 36-7. Exclude review.

Giridharan S, Vakkalanka B, Anwar MS, et al. Multicentre 

prospective audit of surgical outcomes and acute 

complications following short course pre-operative 

radiotherapy for resectable rectal cancer. Colorectal Dis 

2005;7(1):43-6. Exclude falls outside Task Order scope 

Glanz K, Grove J, Lerman C, et al. Correlates of intentions 

to obtain genetic counseling and colorectal cancer gene 

testing among at-risk relatives from three ethnic groups. 

Cancer Epidemiol Biomarkers Prev 1999;8(4 Pt 2):329-36. 

Exclude falls outside Task Order scope 

Glaser F, Schlag P, Herfarth C. Endorectal sonography for 

the assessment of invasion of rectal tumors and lymph node 

involvement. Br J Surg 1990;77:883-7. Exclude no explicit 

QM

Glick SN, Ralls PW, Balfe DM, et al. Screening for 

colorectal cancer. American College of Radiology. ACR 

Appropriateness Criteria. Radiology 2000;215 Suppl(`231-

7). Exclude review. 

Glimelius B, Hoffma K, Fraf W, et al. Quality of Life 

during chemotherapy in patients with symptomatic 

advanced colorectal cancer. Cancer 73(3):556-62. Exclude 

non-U.S. data. 

Gluecker TM, Johnson CD, Harmsen WS, et al. Colorectal 

cancer screening with CT colonography, colonoscopy, and 

double-contrast barium enema examination: prospective 

assessment of patient perceptions and preferences. 

Radiology 2003;227(2):378-84. Exclude falls outside Task 

Order scope.

Goel V. Appraising organised screening programmes for 

testing for genetic susceptibility to cancer. BMJ 

2001;322(7295):1174-8. Exclude no explicit QM.

Goldberg HI, Neighbor WE, Cheadle AD, et al. A 

controlled time-series trial of clinical reminders: using 

computerized firm systems to make quality improvement 

research a routine part of mainstream practice. Health Serv 

Res 2000;34(7):1519-34. Exclude no explicit QM. 

Goldschmidt J. The appropriate use of chemotherapy in 

older adults with colon cancer. J Am Med Dir Assoc 

2004;5(1):47-9. Exclude no data source validity or 

reliability testing. 

Goldstein N. Lymph node recoveries from 2427 pT3 

colorectal resection specimen spanning 45 years: 

recommendations for a minimum number of recovered 

lymph nodes based on predictive probabilities. Am J 

SurgPathol 26(2):179-89. Exclude no explicit QM.

Gondal G, Grotmol T, Hofstad B, et al. Grading of distal 

colorectal adenomas as predictors for proximal colonic 

neoplasia and choice of endoscope in population screening: 

experience from the Norwegian Colorectal Cancer 

Prevention study (NORCCAP). Gut 2003;52(3):398-403. 

Exclude non-U.S. data.

Gondal G, Grotmol T, Hofstad B, et al. The Norwegian 

Colorectal Cancer Prevention (NORCCAP) screening 

study: baseline findings and implementations for clinical 

work-up in age groups 50-64 years. Scand J Gastroenterol 

2003;38(6):635-42. Exclude non-U.S. data. 

Gordon T., Bowman H. , Bass E. , et al. Complex 

gastrointestinal surgery: impact of provider experience on 

clinical and economic outcomes. J Am Coll Surg 

1999;189(1):46-56. Exclude falls outside Task Order scope.

Gosney M. Investigation and treatment of cancer in old 

age. Age Ageing 1998;27(4):417-9. Exclude review. 

Gowen GF. A new method of recording endoscopic and 

operative findings in colorectal neoplasia. Surg Endosc 

1992;6(5):239-43. Exclude no explicit QM. 
C-15

Graham RA, Wang S, Catalano PJ, et al. Postsurgical 

surveillance of colon cancer: preliminary cost analysis of 

physician examination, carcinoembryonic antigen testing, 

chest x-ray, and colonoscopy. Ann Surg 1998;228(1):59-

63. Exclude no explicit QM. 

Grassi A, Casale V, Fracasso P, et al. Medium-large polyps 

of the colon: a contribution for their clinical profile and a 

proper surveillance. J Exp Clin Cancer Res 

1997;16(3):313-9. Exclude non-U.S. data.

Green AJ. Practice parameters for the management of 

colonic cancer I: surgical issues. Recommendations of the 

Colorectal Surgical Society of Australia: comment. Ann N 

Z J Surg 2000;70(1):73-4. Exclude non-U.S. data.

Greene FL. Standard setting for laparoscopic resection of 

colorectal cancer. Surg Endosc 2001;15(2):109. Exclude 

review. 

Greene FL, Stewart AK, Norton HJ. A new TNM staging 

strategy for node-positive (stage III) colon cancer: an 

analysis of 50,042 patients. Ann Surg 2002;236(4):416-21; 

discussion 421. Exclude no explicit QM.

Greff M. Colorectal cancer screening in France: guidelines 

and professional reality. Endocsopy 1999;31(6):471. 

Exclude non-U.S. data. 

Griffin PM, Liff JM, Greenberg RS, et al. 

Adenocarcinomas of the colon and rectum in persons under 

40 years old. A population-based study. Gastroenterology 

1991;100(4):1033-40. Exclude no explicit QM.

Grilli R, Apolone G, Marsoni S, et al. The impact of patient 

management guidelines on the care of breast, colorectal, 

and ovarian cancer patients in Italy. Med Care 

1991;29(1):50-63. Exclude non-U.S. data.

Guadagnoli E, Weitberg A, Mor V, et al. The influence of 

patient age on the diagnosis and treatment of lung and 

colorectal cancer. Arch Intern Med 1990;150(7):1485-90. 

Exclude no explicit QM.

Guest JF, Ruiz FJ, Tang R, et al. Cost implications of postsurgical morbidity following blood transfusion in cancer 

patients undergoing elective colorectal resection: an 

evaluation in the US hospital setting. Curr Med Res Opin 

2005;21(3):447-55. Exclude falls outside Task Order scope.

Gunderson LL, Sargent DJ, Tepper JE, et al. Impact of T 

and N stage and treatment on survival and relapse in 

adjuvant rectal cancer: a pooled analysis. J Clin Oncol 

2004;22(10):1785-96. Exclude review.

Gunnarsson U. Quality assurance in surgical oncology. 

Colorectal cancer as an example. Eur J Surg Oncol 

2003;29(1):89-94. Exclude non-U.S. data.

Gunnarsson U, Seligsohn E, Jestin P, et al. Registration and 

validity of surgical complications in colorectal cancer 

surgery. Br J Surg 2003;90(4):454-9. Exclude non-U.S. 

data. 

Gunter Ollenschlager. Indicators and Guidelines: 

Companions on the Quality Path. Presented at the Joint 

CoE/ESQH/G-I-N Workship. Krakow, April 9, 2003. 

Available at: 

http://www.aezq.de/aezq/kooperationspartner_en/pdf/fogini

ndicatorkakau090403go.pdf. Accessed: May 9, 2005. 

Exclude review.

Gunther K, Radkow T, Reymond MA, et al. Angiogenesis 

and dendritic cell density are not correlated with 

metachronous distant metastasis in curatively operated 

rectal cancer. Int J Colorectal Dis 2003;18(4):300-8. 

Exclude no explicit QM.

Gupta AK, Melton LJ 3rd, Petersen GM, et al. Changing 

trends in the incidence, stage, survival, and screendetection of colorectal cancer: a population-based study. 

Clin Gastroenterol Hepatol 2005;3(2):150-8. Exclude no 

explicit QM. 

Gupta D, Lammersfeld CA, Burrows JL, et al. Bioelectrical 

impedance phase angle in clinical practice: implications for 

prognosis in advanced colorectal cancer. Am J Clin Nutr 

2004;80(6):1634-8. Exclude falls outside Task Order scope.

Gupta D, Lammersfeld CA, Vashi PG, et al. Prognostic 

significance of Subjective Global Assessment (SGA) in 

advanced colorectal cancer. Eur J Clin Nutr 2005;59(1):35-

40. Exclude QOL

Gyrd-Hansen D, Sogaard J, Kronborg O. Colorectal cancer 

screening: efficiency and effectiveness. Health Econ 

1998;7(1):9-20. Exclude falls outside Task Order scope.

Haidet P, Hamel MB, Davis RB, et al. Outcomes, 

preferences for resuscitation, and physician-patient 

communication among patients with metastatic colorectal 

cancer. SUPPORT Investigators. Study to Understand 

Prognoses and Preferences for Outcomes and Risks of 

Treatments. Am J Med 1998;105(3):222-9. Exclude falls 

outside Task Order scope. 

Halasz n. Letters to the Editor. Ann Surg 1999;229(3):443. 

Exclude review.

Hall HI, Van Den Eeden SK, Tolsma DD, et al. Testing for 

prostate and colorectal cancer: comparison of self-report 

and medical record audit. Prev Med 2004;39(1):27-35. 

Exclude falls outside Task Order scope.

Hanauer SB. Surveying surveillance: are 

gastroenterologists consistently inconsistent, inconsistently 

consistent, or poorly educated?. Gastrointest Endosc 

2000;51(2):240-2. Exclude non-U.S. data.
C-16

Harewood GC, Lieberman DA. Prevalence of advanced 

neoplasia at screening colonoscopy in men in private 

practice versus academic and Veterans Affairs medical 

centers. Am J Gastroenterol 2003;98(10):2312-6. Exclude 

falls outside Task Order scope.

Harriss DR, Blake JR. Computerised audit for colorectal 

cancer. Ann R Coll Surg Engl 1993;75(4):268-71. Exclude 

non-U.S. data.

Hart AR, Barone TL, Gay SP, et al. The effect on 

compliance of a health education leaflet in colorectal 

cancer screening in general practice in central England. J 

Epidemiol Community Health 1997;51(2):187-91. Exclude 

non-U.S. data.

Hart AR, Barone TL, Mayberry JF. Increasing compliance 

with colorectal cancer screening: the development of 

effective health education. Health Educ Res 

1997;12(2):171-80. Exclude falls outside Task Order scope.

Hart AR, Jestico B, Antill C, et al. Blood donor centres can 

effectively publicise cancer screening. Public Health 

1996;110(4):247-9. Exclude falls outside Task Order scope. 

Harvey EH, Young MR, Flanigan TL, et al. Complications 

are increased with the need for an abdominal-assisted 

Kraske procedure. Am Surg 2004;70(3):193-6; discussion 

197. Exclude falls outside Task Order scope.

Hasson MA, Fagerstrom RM, Kahane DC, et al. Design 

and evolution of the data management systems in the 

Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer 

Screening Trial. Control Clin Trials 2000;21(6 

Suppl):329S-48S. Exclude falls outside Task Order scope. 

Hatzell TA, Ricketts TC, Tropman SE, et al. Rural 

physicians' understanding of the state-of-the-art in breast, 

colon and rectum cancer treatment. Cancer Causes Control 

1999;10(4):261-7. Exclude no explicit QM.

Havranek E, Steiner J. Valuation of Health States in the US 

versus the UK: Two Measures Divided by a Common 

Language? Med Care 2005;43(3):201-2. Exclude review.

Hawley ST, Foxhall L, Vernon SW, et al. Colorectal cancer 

screening by primary care physicians in Texas: a ruralurban comparison. J Cancer Educ 2001;16(4):199-204. 

Exclude falls outside Task Order scope.

Hawley ST, Levin B, Vernon SW. Colorectal cancer 

screening by primary care physicians in two medical care 

organizations. Cancer Detect Prev 2001;25(3):309-18. 

Exclude falls outside Task Order scope.

Hawley ST, Vernon SW, Levin B, et al. Prevalence of 

colorectal cancer screening in a large medical organization. 

Cancer Epidemiol Biomarkers Prev 2004;13(2):314-9. 

Exclude falls outside Task Order scope. 

Hayes RB, Smith CO, Huang WY, et al. Whole blood 

cryopreservation in epidemiological studies. Cancer 

Epidemiol Biomarkers Prev 2002;11(11):1496-8. Exclude 

falls outside Task Order scope.

Heald RJ, Moran BJ, Ryall RD, et al. Rectal cancer: the 

Basingstoke experience of total mesorectal excision, 1978-

1997. Arch Surg 1998;133:894-9. Exclude non-U.S. data.

Hefferman N, Cella D, Webster K, et al. Measuring Health 

Related Quality of Life in Patients with Hepatobiliary 

Cancers: The Functional Assessment of Cancer TherapyHepatobiliary Questionnaire. J Clin Oncol 

2002;20(9):2229-39. Exclude no explicit QM.

Heldenberg E, Vishne TH, Onaka N, et al. Rectal cancer: 

the impact of a colorectal unit on the preservation of the 

anal sphincter. Isr Med Assoc J 2004;6(8):471-3. Exclude 

non-U.S. data.

Hellinger M. Colon and Rectal Cancer. American Society 

of Colon and Rectal Surgeons. Available at : 

http://www.fascrs.org/displaycommon.cfm?an=1&subarticl

enbr=109. Accessed Feburary 8, 2005. Exclude review.

Hemingway D. A novel indicator for surveillance 

colonoscopy following colorectal cancer resection. 

Colorectal Dis 2003;5(3):270; author reply 271. Exclude 

non-U.S. data.

Hemmings C, Jeffery M, Frizelle F. Changes in the 

pathology reporting of rectal cancer: is it time to adopt 

synoptic reporting? N Z Med J 2003;116(1178):U513. 

Exclude non-U.S. data.

Hendryx MS. The impact of cancer screening promotion by 

rural hospitals on cancer detection. Med Care 

1993;31(6):488-97. Exclude falls outside Task Order scope.

Hermanek P. Impact of surgeon's technique on outcome 

after treatment of rectal carcinoma. Dis Colon Rectum 

1999;42(5):559-62. Exclude non-U.S. data.

Hermanek P, Hermanek P, Hohenberger W, et al. The 

pathological assessment of mesorectal excision: 

implications for further treatment and quality management. 

Int J Colorectal Dis 2003;18(4):335-41. Exclude non-U.S. 

data.

Hermanek P, Hohenberger W. The importance of volume 

in colorectal cancer surgery. Eur J Surg Oncol 

1996;22(3):213-5. Exclude non-U.S. data. 

Hermanek P, Mansmann U, Staimmer DS, et al. The 

German experience: the surgeon as a prognostic factor in 

colon and rectal cancer surgery. Surg Oncol Clin N Am 

2000;9(1):33-49, vi. Exclude non-U.S. data.
C-17

Hermanek P, Wiebelt H, Staimmer D, et al. Prognostic 

factors of rectum carcinoma--experience of the German 

Multicentre Study SGCRC. German Study Group ColoRectal Carcinoma. Tumori 1995;81(3 Suppl):60-4. Exclude 

non-U.S. data. 

Hiatt RA, Klabunde C, Breen N, et al. Cancer screening 

practices from National Health Interview Surveys: past, 

present, and future. J Natl Cancer Inst 2002;94(24):1837-

46. Exclude falls outside Task Order scope. 

Hida J, Yoshifuji T, Tokoro T, et al. Long-term functional 

outcome of low anterior resection with colonic J-pouch 

reconstruction for rectal cancer in the elderly. Dis Colon 

Rectum 2004;47(9):1448-54. Exclude non-U.S. data. 

Hill GJ, Hill HZ. Four-way test of mortality: a useful tool 

for cancer education. J Cancer Educ 2001;16(4):185-7. 

Exclude no explicit QM. 

Hillner B, Smith T, Desch C. Hospital and Physician 

Volume or Specialization and Outcomes in Cancer 

Treatment: Importance in Quality of Cancer Care. J Clin 

Oncol 2000;18(11):2327-40. Exclude no explicit QM.

Hippisley-Cox J, Hardy C, Pringle M, et al. Are patients 

who present late with cancer registered with low referring 

practices?. Br J Gen Pract 1997;47(424):731-2. Exclude 

non-U.S. data.

Hixson LJ, Fennerty MB, Sampliner RE, et al. Prospective 

study of the frequency and size distribution of polyps 

missed by colonoscopy. J Natl Cancer Inst 

1990;82(22):1769-72. Exclude no explicit QM. 

Hocht S. Evaluating toxicity in neoadjuvant radiochemotherapy of rectal cancer. Int J Colorectal Dis 

2003;18(6):500-2. Exclude non-U.S. data.

Hodgson DC, Fuchs CS, Ayanian JZ. Impact of patient and 

provider characteristics on the treatment and outcomes of 

colorectal cancer. J Natl Cancer Inst 2001;93(7):501-15. 

Exclude review.

Hoff G, Blanchard J, Crespi M, et al. Public Awareness and 

Lobbying: Group 3 Report. ESGE/UEGF Colorectal 

Cancer--Public Awareness Campaign. The 

Public/Professional Interface Workshop: Oslo, Norway, 

June 20 - 22, 2003. Endocsopy 2004;36(4):359-61. Exclude 

non-U.S. data.

Hoff G, Sauar J, Vatn MH, et al. Polypectomy of adenomas 

in the prevention of colorectal cancer: 10 years' follow-up 

of the Telemark Polyp Study I. A prospective, controlled 

population study. Scand J Gastroenterol 1996;31(10):1006-

10. Exclude non-U.S. data. 

Hoffmann D, Moore J, Roder D. Trends in survival from 

colonic cancer: the impact of subspecialization. Ann N Z J 

Surg 1997;67(12):842-5. Exclude non-U.S. data.

Holm T, Johansson H, Cedermark B, et al. Influence of 

Hospital and Surgeon-related factors on outcome after 

treatment of rectal cancer with or without preoperative 

radiotherapy. Br J Surg 1997;84(5):657-63. Exclude nonU.S. data. 

Holman JR, Marshall RC, Jordan B, et al. Technical 

competency in flexible sigmoidoscopy. J Am Board Fam 

Pract 2001;14(6):424-9. Exclude falls outside Task Order 

scope.

Hopper W, Kyker KA, Rodney WM. Colonoscopy by a 

family physician: a 9-year experience of 1048 procedures. J 

Fam Pract 1996;43(6):561-6. Exclude falls outside Task 

Order scope. 

Houry S, Amenabar J, Rezvani A, et al. Should patients 

over 80 years old be operated on for colorectal or gastric 

cancer?. Hepatogastroenterology 1994;41(6):521-5. 

Exclude non-U.S. data.

Hristova L, Hakama M. Effect of screening for cancer in 

the Nordic countries on deaths, cost and quality of life up to 

the year 2017. Acta Oncol 1997;36(Suppl 9):1-60. Exclude 

non-U.S. data.

Hundahl SA. Surgical quality in trials of adjuvant cancer 

therapy. J Surg Oncol 2002;80(4):177-80. Exclude review.

Hunt LM, Rooney PS, Hardcastle JD, et al. Endoscopic 

screening of relatives of patients with colorectal cancer. 

Gut 1998;42(1):71-5. Exclude no explicit QM.

Isbister WH. Colorectal surgery in the elderly: an audit of 

surgery in octogenarians. Ann N Z J Surg 1997;67(8):557-

61. Exclude non-U.S. data.

Isbister WH. Audit of definitive colorectal surgery in 

patients with early and advanced colorectal cancer. ANZ J 

Surg 2002;72(4):271-4. Exclude non-U.S. data.

Isenberg G. Differences in the diagnostic criteria used by 

Japanese and Western pathologists to diagnose colorectal 

carcinoma. Gastrointest Endosc 1999;50(3):446-7; 

discussion 447-8. Exclude non-U.S. data.

Isenring E, Bauer J, Capra S. The scored Patient-generated 

Subjective Global Assessment (PG-SGA) and its 

association with quality of life in ambulatory patients 

receiving radiotherapy. Eur J Clin Nutr 2003;57(2):305-9. 

Exclude non-U.S. data.

Iversen LH, Thomsen GH, Thorlacius-Ussing O. Systemic 

coagulation activation and anastomotic leakage after 

colorectal cancer surgery. Dis Colon Rectum 

1999;42(1):56-65. Exclude non-U.S. data.
C-18

Iwama T, Mishima Y. Factors affecting the risk of rectal 

cancer following rectum-preserving surgery in patients with 

familial adenomatous polyposis. Dis Colon Rectum 

1994;37(10):1024-6. Exclude non-U.S. data.

Izbicki JR, Hosch SB, Knoefel WT, et al. Extended 

resections are beneficial for patients with locally advanced 

colorectal cancer. Dis Colon Rectum 1995;38(12):1251-6. 

Exclude non-U.S. data. 

Jackson C. Colorectal cancer screening--what the 

guidelines don't tell you. Aust Fam Physician 2001;30(1):8. 

Exclude review.

Jacobs LA. Health beliefs of first-degree relatives of 

individuals with colorectal cancer and participation in 

health maintenance visits: a population-based survey. 

Cancer Nurs 2002;25(4):251-65. Exclude wrong 

population.

James AS, Campbell MK, Hudson MA. Perceived barriers 

and benefits to colon cancer screening among African 

Americans in North Carolina: how does perception relate to 

screening behavior?. Cancer Epidemiol Biomarkers Prev 

2002;11(6):529-34. Exclude falls outside Task Order scope.

James DS, Bull AD. Information on death certificates: 

cause for concern?. J Clin Pathol 1996;49(3):213-6. 

Exclude non-U.S. data. 

Janda M, Hughes K, Tong S, et al. Faecal occult blood test: 

current practice in a rural Queensland community. Aust J 

Rural Health 2002;10(1):57-64. Exclude non-U.S. data.

Jass J.R. Prognostic factors in rectal cancer. Eur J Cancer 

1995;31A(6):862-3. Exclude non-U.S. data.

Jemal A, Murray T, Ward E, et al. Cancer statistics, 2005. 

CA Cancer J Clin 2005;55(1):10-30. Exclude review. 

Jenkins PJ, Fairclough PD, Richards T, et al. Acromegaly, 

colonic polyps and carcinoma. Clin Endocrinol (Oxf) 

1997;47(1):17-22. Exclude no explicit QM.

Jepson R, Clegg A, Forbes C, et al. The determinants of 

screening uptake and interventions for increasing uptake: a 

systematic review. Health Technol Asses 2000;4(14):i-vii, 

1-133. Exclude review. 

Jess P, Christiansen J, Bech P. Quality of life after anterior 

resection versus abdominoperineal extirpation for rectal 

cancer. Scand J Gastroenterol 2002;37(10):1201-4. Exclude 

non-U.S. data.

Jiwa M, Burr J. GP letter writing in colorectal cancer: a 

qualitative study. Curr Med Res Opin 2002;18(6):342-6. 

Exclude non-U.S. data. 

Jiwa M, Drury D, Hunt L. Improving the outcome in 

colorectal cases. Br J Gen Pract 2002;52(477):329-30. 

Exclude non-U.S. data. 

Johnson BA. Flexible sigmoidoscopy: screening for 

colorectal cancer.[republished in Am Fam Physician. 1999 

Mar 15;59(6):1537-46; PMID: 10193595]. Am Fam 

Physician 1999;59(2):313-24, 327-8. Exclude falls outside 

Task Order scope.

Johnson FE, Longo WE, Vernava AM, et al. How tumor 

stage affects surgeons' surveillance strategies after colon 

cancer surgery. Cancer 1995;76(8):1325-9. Exclude no 

explicit QM. 

Johnson FE, McKirgan LW, Coplin MA, et al. Geographic 

variation in patient surveillance after colon cancer surgery. 

J Clin Oncol 1996;14(1):183-7. Exclude falls outside Task 

Order scope.

Johnson FE, Novell LA, Coplin MA, et al. How practice 

patterns in colon cancer patient follow-up are affected by 

surgeon age. Surg Oncol 1996;5(3):127-31. Exclude no 

explicit QM.

Johnson J, Luo N, Shar J, et al. Valuations of EQ-5D 

Health States: Are the United States and United Kingdon 

Different? Med Care 2005;43(3):221-8. Exclude QOL.

Johnson KA, Trimbath JD, Petersen GM, et al. Impact of 

genetic counseling and testing on colorectal cancer 

screening behavior. Genet Test 2002;6(4):303-6. Exclude 

falls outside Task Order scope.

Joint Test and Technology Transfer Committee Working 

Group ACoMG9RPBMD 20814-3998, United States. 

Genetic testing for colon cancer: joint statement of the 

American College of Medical Genetics and American 

Society of Human Genetics. Joint Test and Technology 

Transfer Committee Working Group. Genet Med 

2000;2(6):362-6. Exclude falls outside Task Order scope.

Jonker DJ, Maroun JA, Kocha W. Survival benefit of 

chemotherapy in metastatic colorectal cancer: a metaanalysis of randomized controlled trials. Br J Cancer 

2000;82(11):1789-94. Exclude non-U.S. data. 

Jorge JM, Wexner SD, James K, et al. Recovery of anal 

sphincter function after the ileoanal reservoir procedure in 

patients over the age of fifty. Dis Colon Rectum 

1994;37(10):1002-5. Exclude falls outside Task Order 

scope.

Joshi AV, Miller LA. Hospital procedure volume and 

outcomes. Ann Intern Med 2004;140(12):1061-2; author 

reply 1062. Exclude no explicit QM.
C-19

Jouret-Mourin A, Working Party fo G.I. Cancer. 

Recommendations for pathological examination and 

reporting for colorectal cancer. Belgian consensus. Acta 

Gastroenterol Belg 2004;67(1):40-5. Exclude non-U.S. 

data.

Kaiser AM, Nunoo-Mensah JW, Wasserberg N. Re: 

Surgical volume and long-term survival following surgery 

for colorectal cancer in the Veterans Affairs Health-Care 

System. Am J Gastroenterol 2005;100(1):250. Exclude falls 

outside Task Order scope.

Kapiteijn E, van De Velde CJ. European trials with total 

mesorectal excision. Semin Surg Oncol 2000;19(4):350-7. 

Exclude non-U.S. data.

Kashtan H, Papa MZ, Stern HS. Is age an independent 

variable in the morbidity and mortality of patients with 

colorectal cancer? A prospective study. Can J Surg 

1991;34(4):374-6. Exclude non-U.S. data.

Kashtan H, Werbin N, Wasserman I, et al. Colorectal 

cancer in patients over 70 years old. A prospective study of 

operative results. Isr J Med Sci 1992;28(12):861-4. Exclude 

non-U.S. data.

Kato I, Tominaga S, Ikari A. The role of socioeconomic 

factors in the survival of patients with gastrointestinal 

cancers. Jpn J Clin Oncol 1992;22(4):270-7. Exclude nonU.S. data.

Kavadas V, Blazeby JM, Conroy T, et al. Development of 

an EORTC disease-specific quality of life questionnaire for 

use in patients with liver metastases from colorectal cancer. 

Eur J Cancer 2003;39(9):1259-63. Exclude non-U.S. data. 

Kaye JA, Shulman LN. Screening program for colorectal 

cancer: effect on stage distribution. HMO Pract 

1992;6(3):13-5. Exclude falls outside Task Order scope. 

Keating J, Lolohea S, Kenwright D. Pathology reporting of 

rectal cancer: a national audit. N Z Med J 

2003;116(1178):U514. Exclude non-U.S. data.

Kee F, Wilson RH, Harper C, et al. Influence of hospital 

and clinician workload on survival from colorectal cancer: 

cohort study. BMJ 1999;318(7195):1381-5. Exclude nonU.S. data. 

Kerr J, Engel J, Schlesinger-Raab A, et al. Doctor-patient 

communication: results of a four-year prospective study in 

rectal cancer patients. Dis Colon Rectum 2003;46(8):1038-

46. Exclude non-U.S. data. 

Kewenter J, Brevinge H. Endoscopic and surgical 

complications of work-up in screening for colorectal 

cancer. Dis Colon Rectum 1996;39(6):676-80. Exclude 

non-U.S. data.

Khuri S, Daley J, Henderson W, et al. Relation of Surgical 

Volume to Outcome in Eight Common Operations. results 

from the VA National Surgical Quality Improvement 

Program. Ann Surg 1999;230(3):414-32. Exclude falls 

outside Task Order scope. 

Khuri SF, Daley J, Henderson W, et al. The National 

Veterans Administration Surgical Risk Study: risk 

adjustment for the comparative assessment of the quality of 

surgical care. J Am Coll Surg 1995;180(5):519-31. Exclude 

falls outside Task Order scope. 

Kievit J. Colorectal cancer follow-up: a reassessment of 

empirical evidence on effectiveness. Eur J Surg Oncol 

2000;26(4):322-8. Exclude non-U.S. data.

Kievit J, Bruinvels DJ. Detection of recurrence after 

surgery for colorectal cancer. Eur J Cancer 1995;31A(7-

8):1222-5. Exclude non-U.S. data.

Kievit J, van de Velde CJ. Utility and cost of 

carcinoembryonic antigen monitoring in colon cancer 

follow-up evaluation. A Markov analysis. Cancer 

1990;65(11):2580-7. Exclude falls outside Task Order 

scope. 

Killingback M, Barron P, Dent O. Elective resection and 

anastomosis for colorectal cancer: a prospective audit of 

mortality and morbidity 1976-1998. ANZ J Surg 

2002;72(10):689-98. Exclude non-U.S. data.

Kim SP, Feinglass J, Bennett CL, et al. Merging claims 

databases with a tumor registry to evaluate variations in 

cancer mortality: results from a pilot study of 698 

colorectal cancer patients treated at one hospital in the 

1990s. Cancer Invest 2004;22(2):225-33. Exclude survival 

only.

Kingston RD, Jeacock J, Walsh S, et al. The outcome of 

surgery for colorectal cancer in the elderly: a 12-year 

review from the Trafford Database. Eur J Surg Oncol 

1995;21(5):514-6. Exclude non-U.S. data. 

Kiran PR, Glass RE. Duration of symptoms and spread of 

colorectal cancer: a short history does not mean early 

disease. Ann R Coll Surg Engl 2002;84(6):381-5. Exclude 

non-U.S. data.

Kjeldsen BJ, Thorsen H, Whalley D, et al. Influence of 

follow-up on health-related quality of life after radical 

surgery for colorectal cancer. Scand J Gastroenterol 

1999;34(5):509-15. Exclude non-U.S. data.

Klein EE, Purdy JA. Quality assurance and dosimetric 

evaluation for an endocavitary unit. Med Dosim 

1994;19(3):151-8. Exclude data sources.
C-20

Klein Kranenbarg E, van de Velde CJ. Surgical trials in 

oncology. the importance of quality control in the TME 

trial. Eur J Cancer 2002;38(7):937-42. Exclude non-U.S. 

data.

Klimberg VS, Langston JD, Maners A, et al. Advantages of 

the Papillon protocol in the preoperative treatment of rectal 

carcinoma. Am J Surg 1992;164(5):433-5; discussion 436. 

Exclude no explicit QM.

Kluger Y, Shiloni E, Jurim O, et al. Subtotal colectomy 

with primary ileocolonic anastomosis for obstructing 

carcinoma of the left colon: valid option for elderly high 

risk patients. Isr J Med Sci 1993;29(11):726-30. Exclude 

non-U.S. data.

Kluka S, Kristjanson LJ. Development and testing of the 

ostomy concerns scale: measuring ostomy-related concerns 

of cancer patients and their partners. J Wound Ostomy 

Continence Nurs 1996;23(3):166-70. Exclude non-U.S. 

data.

Ko CW, Kreuter W, Baldwin LM. Effect of Medicare 

coverage on use of invasive colorectal cancer screening 

tests. Arch Intern Med 2002;162(22):2581-6. Exclude 

wrong population. 

Knopf KB, Warren JL, Feuer EJ, et al. Bowel surveillance 

patterns after a diagnosis of colorectal cancer in Medicare 

beneficiaries. Gastrointest Endosc 2001;54(5):563-71. 

Exclude results combined across modalities. 

Ko CY, Maggard M, Livingston EH. Evaluating health 

utility in patients with melanoma, breast cancer, colon 

cancer, and lung cancer: a nationwide, population-based 

assessment. J Surg Res 2003;114(1):1-5. Exclude QOL.

Ko CY, Rusin LC, Schoetz DJ Jr, et al. Does better 

functional result equate with better quality of life? 

Implications for surgical treatment in familial adenomatous 

polyposis. Dis Colon Rectum 2000;43(6):829-35; 

discussion 835-7. Exclude falls outside Task Order scope.

Koka VK, Potti A, Fraiman GN, et al. An epidemiological 

study evaluating the relationship of distance from a tertiary 

care cancer center to early detection of colorectal 

carcinoma. Anticancer Res . 2002;22(4):2481-3. Exclude 

falls outside Task Order scope.

Kopp I, Bauhofer A, Koller M. Understanding quality of 

life in patients with colorectal cancer: comparison of data 

from a randomised controlled trial, a population based 

cohort study and the norm reference population. Inflamm 

Res 2004;53 Suppl 2(S130-5). Exclude non-U.S. data.

Kopp I, Lorenz W, Rothmund M, et al. Relation between 

severe illness and non-completion of quality-of-life 

questionnaires by patients with rectal cancer. J R Soc Med 

2003;96(9):442-8. Exclude non-U.S. data.

Kotake K, Honjo S, Sugihara K, et al. Changes in 

colorectal cancer during a 20-year period: an extended 

report from the multi-institutional registry of large bowel 

cancer, Japan.[erratum appears in Dis Colon Rectum. 2004 

Jun;47(6):951]. Dis Colon Rectum 2003;46(10 Suppl):S32-

43. Exclude non-U.S. data.

Krieger N. Overcoming the Absence of Socioeconomic 

Data in Medical Records: Validation and Application of a 

Census-Based Methodology. Am J Public Health 

1992;82(5):703-10. Exclude falls outside Task Order scope.

Kronborg O, Fenger C, Olsen J, et al. Randomised study of 

screening for colorectal cancer with faecal-occult-blood 

test. Lancet 1996;348(9040):1467-71. Exclude non-U.S. 

data.

Kuhfahl J, Steinbrecher C, Wagner T, et al. Second-line 

treatment of advanced colorectal cancer with a weekly 

simultaneous 24-hour infusion of 5-fluorouracil and 

sodium-folinate: a multicentre phase II trial. Onkologie 

2004;27(5):449-54. Exclude falls outside Task Order scope.

Kurtz ME, Kurtz JC, Stommel M, et al. The influence of 

symptoms, age, comorbidity and cancer site on physical 

functioning and mental health of geriatric women patients. 

Women Health 1999;29(3):1-12. Exclude falls outside Task 

Order scope. 

Kutner JS, Vu KO, Prindiville SA, et al. Patient age and 

cancer treatment decisions. Patient and physician views. 

Cancer Pract 2000;8(3):114-9. Exclude no explicit QM.

Labianca R, Pancera G, Luporini G. Factors influencing 

response rates for advanced colorectal cancer 

chemotherapy. Ann Oncol 1996;7(9):901-6. Exclude no 

explicit QM.

Lackan NA, Ostir GV, Freeman JL, et al. Hospice use by 

Hispanic and non-Hispanic white cancer decedents. Health 

Serv Res 2004;39(4 Pt 1):969-83. Exclude falls outside 

Task Order scope.

Lambert LA, Colacchio TA, Barth RJ Jr. Interval hepatic 

resection of colorectal metastases improves patient 

selection. Arch Surg 2000;135(4):473-9; discussion 479-

80. Exclude no explicit QM.

Landheer ML, Therasse P, van de Velde CJ. The 

importance of quality assurance in surgical oncology in the 

treatment of colorectal cancer. Surg Oncol Clin N Am 

2001;10(4):885-914, x. Exclude non-U.S. data.

Launoy G, Le Coutour X, Gignoux M, et al. Influence of 

rural environment on diagnosis, treatment, and prognosis of 

colorectal cancer. J Epidemiol Community Health 

1992;46(4):365-7. Exclude non-U.S. data. 
C-21

Law WL, Chan WF, Lee YM, et al. Non-curative surgery 

for colorectal cancer: critical appraisal of outcomes. Int J 

Colorectal Dis 2004;19(3):197-202. Exclude non-U.S. data.

Laxamana A, Solomon MJ, Cohen Z, et al. Long-term 

results of anterior resection using the double-stapling 

technique. Dis Colon Rectum 1995;38(12):1246-50. 

Exclude non-U.S. data. 

Lee AJ, Baker CS, Gehlbach S, et al. Do black elderly 

Medicare patients receive fewer services? An analysis of 

procedure use for selected patient conditions. Med Care 

Res Rev 1998;55(3):314-33. Exclude time period not 

specified. 

Lee FY, Chu W, Chan R, et al. Incidence of deep vein 

thrombosis after colorectal surgery in a Chinese population. 

ANZ J Surg 2001;71(11):637-40. Exclude non-U.S. data.

Lee JY, Soong SJ. Cancer mortality in the South, 1950 to 

1980. South Med J 1990;83(2):185-90. Exclude falls 

outside Task Order scope. 

Lee PY, Fletcher WS, Sullivan ES, et al. Colorectal cancer 

in young patients: characteristics and outcome. Am Surg 

1994;60(8):607-12. Exclude falls outside Task Order scope. 

Leighl NB, Butow PN, Tattersall MH. Treatment decision 

aids in advanced cancer: when the goal is not cure and the 

answer is not clear. J Clin Oncol 2004;22(9):1759-62. 

Exclude no explicit QM.

Leo E, Audisio RA, Belli F, et al. Total rectal resection and 

colo-anal anastomosis for low rectal tumours: comparative 

results in a group of young and old patients. Eur J Cancer 

1994;30A(8):1092-5. Exclude non-U.S. data.

Lev R, Healey J. Colon polyp registries and colorectal 

cancer control. Cancer Detect Prev 1999;23(6):474-8. 

Exclude no explicit QM.

Levin B, Smith RA, Feldman GE, et al. Promoting early 

detection tests for colorectal carcinoma and adenomatous 

polyps: a framework for action: the strategic plan of the 

National Colorectal Cancer Roundtable.[erratum appears in 

Cancer 2002 Dec 15;95(12):2580]. Cancer 

2002;95(8):1618-28. Exclude review.

Levin T, Palitz A. Flexible sigmoindoscopy: An important 

screening option for average-risk individuals. Gastrointest 

Endosc Clin N Am 2002;12(1):23-40. Exclude no explicit 

QM. 

Levin TR, Farraye FA, Schoen RE, et al. Quality in the 

technical performance of screening flexible sigmoidoscopy: 

recommendations of an international multi-society task 

group. Gut 2005;54(6):807-13. Exclude no explicit QM.

Lewis JD, Ginsberg GG, Hoops TC, et al. Flexible 

sigmoidoscopy training and its impact on colorectal cancer 

screening by primary care physicians. Arch Fam Med 

2000;9(5):420-5. Exclude falls outside Task Order scope.

Liang JT, Huang KC, Cheng AL, et al. Clinicopathological 

and molecular biological features of colorectal cancer in 

patients less than 40 years of age. Br J Surg 

2003;90(2):205-14. Exclude non-U.S. data.

Lieberman D. How to screen for colon cancer. Ann Rev 

Med 1998;49:163-72. Exclude falls outside Task Order 

scope.

Lipscomb J. The Outcomes of Cancer Outcomes Research: 

Implications for the Quality of Cancer Care. National 

Cancer Institute. Available at: 

http://www.copufl.edu/departments/phca/Library/1. 

Accessed: May 9, 2005. 2001. Exclude review.

Litle VR, Barbour S, Schrock TR, et al. The continent 

ileostomy: long-term durability and patient satisfaction. J 

Gastrointest Surg 1999;3(6):625-32. Exclude falls outside 

Task Order scope.

Livraghi T, Solbiati L, Meloni F, et al. Percutaneous 

radiofrequency ablation of liver metastases in potential 

candidates for resection: the "test-of-time approach". 

Cancer 2003;97(12):3027-35. Exclude non-U.S. data. 

Longo WE, Johnson FE. The preoperative assessment and 

postoperative surveillance of patients with colon and rectal 

cancer. Surg Clin North Am 2002;82(5):1091-108. Exclude 

no explicit QM.

Longo WE, Virgo KS, Johnson FE, et al. Outcome after 

proctectomy for rectal cancer in Department of Veterans 

Affairs Hospitals: a report from the National Surgical 

Quality Improvement Program. Ann Surg 1998;228(1):64-

70. Exclude falls outside Task Order scope. 

Lonneux M, Reffad AM, Detry R, et al. FDG-PET 

improves the staging and selection of patients with 

recurrent colorectal cancer. Eur J Nucl Med Mol Imaging 

2002;29(7):915-21. Exclude non-U.S. data.

Lopez MJ, Monafo WW. Role of extended resection in the 

initial treatment of locally advanced colorectal carcinoma. 

Surgery 1993;113(4):365-72. Exclude review.

Lovig T, Andersen SN, Thorstensen L, et al. Strong HLADR expression in microsatellite stable carcinomas of the 

large bowel is associated with good prognosis. Br J Cancer 

2002;87(7):756-62. Exclude non-U.S. data.

Lumby J, England K. Patient satisfaction with nursing care 

in a colorectal surgical population. Int J Nurs Pract 

2000;6(3):140-5. Exclude non-U.S. data. 
C-22

Lund JN, Scholefield JH, Grainge MJ, et al. Risks, costs, 

and compliance limit colorectal adenoma surveillance: 

lessons from a randomised trial. Gut 2001;49(1):91-6. 

Exclude non-U.S. data.

Lusky K. Pilot points way to speedier cancer surveillance. 

CAP Today 2005;19(2):5-6. Exclude no explicit QM.

Lyratzopoulos G, West CR, Williams EM. Socioeconomic 

variation in colon cancer tumour factors associated with 

poorer prognosis. Br J Cancer 2003;89(5):828-30. Exclude 

non-U.S. data.

Lytwyn A, Salit IE, Raboud J, et al. Interobserver 

agreement in the interpretation of anal intraepithelial 

neoplasia. Cancer 2005;103(7):1447-56. Exclude no 

explicit QM.

MacFarlane JK. Developing standards for the treatment of 

rectal cancer. Can J Surg 1997;40(5):327-8. Exclude 

review. 

MacGillivray DC, Swartz SE, Robinson AM, et al. 

Adenocarcinoma of the colon and rectum in patients less 

than 40 years of age. Surg Gynecol Obstet 1991;172(1):1-7. 

Exclude survival only.

Mainous AG 3rd, Kern D, Hainer B, et al. The relationship 

between continuity of care and trust with stage of cancer at 

diagnosis. Fam Med 2004;36(1):35-9. Exclude falls outside 

Task Order scope.

Mainprize KS, Mortensen NJ, Warren BF. Dukes' staging 

is poorly understood by doctors managing colorectal 

cancer. Ann R Coll Surg Engl 2002;84(1):23-5. Exclude 

non-U.S. data. 

Makela J, Kiviniemi H, Laitinen S. Prognostic factors after 

surgery in patients younger than 50 years old with 

colorectal adenocarcinoma. Hepatogastroenterology 

2002;49(46):971-5. Exclude non-U.S. data.

Makela JT, Laitinen SO, Kairaluoma MI. Five-year followup after radical surgery for colorectal cancer. Results of a 

prospective randomized trial. Arch Surg 

1995;130(10):1062-7. Exclude no explicit QM.

Malik AI, Tou S, Huang A. Waiting times for cancer 

treatment. J R Soc Med 2004;97(6):311. Exclude non-U.S. 

data.

Mandel JS. Colorectal cancer screening. Cancer Metastasis 

Rev 1997;16(3-4):263-79. Exclude non-U.S. data.

Manos D. New study shows disparities in treatment of 

rectal cancer. Rep Med Guidel Outcomes Res 

2004;15(5):9-10, 12. Exclude no explicit QM.

Manser MD, Levine DF, Pheby DF, et al. Colorectal cancer 

registration: the central importance of pathology. J Clin 

Pathol 2000;53(11):875-7. Exclude non-U.S. data.

Manus B, Adang RP, Ambergen AW, et al. The risk factor 

profile of recto-sigmoid adenomas: a prospective screening 

study of 665 patients in a clinical rehabilitation centre. Eur 

J Cancer Prev 1997;6(1):38-43. Exclude non-U.S. data.

Manus B, Bragelmann R, Armbrecht U, et al. Screening for 

gastrointestinal neoplasia: efficacy and cost of two different 

approaches in a clinical rehabilitation centre. Eur J Cancer 

Prev 1996;5(1):49-55. Exclude non-U.S. data.

Marcella S, Miller JE. Racial differences in colorectal 

cancer mortality. The importance of stage and 

socioeconomic status. J Clin Epidemiol 2001;54(4):359-66. 

Exclude survival only.

Maroun J, Ng E, Berthelot JM, et al. Lifetime costs of 

colon and rectal cancer management in Canada. Chronic 

Dis Can 2003;24(4):91-101. Exclude non-U.S. data.

Marshall T, Mohammed MA. Differences in clinical 

performance. Br J Surg 2002;89(8):948-9. Exclude nonU.S. data.

Martling A, Cedermark B, Johansson H, et al. The surgeon 

as a prognostic factor after the introduction of total 

mesorectal excision in the treatment of rectal cancer. Br J 

Surg 2002;89(8):1008-13. Exclude non-U.S. data.

Martling A, Singnomklao T, Holm T, et al. Prognostic 

significance of both surgical and pathological assessment 

of curative resection for rectal cancer. Br J Surg 

2004;91(8):1040-5. Exclude review.

Marusch F, Koch A, Schmidt U, et al. Hospital caseload 

andthe results achieved in patients with rectal cancer. Br J 

Surg 2001;88(10):1397-402. Exclude non-U.S. data. 

Marusch F, Koch A, Schmidt U, et al. Effect of caseload on 

the short-term outcome of colon surgery: results of a 

multicenter study. Int J Colorectal Dis 2001;16(6):362-9. 

Exclude non-U.S. data.

Mathew J, Shankar P, Aldean IM. Audit on flexible 

sigmoidoscopy for rectal bleeding in a district general 

hospital: are we over-loading the resources? Postgrad Med 

J 2004;80(939):38-40. Exclude non-U.S. data.

Matter SE, Campbell DR. Significance of distal polyps 

detected with flexible sigmoidoscopy in asymptomatic 

patients. Arch Intern Med 1992;152(9):1776-80. Exclude 

falls outside Task Order scope.

Maule WF. Screening for colorectal cancer by nurse 

endoscopists. N Engl J Med 1994;330(3):183-7. Exclude 

non-U.S. data. 
C-23

Mayberry RM, Coates RJ, Hill HA, et al. Determinants of 

black/white differences in colon cancer survival. J Natl 

Cancer Inst 1995;87(22):1686-93. Exclude survival only.

McArdle CS, Hole D. Impact of variability among surgeons 

on postoperative morbidity and mortality and ultimate 

survival. BMJ 1991;302(6791):1501-5. Exclude non-U.S. 

data.

McArdle CS, Hole D, Hansell D, et al. Prospective study of 

colorectal cancer in the west of Scotland: 10-year followup. Br J Surg 1990;77(3):280-2. Exclude non-U.S. data.

McArdle CS, Hole DJ. Influence of volume and 

specialization on survival following surgery for colorectal 

cancer. Br J Surg 2004;91(5):610-7. Exclude non-U.S. data.

McBean AM, Warren JL, Babish JD. Measuring the 

incidence of cancer in elderly Americans using Medicare 

claims data. Cancer 1994;73(9):2417-25. Exclude no 

explicit QM.

McCallion K, Mitchell RM, Wilson RH, et al. Flexible 

sigmoidoscopy and the changing distribution of colorectal 

cancer: implications for screening. Gut 2001;48(4):522-5. 

Exclude non-U.S. data.

McCallum J. The role of the nurse endoscopist. Hosp Med 

2003;64(6):337-9. Exclude review.

McCarthy EP, Burns RB, Davis RB, et al. Barriers to 

hospice care among older patients dying with lung and 

colorectal cancer. J Clin Oncol 2003;21(4):728-35. Exclude 

falls outside Task Order scope.

McCarthy EP, Phillips RS, Zhong Z, et al. Dying with 

cancer: patients' function, symptoms, and care preferences 

as death approaches. J Am Geriatr Soc 2000;48(5 

Suppl):S110-21. Exclude falls outside Task Order scope.

McClish D, Penberthy L. Using Medicare data to estimate 

the number of cases missed by a cancer registry: a 3-source 

capture-recapture model. Med Care 2004;42(11):1111-6. 

Exclude no explicit QM.

McCollum A, Catalano P, Haller D, et al. Outcomes and 

Toxicity in African-American and Caucasian Patients in a 

Randomized Adjuvant Chemotherapy Trial for Colon 

Cancer. J Natl Cancer Inst 2002;94(15):1160-7. Exclude 

survival only. 

McCool J, Morris J. Focus of doctor-patient 

communication in follow-up consultations for patients 

treated surgically for colorectal cancer. J Manag Med 

1999;13(2-3):169-77. Exclude non-U.S. data. 

McDavid K, Schymura MJ, Armstrong L, et al. Rationale 

and design of the National Program of Cancer Registries' 

Breast, Colon, and Prostate Cancer Patterns of Care Study. 

Cancer Causes Control 2004;15(10):1057-66. Exclude no 

explicit QM. 

McDavid K, Tucker TC, Sloggett A, et al. Cancer survival 

in Kentucky and health insurance coverage. Arch Intern 

Med 2003;163(18):2135-44. Exclude survival only.

McFall MR, Woods WG, Miles WF. Colonoscopic 

surveillance after curative colorectal resection: results of an 

empirical surveillance programme. Colorectal Dis 

2003;5(3):233-40. Exclude non-U.S. data.

McFall SL, Warnecke RB, Kaluzny AD, et al. Practice 

setting and physician influences on judgments of colon 

cancer treatment by community physicians. Health Serv 

Res 1996;31(1):5-19. Exclude no explicit QM.

McGlynn E. Selecting common measures of quality and 

system performance. Med Care 2003;41(1 

(supplement)):I39-I47. Exclude review.

McGlynn E, Asch S, Adams J, et al. The quality of health 

care delivered to adults in the United States. N Engl J Med 

2003;348(26):2635-45. Exclude no explicit QM.

McGlynn EA. Choosing chronic disease measures for 

HEDIS: conceptual framework and review of seven clinical 

areas. Manag Care Q 1996;4(3):54-77. Exclude no explicit 

QM.

McGrath DR, Leong DC, Armstrong BK, et al. 

Management of colorectal cancer patients in Australia: the 

National Colorectal Cancer Care Survey. ANZ J Surg 

2004;74(1-2):55-64. Exclude non-U.S. data.

McGrath JS, Ponich TP, Gregor JC. Screening for 

colorectal cancer: the cost to find an advanced adenoma. 

Am J Gastroenterol 2002;97(11):2902-7. Exclude non-U.S. 

data.

McKee M, Hunter D. Mortality league tables: do they 

inform or mislead? Qual Health Care 1995;4(1):5-12. 

Exclude non-U.S. data.

McLeish JA, Giles GG, Thursfield V. Investigation, 

follow-up and recurrence after resection of colorectal 

cancer. Ann N Z J Surg 1992;62(12):931-40. Exclude nonU.S. data.

McLeod RS, Canadian Task Force on Preventive Health 

Care. Screening strategies for colorectal cancer: a 

systematic review of the evidence. Can J Gastroenterol 

2001;15(10):647-60. Exclude falls outside Task Order 

scope.
C-24

Meagher AP. Colorectal cancer: is the surgeon a prognostic 

factor? A systematic review. Med J Aust 1999;171(6):308-

10. Exclude review.

Meagher AP, Ward RL. Current evidence does not support 

routine adjuvant radiotherapy for rectal cancer. ANZ J Surg 

2003;73(4):252-3. Exclude non-U.S. data.

Mella J, Datta SN, Biffin A, et al. Surgeons' follow-up 

practice after resection of colorectal cancer. Ann R Coll 

Surg Engl 1997;79(3):206-9. Exclude non-U.S. data.

Mettlin CJ, Menck HR, Winchester DP, et al. A 

comparison of breast, colorectal, lung, and prostate cancers 

reported to the National Cancer Data Base and the 

Surveillance, Epidemiology, and End Results Program. 

Cancer 1997;79(10):2052-61. Exclude data sources.

Meyerhardt JA, Catalano PJ, Schrag D, et al. Association 

of hospital procedure volume and outcomes in patients with 

colon cancer at high risk for recurrence.[erratum appears in 

Ann Intern Med. 2004 Jun 15;140(12):1063]. Ann Intern 

Med 2003;139(8):649-57. Exclude survival only.

Meyerhardt JA, Mayer RJ. Follow-up strategies after 

curative resection of colorectal cancer. Semin Oncol 

2003;30(3):349-60. Exclude review.

Michael M, Moore MJ. Assessing the impact of 

chemotherapy on tumor-related symptoms in advanced 

colorectal cancer. Oncology (Huntingt) 1998;12(8 Suppl 

6):121-8. Exclude no explicit QM.

Miki C, Inoue Y, Toiyama Y, et al. Deficiency in systemic 

interleukin-1 receptor antagonist production as an operative 

risk factor in malnourished elderly patients with colorectal 

carcinoma. Crit Care Med 2005;33(1):177-80. Exclude 

non-U.S. data. 

Minardi AJ Jr, Sittig KM, Zibari GB, et al. Colorectal 

cancer in the young patient. Am Surg 1998;64(9):849-53. 

Exclude no explicit QM. 

Mitchell PL, White CA, Austin Bowel Cancer Consortium. 

Austin Bowel Cancer Consortium: changing culture in 

bowel cancer care. Med J Aust 2004;180(10 Suppl):S79-

82. Exclude non-U.S. data.

Miyakura Y, Togashi K, Konishi F, et al. Appropriate 

operation for elderly colorectal cancer patients based upon 

an assessment of preoperative risk factors. Surg Today 

2003;33(7):498-503. Exclude non-U.S. data.

Moertel CG, Fleming TR, Macdonald JS, et al. Levamisole 

and fluorouracil for adjuvant therapy of resected colon 

carcinoma. N Engl J Med 1990;322(6):352-8. Exclude no 

explicit QM.

Monnet E, Boutron MC, Faivre J, et al. Influence of 

socioeconomic status on prognosis of colorectal cancer. A 

population-based study in Cote D'Or, France. Cancer 

1993;72(4):1165-70. Exclude non-U.S. data.

Monnet E, Faivre J, Raymond L, et al. Comparability of 

colorectal cancer survival data in three European 

population-based registries. Eur J Cancer Prev 

1998;7(2):127-34. Exclude non-U.S. data.

Moore J, Hewett P, Penfold JC, et al. Practice parameters 

for the management of colonic cancer I: surgical issues. 

Recommendations of the Colorectal Surgical Society of 

Australia.Ann N Z J Surg 1999;69(6):415-21. Exclude no 

explicit QM. 

Morantz C, American Cancer Society. ACS guidelines for 

early detection of cancer. Am Fam Physician 

2004;69(8):2013. Exclude no explicit QM.

Morris JB, Stellato TA, Guy BB, et al. A critical analysis of 

the largest reported mass fecal occult blood screening 

program in the United States. Am J Surg 1991;161(1):101-

5; discussion 105-6. Exclude falls outside Task Order 

scope.

Mosconi P, Torri V, Cifani S, et al. The multi-centre 

assessment of quality of life: the Interdisciplinary Group 

for Cancer Care Evaluation (GIVIO) experience in Italy. 

Stat Med 1998;17(5-7):577-85. Exclude non-U.S. data. 

Mostafa G, Matthews BD, Norton HJ, et al. Influence of 

demographics on colorectal cancer. Am Surg 

2004;70(3):259-64. Exclude data sources.

Mulcahy HE, Patchett SE, Daly L, et al. Prognosis of 

elderly patients with large bowel cancer. Br J Surg 

1994;81(5):736-8. Exclude non-U.S. data.

Munemoto Y, Iida Y, Ohata K, et al. Significance of 

postoperative adjuvant immunochemotherapy after curative 

resection of colorectal cancers: identification of responders 

incorporating the age factor. Oncol Rep 2004;11(3):623-35. 

Exclude falls outside Task Order scope.

Murakami R, Tsukuma H, Kanamori S, et al. Natural 

history of colorectal polyps and the effect of polypectomy 

on occurrence of subsequent cancer. Int J Cancer 

1990;46(2):159-64. Exclude non-U.S. data.

Myers AF, Nichols PH, Beck NE. Age limits for 

surveillance colonoscopy following colorectal cancer 

resection. Colorectal Dis 2004;6(2):131-2. Exclude review.

Myers RE, Fishbein G, Hyslop T, et al. Measuring 

complete diagnostic evaluation in colorectal cancer 

screening. Cancer Detect Prev 2001;25(2):174-82. Exclude 

falls outside Task Order scope.
C-25

Myers RE, Hyslop T, Gerrity M, et al. Physician intention 

to recommend complete diagnostic evaluation in colorectal 

cancer screening. Cancer Epidemiol Biomarkers Prev 

1999;8(7):587-93. Exclude no explicit QM.

Myers RE, Murray J, Weinberg D, et al. Analysis of 

colorectal cancer stage among HMO members targeted for 

screening. Arch Intern Med 1997;157(17):2001-6. Exclude 

falls outside Task Order scope.

Myers RE, Turner B, Weinberg D, et al. Complete 

diagnostic evaluation in colorectal cancer screening: 

research design and baseline findings. Prev Med 

2001;33(4):249-60. Exclude no explicit QM.

Myers RE, Vernon SW, Tilley BC, et al. Intention to screen 

for colorectal cancer among white male employees. Prev 

Med 1998;27(2):279-87. Exclude falls outside Task Order 

scope.

Nadalin V, Cotterchio M, McKeown-Eyssen G, et al. 

Agreement between proxy- and case-reported information 

obtained using the self- administered Ontario Familial 

Colon Cancer Registry epidemiologic questionnaire. 

Chronic Dis Can 2003;24(1):1-8. Exclude non-U.S. data.

Nadel MR, Blackman DK, Shapiro JA, et al. Are people 

being screened for colorectal cancer as recommended? 

Results from the National Health Interview Survey. Prev 

Med 2002;35(3):199-206. Exclude falls outside Task Order 

scope.

Nagashima I, Takada T, Matsuda K, et al. A new scoring 

system to classify patients with colorectal liver metastases: 

proposal of criteria to select candidates for hepatic 

resection. J Hepatobiliary Pancreat Surg 2004;11(2):79-83. 

Exclude non-U.S. data. 

Nagtegaal ID, Kranenbarg EK, Hermans J, et al. Pathology 

data in the central databases of multicenter randomized 

trials need to be based on pathology reports and controlled 

by trained quality managers. J Clin Oncol 2000;18(8):1771-

9. Exclude non-U.S. data. 

Nakama H, Kamijo N, Abdul Fattah AS, et al. Validity of 

immunological faecal occult blood screening for colorectal 

cancer: a follow up study. J Med Screen 1996;3(2):63-5. 

Exclude falls outside Task Order scope. 

Nakamura T, Yoshioka H, Ohno M, et al. 

Clinicopathologic variables affecting survival of distal 

colorectal cancer patients with macroscopic invasion into 

the adjacent organs. Surg Today 1999;29(3):226-32. 

Exclude non-U.S. data. 

National Cancer Institute. Infrastructure needed for cancer 

research: NCI's challenge: quality of cancer care. Available 

at http://plan2003.cancer.gov/infra/quality.htm. Accessed: 

April 6, 2005. 2003. Exclude review. 

National Institute for Clinical Excellence. Guidelines on 

Cancer Services: Improving Outcomes in Colorectal 

Cancers: Manual Update. May 2004. Exclude non-U.S. 

data. 

National Institutes of Health (NIH). NIH consensus 

conference. Adjuvant therapy for patients with colon and 

rectal cancer. JAMA 1990;264(11):1444-50. Exclude no 

explicit QM. 

Neff MJ, ACS. ACS releases guidelines on nutrition and 

physical activity during and after cancer treatment. Am 

Fam Physician 2004;69(7):1803-5. Exclude falls outside 

Task Order scope. 

Nelson H, Petrelli N, Carlin A, et al. Guidelines 2000 for 

Colon and Rectal Cancer Surgery. J Natl Cancer Inst 

2001;93(8):583-96. Exclude review. 

Nelson RL, Dollear T, Freels S, et al. The relation of age, 

race, and gender to the subsite location of colorectal 

carcinoma. Cancer 1997;80(2):193-7. Exclude no explicit 

QM.

Nelson RL, Persky V, Turyk M. Carcinoma in situ of the 

colorectum: SEER trends by race, gender, and total 

colorectal cancer. J Surg Oncol 1999;71(2):123-9. Exclude 

no explicit QM.

Nesbakken A, Nygaard K, Westerheim O, et al. Audit of 

intraoperative and early postoperative complications after 

introduction of mesorectal excision for rectal cancer. Eur J 

Surg 2002;168(4):229-35. Exclude non-U.S. data. 

Ness RM, Holmes A, Klein R, et al. Outcome states of 

colorectal cancer: identification and description using 

patient focus groups. Am J Gastroenterol 1998;93(9):1491-

7. Exclude data sources. 

Ness RM, Rex DK, Imperiale T. Translating clinical 

research to community-based practice: lessons from 

colorectal cancer screening and diagnosis. Endocsopy 

1999;31(2):170-3. Exclude review. 

Nicholson WJ, Davis DL. Analyses of changes in the ratios 

of male-to-female cancer mortality. A hypothesisgenerating exercise. Ann N Y Acad Sci 1990;609:290-9. 

Exclude no explicit QM. 

Nicoli N, Casaril A, Mangiante G, et al. Surgical treatment 

for liver metastases from colorectal carcinoma: results of 

228 patients. Hepatogastroenterology 2004;51(60):1810-4. 

Exclude falls outside Task Order scope. 

Nilsson KR, Berenholtz SM, Dorman T, et al. Preoperative 

predictors of blood transfusion in colorectal cancer surgery. 

J Gastrointest Surg 2002;6(5):753-62. Exclude no explicit 

QM.
C-26

Nordlinger B, Peschaud F, Malafosse R. Resection of liver 

metastases from colorectal cancer--how can we improve 

results? Colorectal Dis 2003;5(5):515-7. Exclude non-U.S. 

data. 

Northover J. Follow-up after curative surgery for colorectal 

cancer. Scand J Surg 2003;92(1):84-9. Exclude non-U.S. 

data. 

Norum J. Prevention of colorectal cancer: a costeffectiveness approach to a screening model employing 

sigmoidoscopy. Ann Oncol 1998;9(6):613-8. Exclude nonU.S. data.

Nwiloh J, Dardik H, Dardik M, et al. Changing patterns in 

the morbidity and mortality of colorectal surgery. Am J 

Surg 1991;162(1):83-5. Exclude no explicit QM. 

O'Bichere A, Sellu D. The quality of operation notes: can 

simple word processors help? Ann R Coll Surg Engl 

1997;79(5 Suppl):204-8; discussion 209. Exclude non-U.S. 

data. 

O'Connell JB, Maggard MA, Ko CY. Colon cancer survival 

rates with the new American Joint Committee on Cancer 

sixth edition staging. J Natl Cancer Inst 2004;96(19):1420-

5. Exclude review. 

Ohlsson B, Palsson B. Follow-up after colorectal cancer 

surgery. Acta Oncol 2003;42(8):816-26. Exclude non-U.S. 

data. 

Oliver MD. Two week rule for cancer referrals. All stages 

of care pathway need speeding up. BMJ 

2001;323(7317):864. Exclude non-U.S. data.

Olynyk JK, Aquilia S, Fletcher DR, et al. Flexible 

sigmoidoscopy screening for colorectal cancer in averagerisk subjects: a community-based pilot project. Med J Aust 

1996;165(2):74-6. Exclude non-U.S. data.

Oppong C, Robertson N, Sherwood A, et al. The use of a 

proforma improves colorectal cancer pathology reporting. 

Ann R Coll Surg Engl 2002;84(4):290; author reply 290. 

Exclude non-U.S. data. 

Paksoy M, Ipek T, Colak T, et al. Influence of age on 

prognosis and management of patients with colorectal 

carcinoma. Eur J Surg 1999;165(1):55-9. Exclude non-U.S. 

data. 

Palitz AM, Selby JV, Grossman S, et al. The Colon Cancer 

Prevention Program (CoCaP): rationale, implementation, 

and preliminary results. HMO Pract 1997;11(1):5-12. 

Exclude falls outside Task Order scope.

Palmer ML, Herrera L, Petrelli NJ. Colorectal 

adenocarcinoma in patients less than 40 years of age. Dis 

Colon Rectum 1991;34(4):343-6. Exclude survival only.

Palmer RC, Schneider EC. Social disparities across the 

continuum of colorectal cancer: a systematic review. 

Cancer Causes Control 2005;16(1):55-61. Exclude review. 

Panageas KS, Schrag D, Riedel E, et al. The effect of 

clustering of outcomes on the association of procedure 

volume and surgical outcomes. Ann Intern Med 

2003;139(8):658-65. Exclude falls outside Task Order 

scope. 

Papagrigoriadis S, Heyman B. Patients' views on follow up 

of colorectal cancer: implications for risk communication 

and decision making. Postgrad Med J 2003;79(933):403-7. 

Exclude non-U.S. data. 

Parry JM, Collins S, Mathers J, et al. Influence of volume 

of work on the outcome of treatment for patients with 

colorectal cancer. Br J Surg 1999;86(4):475-81. Exclude 

non-U.S. data. 

Parry JM, Scott N, Sherlock D, et al. Use of hepatic 

imaging techniques in the management of colorectal 

cancer: a survey of current practice among surgeons in the 

northwest of England. Ann R Coll Surg Engl 

1998;80(4):244-5. 

Exclude non-U.S. data. 

Paszat LF, Brundage MD, Groome PA, et al. A populationbased study of rectal cancer: permanent colostomy as an 

outcome. Int J Radiat Oncol Biol Phys 1999;45(5):1185-91. 

Exclude non-U.S. data. 

Paterson CA, Dozois RR. The ileal pouch-anal 

anastomosis: success and failure. Chirurg 1998;123(6):545-

9. Exclude falls outside Task Order scope. 

Paty PB, Nash GM, Baron P, et al. Long-term results of 

local excision for rectal cancer. Ann Surg 2002;236(4):522-

29; discussion 529-30. Exclude survival only. 

Pauler DK, McCoy S, Moinpour C. Pattern mixture models 

for longitudinal quality of life studies in advanced stage 

disease. Stat Med 2003;22(5):795-809. Exclude data 

sources.

Paunescu V, Ragalie S. Factors of prediction of immediate 

postoperative evolution in cancer of the colon. Rom J 

Intern Med 1991;29(3-4):155-65. Exclude non-U.S. data. 

Paviot BT, Martin C, Clavel L, et al. From DRG databases 

to an epidemiological observatory for colorectal cancer in a 

French small area oncology network. Stud Health Technol 

Inform 2002;95(829-33). Exclude non-U.S. data. 

Pawlega J. Survival studies: Cracow Cancer registry 

experience. Cancer Detect Prev 1993;17(4-5):475-83.

Exclude non-U.S. data. 
C-27

Pawlson G, Scholle SH, Renner P. Pitfalls of converting 

practice guidelines into quality measures. JAMA 

2004;292(11):1301; author reply 1302. Exclude review. 

Peeters KC, van de Velde CJ. Surgical quality assurance in 

breast, gastric and rectal cancer. J Surg Oncol 

2003;84(3):107-12. Exclude non-U.S. data. 

Pennello GA, Devesa SS, Gail MH. Using a mixed effects 

model to estimate geographic variation in cancer rates. 

Biometrics 1999;55(3):774-81. Exclude no explicit QM. 

Persson E, Gustavsson B, Hellstrom AL, et al. Ostomy 

patients' perceptions of quality of care. J Adv Nurs 

2005;49(1):51-8. Exclude non-U.S. data. 

Petrelli NJ, Nagel S, Rodriguez-Bigas M, et al. Morbidity 

and mortality following abdominoperineal resection for 

rectal adenocarcinoma. Am Surg 1993;59(7):400-4. 

Exclude falls outside Task Order scope . 

Pezze M, Gargiulo A, Catarci T, et al. A formal model for 

quality assurance in coloproctologic surgery. Medinfo 

1995;8 Pt 2:1037-41. Exclude non-U.S. data. 

Pfister DG, Benson AB 3rd, Somerfield MR. Clinical 

practice. Surveillance strategies after curative treatment of 

colorectal cancer. N Engl J Med 2004;350(23):2375-82. 

Exclude falls outside Task Order scope. 

Phillips K, Luft H. The Policy Implications of Using 

Hospital and Physician Volumes as "Indicators" of Quality 

of Care in a Changing Health Care Environment. Int J Qual 

Health Care 1997;9(5):341-8. Exclude no explicit QM. 

Phipps E, True G, Harris D, et al. Approaching the end of 

life: attitudes, preferences, and behaviors of AfricanAmerican and white patients and their family caregivers. J 

Clin Oncol 2003;21(3):549-54. Exclude no explicit QM. 

Pickhardt P, Nugent P, Mysliwiec P, et al. Location of 

adenomas missed by optical colonoscopy. Ann Intern Med 

2004;141(5):352-9. Exclude wrong population. 

Pitchforth E, Russell E, Van der Pol M. Access to specialist 

cancer care: is it equitable? Br J Cancer 2002;87(11):1221-

6. Exclude non-U.S. data. 

Platell CF, Semmens JB. Review of survival curves for 

colorectal cancer. Dis Colon Rectum 2004;47(12):2070-5. 

Exclude non-U.S. data. 

Platz EA, Martinez ME, Grodstein F, et al. Parity and other 

reproductive factors and risk of adenomatous polyps of the 

distal colorectum (United States). Cancer Causes Control 

1997;8(6):894-903. Exclude no explicit QM.

Polednak AP. Trends in late-stage breast, cervical and 

colorectal cancers in blacks and whites. Ethn Dis 

2000;10(1):60-8. Exclude no explicit QM. 

Pollard CW, Nivatvongs S, Rojanasakul A, et al. 

Carcinoma of the rectum. Profiles of intraoperative and 

early postoperative complications. Dis Colon Rectum 

1994;37(9):866-74. Exclude no explicit QM.

Pollock AM, Benster R, Vickers N. Why did treatment 

rates for colorectal cancer in south east England fall 

between 1982 and 1988? The effect of case ascertainment 

and registration bias. J Public Health Med 1995;17(4):419-

28. Exclude non-U.S. data. 

Pollock AM, Vickers N. The impact on colorectal cancer 

survival of cases registered by 'death certificate only': 

implications for national survival rates. Br J Cancer 

1994;70(6):1229-31. Exclude non-U.S. data. 

Pollock AM, Vickers N. Reliability of data of the Thames 

cancer registry on 673 cases of colorectal cancer: effect of 

the registration process. Qual Health Care 1995;4(3):184-9. 

Exclude non-U.S. data. 

Pollock AM, Vickers N. Management of colorectal cancer 

in three South Thames District Health Authorities. Public 

Health 1997;111(3):165-70. Exclude non-U.S. data.

Pollock AM, Vickers N. Trends in colorectal cancer care in 

southern England, 1989-1993: using HES data to inform 

cancer services reviews. J Epidemiol Community Health 

1998;52(7):433-8. Exclude non-U.S. data.

Ponz de Leon M, Benatti P, Di Gregorio C, et al. Staging 

and survival of colorectal cancer: are we making progress? 

The 14-year experience of a specialized cancer registry. 

Dig Liver Dis 2000;32(4):312-7. Exclude non-U.S. data. 

Ponz de Leon M, Benatti P, Percesepe A, et al. 

Epidemiology of cancer of the large bowel--the 12-year 

experience of a specialized registry in northern Italy. Ital J 

Gastroenterol 1999;31(1):10-8. Exclude non-U.S. data.

Porter GA, Soskolne CL, Yakimets WW, et al. Surgeonrelated factors and outcome in rectal cancer. Ann Surg 

1998;227(2):157-67. Exclude non-U.S. data. 

Potosky AL, Riley GF, Lubitz JD, et al. Potential for cancer 

related health services research using a linked Medicaretumor registry database. Med Care 1993;31(8):732-48. 

Exclude review. 

Prajapati DN, Saeian K, Binion DG, et al. Volume and 

yield of screening colonoscopy at a tertiary medical center 

after change in medicare reimbursement. Am J 

Gastroenterol 2003;98(1):194-9. Exclude falls outside Task 

Order scope. 

Prandi M, Lionetto R, Bini A, et al. Prognostic evaluation 

of stage B colon cancer patients is improved by an adequate 

lymphadenectomy: results of a secondary analysis of a 

large scale adjuvant trial. Ann Surg 2002;235(4):458-63.

Exclude non-U.S. data. 
C-28

Prior P, Woodman CB, Collins S. International differences 

in survival from colon cancer: more effective care versus 

less complete cancer registration. Br J Surg 

1998;85(1):101-4. Exclude non-U.S. data. 

Provenzale D, Gray RN. Colorectal cancer screening and 

treatment: review of outcomes research. J Natl Cancer Inst 

Monogr 2004;(33):45-55. Exclude QOL. 

Prystowsky J, Bordage G, Feinglass J. Patient outcomes for 

segmental colon resection according to surgeon's training, 

certification, and experience. Surgery 2002;132(4):663-72. 

Exclude wrong population. 

Rabeneck L, El-Serag HB, Davila JA, et al. Outcomes of 

colorectal cancer in the United States: no change in 

survival (1986-1997). Am J Gastroenterol 2003;98(2):471-

7. Exclude survival only.

Rahusen FD, Cuesta MA, Borgstein PJ, et al. Selection of 

patients for resection of colorectal metastases to the liver 

using diagnostic laparoscopy and laparoscopic 

ultrasonography. Ann Surg 1999;230(1):31-7. Exclude nonU.S. data. 

Ramakrishnan K. Technical competency in flexible 

sigmoidoscopy. J Am Board Fam Pract 2002;15(2):173. 

Exclude review.

Ramakrishnan K, Scheid DC. Predictors of incomplete 

flexible sigmoidoscopy. J Am Board Fam Pract 

2003;16(6):478-84. Exclude falls outside Task Order scope. 

Ramsey SD, Berry K, Etzioni R. Lifetime cancerattributable cost of care for long term survivors of 

colorectal cancer. Am J Gastroenterol 2002;97(2):440-5. 

Exclude no explicit QM. 

Ransohoff DF, Lang CA. Screening for colorectal cancer 

with the fecal occult blood test: a background paper. 

American College of Physicians. Ann Intern Med 

1997;126(10):811-22. Exclude falls outside Task Order 

scope. 

Rao BR, Sandilya S, Niculescu R, et al. Mining timedependent patient outcomes from hospital patient records. 

Proc AMIA Symp 2002;632-6. Exclude no explicit QM.

Rao RS, Graubard BI, Breen N, et al. Understanding the 

factors underlying disparities in cancer screening rates 

using the Peters-Belson approach: results from the 1998 

National Health Interview Survey. Med Care 

2004;42(8):789-800. Exclude falls outside Task Order 

scope. 

Raraty MG, Winstanley JH. Variation in the staging of 

colorectal carcinomas: a survey of current practice. Ann R 

Coll Surg Engl 1998;80(3):188-91. Exclude non-U.S. data. 

Ray-Coquard I, Philip T, Lehmann M, et al. Impact of a 

clinical guidelines program for breast and colon cancer in a 

French cancer center. JAMA 1997;278(19):1591-5. 

Exclude non-U.S. data.

Rayson D. Witness. J Clin Oncol 2005;23(10):2429-30. 

Exclude falls outside Task Order scope. 

Redmond K. Assessing patients' needs and preferences in 

the management of advanced colorectal cancer. Br J Cancer 

1998;77 Suppl 2:5-7. Exclude non-U.S. data.

Reeve T. Implementing guidelines about colorectal cancer: 

a national survey of target groups. ANZ J Surg 

2001;71(3):137-8. Exclude non-U.S. data. 

Reinbach DH, McGregor JR, Murray GD, et al. Effect of 

the surgeon's specialty interest on the type of resection 

performed for colorectal cancer. Dis Colon Rectum 

1994;37(10):1020-3. Exclude non-U.S. data. 

Reinus JF. Guidelines for clinical practice. Management of 

colorectal polyps. Dig Dis Sci 1994;39(10):2282-4. 

Exclude review. 

Renehan A, O'Dwyer S. Letters to the Editor. Ann Surg 

1999;229(3):442-3. Exclude non-U.S. data. 

Renehan A, O'Dwyer ST. A selective policy in follow-up 

for bowel cancer. Lancet 1998;351(9119):1891. Exclude no 

explicit QM. 

Renehan AG, Egger M, Saunders MP, et al. Mechanisms of 

improved survival from intensive followup in colorectal 

cancer: a hypothesis. Br J Cancer 2005;92(3):430-3. 

Exclude non-U.S. data.

Renwick AA, Bokey EL, Chapuis PH, et al. Effect of 

supervised surgical training on outcomes after resection of 

colorectal cancer. Br J Surg 2005;92(5):631-6. Exclude 

non-U.S. data. 

Retchin SM, Penberthy L, Desch C, et al. Perioperative 

management of colon cancer under Medicare risk 

programs. Arch Intern Med 1997;157(16):1878-84. 

Exclude wrong population. 

Rex D, Bond J, Winawer S, et al. Quality in the technical 

performance of colonoscopy and the continuous quality 

improvement process for colonoscopy: recommendations 

of the U.S. Multi-Society Task Force on Colorectal Cancer. 

Am J Gastroenterol 2002;97(6):1296-308. Exclude review.

Rex DK. Colonoscopy turning the focus on quality. Dig 

Liver Dis 2002;34(12):831-2. Exclude review. 

Rex DK, Cutler CS, Lemmel GT, et al. Colonoscopic miss 

rates of adenomas determined by back-to-back 

colonoscopies. Gastroenterology 1997;112(1):24-8. 

Exclude no explicit QM. 
C-29

Rex DK, Ness RM. Disease-specific outcomes assessment 

for diagnosis of colorectal neoplasia: from clinical trials to 

community practice. Gastrointest Endosc Clin N Am 

1999;9(4):697-704, ix. Exclude review. 

Rex DK, Rawl SM, Rabeneck L, et al. Colorectal cancer in 

African Americans. Rev Gastroenterol Disord 

2004;4(2):60-5. Exclude review. 

Rex DK, Rahmani E, Haseman J, et al. Relative sensitivity 

of colonoscopy and barium enema for detection of 

colorectal cancer in clinical practice. Gastroenterology 

1997;112:17-23. Exclude data sources.

Reynolds T. Why randomized surgical oncology trials are 

so scarce. J Natl Cancer Inst 1999;91(14):1182-3. Exclude 

review. 

Richards RJ, Minocha A. Screening interval for colorectal 

cancer with colonoscopy: the long and short of it. Am J 

Gastroenterol 1997;92(5):904-5. Exclude review. 

Rickert RR, Compton CC. Protocol for the examination of 

specimens from patients with carcinomas of the anus and 

anal canal: a basis for checklists. Cancer Committee of the 

College of American Pathologists. Arch Pathol Lab Med 

2000;124(1):21-5. Exclude falls outside Task Order scope. 

Riesewyk C, Hawyard C, Enser V, et al. Referral and 

diagnostic process in suspected colorectal cancer needs to 

be improved to achieve two week target. BMJ 

2000;321(7275):1527. Exclude non-U.S. data.

Rigby K, Brown SR, Lakin G, et al. The use of a proforma 

improves colorectal cancer pathology reporting. Ann R 

Coll Surg Engl 1999;81(6):401-3. Exclude non-U.S. data. 

Robinson MH, Marks CG, Farrands PA, et al. Screening 

for colorectal cancer with an immunological faecal occult 

blood test: 2-year follow-up. Br J Surg . 1996;83(4):500-1. 

Exclude falls outside Task Order scope.

Rodney WM, Dabov G, Cronin C. Evolving colonoscopy 

skills in a rural family practice: the first 293 cases. Fam 

Pract Res J 1993;13(1):43-52. Exclude falls outside Task 

Order scope. 

Roetzheim RG, Gonzalez EC, Ramirez A, et al. Primary 

care physician supply and colorectal cancer. J Fam Pract 

2001;50(12):1027-31. Exclude no explicit QM.

Roh MS. Expanding the indications for hepatic resection in 

patients with colorectal liver metastases. Ann Surg Oncol 

2004;11(3):238-9. Exclude no explicit QM. 

Rollins G. PET scans improve management decisions, 

reduce unnecessary surgery in recurrent colorectal cancer. 

Rep Med Guidel Outcomes Res 2002;13(9):7-9. Exclude 

falls outside Task Order scope.

Roman S, Cenni JC, Roy P, et al. Value of rectal ultrasound 

in predicting staging and outcome in patients with rectal 

adenocarcinoma. Dis Colon Rectum 2004;47(8):1323-30. 

Exclude no explicit QM. 

Rose JH, O'Toole EE, Dawson NV, et al. Age differences 

in care practices and outcomes for hospitalized patients 

with cancer. J Am Geriatr Soc 2000;48(5 Suppl):S25-32. 

Exclude falls outside Task Order scope. 

Rosen C, Nagorney D, Taswell H, et al. Perioperative 

Blood Transfusion and Determinants of Survival after liver 

resection for metastatic colorectal carcinoma. Ann Surg 

1992;216(4):493-505. Exclude no explicit QM. 

Rosen L, Stasik JJ Jr, Reed JF 3rd, et al. Variations in colon 

and rectal surgical mortality. Comparison of specialties 

with a state-legislated database. Dis Colon Rectum 

1996;39(2):129-35. . Exclude survival only.

Rothenberger D. The American Cancer Society guidelines 

for colorectal cancer screening: have we gone too far (or 

not far enough)?. Gastroenterology 1998;114(6):1341-3. 

Exclude no explicit QM. 

Rothenberger DA. Evidence-based practice requires 

evidence. Br J Surg 2004;91(11):1387-8. Exclude no 

explicit QM. 

Rothenberger DA, Akbari R, Baxter NN. Are we 

overtreating some patients with rectal cancer?. Oncology 

(Huntingt) 1799;18(14):1789-96; discussion 1796. Exclude 

falls outside Task Order scope. 

Rothenberger D, Garcia-Aguilar J. Management of cancer 

in a polyp. In: Saltz L, ed. Colorectal cancer: multimodality 

management. New Jersey: Saunders. 2002;325-35. Exclude 

review. 

Rozmovits L, Rose P, Ziebland S. In the absence of 

evidence, who chooses? A qualitative study of patients' 

needs after treatment for colorectal cancer. J Health Serv 

Res Pol 2004;9(3):159-64. Exclude no explicit QM. 

Ruo L, Cellini C, La-Calle JP Jr, et al. Limitations of 

family cancer history assessment at initial surgical 

consultation. Dis Colon Rectum 2001;44(1):98-103; 

discussion 103-4. Exclude falls outside Task Order scope. 

Sahay TB, Gray RE, Fitch M. A qualitative study of patient 

perspectives on colorectal cancer. Cancer Pract 

2000;8(1):38-44. Exclude non-U.S. data. 

Salkeld G, Solomon M, Short L, et al. A matter of trustpatient's views on decision-making in colorectal cancer. 

Health Expect 2004;7(2):104. Exclude non-U.S. data.
C-30

Sanchez KM, Richardson JL, Mason HR. The return to 

work experiences of colorectal cancer survivors. AAOHN J 

2004;52(12):500-10. Exclude falls outside Task Order 

scope. 

Sant M, Capocaccia R, Verdecchia A, et al. Comparisons 

of colon-cancer survival among European countries: The 

Eurocare Study. Int J Cancer 1995;63(1):43-8. Exclude 

non-U.S. data.

Sargent DJ, Goldberg RM, Jacobson SD, et al. A pooled 

analysis of adjuvant chemotherapy for resected colon 

cancer in elderly patients. N Engl J Med 

2001;345(15):1091-7. Exclude no explicit QM.

Sarli L, Pietra N, Costi R, et al. Adenocarcinoma of the 

colon and rectum: some reflections on the immediate and 

long-term results of surgery. Ital J Gastroenterol 

1998;30(6):641-7. Exclude non-U.S. data. 

Sato T, Konishi F, Endoh N, et al. Long-term outcomes of 

a neo-anus with a pudendal nerve anastomosis 

contemporaneously reconstructed with an 

abdominoperineal excision of the rectum. Surgery 

2005;137(1):8-15. Exclude non-U.S. data.

Sauer R, Becker H, Hohenberger W, et al. Preoperative 

versus postoperative chemoradiotherapy for rectal cancer. 

N Engl J Med 2004;351(17):1731-40. Exclude no explicit 

QM.

Schag CA, Ganz PA, Heinrich RL. CAncer Rehabilitation 

Evaluation System--short form (CARES-SF). A cancer 

specific rehabilitation and quality of life instrument. Cancer 

1991;68(6):1406-13. Exclude no explicit QM. 

Scheidbach H, Schneider C, Hugel O, et al. Oncological 

quality and preliminary long-term results in laparoscopic 

colorectal surgery. Surg Endosc 2003;17(6):903-10. 

Exclude no explicit QM.

Schiowitz M. Letters to the Editor. Ann Surg 

1999;229(3):442-3. Exclude no explicit QM.

Schmidt CE, Bestmann B, Kuchler T, et al. Impact of age 

on quality of life in patients with rectal cancer. World J 

Surg 2005;29(2):190-7. Exclude non-U.S. data. 

Schmidt CE, Bestmann B, Kuchler T, et al. Prospective 

evaluation of quality of life of patients receiving either 

abdominoperineal resection or sphincter-preserving 

procedure for rectal cancer. Ann Surg Oncol 

2005;12(2):117-23. Exclude non-U.S. data. 

Schmidt CE, Bestmann B, Kuchler T, et al. Ten-year 

historic cohort of quality of life and sexuality in patients 

with rectal cancer. Dis Colon Rectum 2005;48(3):483-92. 

Exclude non-U.S. data. 

Schneider E, Epstein A, Malin J, et al. Developing a 

System to Assess the Quality of Cancer Care: ASCO's 

National Initiative on Cancer Care Quality. J Clin Oncol 

2004;22(15):2985-91. Exclude no explicit QM. 

Schoemaker D, Black R, Giles L, et al. Yearly 

colonoscopy, liver CT, and chest radiography do not 

influence 5-year survival of colorectal cancer patients. 

Gastroenterology 1998;114(1):7-14. Exclude no explicit 

QM. 

Schoenfeld PS, Cash B, Kita J, et al. Effectiveness and 

patient satisfaction with screening flexible sigmoidoscopy 

performed by registered nurses. Gastrointest Endosc 

1999;49(2):158-62. Exclude falls outside Task Order scope. 

Schoetz DJ Jr, Bockler M, Rosenblatt MS, et al. "Ideal" 

length of stay after colectomy: whose ideal?. Dis Colon 

Rectum 1997;40(7):806-10. Exclude no explicit QM. 

Scholefield JH, Steele RJ, British Society For 

Gastroenterology, et al. Guidelines for follow up after 

resection of colorectal cancer. Gut 2002;51(Suppl 5):V3-5. 

Exclude non-U.S. data.

Schorr M, Siebeck M, Hohenbleicher F, et al. Colorectal 

cancer: A rational follow-up program adapted to patient's 

individual risk of recurrence and review of the literature. 

Eur J Med Res 1999;4(7):275-82. Exclude no explicit QM. 

Schrag D. Communication and coordination: the keys to 

quality. Jl Clin Oncol 2005;23(27):tbd. Exclude no explicit 

QM.

Schroy PC 3rd, Wilson S, Afdhal N. Feasibility of highvolume screening sigmoidoscopy using a flexible fiberoptic 

endoscope and a disposable sheath system. Am J 

Gastroenterol 1996;91(7):1331-7. Exclude falls outside 

Task Order scope. 

Schwartz KL, Crossley-May H, Vigneau FD, et al. Race, 

socioeconomic status and stage at diagnosis for five 

common malignancies. Cancer Causes Control 

2003;14(8):761-6. Exclude no explicit QM. 

Schwartz MH, Tartter PI. Decreased length of stay for 

patients with colorectal cancer: implications of DRG use. J 

Healthc Qual 1998;20(4):22-5. Exclude falls outside Task 

Order scope. 

Schwenk W, Neudecker J, Haase O, et al. Comparison of 

EORTC quality of life core questionnaire (EORTC-QLQC30) and gastrointestinal quality of life index (GIQLI) in 

patients undergoing elective colorectal cancer resection. Int 

J Colorectal Dis 2004;19(6):554-60. Exclude QOL. 
C-31

Secco GB, Fardelli R, Gianquinto D, et al. Efficacy and 

cost of risk-adapted follow-up in patients after colorectal 

cancer surgery: a prospective, randomized and controlled 

trial. Eur J Surg Oncol 2002;28(4):418-23. Exclude nonU.S. data. 

Seeff LC, Shapiro JA, Nadel MR. Are we doing enough to 

screen for colorectal cancer? Findings from the 1999 

Behavioral Risk Factor Surveillance System. J Fam Pract 

2002;51(9):761-6. Exclude falls outside Task Order scope.

Seitz U, Bohnacker S, Seewald S, et al. Long-term results 

of endoscopic removal of large colorectal adenomas. 

Endocsopy 2003;35(8):S41-4. Exclude no explicit QM. 

Senagore AJ, Warmuth AJ, Delaney CP, et al. POSSUM, 

p-POSSUM, and Cr-POSSUM: implementation issues in a 

United States health care system for prediction of outcome 

for colon cancer resection. Dis Colon Rectum 

2004;47(9):1435-41. Exclude no explicit QM. 

Seymour MT, Stenning SP, Cassidy J. Attitudes and 

practice in the management of metastatic colorectal cancer 

in Britain. Colorectal Cancer Working Party of the UK 

Medical Research Council. Clin Oncol (R Coll Radiol) 

1997;9(4):248-51. Exclude non-U.S. data. 

Shankar A, Taylor I. Treatment of colorectal cancer in 

patients aged over 75. Eur J Surg Oncol 1998;24(5):391-5. 

Exclude non-U.S. data. 

Shavers VL, Brown ML. Racial and ethnic disparities in 

the receipt of cancer treatment. J Natl Cancer Inst 

2002;94(5):334-57. Exclude review. 

Shaw JW, Johnson JA, Coons SJ. US Valuation of the EQ5D Health States:development and testing of the D1 

valuation model. Med Care 2005;43(3):203-20. Exclude 

QOL. 

Shepherd NA, Quirke P. Colorectal cancer reporting: are 

we failing the patient? J Clin Pathol 1997;50(4):266-7. 

Exclude non-U.S. data. 

Shibata S, Shiroma T, Tsukiji M, et al. Report of colon 

cancer detection in mass surveys using immunological 

occult blood test (Latex method). J Med Syst 1993;17(3-

4):153-6. Exclude falls outside Task Order scope. 

Simon JB. Should all people over the age of 50 have 

regular fecal occult-blood tests? Postpone population 

screening until problems are solved. N Engl J Med 

1998;338(16):1151-2; discussion 1154-5. Exclude falls 

outside Task Order scope. 

Simons BD, Morrison AS, Lev R, et al. Relationship of 

polyps to cancer of the large intestine. J Natl Cancer Inst 

1992;84(12):962-6. Exclude no explicit QM. 

Simunovic M, To T, Baxter N, et al. Hospital procedure 

volume and teaching status do not influence treatment and 

outcome measures of rectal cancer surgery in a large 

general population. J Gastrointest Surg 2000;4(3):324-30. 

Exclude non-U.S. data.

Singer RB. Fecal occult blood testing and the incidence of 

colorectal cancer. J Insur Med 2002;34(2):89-91. Exclude 

falls outside Task Order scope. 

Singh GK, Miller BA, Hankey BF. Changing area 

socioeconomic patterns in U.S. cancer mortality, 1950-

1998: Part II--Lung and colorectal cancers. J Natl Cancer 

Inst 2002;94(12):916-25. Exclude no explicit QM. 

Singh SM, Paszat LF, Li C, et al. Association of 

socioeconomic status and receipt of colorectal cancer 

investigations: a population-based retrospective cohort 

study. CMAJ 2004;171(5):461-5. Exclude non-U.S. data. 

Siu AL, McGlynn EA, Morgenstern H, et al. Choosing 

quality of care measures based on the expected impact of 

improved care on health. Health Serv Res 1992;27(5):619-

50. Exclude no explicit QM. 

Sloan J, Loprinzi C, Kuross S, et al. Randomized 

Comparison of Four Tools Measuring Overall Quality of 

Life in Patients with Advanced Cancer. J Clin Oncol 

1998;16(11):3662-73. Exclude QOL. 

Smith J, King P, Lane R, et al. Evidence of the effect of 

specialization on the management, surgical outcome and 

survival form colorectal cancer in Wessex. Br J Surg 

2003;90:583-92. Exclude non-U.S. data. 

Smith JJ, Lee J, Burke C, et al. Major colorectal cancer 

resection should not be denied to the elderly. Eur J Surg 

Oncol 2002;28(6):661-6. Exclude non-U.S. data.

Smith LE, Ko ST, Saclarides T, et al. Transanal endoscopic 

microsurgery. Initial registry results. Dis Colon Rectum 

1996;39(10 Suppl):S79-84. Exclude no explicit QM. 

Smith TJ, Hillner BE, Bear HD. Taking action on the 

volume-quality relationship: how long can we hide our 

heads in the colostomy bag?. J Natl Cancer Inst 

2003;95(10):695-7. Exclude no explicit QM.

Solomon MJ, Pager CK, Keshava A, et al. What do patients 

want? Patient preferences and surrogate decision making in 

the treatment of colorectal cancer. Dis Colon Rectum 

2003;46(10):1351-7. Exclude non-U.S. data. 

Solomon MJ, Pager CK, Young JM, et al. Patient entry into 

randomized controlled trials of colorectal cancer treatment: 

factors influencing participation. Surgery 2003;133(6):608-

13. Exclude no explicit QM. 
C-32

Solomon MJ, Thomas RJ, Gattellari M, et al. Does type of 

surgeon matter in rectal cancer surgery? Evidence, 

guideline consensus and surgeons' views. ANZ J Surg 

2001;71(12):711-4. Exclude non-U.S. data. 

Soloway G, Reinus JF. Flexible sigmoidoscopy may be 

ineffective for secondary prevention of colorectal cancer in 

asymptomatic, average-risk men. Gastrointest Endosc 

1992;38(2):205-6. Exclude falls outside Task Order scope. 

Sorensen HT, Ejlersen E, Moller-Petersen J, et al. Overall 

use of proctoscopy in general practice and possible relation 

to the stage of rectal cancer. Fam Pract 1992;9(2):145-8. 

Exclude non-U.S. data. 

Soybel DI. Surgeon-related factors and outcome in rectal 

cancer. Gastroenterology 1999;116(1):220-1. Exclude 

review. 

Spatz H, Engel J, Holzel D, et al. The surgical discharge 

summary: a lack of substantial clinical information may 

affect the postop treatment of rectal cancer patients. 

Langenbecks Arch Surg 2001;386(5):350-6. Exclude nonU.S. data.

Sprangers MA. Quality-of-life assessment in colorectal 

cancer patients: evaluation of cancer therapies. Semin 

Oncol 1999;26(6):691-6. Exclude review. 

Sprangers MA, te Velde A, Aaronson NK. The 

construction and testing of the EORTC colorectal cancerspecific quality of life questionnaire module (QLQ-CR38). 

European Organization for Research and Treatment of 

Cancer Study Group on Quality of Life. Eur J Cancer 

1999;35(2):238-47. Exclude non-U.S. data. 

Staib L, Link KH, Blatz A, et al. Surgery of colorectal 

cancer: surgical morbidity and five- and ten-year results in 

2400 patients--monoinstitutional experience. World J Surg 

2002;26(1):59-66. Exclude non-U.S. data. 

Staib L, Schirrmeister H, Reske SN, et al. Is (18)Ffluorodeoxyglucose positron emission tomography in 

recurrent colorectal cancer a contribution to surgical 

decision making? Am J Surg 2000;180(1):1-5. Exclude 

non-U.S. data. 

Steele G. National Cancer Database: Annual review of 

patient care, 1993. Atlanta, Ga. American Cancer Society 

1993;20-36. Exclude publication duplicate. 

Stevenson G. Radiology in the detection and prevention of 

colorectal cancer. Eur J Cancer 1995;31A(7-8):1121-6. 

Exclude non-U.S. data.

Stewart BT, Keck JO, Duncan AV, et al. Difficult or 

incomplete flexible sigmoidoscopy: implications for a 

screening programme. Ann N Z J Surg 1999;69(1):19-21. 

Exclude non-U.S. data. 

Stewart SL, King JB, Thompson TD, et al. Cancer 

mortality surveillance--United States, 1990-2000. MMWR 

Surveill Summ 2004;53(3):1-108. Exclude survival only.

Stiggelbout AM, de Haes JC, Vree R, et al. Follow-up of 

colorectal cancer patients: quality of life and attitudes 

towards follow-up. Br J Cancer 1997;75(6):914-20. 

Exclude non-U.S. data.

Stockton D, Davies T. Multiple cancer site comparison of 

adjusted survival by hospital of treatment: an East Anglian 

study. Br J Cancer 2000;82(1):208-12. Exclude non-U.S. 

data.

Sugarman JR, Dennis LK, White E. Cancer survival among 

American Indians in western Washington State (United 

States). Cancer Causes Control 1994;5(5):440-8. Exclude 

survival only.

Summerhayes M. Choice of chemotherapy regimen for 

advanced colorectal cancer. Clin Oncol (R Coll Radiol) 

2002;14(6):520-1. Exclude non-U.S. data. 

Swanson RS. Palliation of rectal cancer: expertise and 

selection are the keys. Ann Surg Oncol 2002;9(10):937-8. 

Exclude review. 

Swanson RS, Compton CC, Stewart AK, et al. The 

prognosis of T3N0 colon cancer is dependent on the 

number of lymph nodes examined. Ann Surg Oncol 

2003;10(1):65-71. Exclude no explicit QM. 

Swedish Revtal Cancer Trial. Improved survival with 

preoperative radiotherapy in resectable rectal cancer. N 

Engl J Med 1997;336(14):980-7. Exclude no explicit QM. 

Tagore KS, Lawson MJ, Yucaitis JA, et al. Sensitivity and 

specificity of a stool DNA multitarget assay panel for the 

detection of advanced colorectal neoplasia. Clin Colorectal 

Cancer 2003;3(1):47-53. Exclude falls outside Task Order 

scope. 

Taylor I. Quality of follow-up of the cancer patient 

affecting outcome. Surg Oncol Clin N Am 2000;9(1):21-5, 

vi. Exclude non-U.S. data.

Taylor KL, Shelby R, Gelmann E, et al. Quality of life and 

trial adherence among participants in the prostate, lung, 

colorectal, and ovarian cancer screening trial. J Natl Cancer 

Inst 2004;96(14):1083-94. Exclude falls outside Task Order 

scope. 

Taylor P. One stop to the best patient care. Interview by 

Charlotte Alderman. Nurs Stand 1998;12(52):28-30. 

Exclude falls outside Task Order scope. 

Tekkis P, Poloniecki J, Thompson M, et al. Operative 

Mortality in colorectal cancer: prospective national study. 

Br Med J 2003;327(7425):1196-201. Exclude non-U.S. 

data. 
C-33

Tekkis PP, Senagore AJ, Delaney CP, et al. Evaluation of 

the learning curve in laparoscopic colorectal surgery: 

comparison of right-sided and left-sided resections. Ann 

Surg 2005;242(1):83-91. Exclude falls outside Task Order 

scope. 

Temple LK, Bacik J, Savatta SG, et al. The development of 

a validated instrument to evaluate bowel function after 

sphincter-preserving surgery for rectal cancer. Dis Colon 

Rectum 2005;48(7):1353-65. Exclude QOL. 

Tepper JE, O'Connell MJ, Niedzwiecki D, et al. Impact of 

number of nodes retrieved on outcome in patients with 

rectal cancer. J Clin Oncol 2001;19(1):157-63. Exclude no 

explicit QM.

Termuhlen PM, Kemeny MM. Surgery in the older patient. 

Oncology (Huntingt) 2002;16(2):183-9; discussion 194-6, 

199. Exclude review.

Thompson GA, Cocks JR, Collopy BT, et al. Clinical 

indicators in colorectal surgery. J Qual Clin Pract 

1996;16(4):229-30. Exclude non-U.S. data. 

Thompson GA, Cocks JR, Collopy BT, et al. Colorectal 

resection in Victoria:a comparison of hospital based and 

individual audit. Ann N Z J Surg 1996;66(8):520-4. 

Exclude non-U.S. data.

Thompson W. Wessex regional radiology audit: barium 

enema in colo-rectal carcinoma. Clin Radiol 

1996;51(7):525. Exclude non-U.S. data.

Tilney HS, Trickett JP, Edwards DP. A novel indicator for 

surveillance colonoscopy following colorectal cancer 

resection. Colorectal Dis 2003;5(3):270; author reply 271. 

Exclude falls outside Task Order scope.

Toby G, Bowman H, Tielsch J, et al. Statewide 

Regionalization of Pancreaticoduodenectomy and its Effect 

on In-Hospital Mortality. Ann Surg 1998;228(1):71-8. 

Exclude falls outside Task Order scope.

Topham C, Moore J. Patient preferences for chemotherapy 

schedules used in the treatment of advanced colorectal 

cancer--a pilot study. Eur J Cancer Care (Engl) 

1997;6(4):291-4. Exclude non-U.S. data. 

Torfs K, Pocceschi S. A retrospective study of resource 

utilisation in the treatment of advanced colorectal cancer in 

Europe. Eur J Cancer 1996;32A Suppl 5:S28-31. Exclude 

non-U.S. data

Towler B, Irwig L, Glasziou P, et al. A systematic review 

of the effects of screening for colorectal cancer using the 

faecal occult blood test, hemoccult. BMJ 

1998;317(7158):559-65. Exclude review.

Triantafillidis JK, Papatheodorou K, Kogevinas M, et al. 

Prognostic factors affecting the survival of operated 

patients with colorectal cancer: significance of delayed 

hypersensitivity skin reactions and nutritional status. Ital J 

Gastroenterol 1995;27(8):419-24. Exclude non-U.S. data.

Triantafyllou K, Papatheodoridis GV, Paspatis GA, et al. 

Predictors of the early development of advanced 

metachronous colon adenomas. Hepatogastroenterology 

1997;44(14):533-8. Exclude non-U.S. data.

Trillet-Lenoir V, Freyer G, Kaemmerlen P, et al. 

Assessment of tumour response to chemotherapy for 

metastatic colorectal cancer: accuracy of the RECIST 

criteria. Br J Radiol 2002;75(899):903-8. Exclude non-U.S. 

data.

Trimble EL, Carter CL, Cain D, et al. Representation of 

older patients in cancer treatment trials. Cancer 1994;74(7 

Suppl):2208-14. Exclude falls outside Task Order scope. 

Troxel AB. A comparative analysis of quality of life data 

from a Southwest Oncology Group randomized trial of 

advanced colorectal cancer. Stat Med 1998;17(5-7):767-79. 

Exclude data sources.

Tsalic M, Bar-Sela G, Beny A, et al. Severe toxicity related 

to the 5-fluorouracil/leucovorin combination (the Mayo 

Clinic regimen): a prospective study in colorectal cancer 

patients. Am J Clin Oncol 2003;26(1):103-6. Exclude nonU.S. data. 

Tulsky JA. Beyond advance directives: importance of 

communication skills at the end of life. JAMA 

2005;294(3):359-65. Exclude falls outside Task Order 

scope. 

Turner B, Myers RE, Hyslop T, et al. Physician and patient 

factors associated with ordering a colon evaluation after a 

positive fecal occult blood test. J Gen Intern Med 

2003;18(5):357-63. Exclude no explicit QM. 

Tveit KM, ESMO Guidelines Task Force. ESMO 

Minimum Clinical Recommendations for diagnosis, 

treatment and follow-up of rectal cancer. Ann Oncol 

2003;14(7):1006-7. Exclude non-U.S. data. 

Tyldesley S, Zhang-Salomons J, Groome PA, et al. 

Association between age and the utilization of radiotherapy 

in Ontario. Int J Radiat Oncol Biol Phys 2000;47(2):469-

80. Exclude non-U.S. data. 

Ubhi SS, Kent SJ. Which surgeons in a district general 

hospital should treat patients with carcinoma of the rectum? 

J R Coll Surg Edinb 1995;40(1):52-4. Exclude non-U.S. 

data. 
C-34

UK Department of Health. Manual of quality measures for 

cancer peer review 2004-draft for consultation: (manual for 

cancer services assessment). Available at: 

http://www.dh.gov.uk/policyandguidance/healthandsocialc

aretopics/cancer/cancerarticle. Accessed April 6, 2005. 

2004. Exclude review. 

Ulander K, Jeppsson B, Grahn G. Quality of life and 

independence in activities of daily living preoperatively 

and at follow-up in patients with colorectal cancer. Support 

Care Cancer 1997;5(5):402-9 . Exclude non-U.S. data. 

Urbach DR, Bell CM, Austin PC. Differences in operative 

mortality between high- and low-volume hospitals in 

Ontario for 5 major surgical procedures: estimating the 

number of lives potentially saved through regionalization. 

CMAJ 2003;168(11):1409-14. Exclude non-U.S. data.

Urban M, Rosen HR, Holbling N, et al. MR imaging for the 

preoperative planning of sphincter-saving surgery for 

tumors of the lower third of the rectum: use of intravenous 

and endorectal contrast materials. Radiology 

2000;214(2):503-8. Exclude no explicit QM.

Valentini V, Genovesi D, Polico C, et al. Quality assurance 

procedures in radiotherapy of rectal neoplasms. Rays 

1996;21(4):649-62. Exclude non-U.S. data.

van den Brink M, van den Hout WB, Stiggelbout AM, et al. 

Cost measurement in economic evaluations of health care: 

whom to ask? Med Care 2004;42(8):740-6. Exclude nonU.S. data.

van Harten WH, van Noort O, Warmerdam R, et al. 

Assessment of rehabilitation needs in cancer patients. Int J 

Rehabil Res 1998;21(3):247-57. Exclude non-U.S. data.

Vanchieri C. Colon cancer update triggers changes in 

practice. J Natl Cancer Inst 1990;82(11):898. Exclude falls 

outside Task Order scope. 

Vaughn DJ, Haller DG. Adjuvant therapy for colorectal 

cancer: past accomplishments, future directions. Cancer 

Invest 1997;15(5):435-47. Exclude review.

Verma S, Giaffer MH. Open-access versus hospitalinitiated flexible sigmoidoscopy: a comparative audit of 

efficacy. Eur J Gastroenterol Hepatol 2001;13(6):655-8. 

Exclude non-U.S. data.

Vernava AM 3rd, Longo WE, Virgo KS, et al. Current 

follow-up strategies after resection of colon cancer. Results 

of a survey of members of the American Society of Colon 

and Rectal Surgeons. Dis Colon Rectum 1994;37(6):573-

83. Exclude falls outside Task Order scope.

Verne JE, Aubrey R, Love SB, et al. Population based 

randomized study of uptake and yield of screening by 

flexible sigmoidoscopy compared with screening by faecal 

occult blood testing. BMJ 1998;317(7152):182-5. Exclude 

non-U.S. data. 

Vernon SW. Participation in colorectal cancer screening: a 

review. J Natl Cancer Inst 1997;89(19):1406-22. Exclude 

review.

Vernon SW, Hughes JI, Heckel VM. HMO versus fee for 

service practice: a four-year retrospective analysis of 

colorectal cancer diagnosis and treatment. Is there a 

difference in quality of care? Prog Clin Biol Res 

1990;339:341-50. Exclude no explicit QM. 

Vernon SW, Meissner H, Klabunde C, et al. Measures for 

ascertaining use of colorectal cancer screening in 

behavioral, health services, and epidemiologic research. 

Cancer Epidemiol Biomarkers Prev 2004;13(6):898-905. 

Exclude falls outside Task Order scope. 

Vickers N, Pollock A. Incompleteness and retrieval of case 

notes in a case note audit of colorectal cancer. Qual Health 

Care 1993;2(3):170-4. Exclude non-U.S. data.

Vincenzi B, Cesa AL, Santini D, et al. Predictive factors for 

response to chemotherapy in colorectal cancer patients. Crit 

Rev Oncol Hematol 2004;52(1):45-60. Exclude non-U.S. 

data. 

Violi V, Pietra N, Grattarola M, et al. Curative surgery for 

colorectal cancer: long-term results and life expectancy in 

the elderly. Dis Colon Rectum 1998;41(3):291-8. Exclude 

non-U.S. data. 

Virgo KS, Vernava AM, Longo WE, et al. Cost of patient 

follow-up after potentially curative colorectal cancer 

treatment. JAMA 1995;273(23):1837-41. Exclude no 

explicit QM. 

Virgo KS, Wade TP, Longo WE, et al. Surveillance after 

curative colon cancer resection: practice patterns of 

surgical subspecialists. Ann Surg Oncol 1995;2(6):472-82. 

Exclude falls outside Task Order scope. 

Vironen JH, Sainio P, Husa AI, et al. Complications and 

survival after surgery for rectal cancer in patients younger 

than and aged 75 years or older. Dis Colon Rectum 

2004;47(7):1225-31. Exclude non-U.S. data. 

Vukasin P, Ortega AE, Greene FL, et al. Wound recurrence 

following laparoscopic colon cancer resection. Results of 

the American Society of Colon and Rectal Surgeons 

Laparoscopic Registry. Dis Colon Rectum 1996;39(10 

Suppl):S20-3. Exclude no explicit QM.
C-35

Wagman RT, Minsky BD. Conservative management of 

rectal cancer with local excision and adjuvant therapy. 

Oncology (Huntingt) 2001;15(4):513-9, 524;discussion 

524-8. Exclude review. 

Wakefield SE, Baigrie RJ, MacLeod AJ, et al. Colorectal 

cancer surgery in a district general hospital. J R Coll Surg 

Edinb 1996;41(6):374-8. Exclude non-U.S. data. 

Walker MS, Ristvedt SL, Haughey BH. Patient care in 

multidisciplinary cancer clinics: does attention to 

psychosocial needs predict patient satisfaction?. 

Psychooncology. 2003;12(3):291-300. Exclude no explicit 

QM.

Walsh JM, Terdiman JP. Colorectal cancer screening: 

scientific review. JAMA 2003;289(10):1288-96. Exclude 

falls outside Task Order scope. 

Walsh S, Bruce C, Bennington S, et al. The fourteen-day 

rule and colorectal cancer. Ann R Coll Surg Engl 

2002;84(6):386-8. Exclude non-U.S. data.

Walter LC, Davidowitz NP, Heineken PA, et al. Pitfalls of 

converting practice guidelines into quality measures: 

lessons learned from a VA performance measure. JAMA 

2004;291(20):2466-70. Exclude no explicit QM.

Wanebo HJ, Chu QD, Vezeridis MP, et al. Patient selection 

for hepatic resection of colorectal metastases. Arch Surg 

1996;131(3):322-9. Exclude no explicit QM. 

Ward WL, Hahn EA, Mo F, et al. Reliability and validity of 

the Functional Assessment of Cancer Therapy-Colorectal 

(FACT-C) quality of life instrument. Qual Life Res 

1999;8(3):181-95. Exclude data sources. 

Warren JL, Harlan LC. Can cancer registry data be used to 

study cancer treatment? Med Care 2003;41(9):1003-5. 

Exclude review.

Warren JL, Harlan LC, Fahey A, et al. Utility of the SEERMedicare data to identify chemotherapy use. Med Care 

2002;40(8 Suppl):IV-55-61. Exclude data sources.

Warsi AA, White S, McCulloch P. Completeness of data 

entry in three cancer surgery databases. Eur J Surg Oncol 

2002;28(8):850-6. Exclude non-U.S. data.

Weeks J, Nelson H, Gelber S, et al. Short-term quality of 

life outcomes following laparoscopic-assisted colectomy vs 

open colectomy for colon cancer: a randomized trial. 

JAMA 2002;287(3):321-8. Exclude QOL.

Weeks JC. Preferences of older cancer patients: can you 

judge a book by its cover?. J Natl Cancer Inst 

1994;86(23):1743-4. Exclude review. 

Wei EK, Ryan CT, Dietrich AJ, et al. Improving colorectal 

cancer screening by targeting office systems in primary 

care practices: disseminating research results into clinical 

practice. Arch Intern Med 2005;165(6):661-6. Exclude 

wrong population. 

Weissfeld JL, Ling BS, Schoen RE, et al. Adherence to 

repeat screening flexible sigmoidoscopy in the Prostate, 

Lung, Colorectal, and Ovarian (PLCO) Cancer Screening 

Trial. Cancer 2002;94(10):2569-76. Exclude falls outside 

Task Order scope. 

Wexner SD, Forde KA, Sellers G, et al. How well can 

surgeons perform colonoscopy? Surg Endosc 

1998;12(12):1410-4. Exclude falls outside Task Order 

scope. 

Wexner SD, Rotholtz NA. Surgeon influenced variables in 

resectional rectal cancer surgery. Dis Colon Rectum 

2000;43(11):1606-27. Exclude review.

Whittle J, Steinberg EP, Anderson GF, et al. Accuracy of 

Medicare claims data for estimation of cancer incidence 

and resection rates among elderly Americans. Med Care 

1991;29(12):1226-36. Exclude data sources.

Whynes DK, Frew EJ, Manghan CM, et al. Colorectal 

cancer, screening and survival: the influence of socioeconomic deprivation. Public Health 2003;117(6):389-95. 

Exclude non-U.S. data.

Whynes DK, Neilson AR. Symptoms before and after 

surgery for colorectal cancer. Qual Life Res 1997;6(1):61-

6. Exclude non-U.S. data. 

Wibe A, Eriksen MT, Syse A, et al. Total mesorectal 

excision for rectal cancer--what can be achieved by a 

national audit? Colorectal Dis 2003;5(5):471-7. Exclude 

non-U.S. data. 

Wibe A, Moller B, Norstein J, et al. A national strategic 

change in treatment policy for rectal cancer--

implementation of total mesorectal excision as routine 

treatment in Norway. A national audit. Dis Colon Rectum 

2002;45(7):857-66. Exclude non-U.S. data.

Wichmann MW, Muller C, Hornung HM, et al. Gender 

differences in long-term survival of patients with colorectal 

cancer. Br J Surg 2001;88(8):1092-8. Exclude non-U.S. 

data.

Widder J, Sedlmayer F, Stanek C, et al. Quality assurance 

in preoperative radiotherapy of rectal cancer: evaluation of 

a pre-trial dummy-run. Radiother Oncol 2000;56(3):341-7. 

Exclude non-U.S. data.

Wiggers T, van de Velde CH. Reduction "by half". The 

need for standardised surgical technique in studies of 

radiotherapy for rectal cancer. Eur J Surg 1999;165(5):407-

9. Exclude no explicit QM.
C-36

Wils J. Adjuvant treatment of colon cancer. A plea for a 

large-scale European trial. Neth J Med 1994;44(2):37-40. 

Exclude non-U.S. data. 

Winawer S, Fletcher R, Rex D, et al. Colorectal cancer 

screening and surveillance: clinical guidelines and 

rationale-Update based on new evidence. Gastroenterology 

2003;124(2):544-60. Exclude falls outside Task Order 

scope. 

Winawer SJ. Appropriate intervals for surveillance. 

Gastrointest Endosc 1999;49(3 Pt 2):S63-6. Exclude 

review. 

Winawer SJ, Zauber A, O'Brien M, et al. Randomized 

comparison of surveillance intervals after colonoscopie 

removal of newly diagnosed adenomatous polyps. N Engl J 

Med 1993;328:901-6. Exclude no explicit QM. 

Winchester DP, Stewart AK, Bura C, et al. The National 

Cancer Data Base: a clinical surveillance and quality 

improvement tool. J Surg Oncol 2004;85(1):1-3. Exclude 

review.

Wolf DC. Coding for screening and surveillance of 

colorectal cancer in the Medicare population. Inflamm 

Bowel Dis 1998;4(3):255-6. Exclude falls outside Task 

Order scope.

Wolters U, Isenberg J, Stutzer H. Colorectal carcinoma--

aspects of surgery in the elderly. Anticancer Res 

1997;17(2B):1273-6. Exclude non-U.S. data.

Womack C, Thornton-Holmes J, Wells AD. Colorectal 

cancer reporting. J Clin Pathol 1997;50(10):881-2. Exclude 

non-U.S. data.

Wong RK, Gafni A, Whelan T, et al. Defining patientbased minimal clinically important effect sizes: a study in 

palliative radiotherapy for painful unresectable pelvic 

recurrences from rectal cancer. Int J Radiat Oncol Biol 

Phys 2002;54(3):661-9. Exclude non-U.S. data.

Woolf SH. Shared decision-making: the case for letting 

patients decide which choice is best. J Fam Pract 

1997;45(3):205-8. Exclude review.

Worthington TR, Wilson T, Padbury R. Case for 

postoperative surveillance following colorectal cancer 

resection. ANZ J Surg 2004;74(1-2):43-5. Exclude nonU.S. data. 

Wulf J, Kramer K, van Aaken C, et al. Outcome of 

postoperative treatment for rectal cancer UICC stage II and 

III in day-to-day clinical practice. Results from a 

retrospective quality control analysis in six institutions in 

North Bavaria (Germany). Strahlenther Onkol 

2004;180(1):5-14. Exclude non-U.S. data.

Wullenweber HP, Sutter C, Autschbach F, et al. Evaluation 

of Bethesda guidelines in relation to microsatellite 

instability. Dis Colon Rectum 2001;44(9):1281-9. Exclude 

non-U.S. data.

Yabroff KR, Klabunde CN, Myers R, et al. Physician 

recommendations for follow-up of positive fecal occult 

blood tests. Med Care Res Rev 2005;62(1):79-110. Exclude 

no explicit QM.

Yabroff KR, Washington KS, Leader A, et al. Is the 

promise of cancer-screening programs being compromised? 

Quality of follow-up care after abnormal screening results. 

Med Care Res Rev 2003;60(3):294-331. Exclude review.

Yamamoto S, Akasu T, Fujita S, et al. Postsurgical 

surveillance for recurrence of UICC stage I colorectal 

carcinoma: is follow-up by CEA justified? 

Hepatogastroenterology 2005;52(62):444-9. Exclude falls 

outside Task Order scope.

Yamamoto S, Watanabe M, Hasegawa H, et al. Prospective 

evaluation of laparoscopic surgery for rectosigmoidal and 

rectal carcinoma. Dis Colon Rectum 2002;45(12):1648-54. 

Exclude non-U.S. data.

Yancik R, Wesley M, Ries L, et al. Comorbidity and Age s 

Predictors of Risk for Early Mortality of Male and Female 

Colon Carcinoma Patients. Cancer 1998;82(11):2123-34. 

Exclude survival only.

Yancik R, Wesley MN, Ries LA, et al. Comorbidity and 

age as predictors of risk for early mortality of male and 

female colon carcinoma patients: a population-based study. 

Cancer 1998;82(11):2123-34. Exclude no explicit QM.

Yee J. Screening CT colonography. Semin Ultrasound CT 

MR 2003;24(1):12-22. Exclude review. 

Yoong KK, Heymann T. Colonoscopy in the very old: why 

bother? Postgrad Med J 2005;81(953):196-7. Exclude falls 

outside Task Order scope. 

Zauber AG. Quality control for flexible sigmoidoscopy: 

which polyps count? Gastroenterology 2004;126(5):1474-

7. Exclude falls outside Task Order scope. 

Zauber AG, Winawer SJ. Initial management and followup surveillance of patients with colorectal adenomas. 

Gastroenterol Clin North Am 1997;26(1):85-101. Exclude 

review.

Zealley IA, Skehan SJ, Rawlinson J, et al. Selection of 

patients for resection of hepatic metastases: improved 

detection of extrahepatic disease with FDG pet. 

Radiographics 2001;21 (Spec No):S55-69. Exclude nonU.S. data.
C-37

Zelek L, Piedbois P, Buyse M. Contribution of metaanalyses to the evaluation of treatments for advanced 

colorectal cancer. Expert Rev Anticancer Ther 

2002;2(4):417-25. Exclude review. 

Zenni GC, Abraham K, Harford FJ, et al. Characteristics of 

rectal carcinomas that predict the presence of lymph node 

metastases: implications for patient selection for local 

therapy. J Surg Oncol 1998;67(2):99-103. Exclude no 

explicit QM. 

Zielinski SL. Last decade brought new guidelines, 

detection methods for colon cancer screening. J Natl 

Cancer Inst 2004;96(21):1575. Exclude review. 

Zieren HU, Muller JM, Zieren J. Resection of colorectal 

liver metastases in old patients. Hepatogastroenterology 

1994;41(1):34-7. Exclude non-U.S. data. 

Zieren HU, Muller JM, Zieren J, et al. The impact of 

patient's age on surgical therapy of colorectal liver 

metastases. Int Surg 1993;78(4):288-91. Exclude non-U.S. 

data.

Ziogas A, Anton-Culver H. Validation of family history 

data in cancer family registries. Am J Prev Med 

2003;24(2):190-8. Exclude no explicit QM.

Zorcolo L, Covotta L, Carlomagno N, et al. Safety of 

primary anastomosis in emergency colo-rectal surgery. 

Colorectal Dis 2003;5(3):262-9. Exclude non-U.S. data. 
D-1

Appendix D: Sample Data Abstraction Forms* Blank Data Abstraction Form/ET for CRC Care Quality Evidence Report (to be used for published quality measures only) General instructions: 

1) If an article includes more than one quality measure, please complete a separate evidence table entry (row) for each measure. 2) If an article includes a quality measure having to do with the quality of a data source, please note this under “General comment

column, but do not abstract the data pertaining to this measure (we will do that separately). 

3) If an article reports on long-term survival/mortality as a quality measure, please note this under “General comments” in the Com

abstract the data pertaining to this measure (we will do that separately). Please do abstract data on short-term post-surgical mo

Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

First author, date, Pro-Cite# 

Quality measure (QM): [include formal name of measure, if any, or give description (“the percentage of ...”)] Basis of QM: [delete all but one] Clinical practice guideline Other (please specify): None If basis was clinical practice guideline or “other,”was this named/cited in the article? Yes/No If yes, please specify here: If no, do you think you know what it was?:  Yes (please specify):  No Type of QM: [delete all but one in each line] (a) Structure/Process/Outcome (b) General/Technical For structure and process measures, please indicate the Outcome to which the QM is linked:Intent of QM: [delete as appropriate] Quality improvement Accountability Research Other (please specify): 

Study population: N: Age: Race: Sex: Tumor stage: Performance status: Other: Geographic location:

Dates:Healthcare setting:Is this a formal validation study?: Yes/No  If no, does it cite a formal validation study?:  Yes (give citation #)/No Results:

If article should be EXC

why: [delete as approprNon-US data Review article Wrong study population:  Screening of asymp Familial polyposis s Ulcerative colitis  Colorectal CA with  Multiple metastases  Other (please specNo explicit QM with posiNo data source testing oOther (falls outside scopGeneral comments:Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: - Usability: - Scientific acceptability  Precise specificatio Reliability:  Validity:  Adaptability:  Adequacy of risk ad
D-2

Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Not specified: Definition of denominator/numerator:Denominator: Numerator: Data sources:Recommended frequency of data collection:

This QM is relevant toappropriate] Question 1a (imaging, viQuestion 1b (pathologic Question 2a (polypectomQuestion 2b (surgical theQuestion 2c (adj chemo Question 2d (radiation thQuestion 3 (colonoscopiQuestion 4 (documentatQuestion 5a (patient popQuestion 5b (demograph

* The data abstraction form reproduced here was used for published quality measures. The form used for unpubidentical to the above, except that it omitted the center column, on “Testing of Quality Measure.” 
E-1

Appendix E: Evidence Tables Evidence Table 1 – Question 1a (Published Literature): What quality-of-care measures are available and what evidence is ava

to assess the quality of diagnosis of colorectal cancer, including appropriate use of colon imaging, endoscopic visualization, and biopsy?Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Baig, Myers, Turner, et al., 2003 #33210 

Quality measure (QM): Percentage of patients with a positive fecal occult blood test (+FOBT) who underwent a complete diagnostic evaluation (colonoscopy or double contrast barium enema [DCBE]) Basis of QM: Clinical practice guideline; not cited, but appears to be Multi-Society Task Force (Winawer et al., Gastroenterology 1997, updated 2003; maybe also U.S. Preventive Services Task Force (Pignone et al., Ann Intern Med ~2002) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Mortality in prior trials – no link within this particular paper Intent of QM: Not specified (aim of the study was to determine reasons for non-performance of a complete diagnostic evaluation) Definition of denominator/numerator:Denominator: Number of Aetna Health (an HMO in greater Philadelphia, PA) participants aged 50 and older who had a +FOBT from Aug-Nov 1998 Numerator: Number above subjects who had a colonoscopy or DCBE at 60 days from the +FOBT, as indicated by mailed survey and claims data Data sources: Provider survey, MCO claims data Recommended frequency of data collection: No recommendation 

Study population: HMO patients; primary care practices N: 544 Age: ≥ 50 Race: NR Sex: NR Tumor stage: NA Geographic location: Greater Philadelphia, PA 

Dates: FOBT result, Aug-Nov 1998; follow up at 60 days Healthcare setting: HMO primary care practices Results:Of the 544 +FOBT patients, 46% did not receive colonoscopy or DCBE. 

General comments:

Measure has face validity with colonoscopy (or DCBguidelines, which themsewell-designed RCTs with 

as the end point. Rating of quality measustudy (scale of 1-5, wherideal): - Importance: 5 - Usability: 4 - Scientific acceptability (f Precise specification Reliability: 4  Validity: 4  Adaptability: 4  Adequacy of risk adj
E-2

Evidence Table 1 – Question 1a (Published Literature) – continued 

Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Beart, Steele Jr., Menck, et al., 1995 #990 

Quality measure (QM): Percentage of patients with colon (or rectal) cancer undergoing colonoscopy as part of their initial evaluation Basis of QM: None Type of QM:(a) Process (b) General Outcome to which the QM is linked: None is specified, but presumably it would be the finding of synchronous lesions that should be dealt with during surgery as well as the primary tumor Intent of QM: Not specified (aim of the study was to identify current trends in the management of patients with carcinoma of the colon or rectum and to identify changes in patterns of care and survival) Definition of denominator/numerator:Denominator: Number of cases of colon and rectal cancer (up to 25 per program or facility) reported by over 1,200 approved cancer programs and 800 other facilities on the Commission on Cancer mailing list that were invited to participate. Numerator: Number of above cases having colonoscopy as part of their initial evaluation. Data sources: Hospitals sent in up to 25 consecutive cases from their medical records Recommended frequency of data collection: No recommendation 

Study population: N: 39,502 (29,209 colon; 10,293 rectal) 

 Colon Rectum 

 1983 1988 1983 1988 Cases 12,682 16,527 4,597 5,696 < 50 yrs 5.6 5.3 6.1 6.1 50-69 39.4 37.4 46.1 46.0 70-79 33.4 33.9 31.5 31.2 80+ 2106 23.4 16.2 16.7 Male % 47.8 49.3 55.9 57.2 Tumor Stage 0 3.7 5.5 5.6 6.1 I 21.2 22.9 27.4 28.3 II 29.9 28.6 22.7 21.6 III 22.4 22.1 21.9 22.0 IV 20.1 17.7 17.0 16.7 Unkown 2.7 3.3 5.4 5.4 Nonhispanic white 86.1 87.0 Hispanic 2.7 3.5 AA 9.1 6.5 Asian 1.1 1.6 

Performance status: NR Geographic location: Entire U.S. Dates: Calendar years 1983 and 1988 Healthcare setting: Hospitals Results:44% of colon cancer patients in 1983 and 63.8 in 1988 underwent colonoscopy. 30.1% of rectal cancer patients in 1983 and 54.6% in 1988 underwent colonoscopy. 

General comments:

The data for this study weso it may not reflect more for management. This study reports the freof diagnostic tests (chest primary site, barium enemCT scan of primary site othat were performed for pcancer, but it is never stabeen considered componTherefore, it is not possibquality measures, and nothis abstraction except co

have face validity as a quRating of quality measustudy (scale of 1-5, wherideal): - Importance: 4 - Usability: 5 - Scientific acceptability (f Precise specification Reliability: 4  Validity: 4  Adaptability: 4  Adequacy of risk adj
E-3

Evidence Table 1 – Question 1a (Published Literature) – continued 

Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Harewood, Sharma, and de Garmo, 2003 #33050 

Quality measure (QM): Percentage of patients with adequate preparation of bowel prior to colonoscopy Basis of QM: None Type of QM:(a) Process (b) Technical Outcome to which the QM is linked: Detection of colonic lesions Intent of QM: Not specified (aim of the study was to characterize the impact of bowel preparation adequacy on detection of colonic lesions) Definition of denominator/numerator:Denominator: Number of colonoscopies performed from January 1, 2000, to December 31, 2001, that were completed and had complete documentation of age, sex, preparation quality, and endoscopic findings. Numerator: Number of above colonoscopies for which bowel preparation was rated as adequate. Data sources: Clinical Outcomes Research Initiative (CORI) database, which collects data from endoscopic procedures from a consortium of 580 specialists in GI diseases at 88 sites in 24 states Recommended frequency of data collection: No recommendation 

Study population: N: 93,004 Age: NR Race: NR Sex: NR Tumor stage: NR; most cases were not colorectal cancer Performance status: NR Geographic location: 88 sites in 24 states 

Dates: Jan 1, 2000 to Dec 31, 2001 Healthcare setting: Varied (community, academic, and VA institutions) Results: 76.9% of colonoscopies with complete data were rated as having an adequate preparation. Regarding the relationship of adequacy of bowel preparation to ability to detect colonic polyps/cancers: Suspected neoplasia was detected in 29.1% of colonoscopies with adequate preparation and 26.4% with inadequate preparation (P < 0.0001). Polyps > 9 mm were identified with equal frequency (7.3%) regardless of adequacy of preparation. Polyps ≤ 9mm were identified in 21.8% of 

colonoscopies with adequate preparation and 19.0% with inadequate preparation (P < 0.0001). After adjusting for age and sex, adequate preparation was predictive of detection of suspected neoplasia. 

General comments:The ability to achieve adepreparation is predominaThis paper also does not cancer and does not disccolonoscopy was performRating of quality measustudy (scale of 1-5, wherideal): - Importance: 2 - Usability: 5 - Scientific acceptability (f Precise specification Reliability: 3  Validity: 3  Adaptability: 3  Adequacy of risk adj
E-4

Evidence Table 1 – Question 1a (Published Literature) – continued 

Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Haseman, Lemmel, Rahmani, et al., 1997 #35910 

Quality measure (QM): Percentage of colon cancers not detected by colonoscopy in symptomatic patients. (Only 7% of the patients in this study had routine screening colonoscopies.) Basis of QM: None Type of QM:(a) Process (b) Technical Outcome to which the QM is linked: Not stated but presumably linked to stage of colon cancer that a patient is ultimately diagnosed with Intent of QM: Not specified, but presumably to help with future quality improvement projects (aim of the study was to understand the reasons for colonoscopic failures to detect colon cancer and polyps) Definition of denominator/numerator:Denominator: Number of cases of colorectal cancer in which colonoscopy was used within 3 years of diagnosis. Numerator: Number of above cases of colon cancer that were missed by the colonoscopy. Data sources: Data were collected from 20 hospitals that agreed to participate in this project Recommended frequency of data collection: No recommendation 

Study population: N: 941 Age: 69.8 Race: NR Sex: NR Tumor stage: Various Performance status: NR Geographic location: Hospitals in central Indiana 

Dates: 1988-1993 Healthcare setting: Hospitals Results:Failure to detect colorectal cancer by colonoscopy occurred in 5%. All undetected tumors were “missed” rather than not reached. 

General comments:

Data for this study were c“agreed to participate,” wwhether bias may occur. rates might want to particavoid “bad publicity.” Cecal intubation rates webe relevant because failufactor in failed detection. Data on size of tumors wRating of quality measustudy (scale of 1-5, wherideal): - Importance: 5 - Usability: 5 - Scientific acceptability (f Precise specification Reliability: 4  Validity: 3  Adaptability: 4  Adequacy of risk adj
E-5

Evidence Table 1 – Question 1a (Published Literature) – continued 

Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Levin, Hess, and Johnson, 1997 #11880 

Quality measure (QM): Number of participants undergoing adequate diagnostic follow up of positive fecal occult blood test (+FOBT) Basis of QM: Clinical practice guideline (ACS guidelines [ref 16]) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Detection of colonic neoplasms Intent of QM: Not specified (aim of the study was to compare three FOBTs and assess patient and physician compliance with ACS guidelines on recommended diagnostic workup of participants with a +FOBT) Definition of denominator/numerator:Denominator: Number of participants with +FOBT tests for whom followup information was available. Numerator: Number of above participants undergoing adequate diagnostic follow up (colonoscopy or double-contrast barium enema with flexible sigmoidoscopy) Data sources: Telephone calls to participants; questionnaires sent to physicians Recommended frequency of data collection: No recommendation 

Study population: N: 934 Age: 50-94 Race: 90% white, 4.8% black, 2.9% Hispanic, 1.4% Asian, < 1% American Indian Sex: 43.3% male, 56.7% female Tumor stage: NR Performance status: NR Geographic location: Houston, TX 

Dates: 4-week period in October 1993 Healthcare setting: Not stated; all were outpatients when participating in the study Results:Rate of adequate diagnostic followup was 59% overall and varied depending on specialty of the physician who was consulted:  Family physician: 45%  Other physician: 55%  Gastroenterologist: 85% Neoplasia was discovered in 56/553 (10%) participants who received adequate diagnostic follow up regardless of physician type, but in only 4/553 (1%) participants who did not get adequate followup care. This difference was statistically significant (p < 0.001). 

General comments:

The purpose of this papedifferent hemoccult tests blood, but it also obtained diagnostic followup was oRating of quality measustudy (scale of 1-5, wherideal): - Importance: 5 - Usability: 5 - Scientific acceptability (f Precise specification Reliability: 3  Validity: 3  Adaptability: 3  Adequacy of risk adj
E-6

Evidence Table 1 – Question 1a (Published Literature) – continued 

Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Mandel, Church, Bond, et al., 2000 #16070 

Quality measure (QM): Percentage of patients who have a followup evaluation of a positive fecal occult blood test (+FOBT) Basis of QM: Clinical practice guideline Type of QM:(a) Process (b) General Outcome to which the QM is linked: Positive diagnosis of colorectal cancer Intent of QM: Quality improvement, research Definition of denominator/numerator:Denominator: Number of patients with a +FOBT. Numerator: Number of above patients who had appropriate diagnostic evaluation. Data sources: Database of the Minnesota Colon Cancer Control study Recommended frequency of data collection: No recommendation 

Study population: 

 Annual 

Screening 

Biennial Screening 

Male 7,474 7,430 Female 8,058 8,120 Total 15,532 15,550 

Average age: 50-80 years old Geographic location: Minnesota Dates: 1976-1992 Healthcare setting: Randomized trial Results:83% of patients in the annual-screening group and 84% of patients in the biennial-screening group underwent diagnostic followup, including a complete examination of the large bowel by colonoscopy or the combination of double-contrast enema and flexible sigmoidoscopy. In each group about 11% of the subjects with positive screening tests underwent flexible sigmoidoscopy or barium enema or underwent another FOBT. Five percent of the subjects with positive tests declined to consult a physician. 

General comments:

The purpose of this papecumulative incidence of cindividual groups who webiennial, or usual care. It 

of patients who had follow Rating of quality measustudy (scale of 1-5, wherideal): - Importance: 5 - Usability: 5 - Scientific acceptability (f Precise specification Reliability: 5  Validity: 5  Adaptability: 3  Adequacy of risk adj
E-7

Evidence Table 1 – Question 1a (Published Literature) – continued 

Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Marble, Banerjee, and Greenwald, 1992 #4910 

Quality measure (QM): Time from patient presentation to physician diagnosis Basis of QM: None Type of QM:(a) Process (b) General Outcome to which the QM is linked: More rapid diagnosis presumed to be linked to earlier stage at diagnosis and hence improved cure rates. (Unlikely that there are data to support this idea over the typical range of time between presentation and diagnosis.) Intent of QM: Not specified (aim of the study was to determine whether younger patients with colorectal cancer had a poorer prognosis than their older counterparts, and if they did, the reasons underlying this) Definition of denominator/numerator:Denominator: Numerator: Data sources: Saint Francis Hospital and Medical Center Tumor Registry Recommended frequency of data collection: No recommendation 

Study population:N: 50/50 Age: ≤ 40/ > 40 Race: 94% white/98% white Sex: 58% female/52% female Tumor stage: NR Performance status: NR Geographic location: Hartford, CT 

Dates: 1935-1988 Healthcare setting: Hospital Results:There was no difference in the interval from presentation to diagnosis in older vs. younger patients (the appropriate diagnosis was made in < 1 week in > 90% of the patients in both groups). 

General comments:

A small study of limited uRating of quality measustudy (scale of 1-5, wherideal): - Importance: 3 - Usability: 4 - Scientific acceptability (f Precise specification Reliability: 1  Validity: 1  Adaptability: 3  Adequacy of risk adj
E-8

Evidence Table 1 – Question 1a (Published Literature) – continued 

Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Myers, Turner, Weinberg, et al., 2004 #30300 

Quality measure (QM): Percentage of patients receiving complete diagnostic evaluation (CDE) for positive fecal occult blood test (+FOBT), defined as colonoscopy or barium enema + flexible sigmoidoscopy (BEFS) Basis of QM: Clinical practice guideline (Winawer et al., Gastroenterology, 2003) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Reduced colorectal cancer-related mortality Intent of QM: Not specified (aim of the study was to evaluate the impact of a guideline on CDE performance rates in primary care practices) Definition of denominator/numerator:Denominator: Number of +FOBT patients eligible for CDE recommendation and performance. Exclusion criteria: CDE procedures completed within 3 years prior to +FOBT result; patient had a medical condition that contraindicated CDE; patient deceased; patient unknown to practice and patient left practice before CDE could be recommended. Numerator: Number of above patients whose physicians had recommended CDE; number of patients who underwent CDE within 180 days after the +FOBT result. Results were coded as dichotomous variables, 

yes or no (recommendation was made or treatment was provided). Data sources: ICA forms; MCO administrative 

claims data Recommended frequency of data collection: No recommendation 

Study population: N: 2,992 Age: ≥ 50 Geographic location: Southern PA and NJ Dates: 1994-2000, 3 study periods Healthcare setting: Aetna U.S. Healthcare and various MCO-based cancer screening sites Results:

 Intervention 

group 

Control group 

CDE recommendation rate 

79.6 67.3 

CDE performance rate 

63.3 53.7 

For study period 3 (5/99 to 2/00), the differences in CDE recommendation and performance rates between the intervention group and control group was statistically significant. Use of a physicianoriented intervention substantially and significantly increased CDE recommendation and performance in intervention group practice as compared to control group practices. Targeting PCPs for delivery of a combined CDE reminder feedback and educational outreach intervention can have a meaningful impact on physician behavior and patient followup in colorectal cancer screening. 

General comments: No

Rating of quality measustudy (scale of 1-5, wherideal): - Importance: 5 - Usability: 4 - Scientific acceptability (f Precise specification Reliability: 4  Validity: 4  Adaptability: 4  Adequacy of risk adj
E-9

Evidence Table 1 – Question 1a (Published Literature) – continued 

Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Nelson, McQuaid, Bond, et al., 2002 #45970 

Quality measure (QM): Postprocedural complication rate Basis of QM: Not specified Type of QM:(a) Outcome (b) Technical Intent of QM: Not specified (aim of the study was to report and identify predictive variable for procedural success and complication rates of screening colonoscopy in a large asymptomatic cohort) Definition of denominator/numerator:Denominator: Number of asymptomatic patients between 50 and 75 years undergoing a screening colonoscopy at 13 VAMCs between 2/1994-1/1997. Exclusion criteria: Patients were excluded if they reported symptoms of lower GI tract disease, including rectal bleeding on more than one occasion in the prior 6 months, significant change in bowel habits or lower abdominal pain that would require evaluation. Also excluded were those with prior colonic disease; prior colon examination within 10 years; significant medical problems that would increase risk of colonoscopy, or a medical condition that would preclude benefit from screening; need for special precautions for colonoscopy; women with childbearing potential. Numerator: Number of above cases where postprocedural complications within 30 days occurred, including perforation, GI bleeding with hospitalization, new arrhythmia, MI/CVA, death within 30 days. Data sources: Medical chart Recommended frequency of data collection: No recommendation 

Study population: n = 3,196 

Mean age 63.0 

% male 96.8 % white 83.5 Comorbidities: Coronary heart disease 

21.1% 

CVA/TIA 8.2% Diabetes 20.8% COPD 8.5% 

Geographic location: 13 VAMCs across U.S. Dates: 1994-1997 Healthcare setting: General medicine clinics Results:No statistically significant complications occurred. 

General comments: No

Rating of quality measustudy (scale of 1-5, wher

ideal): - Importance: 5 - Usability: 3 - Scientific acceptability (f Precise specification Reliability: 4  Validity: 4  Adaptability: 4  Adequacy of risk adj
E-10

Evidence Table 1 – Question 1a (Published Literature) – continued 

Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Nelson, McQuaid, Bond, et al., 2002 #45970 

Quality measure (QM): Percent of patients with “adequate” or better bowel preparation Basis of QM: Not specified Type of QM:(a) Outcome (b) Technical Outcome to which the QM is linked: Intent of QM: Not specified (aim of the study was to report and identify predictive variable for procedural success and complication rates of screening colonoscopy in a large asymptomatic cohort) Definition of denominator/numerator:Denominator: Number of asymptomatic patients between 50 and 75 years undergoing a screening colonoscopy at 13 VAMCs between 2/1994-1/1997. Exclusion criteria: Patients were excluded if they reported symptoms of lower GI tract disease, including rectal bleeding on more than one occasion in the prior 6 months, significant change in bowel habits or lower abdominal pain that would require evaluation. Also excluded were those with prior colonic disease; prior colon examination within 10 years; significant medical problems that would increase risk of colonoscopy, or a medical condition that would preclude benefit from screening; need for special precautions for colonoscopy; women with childbearing potential. Numerator: Number of above cases where bowel preparation was rated by the endoscopist as “good” (mucosa well seen throughout), “fair” (liquid, contents; exam adequate); or ”poor” (solid contents, exam compromised). Data sources: Medical chart Recommended frequency of data collection: No 

Study population: n = 3,196 

Mean age 63.0 

% male 96.8 % white 83.5 Comorbidities: Coronary heart disease 

21.1% 

CVA/TIA 8.2% Diabetes 20.8% COPD 8.5% 

Geographic location: 13 VAMCs across U.S. Dates: 1994-1997 Healthcare setting: General medicine clinics Results:By using a polyethylene glycol-based electrolyte solution, the bowel preparation was described as good in 81.4%, fair in 15.8% and poor in 2.7% of patients. 

General comments: No

Rating of quality measustudy (scale of 1-5, wher

ideal): - Importance: 5 - Usability: 3 - Scientific acceptability (f Precise specification Reliability: 4  Validity: 4  Adaptability: 4  Adequacy of risk adj
E-11

Evidence Table 1 – Question 1a (Published Literature) – continued 

Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

recommendation. 

Nelson, McQuaid, Bond, et al., 2002 #45970 

Quality measure (QM): Percent of patients with a 

colonoscopy with successful cecal intubation Basis of QM: Not specified 

Type of QM:(a) Process (b) Technical Outcome to which the QM is linked: Intent of QM: Not specified (aim of the study was to report and identify predictive variable for procedural success and complication rates of screening colonoscopy in a large asymptomatic cohort) Definition of denominator/numerator:Denominator: Number of asymptomatic patients between 50 and 75 years undergoing a screening colonoscopy at 13 VAMCs between 2/1994-1/1997. Exclusion criteria: Patients were excluded if they reported symptoms of lower GI tract disease, including rectal bleeding on more than one occasion in the prior 6 months, significant change in bowel habits or lower abdominal pain that would require evaluation. Also excluded were those with prior colonic disease; prior colon examination within 10 years; significant medical problems that would increase risk of colonoscopy, or a medical condition that would preclude benefit from screening; need for special precautions for colonoscopy; women with childbearing potential. Numerator: Number of above cases where colonoscopy with procedure was successful; that is, confirmation of cecal intubation. Data sources: Medical chart Recommended frequency of data collection: No 

Study population: n = 3,196 

Mean age 63.0 

% male 96.8 % white 83.5 Comorbidities: Coronary heart disease 

21.1% 

CVA/TIA 8.2% Diabetes 20.8% COPD 8.5% 

Geographic location: 13 VAMCs across U.S. Dates: 1994-1997 Healthcare setting: General medicine clinics Results: Colonoscopy with cecal intubation was successful on the first attempt in 97.2% of the cases. This includes 69 cases in which the quality of the preparation was felt to be inadequate to visualize the entire colonic mucosa. 53.8% of these cases had at lest 1 polyp resected. 

General comments: No

Rating of quality measustudy (scale of 1-5, wher

ideal): - Importance: 5 - Usability: 3 - Scientific acceptability (f Precise specification Reliability: 4  Validity: 4  Adaptability: 4  Adequacy of risk adj: 
E-12

Evidence Table 1 – Question 1a (Published Literature) – continued 

Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

recommendation. 

Shehadeh, Rebala, Kumar, et al., 2002 #18520 

Quality measure (QM): Miss rate for followup colonoscopy after polypectomy Basis of QM: None Type of QM:(a) Process (b) Technical Outcome to which the QM is linked: Presumably, improved survival via earlier detection of malignancy Intent of QM: Not specified (aim of the study was to evalute the miss rates of advanced adenomas) Definition of denominator/numerator:Denominator: Number of patients who had polypectomy on initial colonoscopy and who had at least one followup colonoscopy from July 1, 1992 to June 30, 1999 at the Dayton VAMC. Colonoscopy had to be complete and bowel prep had to be satisfactory. Numerator: Number of patients with missed advanced adenomas (≥10 mm) found on repeat colonoscopy. Data sources: Computerized database and paper charts at VAMC Recommended frequency of data collection: No recommendation 

Study population: N: 122 Age: NR Race: NR Sex: 100% male Tumor stage: NA Performance status: NR Geographic location: Dayton, Ohio 

Dates: 1992-1999 Healthcare setting: Veteran’s Affairs Medical Center (VAMC) Results:122 patients had 338 colonoscopies. Miss rates were calculated as in the National Polyp Study. Missed adenomas were defined as adenomas (especially big ones) found on repeat colonoscopy. 122 patients had 2 colonoscopies and 60 patients had a third colonoscopy. The calculated miss rate of advanced adenomas for the second colonoscopy was 2.5% (4/122); for the third colonoscopy, it was 3.3 %(2/60). 

General comments:

Because fellows in trainincolonoscopies (albeit with supervision), the results mroutine clinical practice. Miss rates could be used improvement purposes (aRating of quality measustudy (scale of 1-5, wherideal): - Importance: 5 - Usability: 5 - Scientific acceptability (f Precise specification Reliability: 3  Validity: 5 

 Adaptability: 3  Adequacy of risk adj
E-13

Evidence Table 1 – Question 1a (Published Literature) – continued 

Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Shields, Weiner, Henry, et al., 2001 #23040 

Quality measure (QM): Percentage of patients aged 40 or older participating in a mass screening program who had an “adequate evaluation” (colonoscopy or barium enema + flexible sigmoidoscopy [BEFS]) for a positive fecal occult blood test (+FOBT; Hemeoccult II) Basis of QM: Clinical practice guideline (American Cancer Society [ref 14, 15 1980 CA, 1996 JAMA]) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Reduced mortality from colorectal cancer Intent of QM: Not specified (aim of the study was to determine whether factors like age, sex, and family history [among others] influence adequate evaluation of a +FOBT) Definition of denominator/numerator:Denominator: Number of patients who participated in a 1986 mass screening program with FOBT (pharmacy-based, processed at Beth Israel Hospital (Boston MA), who returned their cards, had at least 1 positive window (out of 6) and who provided (or their physician provided) followup data (90%). Numerator: Number of the above subjects who in addition had an 

“adequate evaluation” (colonoscopy or barium enema + flexible sigmoidoscopy [BEFS]) Data sources: Physician survey “checklist,” patient 

or physician phone call (if survey/checklist not returned), pathology reports, patient survey at 3 yearsRecommended frequency of data collection: No recommendation 

Study population: N: 940 

 percent 

40-49 yrs 11 50-59 yrs 18 60-69 yrs 37 70-79 yrs 28 80+ yrs 6 male 48 female 47 unspecified 5 

Race: NR Geographic location: Boston area, MA Dates: 1986-1988 Healthcare setting: Variety Results:59% of patients with a +FOBT were adequately evaluated. 11.2% of adequately evaluated patients had new colon cancers discovered. The effects of age and sex on the adequacy of follow up were considered, but no significant differences were found by age group or between men and women. 

General comments: No

Rating of quality measustudy (scale of 1-5, wher

ideal): - Importance: 5 - Usability: 4 - Scientific acceptability (f Precise specification Reliability: 3  Validity: 4  Adaptability: 3  Adequacy of risk adj
E-14

Evidence Table 1 – Question 1a (Published Literature) – continued 

Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Ure, Dehghan, Vernava 3rd, et 

al., 1995 #960 

Quality measure (QM): Complication rate of colonoscopy in the elderly and non-elderly Basis of QM: None Type of QM:(a) Outcome (b) Technical Intent of QM: Not specified (aim of the study was to evaluate the utility, morbidity, and patient tolerance of colonoscopy in elderly patients as compared to a similar group of younger patients) Definition of denominator/numerator:

Denominator: Number of elderly/non-elderly patients undergoing colonoscopy. Numerator: Number of the above patients with resulting complications. Data sources: Not specified Recommended frequency of data collection: No recommendation 

Study population: Eldlerly/Non-elderly N: 354/302 Age: ≥ 70 / 50-70 

Race: NR Sex: NR Tumor stage: NA Performance status: NR Geographic location: St. Louis area Dates: “A recent 48-month period” Healthcare setting: University Hospital 

Results:Overall morbidity was similar in elderly and nonelderly patients (24% vs. 16%, p = NS). Elderly patients were significantly more likely to require termination of the procedure because of inadequate bowel preparation or pain (17% vs. 1%, p < 0.05). Colonoscopy was successful to the cecum or ileocolic anastomosis in 85% (555/656). No patient had a perforation; 2% postprocedure hemorrhage in those undergoing polypectomy. 

General comments:

This paper reports the ratmeasures of the performacolonoscopy completion, postpolypectomy bleedinundergoing colonoscopy Rating of quality measustudy (scale of 1-5, wherideal): - Importance: 5 - Usability: 5 - Scientific acceptability (f Precise specification Reliability: 5  Validity: 5  Adaptability: 3  Adequacy of risk adj
E-15

Evidence Table 2 – Question 1a (Grey Literature): What quality-of-care measures are available and what evidence is available assess the quality of diagnosis of colorectal cancer, including appropriate use of colon imaging, endoscopic visualization, and biopsy? 

Study Characteristics of Quality Measure Comments/Quality Scoring 

Colon Cancer Workgroup, 2003 #36650 

Quality measure (QM): Colonoscopy to the ileocecal valve is performed prior to surgical resection of colorectal cancer Basis of QM: Clinical practice guideline (National Comprehensive Cancer Network [NCCN] guidelines) Type of QM:(a) Process (b) Technical Outcome to which the QM is linked: Better surgical outcome Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of patients having colon cancer surgery. Numerator: Number of above patients with documentation of colonoscopy of the entire colon to the ileocecal valve before surgery except in cases of obstructing carcinoma or perforation. Data sources: Medical records (operative report, history and physical, or other preoperative testing documentation) Recommended frequency of data collection: No recommendation 

General comments:It should be noted that colonoscopy will not be perfocases of obstructing or perforated carcinoma.Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-16

Evidence Table 2 – Question 1a (Grey Literature) – continued Study Characteristics of Quality Measure Comments/Quality Scoring Minnesota Gastroenterology PA, 2004 #36590 

Quality measure (QM): Percentage of colonoscopy cases performed by physicians completing residency/fellowship training in gastroenterology, colorectal surgery, general surgery or pediatric surgery Basis of QM: Practice guideline (ASGE recommendation) Type of QM:(a) Structure (b) General Outcome to which the QM is linked: Improved health status Intent of QM: Quality improvement, accountability Definition of denominator/numerator:Denominator: Number of patients undergoing endoscopy. Numerator: Number of above patients undergoing endoscopy by a physician having completed residency or fellowship training in gastroenterology, colorectal surgery, general surgery or pediatric surgery and who have specific proctored training in endoscopy. Reference (Feb 2002, ASGE Policy and Procedure Manual) Data sources: Physician credentials, endoscopy notes Recommended frequency of data collection: No recommendation 

General comments: None Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-17

Evidence Table 2 – Question 1a (Grey Literature) – continued Study Characteristics of Quality Measure Comments/Quality Scoring Minnesota Gastroenterology PA, 2004 #36590 

Quality measure (QM): Unplanned reversal of sedation medication Basis of QM: Internal quality assurance committee Type of QM:(a) Process (b) Technical Outcome to which the QM is linked: Survival Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of endoscopies performed. Numerator: Number of above cases where sedation medication was reversed. Data sources: Medical chart Recommended frequency of data collection: Continually 

General comments: None Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-18

Evidence Table 2 – Question 1a (Grey Literature) – continued Study Characteristics of Quality Measure Comments/Quality Scoring Minnesota Gastroenterology PA, 2004 #36590 

Quality measure (QM): Postendoscopic procedure complication rates 

Basis of QM: Internal quality assurance committee Type of QM

(a) Outcome (b) Technical Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of endoscopies performed. Numerator: Number of above cases where postendoscopic procedure complications occurred, including death within 30 days, perforation, bleeding with transfusion, cardiopulmonary arrest, unplanned surgery. Data sources: Medical chart Recommended frequency of data collection: Continually 

General comments: None Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-19

Evidence Table 2 – Question 1a (Grey Literature) – continued Study Characteristics of Quality Measure Comments/Quality Scoring Minnesota Gastroenterology PA, 2004 #36590 

Quality measure (QM): Effectiveness of bowel prep prior to colonoscopyBasis of QM: Internal quality assurance committee Type of QM:(a) Process (b) General Outcome to which the QM is linked: High-quality exam Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of endoscopies performed. Numerator: Number of above cases where no fecal material obscured a lesion 5 mm or more in diameter and a high quality exam takes place. Data sources: Electronic chart Recommended frequency of data collection: Continually 

General comments: None Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-20

Evidence Table 2 – Question 1a (Grey Literature) – continued Study Characteristics of Quality Measure Comments/Quality Scoring Minnesota Gastroenterology PA, 2004 #36590 

Quality measure (QM): Percent of examinations complete to the end of colon 

(cecal intubation rate) Basis of QM: Internal quality assurance committee (goal is less than > 90 

percent) Type of QM:(a) Process (b) Technical Outcome to which the QM is linked: Improved health outcomes Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of endoscopies performed, Numerator: Number of above cases where end of colon was reached. Data sources: Medical chart Recommended frequency of data collection: Continually 

General comments: None Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-21

Evidence Table 2 – Question 1a (Grey Literature) – continued Study Characteristics of Quality Measure Comments/Quality Scoring Minnesota Gastroenterology PA, 2004 #36590 

Quality measure (QM): Rate of adenomas removed for patients > 50 years of age Basis of QM: Clinical practice guideline (Multi-Society Task Force on Colon Cancer); goal: men > 50, 25%; women > 50, 15% Type of QM:(a) Process (b) Technical Outcome to which the QM is linked: Survival Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of endoscopies performed. Numerator: Number of above cases where one or more adenomas (polyps > 1 cm or with advanced histology) were removed. Data sources: Medical chart Recommended frequency of data collection: Continually 

General comments: None Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-22

Evidence Table 2 – Question 1a (Grey Literature) – continued Study Characteristics of Quality Measure Comments/Quality Scoring Minnesota Gastroenterology PA, 2004 #36590 

Quality measure (QM): Acceptable cycle time for colonoscopic examination 

Basis of QM: No national, internal goals 

Type of QM:(a) Process (b) General Outcome to which the QM is linked: Quality improvement

Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of endoscopies performed. Numerator: Number of above cases where exam was completed in a timely fashion acceptable to patient and referring physician according to internal practice standards. Data sources: Medical chart Recommended frequency of data collection: Continually 

General comments: None Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-23

Evidence Table 2 – Question 1a (Grey Literature) – continued Study Characteristics of Quality Measure Comments/Quality Scoring National Quality Measures Clearinghouse (measure proposed by Accreditation Association for Ambulatory Health Care Institute for Quality Improvement), 2003 #36640 

Quality measure (QM): Intraprocedure colonoscopy complication rate 

(percentage of patients who developed one or more intraprocedure complications) Basis of QM: None 

Type of QM:(a) Outcome (b) Technical Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of patients undergoing colonoscopy procedure (CPT codes 45378-45385) at the ambulatory health care organization. Numerator: Number of above patients who developed one or more intraprocedure complications (including arrhythmia, bleeding requiring treatment, extended recovery, hospital transfer, hypotension, hypoxia, noted perforations, respiratory arrest). Data sources: Patient survey Recommended frequency of data collection: No recommendation 

General comments:Patients could be undergoing colonoscopy for screesurveillance. Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-24

Evidence Table 2 – Question 1a (Grey Literature) – continued Study Characteristics of Quality Measure Comments/Quality Scoring National Quality Measures Clearinghouse (measure proposed by Accreditation Association for Ambulatory Health Care Institute for Quality Improvement), 2003 #36640 

Quality measure (QM): Patient understanding of colonoscopy procedure 

(percentage of patients answering “yes” to the postprocedure telephone interview 

question, “Did you have an adequate understanding of your procedure?”) Basis of QM: None Type of QM:(a) Outcome (b) General Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of patients undergoing colonoscopy procedure (CPT codes 45378-45385) at the ambulatory health care organization who were reached for the telephone survey and who responded to the question “Did you have an adequate understanding of your procedure?” Numerator: Number of above patients answering “yes” to the postprocedure telephone interview question “Did you have an adequate understanding of your procedure?”Data sources: Patient survey Recommended frequency of data collection: No recommendation 

General comments:Patients could be undergoing colonoscopy for screesurveillance. Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-25

Evidence Table 3 – Question 1b (Published Literature): What quality-of-care measures are available and what evidence is av

to assess the quality of diagnosis of colorectal cancer, including availability and accuracy of pathologic staging? Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Baxter, Virnig, Rothenberger, et al., 2005 #35800 

Quality measure (QM): Adequate lymph node evaluation Basis of QM: Clinical practice guideline (American Joint Committee on Cancer, International Union against Cancer) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Recurrence Intent of QM: Not specified (aim of the study was to determine the proportion of colorectal cancer patients in the U.S. who receive adequate lymph node evaluation) Definition of denominator/numerator:Denominator: Number of patients ≥ 18 yr, diagnosed with localized invasive adenocarcinoma of colon or rectum from Jan 1988-Dec 2001. Exclusion criteria: Patients with cancer colon not otherwise localized, patients who underwent radical surgical resection, and postoperative radiation. Numerator: Number of above patients with adequate lymph node evaluation (adequate = at least 12 nodes examined) . Data sources: NCI’s SEER registry Recommended frequency of data collection: No recommendation 

Study population: N: 116,995 

 Anatomic site of tumor 

Characteristics All pts (N = 

116,995) 

Right colon (49,613) 

Left colon (50,536) 

Rectum (16,846) 

Median age (yr) 

71 74 69 68 

Males 50% 44% 53% 58% Race/ ethnicity 

White 83% 84% 82% 84% 

Nonwhite 17% 16% 18% 16% 

Tumor stage 

 Stage I 25% 19.5% 27% 33% 

Stage II 40% 45.5% 39% 29% 

Stage III 35% 35% 34% 38% 

Tumor Grade 

Well or 

mod diff. 

82% 76% 87% 83% 

Poorly 

diff. 

18% 24% 13% 17% 

Node No. (Median) 

9 11 7 8 

Geographic location: U.S. Dates: Jan 1988-Dec 2001 Healthcare setting: Multiple Results:No lymph nodes were examined in 6.5% patients. (continued on next page) 

General comments: NoRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 4  Adequacy of risk ad
E-26

Evidence Table 3 – Question 1b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Characteristic % of all 

pts 

Rates among pts with stage I disease (N = 27,323) 

Rates among pts with stage II disease (N = 44,771) 

Rates among pts with stage III disease (N = 38,660) 

Overall rates 

37% 25% 41% 46% 

Patient age (yr) 

≤ 50 51% 35% 56% 57% 

51-60 40% 27% 46% 48% 61-70 36% 23% 41% 45% 

≥ 71 35% 25% 37% 43% 

Year of diagnosis 

1988-90 33% 22% 37% 40% 

1991-3 35% 23% 38% 44% 1994-6 36% 24% 40% 45% 1997-9 40% 26% 43% 49% 2000-1 43% 31% 45% 52% 
E-27

Evidence Table 3 – Question 1b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SChiaverini, Fulton, and Darcy, 2002 #17720 

Quality measure (QM): Percentage of newly 

diagnosed CRC cases that were staged using the AJCC system Basis of QM: Clinical practice guideline (implicit that 

AJCC staging system would follow the American Joint Committee on Cancer guidelines but these are not actually cited in the article. The authors do mention the rules and regulations of the Rhode Island Cancer Registry and the Rhode Island Cancer Control Plan [ref 4]). Type of QM:(a) Structure (b) Process and technical Outcome to which the QM is linked: No outcome linked, but measure is said to be important for choosing appropriate treatments Intent of QM: NA (aim of the study was to describe progress in the control of colorectal cancer in Rhode Island, 1987-2000) Definition of denominator/numerator:Denominator: Number of Rhode Island men and women newly diagnosed with colorectal cancer in 1989-1999 compared to 2000. Numerator: Number of above patients who were staged using the AJCC system. Data sources: Rhode Island Cancer Registry Recommended frequency of data collection: No recommendation 

Study population: N: NR Age: NR Race: NR Sex: NR Tumor stage: NR Performance status: NR Geographic location: Rhode Island 

Dates: 1989-2000 Healthcare setting: Multiple Results:The proportion of cases staged with AJCC staging methodology increased from 65% in 1989 to 92% in 2000 for men, and from 65% in 1989 to 90% in 2000 for women. 

General comments: NoRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 4 - Usability: 4 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 3  Adequacy of risk ad
E-28

Evidence Table 3 – Question 1b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SGalvis, Raab, D’Amico, et al., 2001 #20330

Quality measure (QM): Lymph node retrieval rate Basis of QM: None (but see refs 1-10) Type of QM(a) Process (b) Technical Outcome to which the QM is linked: Recurrence Intent of QM: Not specified (aim of the study was to measure the quality of pathologists’ assistants’ surgical gross examination [vs. that of pathology residents]) Definition of denominator/numerator:Denominator: Number of colorectal specimens submitted for tumor from Dec 1997-Nov 1999 (all cases for which 5 or more cassettes of tissue were initially submitted for histologic examination). Numerator: Number of positive lymph nodes retrieved from above specimens. Data sources: Retrospective review of surgical pathology reports Recommended frequency of data collection: No recommendation 

Study population: N: 176 specimens Geographic location: PA 

Dates: Dec 1997-Nov 1999 Healthcare setting: University hospital Results:

Mean # of lymph nodes 

Examiner 

Total 

number 

of cases 

Retrieved per specimen 

Positive per specimen 

Pathologist’s assistant 50 31.3 1.1 Pathology resident 126 18.7 2.2 

General comments: NoRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 4 - Scientific acceptability  Precise specificatio Reliability: 2  Validity: 4  Adaptability: 4  Adequacy of risk ad
E-29

Evidence Table 3 – Question 1b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SSteele, 1994, #35840 

Quality measure: Percentage of newly diagnosed CRC cases who were staged using the AJCC system Basis of QM: Clinical practice guideline (implicit that the AHCC guidelines be followed but these are not actually cited) Type of QM:(a) Process (b) Technical Outcome to which the QM is linked: No outcome is exactly linked to the above but is said to be a marker of appropriate cancer diagnosis and treatment. No citation is provided for this statement. Intent of QM: Not specified (aim of the study was to assess cancer patient care and outcomes on a national basis) Definition of denominator/numerator:Denominator: Number of patients in the National Cancer Data Base (NCDB) with a new diagnosis of colon cancer or rectal cancer (rectosigmoid junction, rectum and anal canal) in the years 1985-1986 or 1991. Numerator: Number of the above patients who were staged using the AJCC system. Data sources: National Cancer Data Base (NCDB)Recommended frequency of data collection: No recommendation 

Study population: 

 Colon Rectal 

 1985-

86 1991 

1985-

86 1991 

cases 40384 39751 18418 17348 Age ≤54 10.6 11.2 13.5 14.8 55-64 19.8 17.2 23.8 21.6 65-74 32.3 21.9 33.9 32.9 ≥75 37.3 39.7 28.8 30.7 %male 48.7 49.6 55.3 55.7 race White 86.9 85.9 88.0 86.9 hispanic 

 1.1 1.8 1.6 2.3 

AA 6.6 8.1 4.7 6.3 Asian 0.9 1.5 1.2 2.2 Unk. 4.5 2.7 4.5 2.3 

Tumor 

stage 

0 3.7 6.5 5.8 5.0 1 14.4 19.3 15.2 18.6 2 13.8 26.2 10.5 17.4 3 12.9 20.3 12.2 20.0 4 9.2 15.8 5.0 7.8 

Unk. 46.0 11.9 51.3 31.2 

Geographic location: National sample; note that the sample is not probabilistic and represented different hospitals in different years Dates: 1991 and outcomes for 1985-86 Healthcare setting: 464 hospital in 1985, 474 hospitals in 1986, 937 hospitals in 1991 Results: The percent of cases reported as having an “unknown stage” decreased from 46% to 11.9% and 51.3% to 31.2% for colon and rectal cancer cases respectively, between 1985-86 and 1991. 

General comments: NoRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 3 - Usability: 4 - Scientific acceptability  Precise specificatio Reliability: 4  Validity: 3  Adaptability: 3  Adequacy of risk ad
E-30

Evidence Table 4 – Question 1b (Grey Literature): What quality-of-care measures are available and what evidence is available 

assess the quality of diagnosis of colorectal cancer, including availability and accuracy of pathologic staging? Study Characteristics of Quality Measure Comments/Quality Scoring 

Colon Cancer Workgroup, 2003 #36650 

Quality measure (QM): Surgical resection includes at least 8 lymph nodes 

Basis of QM: Clinical practice guideline (CAP, National Comprehensive Cancer 

Network [NCCN] guidelines, NCI guidelines) Type of QM:(a) Process (b) Technical Outcome to which the QM is linked: Improved staging and subsequent treatment planning Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of patients having colon cancer surgery. Numerator: Number of above patients in whom 8 or more lymph nodes were resected. Data sources: Pathology, surgery reports Recommended frequency of data collection: No recommendation 

General comments:Measure notes that some patients would have feweMeasure mentions only “surgical resection,” but the “number of lymph nodes positive” is important for stadata are important for staging. Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-31

Evidence Table 4 – Question 1b (Grey Literature) – continued Study Characteristics of Quality Measure Comments/Quality Scoring Institute of Medicine, 2005 #36680 

Quality measure (QM): Appropriate histological assessment of colorectal cancer Basis of QM: Clinical practice guideline (National Comprehensive Cancer 

Network [NCCN] guidelines) NCCN recommends that a minimum of 14 regional lymph nodes be removed during surgical resection. CAP recommends removal of at least 12 nodes and urges that additional techniques (i.e., visual enhancement) be considered if fewer 

than 12 nodes are found. Type of QM:(a) Process (b) General Outcome to which the QM is linked: Staging Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of colorectal cancer surgery patients. Numerator: Number of above patients with a surgical resection that included at least 12 lymph nodes. Data sources: Surveillance, Epidemiology and End results Program (SEER)- GCCR, baseline studies of pathology reports and medical records Recommended frequency of data collection: No recommendation 

General comments:Numerous studies reveal that surgical and pathology variable quality. Information on margins and the numoften missing. Key references cited: Compton (CAP), 2004 Compton, 2003 LeVoyer et al., 2003 NCCN, 2004 Stocchi et al., 2001 Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-32

Evidence Table 4 – Question 1b (Grey Literature) – continued Study Characteristics of Quality Measure Comments/Quality Scoring National Committee for Quality Assurance, 2005 #36580 

Quality measure (QM): Percent of colon cancer patients receiving surgical treatment that had at least 14 lymph nodes examined Basis of QM: Clinical practice guideline (National Comprehensive Cancer Network [NCCN] guidelines) Type of QM:(a) Process (b) Technical Outcome to which the QM is linked: Improved health status Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of patients with colon cancer surgery over a 12-month period. Numerator: Number of above patients who had 14 or more lymph nodes examined at the time of surgery. Data sources: Cancer registry, claims data Recommended frequency of data collection: No recommendation 

General comments:Not all data may be available. Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-33

Evidence Table 5 – Question 2b (Published Literature): As appropriate to specific stages of colorectal cancer, what quality-of-

available and what evidence is available for measures of quality of care of treatment of colorectal cancer, including surgical therapy for c

Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Agarwal, Leighton, Mandile, et al., 1990 #7530 

Quality measure (QM): 30-day operative mortality 

rate Basis of QM: None 

Type of QM:(a) Outcome (b) General Intent of QM: Quality improvement (aim of the study was to determine the relationship between age and 5-year survival, taking into consideration the site, stage, and type of surgical treatment rendered) Definition of denominator/numerator:Denominator: Number of patients treated for colorectal cancer during 1975-1980 with histologically confirmed adenocarcinoma. Exclusion criteria: Carcinoma in situ. Numerator: Number of above patients who died within 30 days following operation. Data sources: Tumor registry Recommended frequency of data collection: No recommendation 

Study population: N: 271 

 30-day 

mortality n = 29 (%) 

P value 

Age < 0.001 < 80 6 ≥ 80 25 Tumor stage < 0.05 I 5 II 8 III 4 IV 19 Treatment < 0.01 Colectomy 9 Low anterior 2 Abdominoperineal 0 Colostomy 24 Miscellaneious 23 

Geographic location: New York Dates: 1975-1980 Healthcare setting: Medical center Results: 30-day mortality was related significantly to stage and treatment but not to site of cancer. An increased proportion of patients who had stage IV cancers or who received colostomy and miscellaneous treatment died within 1 month. Patients ≥ 80 yr had a significantly higher 30-day mortality compared with patients < 80 yr (p < 0.0010). 

General comments: NoRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 4  Validity: 4  Adaptability: 3  Adequacy of risk ad
E-34

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Coburn, Pricolo, and Soderbert, 1994 #2920 

Quality measure (QM): Anastomotic leak rate Basis of QM: None

Type of QM:

(a) Outcome (b) Technical Intent of QM: Not specified (aim of the study was to perform univariate and multivariate analyses of poor indicators in the elderly undergoing colorectal operation, and to compare clinical, pathologic, and therapeutic factors in patients younger than and older than 80 years of age) Definition of denominator/numerator:Denominator: Number of subjects who had received operative treatment for colon and rectal cancer between 1961 and 1987. Numerator: Number of above subjects that had an anastomotic leak after surgery (within a unspecified period). 

Comparisons were made between patients under and over 80 years. Data sources: Inpatient, outpatient, and office records Recommended frequency of data collection: No 

recommendation 

Study population: N: 800 

< 80 yr > 80 yr 

N 177 623 Sex (%F/M) 48/52 56/44 Rectal lesions 36 23 Duke stage  A 9.0 6.0  B1 16.7 12.8  B2 31.2 42.7  C1 4.5 0  C2 21.5 21.8  D 17.1 16.7 

Geographic location: Rhode Island, U.S. Dates: 1961 and 1987 Healthcare setting: Academic institute Results:Postanastomotic leak rate was 6% for patients > 80 yr and 3% for patients < 80 yr. Long-term survival (at 5 and 8 yrs) and rate of recurrence after curative therapy used to validate the measure. There was a statistically significant difference in survival rates: 48% for patients < 80 vs. 32% for patients > 80. There was no statistically significant difference between the groups with regard to recurrence after curative therapy. 

General comments: Data collected between Other factors that affecinclude: more aggressof the colon and rectum local disease and less tdissemination, greater colon to be right-sided, administered to older prate in elderly patients and greater frequency lesions. Rating of quality meastudy (scale of 1-5, wh= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 4  Validity: 4  Adaptability: 4  Adequacy of risk a
E-35

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Coburn, Pricolo, and Soderbert, 1994 #2920 

Quality measure (QM): 30-day mortality Basis of QM: None Type of QM:(a) Outcome (b) General Outcome to which the QM is linked: Mortality Intent of QM: Not specified (aim of the study was to perform univariate and multivariate analyses of poor indicators in the elderly undergoing colorectal operation, and to compare clinical, pathologic, and therapeutic factors in patients younger than and older than 80 years of age) Definition of denominator/numerator:Denominator: Number of subjects who had received operative treatment for colon and rectal cancer between 1961 and 1987. Numerator: Number of above subjects that died within 30 days of operation Comparisons were made between patients under and over 80 years. Data sources: Inpatient, outpatient, and office records Recommended frequency of data collection: No recommendation 

Study population:N: 800 

< 80 yr > 80 yr 

N 177 623 Sex (%F/M) 48/52 56/44 Obstruction/ Perforation 

12 22 

Normal CEA at diagnosis 

77 68 

Rectal lesions 36 23 Duke stage  A 9.0 6.0  B1 16.7 12.8  B2 31.2 42.7  C1 4.5 0  C2 21.5 21.8  D 17.1 16.7 

Geographic location: Rhode Island, U.S. Dates: 1961 and 1987 Healthcare setting: Academic institute Results:30-day mortality rate was:  6.2% for patients > 80 yr and  2.4% for patients < 80 yr Long-term survival (at 5 and 8 yrs) and rate of recurrence after curative therapy used to validate the measure. There was a statistically significant difference in survival rates: 48% for patients < 80 vs. 32% for patients > 80. There was no statistically significant difference between the groups with regard to recurrence after curative therapy. 

General comments: Data collected between Other factors that affecinclude: more aggressof the colon and rectum local disease and less tdissemination, greater colon to be right-sided, administered to older prate in elderly patients and greater frequency lesions. Rating of quality meastudy (scale of 1-5, wh= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 4  Validity: 4  Adaptability: 4  Adequacy of risk a
E-36

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Coburn, Pricolo, and Soderbert, 1994 #2920 

Quality measure (QM): Surgical complication rate Basis of QM: None Type of QM:(a) Outcome (b) General Intent of QM: Not specified (aim of the study was to perform univariate and multivariate analyses of poor indicators in the elderly undergoing colorectal operation, and to compare clinical, pathologic, and therapeutic factors in patients younger than and older than 80 years of age) Definition of denominator/numerator:Denominator: Number of subjects who had received operative treatment for colon and rectal cancer between 1961 and 1987. Numerator: Number of above subjects that had postsurgical complications (within a unspecified period) received adjuvant therapy (chemo- or radiotherapy). Comparisons were made between patients under and over 80 years. Data sources: Inpatient, outpatient, and office records Recommended frequency of data collection: No recommendation 

Study population: N: 800 

< 80 

yrs 

> 80 yrs 

N 177 623 Sex (%F/M) 48/52 56/44 Obstruction/ Perforation 

12 22 

Normal CEA at diagnosis 

77 68 

Rectal lesions 36 23 Right sided lesions 9 19 Mucin producing tumors 

13 23 

Invasion adjacent organs 

13 23 

Single liver metastasis 20 38 Duke stage  A 9.0 6.0  B1 16.7 12.8  B2 31.2 42.7  C1 4.5 0  C2 21.5 21.8  D 17.1 16.7 

Geographic location: Rhode Island, U.S. Dates: 1961 and 1987 Healthcare setting: Academic institute Results:Complication rate was 55% for patients > 80 yr and 35% for patients < 80 yr. There was a statistically significant difference in survival rates: 48% for patients < 80 vs. 32% for patients > 80. There was no statistically significant difference between the groups with regard to recurrence after curative therapy. 

General comments:

Data collected between Long-term survival (at 5 recurrence after curativthree measures. Other factors that affecinclude: more aggressof the colon and rectum local disease and less tdissemination, greater colon to be right-sided, administered to older prate in elderly patients and greater frequency lesions. Rating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 4  Validity: 4  Adaptability: 4  Adequacy of risk a
E-37

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Cooper, Yuan, Landefeld, et al., 1996 #14030

Quality measure (QM): 30-day mortality Basis of QM: None Type of QM:(a) Outcome (b) General Outcome to which the QM is linked: Survival Intent of QM: Not specified (aim of the study was to determine whether race was associated with differences in two aspects of treatment of colorectal cancer: potentially curative surgical resections and survival after surgery) Definition of denominator/numerator:Denominator: Number of Medicare beneficiaries with a first documented discharge diagnosis of colon or rectal 

cancer. Exclusion criteria: Patients less than 65 yr of age enrolled in Medicare for ESRD or chronic disability, or 

whose race was not specified, and those with ulcerative colitis, hospital identifier not matching an Americal Hospital Association number and preexisting colorectal malignancy according to the 1984 through 1987 files. Numerator: Number of above patients with deaths within 30 days postadmission. Data sources: 1987 Medicare provider analysis and review files Recommended frequency of data collection: No recommendation 

Study population: 

 % White 

patients (n = 75,865) 

% Black patients (n = 5714) 

% Total (n = 81,579) 

Age, y 65-69 20.9 24.0 21.1 70-74 23.0 23.0 23.0 75-79 22.8 22.2 22.8 80-84 17.6 16.2 17.5 > 85 15.7 14.6 15.6 Female 53.7 57.6 54.0 Male 46.3 42.4 46.0 

Geographic location: Nationwide Dates: 1984-1987 Healthcare setting: Mixed Results: Death rates at 30 days were higher for black patients (6.1% vs. 4.6% for white patients). Death rates were also higher for black patents at 1 year (26.6%% vs. 21.7% for white patients) and at 2 years (40% vs. 33.5% for white patients). The disparity persisted after controlling for sex, age, number of comorbidities, and extent and location of cancer. 

General comments:

Race-related differencemanagement of colon ctherapy and surveillanccontributed to the obseobserved survival differperi- and postoperative Rating of quality meastudy (scale of 1-5, wh= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 3  Validity: 3  Adaptability: 4  Adequacy of risk a
E-38

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Cooper, Yuan, Landefeld, et al., 1996 #14030 

Quality measure (QM): Surgical resection rate Basis of QM: None Type of QM:(a) Outcome (b) General Outcome to which the QM is linked: Survival Intent of QM: Not specified (aim of the study was to determine whether race was associated with differences in two aspects of treatment of colorectal cancer: potentially curative surgical resections and survival after surgery) Definition of denominator/numerator:Denominator: Number of Medicare beneficiaries with a first documented discharge diagnosis of colon or rectal 

cancer. Exclusion criteria: Patients less than 65 yr of age enrolled in Medicare for ESRD or chronic disability, or 

whose race was not specified, and those with ulcerative colitis, hospital identifier not matching an Americal Hospital Association number and preexisting colorectal malignancy according to the 1984 through 1987 files. Numerator: Number of above patients who underwent surgical resection. Data sources: 1987 Medicare provider analysis and review files Recommended frequency of data collection: No recommendation 

Study population: 

 % White 

patients (n = 75,865) 

% Black patients (n = 5714) 

% Total (n = 81,579) 

Age, y 65-69 20.9 24.0 21.1 70-74 23.0 23.0 23.0 75-79 22.8 22.2 22.8 80-84 17.6 16.2 17.5 >-85 15.7 14.6 15.6 Gender  Female 53.7 57.6 54.0  Male 46.3 42.4 46.0 

Geographic location: Nationwide Dates: 1984-1987 Healthcare setting: Mixed Results:Surgical resection of colon cancer was performed less often in black patients (68%) than in white patients (78%). After controlling for sex, age, number of comorbidities, and extent and location of cancer, black patients were less likely to undergo surgical resection. Surgical resection was performed less often for black patients than white patients in teaching hospitals, non-teaching hospitals, private hospitals and public hospitals. Death rates were also higher for black patents at 1 year (26.6%% vs. 21.7% for white patients) and at 2 years (40% vs. 33.5% for white patients). The disparity persisted after controlling for gender, age, number of comorbidities, and extent and location of cancer. 

General comments:

Race-related differencemanagement of colon ctherapy and surveillanccontributed to the obseobserved survival differperi- and postoperative Rating of quality meastudy (scale of 1-5, wh= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 3  Validity: 3  Adaptability: 4  Adequacy of risk a
E-39

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Demissie, Oluwole, Balasubramanian, et al., 2004 #31380 

Quality measure (QM): Non-receipt of standard surgical treatment Basis of QM: Clinical practice guideline (NIH guidelines) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Survival Intent of QM: Not specified (aim of the study was to examine colorectal cancer treatment differences between races with comparable disease at presentation) Definition of denominator/numerator:Denominator: Number of white or black subjects diagnosed with rectal cancer between Jan 1, 1988 and Dec 31, 1997. Exclusion criteria: Subjects neither white nor black, unknown treatment history, cancer diagnosed at autopsy, in-situ tumor or carcinoid tumor, rare histological types. Numerator: Number of above subjects not receiving standard surgical treatment. Comparisons were made between blacks and whites. Data sources: SEER database Recommended frequency of data collection: No recommendation 

Study population: 

Charac-teristic 

Male Female 

 White Black White Black n 49,359 4383 47,803 4832 < 84 yr 92.5 95.3 84 92 AJJCC Tumor Stage II 26.84 23.07 24.18 20.99 II 29.54 26.44 32.49 28.04 III 22.66 23.02 23.77 24.71 IV 16.76 22.36 15.70 20.88 Unknown 4.2 5.11 3.87 5.38 

Geographic location: SEER geographic regions Dates: 1988-1997 Healthcare setting: Mixed Results:The black-to-white disparities in surgical treatment for stage I and IV colon cancer were small, but persisted controlling for age, location of cancer, histologic type, and tumor grade. No significant racial disparities in receipt of surgical treatment were apparent for stage II and III and unstaged colon cancer patients. Odds of non-receipt of surgical treatment were higher for black males compared with white males for stages I, III, and IV rectal cancer. A higher likelihood of nonreceipt of surgical treatment was also seen for stage I and II rectal cancer among black females. Black-to-white disparity in non-receipt was more prominent when actual treatment received rather than intent-to-treat was considered. Patients refused cancer-directed surgery: Stage I: 32.8% black and 9.2% white males.19.4% black and 12.8% white females. 

General comments:

Intent-to-treat was consexploratory surgical proactual removal of tumosurgical treatment.” “Sfor rectal and colon canstudy. Report states that highstandard treatment in bhigher treatment refusaRating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 3  Validity: 3  Adaptability: 4  Adequacy of risk a
E-40

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Dimick, Cowan, Upchurch, et al., 2003 #33220 

Quality measure (QM): In-hospital mortality Basis of QM: None Type of QM:(a) Structure (b) General Outcome to which the QM is linked: Mortality Intent of QM: Not specified (aim of the study was to determine the impact of hospital volume on mortality for patients of different age groups to determine whether elderly patients would derive more benefit from selective referral policies) Definition of denominator/numerator:Denominator: Number of patients who underwent colon resection for cancer by hospital volume. Numerator: Number of above patients who died while in hospital. Comparisons were made across hospitals: Quartiles of hospitals: Low volume = < 55/yr Medium volume= 55-100/yr High volume = 101-150/yr Very high volume = > 150/yr Data sources: Nationwide Inpatient Sample (NIS) Recommended frequency of data collection: No recommendation 

Study population: 

 Volume 

 Low Medium High V high No.of hospitals 

536 (64%) 

149 (18%) 

97 (12%) 

60 (7%) 

Bed size Small 

271 (51%) 

21 (14%) 

12 (12%) 

2 (3%) 

Med. 

190 (35%) 

57 (38%) 

32 (33%) 

11 (18%) 

Large 

75 (14%) 

69 (46%) 

53 (55%) 

47 (78%) 

Geographic location: U.S. Dates: 1997 Healthcare setting: Mixed Results: Adjusting for several significant patient covariates, low hospital volume was associated with a 50% (95% CI, 10-100%, p = 0.005) increased risk of in-hospital mortality compared to very high volume hospitals.In the multivariate analysis adjusting for several significant patient covariates, low hospital volume was associated with increased risk of in-hospital mortality compared to very high volume hospitals. Other significant independent variables in the multivariate analysis included patient age, nature of admission (urgent or emergent), comorbid disease, annual income, and patient sex. 

General comments:

Risk-adjustment is esseconclusions regarding ton surgical outcomes. Study suggests that oldidentifiable high-risk groreferral to high-volume strategy would realize tand will not affect low-vRating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 3  Validity: 1  Adaptability: 2  Adequacy of risk a
E-41

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Dominitz, Samsa, Landsman, et al., 1998 #9380 

Quality measure (QM): Percentage of patients with colorectal cancer undergoing surgery Basis of QM: None Type of QM:(a) Process (b) General Outcome to which the QM is linked: Survival Intent of QM: Not specified (aim of the study was to assess the influence of race on the treatment and survival of patients with colorectal carcinoma) Definition of denominator/numerator:Denominator: Number of white/black patients with colorectal cancer identified from the VA Patient Treatment File (PTF) who were discharged from VA hospitals with a diagnosis of CRC (ICD-9-CM codes 153.0-153.4, 153.6-154.1) during fiscal year 1989. To exclude prevalent cases, any patient with a diagnosis of CRC (ICD-9-CM codes 153.0-153.4, 153.6-154.1, V10.05-V10.06 ) for hospitalization during FY 1984-1988, or a personal history of CRC (ICD-9-CM codes V10.05-V10.06) during the index hospitalization, was excluded. Male veterans only. No ulcerative colitis or Crohn’s disease. Numerator: Number of above white/black patients treated with surgery, as indicated by specific ICD-9-CM codes in 

the PTF. Data sources: VA PTF 

Recommended frequency of data collection: No recommendation

Study population: N: 2607 whites + 569 blacks Age: 67.1 ± 9.1 (whites); 66.4 ± 9.9 (blacks) Race: See above Sex: 100% male Tumor stage: All stages Performance status: NR Geographic location: VA Medical Centers Dates: 1989 Healthcare setting: VA Hospitals Results:No statistically significant differences in rates of surgical resection in blacks vs. whites. No difference in 5-year relative survival or overall survival in blacks vs. whites. 

General comments:

The article compares raand chemotherapy amoattempt is made to coma guideline) to determinor whites are optimal. The study is limited becadministrative database and did not allow for adRating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 3  Validity: 1  Adaptability: 1  Adequacy of risk a
E-42

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Govindarajan, Shah, Erkman, et al., 2003 #35310 

Quality measure (QM): Percentage of patients with stage II, III colorectal cancer undergoing surgery Basis of QM: None Type of QM:(a) Process (b) General Outcome to which the QM is linked: Not firmly linked to any outcome, but presumably to survival, quality of life, or rates of local recurrence Intent of QM: Not specified (aim of the study was to analyze disease stage, treatment received, and socioeconomic factors to better understand the factors influencing survival differences between AfricanAmericans and Caucasians with colorectal cancer) Definition of denominator/numerator:Denominator: Number of African-Americans (AAs)/Caucasians, aged 18 older, with CRC treated between 1984 and 1997 at a state-funded University hospital located in Arkansas, identified through the hospital tumor registry using appropriate tumor registry codes. Numerator: Number of above AAs/Caucasians treated with surgery. Data sources: Tumor registry at Univ. of Arkansas. Recommended frequency of data collection: No recommendation 

Study population: N: 617 

Colorectal Caucasian AA 

N 427 190 Median age 60.0 61.5 Male (%) 47.1 39.5 Tumor stage (%) 0-I 23.4 21.6 II 20.5 15.3 III 26.2 26.7 IV 30.0 36.4 

Performance status: NR Geographic location: Arkansas Dates: 1984-1997 Healthcare setting: University hospital Results:Authors show that significantly higher percentages of Caucasians received surgery, compared with AAs, for patients with all stages of CRC, and for patients with only stages II and III disease. Cancer-specific mortality was higher in AfricanAmericans than in Caucasians for all stages. 

General comments: N

Rating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 3  Validity: 1  Adaptability: 1  Adequacy of risk a
E-43

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Harmon, Tang, Gordon, et al, 1999 #35570 

Quality measure (QM): In-hospital death rate Basis of QM: None Type of QM:(a) Outcome (b) General Outcome to which the QM is linked: Mortality Intent of QM: Not specified (aim of the study was to examine the association of surgeon and hospital case volumes with the short-term outcomes in-hospital death, total hospital charges, and length of stay for resection of colorectal carcinoma) Definition of denominator/numerator:Denominator: Number of patients who underwent colorectal resection (partial colectomy, total colectomy, abdominoperineal resection, other rectal resections including anterior resection). Numerator: Number of above patients who died in hospital. Comparisons were made across hospital and surgeon volume. Surgeon case volume groups: low = < 5 

cases/yr, medium = 5-10 cases/yr, high = >10 cases/yr. Hospital volume groups: low = <40 cases/yr, medium = 40-70 cases/yr, high => 70 cases/yr. 

Data sources: Health Services Cost Review Commission Recommended frequency of data collection: No recommendation 

Study population:

Characteristics 

Surgeon Volume Groups 

Hospital Volume Groups 

 Low/ Med/ 

High 

Low/ Med/ High 

Mean age (years) 

68.7/ 69.2/ 69.3 

69.1/ 69.5/ 68.8 

Sex (% male) 47.0/ 48.6/ 

47.2 

47.8/46.6/ 48.7 

Race-%white 74.4/ 83.8/ 

84.1 

74.9/ 81.7/ 84.7 

Race-% black 23.8/ 14.8/ 

13.7 

23.3/ 16.5/ 13.5 

Geographic location: Maryland Dates: 1992-1996 Healthcare setting: Mixed Results:

Characteristics State 

Total 

Surgeon Volume Groups 

Hospital Volume Groups 

 Low/ 

Med/ High 

Low/ Med/ High 

Mortality Crude % 3.5 4.5/ 3.3/ 

2.6 

4.7/ 3.0/ 3.0 

Adjusted relative risk 

 1.00/0.79/ 

0.64 

1.00/0.79/ 0.78 

A positive relation between high surgeon case volume and favorable outcomes was observed. Medium volume surgeons achieved excellent outcomes similar to high-volume surgeons when operating in medium- or high-volume hospitals, but not in low-volume hospitals. 

General comments:

Hospital case volume csurgeon case volume. hospitals had indistinguRating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 4  Validity: 4  Adaptability: 3  Adequacy of risk a
E-44

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality 

Results of low-volume surgeons improved with increasing hospital volume, but never equated those of high-volume surgeons Besides surgeon and hospital volume, other independent variables had a significant impact: an urgent or emergent admission, increased age, presence of organ metastasis, increased comorbidity index, male sex, undergoing a total colectomy or rectal resection and Medicaid payer status were associated with poorer outcomes. 

Hodgson, Zhang, Zaslavsky, et al., 2003 #34120 

Quality measure (QM): Postoperative (30-day) mortality Basis of QM: None Type of QM:(a) Outcome (b) General Intent of QM: Not specified (aim of the study was to assess the association between hospital volume and colostomy rates, postoperative mortality, and overall survival for patients with rectal cancer) Definition of denominator/numerator:Denominator: Number of patients diagnosed with rectal cancer or rectosigmoid cancer (ICD for Oncology codes 20.9 and 19.9), stage I-III, in California who underwent a major surgical procedure and had available hospital, demographic, and followup data. Numerator: Number of above patients who died within 30 days of the surgical procedure. Data sources: California Cancer Registry Recommended frequency of data collection: No recommendation 

Study population: N: 7,257 

Median age 68.1 

Male (%) 55.5 Race (%) White 75.0 Black 4.9 Hispanic 10.9 Asian 8.8 Tumor stage (%) I 34.0 II 33.3 III 32.7 

Performance status: NR Geographic location: California Dates: January 1, 1994-December 31, 1997 Healthcare setting: Hospital Results:The 30-day mortality rate varied from 1.6% at highvolume hospitals to 4.8% at low-volume hospitals. 

General comments:

30-day mortality seems indicator of quality of sunumber of factors that mhospital volume. The inmultivariate analysis anremained a significant iare some factors that mthe model. Rating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 4  Validity: 4  Adaptability: 3  Adequacy of risk a
E-45

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Hodgson, Zhang, Zaslavsky, et al., 2003 #34120 

Quality measure (QM): Rate of colostomy for patients with rectal and rectosigmoid cancers Basis of QM: None Type of QM:(a) Outcome (b) General Intent of QM: Not specified (aim of the study was to assess the association between hospital volume and colostomy rates, postoperative mortality, and overall survival for patients with rectal cancer) Definition of denominator/numerator:Denominator: Number of patients diagnosed with rectal cancer or rectosigmoid cancer (ICD for Oncology codes 20.9 and 19.9), stage I-III, in California who underwent a major surgical procedure and had available hospital, demographic, and followup data. Numerator: Number of above patients requiring a colostomy. Data sources: California Cancer Registry Recommended frequency of data collection: No recommendation 

Study population: N: 7,257 

Median age 68.1 

Male (%) 55.5 Race (%) White 75.0 Black 4.9 Hispanic 10.9 Asian 8.8 Tumor stage (%) I 34.0 II 33.3 III 32.7 

Performance status: NR Geographic location: California Dates: January 1, 1994-December 31, 1997 Healthcare setting: Hospital Results:33.1% of patients underwent a permanent colostomy, and the rate increased statistically as hospital volume decreased. 

General comments:

The relation of colostomseems to have face valmay not indicate poor qpatients have tumors thexcised with a reanastacolostomy could be temanastomosis. The authtemporary colostomies codes for reversal withimisplace some patientsRating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 5  Validity: 2  Adaptability: 2  Adequacy of risk a
E-46

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Hyman, Labow, and Vermont Chapter of the American College of Surgeons, 2002 #18980

Quality measure (QM): Curative resection rate Basis of QM: Not specified Type of QM:(a) Process (b) General Outcome to which the QM is linked: Mortality Intent of QM: Quality improvement (aim of the study was to assess the feasibility of performing a quality study of the surgical management of colorectal cancer using a voluntary registry) Definition of denominator/numerator:Denominator: Number of colorectal cancer patients reported to registry. Numerator: Number of above patients curatively resected. Curative resection defined as “no residual disease and 

no unresected metastatic disease.” Data sources: Prospective statewide voluntary 

registry created by the Vermont chapter of ACS with the Vermont Program for Quality Health Care Recommended frequency of data collection: No recommendation 

Study population: N: 364 (33 surgeons) Age: Mean 68.7 Stage of tumor: Stage I = 24%, stage II = 32%, stage II = 28%, stage IV = 16% Operative Procedures Performed in 364 Patients with Colorectal Cancer 

Procedure Patients, No. 

Right hemicolectomy 125 Transverse colectomy 14 Left hemicolectomy 23 Sigmoid resection 55 Subtotal colectomy 13 Total proctocolectomy 4 Anterior resection 58 Coloanal anastomosis 11 Abdominoperineal resection 

15 

Local excision 20 Diverting stoma only 13 Intestinal bypass 3 Exploratory laparotomy only 

2 

Other/multiple procedures 

8 

Total 364 

Geographic location: Vermont Dates: April 1 1999 to March 31, 2001 Healthcare setting: Mixed Results:85% patients had a potentially curative resection (no residual local disease and no unresected metastatic disease). 

General comments:

The study was not desito compare quality of csurgeons (although it ccomparison with nationA voluntary registry waRating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 2  Validity: 4  Adaptability: 3  Adequacy of risk a
E-47

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Ko, Chang, Chaudhry, et al., 2002 #17540

Quality measure (QM): In-hospital death by hospital volume Basis of QM: None Type of QM:(a) Structure (b) General Outcome to which the QM is linked: In-hospital mortality Intent of QM: Not specified (aim of the study was to evaluate the importance of volume variables relative to other factors in an attempt to target specific areas for improving outcomes for colon cancer resections) Definition of denominator/numerator:Denominator: Number of patients with colon cancer resection. Numerator: Number of above patients who died in hospital. Comparisons were made by hospital volume. Data sources: 1996 HCUP nationwide inpatient sample Recommended frequency of data collection: No recommendation 

Study population: N: 22,408 

Average age 70.4 +/- 12.2 male 51% Caucasian 86% black 8% Hispanic 4% Asian 1% Average discharges 

15,437 +/- 11,000 

Colorectal cases 60 +/- 41 Surgeon cases 10.8 +/- 7.8 

Geographic location: U.S. Dates: 1996 Healthcare setting: Inpatient hospital setting Results:Significant predictors of mortality were age, sex, comorbid disease and volume (both hospital and surgeon). The predicted decrease in mortality by going to a high-volume hospital was 0.13%. Overall, the volume variables, although statistically significant, have a relatively smaller effect on outcome compared with other factors. 

General comments:

The analysis suggests in the greatest increase mortality included havinhaving an emergency omore than a 3-fold incre(1.22%). Female sex, volume and increased associated with a decreRating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 4  Validity: 3  Adaptability: 3  Adequacy of risk a
E-48

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Ko, Chang, Chaudhry, et al., 2002 #17540

Quality measure (QM): In-hospital death by surgeon volume Basis of QM: None Type of QM:(a) Structure (b) General Outcome to which the QM is linked: In-hospital mortality Intent of QM: Not specified (aim of the study was to evaluate the importance of volume variables relative to other factors in an attempt to target specific areas for improving outcomes for colon cancer resections) Definition of denominator/numerator:Denominator: Number of patients with colon cancer resection. Numerator: Number of above patients who died in hospital. Comparisons were made by surgeon volume. Data sources: 1996 HCUP nationwide inpatient sample Recommended frequency of data collection: No recommendation 

Study population: N: 22,408 

Average age 70.4 +/- 12.2 male 51% Caucasian 86% black 8% Hispanic 4% Asian 1% Average discharges 

15,437 +/- 11,000 

Colorectal cases 60 +/- 41 Surgeon cases 10.8 +/- 7.8 

Geographic location: U.S. Dates: 1996 Healthcare setting: Mixed Results:Significant predictors of mortality were age, sex, comorbid disease and volume (both hospital and surgeon). The predicted decrease in mortality by going to a high-volume surgeon was 0.17%. Overall, the volume variables, although statistically significant, have a relatively smaller effect on outcome compared with other factors. 

General comments:

The analysis suggests in the greatest increase mortality included havinhaving an emergency omore than a 3-fold incre(1.22%). Female sex, volume and increased associated with a decreRating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 4  Validity: 3  Adaptability: 3  Adequacy of risk a
E-49

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Merrill, Brown, Potosky, et al., 1999 #27110 

Quality measure (QM): Percentage of CRC patients who had surgical resection Basis of QM: None Type of QM:(a) Structure (b) General Outcome to which the QM is linked: Presumably survival Intent of QM: Not specified (aim of the study was to compare treatment utilization and long-term survival in HMO and fee-for-service [FFS] settings for Medicare colorectal cancer cases) Definition of denominator/numerator:Denominator: Number of CRC patients in each subgroup of the dataset (see Data sources, below). Numerator: Number of above patients who had surgical resection. Data sources:1. Two tumor registries on colon and rectum cancer patients diagnosed 1985-92 and followed up through 31 December 1994. Both registries are part of SEER. 2. Medicare inpatient claims data. 3. Databases of two HMO plans, Kaiser Permanente of Northern California and Group Health Cooperative of Puget Sound. 4. 1990 U.S. Census Recommended frequency of data collection: No recommendation 

Study population: N: 15,352 total Age: ≥ 65 Race: 81.2 to 95.3% white depending on subgroup Sex: 46.8 to 54.3% male, depending on subgroup Tumor stage: 0-IV Performance status: NR Comorbidity Index: 0-2+ Geographic location: San-Francisco-Oakland area, Seattle-Puget Sound area Dates: 1985-1992 Healthcare setting: 2 tumor registries participating in the SEER Program augmented with Medicare data along with HMOs KPNC and GHC. Results:Only statistical difference in rates of surgical resection between HMO and FFS cases was observed for stage I cases, in which the percentage receiving resection was 75% (95% CI 70-79%) for HMO cases and 82% (95% CI 80-84%) for FFS cases. Standardizing to the FFS cases gave an adjusted HMO rate of 74% (95% CI 70-79%), which was no longer statistically different. No differences were seen in the use of surgical resection for rectal cases. Cancer-specific mortality rates were similar in both settings. 

General comments:

A very high-quality studRating of quality meastudy (scale of 1-5, wh= ideal): - Importance: 5 - Usability: 4 - Scientific acceptability  Precise specificat Reliability: 3  Validity: 4  Adaptability: 4  Adequacy of risk a
E-50

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Morris, Billingsley, Baxter, et al., 2004 #32200 

Quality measure (QM): Rate of sphincter-preserving procedures (SSP) Basis of QM: Not specified, but appears to be NIH Consensus conference Type of QM:(a) Process (b) General Outcome to which the QM is linked: No explicit outcome link, but presumably to decreased rates of LRR or improved OS Intent of QM: Aim of the study was to investigate racial variation in the performance of sphincter-sparing procedures for patients with stage II and III rectal cancer who underwent surgery, and to study race and delivery of any surgical treatment, neoadjuvant therapy, and radiation therapy to these patients, with implications for quality of care. Definition of denominator/numerator:Denominator: Number of blacks/whites diagnosed with rectal cancer and entered into the SEER database 1988-1999. Exclusion criteria: Prior diagnosis of CRC, prior diagnosis of cancer requiring surgery or radiotherapy in the pelvis, or < age 35. Numerator: Number of above patients who received various treatments for CRC including surgery (also type of operation) and radiation (timing of RT also evaluated). Data sources: SEER registry Recommended frequency of data collection: No recommendation 

Study population: N = 52,864 

 White Black 

n 44,010 3,851 Age (%) 35-64 35 49 65-79 46 40 80+ 20 12 Male (%) 55 52 Tumor stage (%) In situ 9 8 I 3 28 II 21 20 III 19 18 IV 14 17 Unstaged 8 10 

Geographic location: SEER regions Dates: 1988-1999 Healthcare setting: U.S. healthcare facilities Results:

N = 15,351 

Rate of operation 

 Any None 

Adjusted 

OR (95% 

CI) 

White 96 4 Black 94 6 

1.30 (1.12-

1.95) 

N = 14,405 

SSP APR 

Adjusted 

OR (95% 

CI) 

White 63 37 Black 57 43 

1.42 (1.23-

1.65) 

General comments: N

Rating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 4 - Usability: 4 - Scientific acceptability  Precise specificat Reliability: 3  Validity: 4  Adaptability: 4  Adequacy of risk a
E-51

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Newcomb and Carbone, 1993 #3760 

Quality measure (QM): Rate of surgery for colorectal cancer Basis of QM: None mentioned, but presumably practice guideline Type of QM:(a) Process (b) General Outcome to which the QM is linked: Not specified, but presumably survival or rate of bleeding or obstruction Intent of QM: Not specified (aim of the study was to evaluate the selection of cancer treatment among the elderly) Definition of denominator/numerator:Denominator: Number of women residents of Wisconsin aged 20-74 at the time of diagnosis with a new diagnosis of carcinoma of the large bowel during 1989-1991, and who had a listed telephone number and spoke English, and whose physician consented to their participation, 

and who agreed to a telephone interview, and were among the 628 consecutive participants interviewed September 1, 1990 through November 30, 1990. Numerator: Number of above subjects having surgery. 

Data sources: Registry for staging information and patient interviews (for the treatment information) Recommended frequency of data collection: No recommendation 

Study population: N: 121 with colorectal Age: 20-74 Race: NR Sex: Female Tumor stage: All stages Performance status: NR Geographic location: Wisconsin Dates: Sep 1 through Nov 30, 1990 Healthcare setting: None (all participants were outpatients contacted by phone) Results: 98% of patients less than 65 and 94% ≥ 65 received 

surgery. 

General comments:Relying on recall of pata treatment is open to bRating of quality meastudy (scale of 1-5, wh= ideal): - Importance: 5 - Usability: 3 - Scientific acceptability  Precise specificat Reliability: 4  Validity: 4  Adaptability: 5  Adequacy of risk a
E-52

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality O’Connell, Maggard, Liu, et al., 2004 #45210 

Quality measure (QM) Cancer-directed surgery (CDS) rate Basis of QM: NIH Consensus Conference Guidelines 

for adjuvant therapy for colorectal cancer Type of QM:(a) Process (b) General Intent of QM: Not specified (aim of the study was to compare rectal cancer patient outcomes between young and older populations) Definition of denominator/numerator:Denominator: Patients in the SEER database diagnosed with rectal cancer between 1991 and 1999. Tumors identified as “rectum” in location were selected. Specific histologies were chosen to include only adenocarcinomas. Exclusion criteria: Patients with tumors identified as rectosigmoid. Mucinous, signet ring cell, carcinoid, sarcoma and lumphoma histologies were excluded, along with those tumors classified as benign or in situ. Numerator: Number of above patients who received CDS subsequent to diagnosis. Data sources: SEER database Frequency of data collection: No recommendation 

Study population: N = 11,778 

 20-40 yrs 

(young) N = 466 

60-80 yrs (old) N = 11,312 

P-value 

Age 34.1± 4.5 70± 5.5 Male % 54.9 60 0.03 Race White 62.6 82.3 < 0.001 Black 12.7 6.1 < 0.001 Hispanic 8.8 3.7 < 0.001 Asian 7.5 6.4 NS other 8.4 1.5 < 0.001 AJCC stage I 17 23.7 < 0.001 II 15.5 17.8 NS III 27 20 < 0.001 IV 17.4 13.6 0.02 Grade Well 7.5 9.4 NS Moderate 50.4 61.2 < 0.001 Poorly 24.3 14 < 0.001 Anaplastic 1.7 .7 0.008 unknown 16.1 14.7 NS 

Geographic location: U.S. Dates: 1991- 1999 Healthcare setting: Hospital Results:Young patients overall were as likely to receive CDS as their older counterparts (85.4% vs. 85.5%). Similar findings were seen for stages I, II and III disease (96.2-99.2% received CDS). However, both young and older patients with stage IV disease were less likely to receive CDS (51.9 vs. 51.8; p = NS). Median survival was 24 months for the young group and 27 months for the older group. Univariate 

General comments:

The SEER registry maimeasures to prevent co

one of the best populathowever, miscoding anpresent. The registry drelevant data, such as factors, etc. A third limtumor stage or grade wpatients. Another limitasubstantially smaller sapatients compared with patients, leading to oveRating of quality meastudy (scale of 1-5, wh= ideal): Importance: 4 Usability: 5 Scientific acceptability  Precise specification Reliability: 3  Validity: 3  Adaptability: 3  Adequacy of risk ad
E-53

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality 

analysis revealed overall 5-year survival was similar for the young group compared with the older (63.2 vs. 62.1%; p = NS). Five-year, stage-specific survival also was similar for all stages. Multivariate Cox regression analysis revealed that young, rectal cancer patients had a lower hazard of dying (hazard ratio, 0.693, p = 0.0004) when controlling for tumor stage, patient demographics, tumor characteristics, and stage and treatment. 
E-54

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality O’Connell, Maggard, Liu, et al., 2004 #29710 

Quality measure (QM): Percentage of patients with colon cancer who underwent cancer-directed surgery Basis of QM: None

Type of QM:(a) Process (b) General Outcome to which the QM is linked: Mortality Intent of QM: Not specified (aim of the study was to analyze differences in patient demographics, cancerrelated data, and survival between young and older colon cancer patients) Definition of denominator/numerator:Denominator: Number of patients with colon cancer diagnosed between 1991 and 1999 in a SEER database (which included only those patient living in a SEER area at the time of diagnosis) who were either aged 20-40 or aged 60-80. Exclusion criteria: In situ cancers, rectal cancers. Numerator: Number of the above subjects who also underwent “cancer-directed surgery” (a variable in the SEER database). Data sources: SEER database Recommended frequency of data collection: No recommendation 

Study population: N: 47,791 

 Young 

(20-40 yrs) 

Older 

(60-80 yrs) 

N 1,334 46,457 Age 34.1 70.8 Male (%) 52.1 51.5 Race (%) White 64.2 81.2 Black 16.4 9.0 Hispanic 6.5 3.0 Asian 7.4 5.6 Other 5.5 1.2 AJCC Stage (%) I 10.6 18.6 II 23.0 29.0 III 31.5 22.8 IV 24.5 17.3 Unstaged 10.4 12.3 

Geographic location: SEER regions not specifically listed in this publication) Dates: Diagnosis 1991-1999; followed through 2004 (?) Healthcare setting: Variety, Medicare, non-HMO Results:Young patients were as likely to undergo cancerdirected surgical resection as their older counterparts, with 91.4% of young patients and 91.2% of older patients having cancer-directed surgery (p = NS). Similar findings were seen specifically for surgery for stage I, II, and III disease (99.2-99.7% underwent surgery). However, older patients with stage IV disease had surgery significantly less often than young patients (74.6% vs. 83.2%, p < 0.001). Overall 5-year cancer-specific survival for young patients was poorer than for older patients (62% vs. 65%). 

General comments:

The data for other poteare much weaker, poornecessarily follow guideradiation therapy are toQM (i.e., not stratified bfor colon cancer anywaexcluded from this studRating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 4 - Usability: 4 - Scientific acceptability  Precise specificat Reliability: 3  Validity: 2  Adaptability: 3  Adequacy of risk a
E-55

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Purves, Pietrobon, Hervey, et al. 2005 #43660 

Quality measure (QM): Rate of sphincter-sparing procedures (SSP) and abdominoperineal resection (APR) procedures performed among surgeons, by caseload volume Basis of QM: Not specified Type of QM:(a) Process (b) General Outcome to which the QM is linked: No explicit outcome link, but presumably to decreased rates of LRR or improved OS. Intent of QM: Not specified (aim of the study was to determine whether the rate of SSP increased among surgeons who performed a larger number of rectal cancer surgeries in one year, while controlling for patient characteristics, including comorbidity) Definition of denominator/numerator:Denominator: Patients with a diagnostic ICD-9 code of 154.1 (malignant neoplasm of the rectum) as primary diagnosis. Exclusion criteria: Patients with anal cancer (154.2, 154.3); patients with rectum or colon cancer (154.0) or patients with rectal cancer contiguous with the rectosigmoid junction or anus (154.8). Also excluded were patients who were not surgically treated for their rectal cancer during the hospital admission of record. Prior diagnosis of CRC, prior diagnosis of cancer 

requiring surgery or radiotherapy in the pelvis, or < age 35. Numerator: Number of above patients who underwent either APR or SSP procedure. Data sources: 1997 National Inpatient Sample (NIS) 

of the Healthcare Cost and Utilization Project (HCUP) database 

Study population: N = 477 

 SSP APR 

n 259 218 Surgeons 120 109 Patient Age (%) ≤ 59 26.3 22.0 60-69 26.6 29.8 ≥ 70 47.1 47.2 Male (%) 56 60 White (%) 70 71.1 Non-White (%) 

12.6 10.1 

Geographic location: 22 states in U.S. Dates: 1997 Healthcare setting: U.S. community hospitals Results:

SSP APR 

Deyo score (%) 

 P<.05 

0-7 29.8 18.2 I8-15 20.1 23.5 ≥ 16 4.4 4.0 Surgeon volume (%) 

 P<.05 

1-3 12.4 16.4 4-9 9.4 7.1 ≥ 10 9.0 2.7 Missing 23.5 19.5 mean 1.27 .73 

(continued on next page) 

General comments:

Investigators did not haspecific characteristics 

such as grading, tumor metastases and other tto staging. Rating of quality meastudy (scale of 1-5, wh= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 4  Validity: 3  Adaptability: 4  Adequacy of risk a
E-56

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality 

Recommended frequency of data collection: No recommendation 

Surgeon caseload volume 

Adjusted for age, sex, race, comorbidity, income odds ratio 

Reduced adjusted (race only) odds ratio 

1-3 1.00 1.00 4-9 1.73 2.02 ≥ 10 5.05 4.55 

SSP accounted for 259 o 54.3 percent of all resections. APR accounted for 45.7%. SSP patients did not differ from those undergoing APR in terms of age, race, sex or income. Patients undergoing SSP, however, had a lower average Deyo comorbidity score than those undergoing APR (6.4 vs. 7.7, p = 0.01). The mean surgeon caseload volume for patients undergoing SSP was significantly more than that for APR patients (7.7 vs. 4.6, p = 0.0002). Rectal cancer patients treated by a high-caseload surgeon (≥ 10 outcome procedures per year) are more likely to receive an SSP than are patients treated by lowcaseload surgeons (1-3 outcome procedures per year). In a multivariable analysis, patients undergoing procedures performed by surgeons in the highest volume category were five times more likely to undergo SSP than patients treated by surgeons in the lowest volume category, when analysis was controlled for age, race, sex, and Deyo comorbidity score. In a reduced model, controlling only for race, the odds ratio of SSP for patients treated by surgeons in the highest-volume category compared with lowestvolume category was 4.55. 
E-57

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Rabeneck, Davila, Thompson, et al., 2004 #44670 

Quality measure (QM): 30 day operative mortality 

rate Basis of QM: None 

Type of QM:(a) Outcome (b) General Intent of QM: Not specified (aim of the study was to compare 30-day and 5-year mortality in elderly vs. younger patients following surgical resection for colorectal cancer) Definition of denominator/numerator:Denominator: Patients admitted to VA hospitals with a new diagnosis of CRC who underwent surgical resection within 6 months from discharge date of first hospitalization between October 1990 and September 2000. Exclusion criteria: Patients with previous CRC diagnosis within last 5 years, those with a prior diagnosis of ulcerative colitis or Crohn’s disease. Numerator: Number of above patients who died within 30 days of surgery. Data sources: National VA Patient Treatment File (PTF) Frequency of data collection: No recommendation 

Study population: N = 22,633 Mean age: 68, with two-thirds ≥ 65 Sex: 98% male Race: 75% white; 17% black, 4% Hispanic 57% comorbidity score of 0; 12% had metastatic disease Geographic location: U.S. Dates: FY 1991- 2000 Healthcare setting: 172 VA medical centers Results:

Hazard Ratio 

 Rectal 

cancer 

P-value Colon 

cancer 

P-value 

< 65 1.00 ≥ 65 2.64 <.0001 2.43 <.0001 Deyo score 0 1.00 1-2 1.77 

0.0002 

1.58 <.0001 

3-4 3.33 <.0001 2.51 <.0001 >5 5.59 <.0001 2.56 .0002 

Results show 30-day postoperative mortality for patients with rectal cancer following surgical resection was more than 2.5 times greater among older patients than among younger when adjusting for sex, race, comorbidity score, marital status, and hospital surgical volume. The analysis was repeated for patients with colon cancer who underwent surgical resection during 1991-2000 and the results were similar (2.5 times greater in elderly). 

General comments:

Some misclassification because of coding erropopulation was predomnot be generalized to wascertainment of chemunknown. Therefore, ovariables might be incocomorbidity may be incadministrative data, anseverity may be incompinformation is an imporRating of quality meastudy (scale of 1-5, wh= ideal): Importance: 5 Usability: 5 Scientific acceptability  Precise specification Reliability: 4  Validity: 4  Adaptability:3  Adequacy of risk ad
E-58

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Read, Myerson, Fleshman, et al., 2002 #17970 

Quality measure (QM): Local control rate 

Basis of QM: Clinical practice guideline (NIH 

Consensus Statement) Type of QM:(a) Structure (b) General Outcome to which the QM is linked: Local tumor recurrence (not well defined) Intent of QM: Quality improvement (aim of the study was to determine the effect of surgeon specialty on disease-free survival and local control in patients with adenocarcinoma of the rectum) Definition of denominator/numerator:Denominator: Number of patients undergoing curative treatment for primary rectal adenocarcinoma with neoadjuvant radiotherapy followed by proctectomy. Patients undergoing endocavitary radiation or transanal local excision were excluded. Numerator: Number of above patients who failed to have local tumor recurrence (not well specified). Data sources: Medical records Recommended frequency of data collection: No recommendation 

Study population: N: 213 Age: 64 mean (19-97) Race: NR Sex: 100% male Tumor stage: NR Performance status: NR Geographic location: U.S. (multi-institutional) Dates: 1977-1995 Healthcare setting: Hospital Results: Background of the surgeon and the pathologic stage were significantly correlated with local control (and disease- free survival) at 5 years. 

No. of 

cases 

% 

without 

relapse 

at 5 yr 

% 

local 

control 

at 5 yr 

Surgeon Colorectal specialist 

251 77 93 

Other 133 68 84 P value 0.004 0.005 Histologic stage I 145 83 95 II 137 75 91 III 102 57 81 P value < 0.001 0.005 

General comments: N

Rating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 4 - Scientific acceptability  Precise specificat Reliability: 4  Validity: 4  Adaptability: 5  Adequacy of risk a
E-59

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Roetzheim, Pal, Gonzalez, et al., 2000 #23910 

Quality measure (QM): Percentage of patients with a colon or rectal cancer diagnosis who received cancerdirected surgery (i.e., performed for cure not palliation) Basis of QM: None stated Type of QM:(a) Process (b) General Outcome to which the QM is linked: Mortality Intent of QM: Not specified (aim of the study was to explore the influence of race/ethnicity and insurance payer on the treatment of and outcomes for colorectal cancer patients in Florida) Definition of denominator/numerator:Denominator: Number of incident cases of colon or rectal cancer occurring in Florida in 1994 as listed in the state tumor registry. Numerator: Number of the above patients who also received cancer-directed surgery within 4 months of initiation of therapy. Data sources: Florida state cancer registry (Florida Cancer Data System = FCDS); Florida Agency for Health Care Administration AHCA discharge abstracts (admissions to all nonfederal acute care hospitals and patient visits to ambulatory surgical centers, freestanding radiation therapy centers and diagnostic imaging centers) Recommended frequency of data collection: No recommendation 

Study population: N: 9,548 

Patient characteristic Percent 

Male 51 Race  White 85  Black 6  Hispanic 8 Diagnosis stage  In situ 6  Local 30  Regional 42  Distant 16  Unstaged 6 Anatomic site  Colon 84  Rectal 16 Comorbidity index  0 71  1 21 

≥ 2 8 

Geographic location: Florida Dates: 1994 Healthcare setting: Multiple Results:Used 2 sources for the procedure data: the cancer registry and the state AHCA discharge summaries and in 94.2% there was agreement regarding surgical treatments performed. Treatments received varied according to insurance payer. Patients with CRC who were uninsured or insured by Medicaid or commercial HMO’s had higher mortality rates than patients with commercial FFS insurance. Mortality was also higher among nonHispanic African-American patients even after controlling for stage of disease at diagnosis and treatment modality. Patients who needed co-directed 

General comments: NRating of quality meastudy (scale of 1-5, wh= ideal): - Importance: 4 - Usability: 3 - Scientific acceptability  Precise specificat Reliability: 4  Validity: 3  Adaptability: 3  Adequacy of risk a
E-60

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality 

surgery were similar in age to those who did not. There were no differences between races in receipt of definitive surgical treatment. Among Medicare patients, those with HMO insurance were more likely to receive definitive surgical treatment, the differences increased with advancing cancer stage. Among nonMedicare patients those having Medicaid, those were uninsured, and those having other forms of health insurance were less likely to receive surgical treatment than were private FFS patients. Use of definitive surgery was more common in younger patients, those with higher levels of education, and those who were married. 

Rogers, Ray, and Smalley, 2004 #30130 

Quality measure (QM): Percentage of patients with a diagnosis of colorectal cancer receiving surgery Basis of QM: Not stated Type of QM:(a) Process (b) General Outcome to which the QM is linked: Not stated, but presumably survival or recurrence of colorectal cancer 

Intent of QM: Not specified (aim of the study was to examine the effect of race on CRC outcomes in patients who had identical health care coverage) Definition of denominator/numerator:

Denominator: Number of patients enrolled in both Tennessee Medicaid and Medicare and hospitalized with a diagnosis of colorectal cancer. Numerator: Number of above patients who received surgery. Data sources: Medical records, administrative files from the Tennessee Medicaid program, Tennessee death certificates, and the National Death Index Recommended frequency of data collection: No recommendation 

Study population: N: 969 total (19.3% rectal cancer) 

 White Black 

Cases (%) 72 28 Age 78.9 78.9 Male (%) 26.1 27.6 Tumor stage I 16.4 9.2 II 39.7 36.0 III 24.4 27.9 IV 9.8 12.9 Unknown 9.8 14.0 CharlsonDeyo score (%) 64 69 

Geographic location: Tennessee Dates: 1984-1994 Healthcare setting: Patients identified from hospitalizations Results:91% of whites and 86% of blacks had surgery. 

General comments:

Study concluded that wcare, outcomes were eRating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 5  Validity: 5  Adaptability: 3  Adequacy of risk a
E-61

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Schrag, Cramer, Bach, et al., 2000 #23710 

Quality measure (QM): 30-day mortality in low- and 

high-volume hospitals Basis of QM: Not stated 

Type of QM:(a) Structure (b) General Outcome to which the QM is linked: Mortality Intent of QM: Quality improvement (aim of the study was to determine whether hospital procedure volume predicts short- and long-term survival following primary colon cancer surgery) Definition of denominator/numerator:Denominator: Number of patients with stage III tumors who survived 3 months postoperatively, were enrolled in Medicare Part B, were 65 years and older, were diagnosed as having primary adenocarcinoma of the colon in a SEER area, and underwent surgery followed by at least one claim for chemotherapy. Exclusion criteria: Patients were excluded if diagnoses were exclusively noted on death certificates or at autopsy; month of diagnosis was unknown; or patient was enrolled in an HMO. Also, patients who were operated on exclusively for local resection or creation of an ostomy were excluded. Numerator: Number of patients who died within 30 days following surgery. Data sources: SEER for patients 65 and older Medicare Claims Part B Recommended frequency of data collection: No recommendation 

Study population:  Overall 

 No. or % 

Total patients 27,986 Total hospitals 611 Sex  Male 44.8  Female 55.2 Age  65-69 19.5  70-74 23.7  75-79 23.2  80+ 33.6 Race  White 84.6  Black 7.1  Other 8.3 AJCC stage  I 19.2  II 36.2  III 25.6  IV 13.9  Unstaged 5.1 Romano Charlson comorbid 0 65.2 1 24.3 2 10.5 

(continued on next page) 

General comments: N

Rating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 3  Validity: 4  Adaptability: 4  Adequacy of risk a
E-62

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality 

 By hospital volume (figures are numbers or %) 

Low Med High 

Very high 

Total patients 6,837 7,105 6,947 7,097 Total hospitals 440 89 51 31 Sex  Male 44.9 44.9 45.6 44.0  Female 55.1 55.1 54.4 56.0 Age  65-69 20.8 18.9 19.2 19.2  70-74 23.9 24.0 23.2 23.9  75-79 22.1 23.3 24.1 23.3  80+ 33.2 33.8 33.6 33.7 Race  White 80.3 83.6 86.1 88.4  Black 7.1 7.2 5.5 8.6  Other 12.6 9.2 8.4 3.0 AJCC stage  I 18.0 18.4 20.0 20.4  II 35.5 36.3 35.5 37.5  III 24.8 26.3 25.0 26.1  IV 13.9 13.6 14.4 13.6 Unstaged 7.8 5.3 5.0 2.5 Romano Charlson comorbid 0 64.8 64.8 65.5 65.9 1 24.4 24.6 24.3 23.8 2 10.8 10.6 10.2 10.3 

Geographic location: U.S. Dates: 1991-1996 Healthcare setting: All U.S. Results:Adjusting for age, sex, race, cancer stage, comorbid illness, socioeconomic status, and acuity of hospitalization, very high-volume hospitals had a 
E-63

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality 

lower 30-day mortality than low-volume hospitals. A consistent association between higher postoperative mortality and lower surgical volume was evident; this persisted after inclusion of potential confounders. This association was evident for stages I-III, but not for patients with stage IV or unstaged disease (due to smaller sample sizes). The survival curves illustrate a clear association between procedure volume and overall survival. The difference in 5-year mortality for patients operated on at very high- vs. low-volume hospitals was 4.4%. 
E-64

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Schrag, Panageas, Riedel, et al., 2002 #17000 

Quality measure (QM): 30-day mortality rate 

Basis of QM: None Type of QM:

(a) Outcome (b) General Intent of QM: Research (aim of the study was to compare surgeon and hospital procedure volume as predictors of outcomes for patients with rectal cancer) Definition of denominator/numerator:Denominator: Number of Medicare patients undergoing rectal cancer surgery for a primary tumor diagnosed in 1992-1996 in a SEER area. Exclusion criteria: Excluded patients with cancers arising in the rectosigmoid; had tumors with a predominant squamous cell component that could have been anal in origin; HMO enrollees; with diagnoses noted exclusively on death certificates or at autopsy; where the month of diagnosis was unknown. Numerator: Number of above patients who died within 30 days of rectal sphincter-sparing surgery. Patient data were case -adjusted based on age, sex, race, disease stage, comorbidity, socioeconomic status and whether surgery was emergent. Data sources: SEER, Medicare claims Recommended frequency of data collection: No recommendation 

Study population: N: 2,815 

Stage % 

 I 30  II 28  III 29  IV 8  Unstaged 5 Age  65-69 25  70-74 27  75-79 23  80-84 16  85+ 9 Male/female 54/46 Race  White 86  Black 4  Other 10 Comorbidity  0 67  1 25  2+ 8 Nonemergent admit 

91 

Geographic location: U.S. Dates: 1992-1996 Healthcare setting: All Results:Hospitals and surgeons were ranked by their volumes as very low, low, medium and high. There was no statistically significant association between 30-day mortality and hospital or surgeon volume. Over the longer term, differences in surgical outcomes emerged 2 years following resection; these differences were significant and persisted after adjusting for differences in baseline patient 

General comments: N

Rating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 3  Validity: 4  Adaptability: 4  Adequacy of risk a
E-65

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality 

characteristics. However, hospital procedure volume ceased to become a significant predictor of outcomes once surgeon procedure volume had been considered. Surgeon-specific procedure volume remained important as a predictor of outcome after adjusting for differences in hospital procedure volume. 

Hospital Procedure volume 

30-day mortality (%) 

Very low 3.9 Low 4.0 Medium 2.4 High 3.3 P-value 0.79 Surgeon procedure volume Very low 4.3 Low 3.7 Medium 3.5 High 1.7 P-value 0.20 
E-66

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Schrag, Panageas, Riedel, et al., 2002 #17000 

Quality measure (QM): Abdominoperineal (APR) 

resection rate Basis of QM: None Type of QM:

(a) Process (b) General Outcome to which the QM is linked: Mortality Intent of QM: Research (aim of the study was to compare surgeon and hospital procedure volume as predictors of outcomes for patients with rectal cancer) Definition of denominator/numerator:Denominator: Number of Medicare patients undergoing rectal cancer surgery for a primary tumor diagnosed in 1992-1996 in a SEER area. Exclusion criteria: Excluded patients with cancers arising in the rectosigmoid; had tumors with a predominant squamous cell component that could have been anal in origin; HMO enrollees; with diagnoses noted exclusively on death certificates or at autopsy; where the month of diagnosis was unknown. Numerator: Number of above patients for whom surgeons performed abdominoperineal surgery. Patient data were case -adjusted based on age, sex, race, disease stage, comorbidity, socioeconomic status and whether surgery was emergent. Data sources: SEER, Medicare claims Recommended frequency of data collection: No recommendation 

Study population: N: 2,815 

Stage % 

 I 30  II 28  III 29  IV 8  Unstaged 5 Age  65-69 25  70-74 27  75-79 23  80-84 16  85+ 9 Male/female 54/46 Race  White 86  Black 4  Other 10 Comorbidity  0 67  1 25  2+ 8 Nonemergent admit 

91 

Geographic location: U.S. Dates: 1992-1996 Healthcare setting: All Results:APR rates were compared by providers with different case volumes. After case-mix adjusting, the association was not significant, and was further attenuated when adjusted for surgeon volume, p = 0.15. (continued on next page) 

General comments: N

Rating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 3  Validity: 4  Adaptability: 4  Adequacy of risk a
E-67

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality 

Hospital Procedure volume 

APR rate (%) 

Very low 51 Low 53 Medium 49 High 44 P-value 0.04 Surgeon procedure volume Very low 49 Low 54 Medium 51 High 44 P-value 0.15 
E-68

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Schrag, Panageas, Riedel, et al., 2003 #34030 

Quality measure (QM): 30-day mortality rate Basis of QM: None stated Type of QM:(a) Outcome (b) General Intent of QM: Quality improvement (aim of the study was to compare surgeon and hospital procedure volume as predictors of outcomes following colon cancer resection) Definition of denominator/numerator:Denominator: Number of Medicare eligible patients aged 65 or older diagnosed in SEER regions with primary colon cancer in 1991-1996. Exclusion criteria: Excluded were patients enrolled in an HMO; operated on exclusively for intestinal bypass; diagnosis noted exclusively on death certificate or at autopsy; month of death was unknown. Numerator: Number of above patients who died within 30 days of operation (not clearly defined). Adjustment was made for sex, race, age, stage of tumor, comorbidity and socioeconomic status, whether hospitalization was emergent and whether obstruction or perforation was present. Data sources: SEER, Medicare claims from HCFA Recommended frequency of data collection: No recommendation 

Study population: N = 24,166 

Characteristic Percent 

Male/female 44.1/55.9 Age  65-69 18.6  70-74 23.6  75-79 23.6  80+ 34.2 Race  White 87.1  Black 6.6  other 6.3 AJCC stage I 19.9  II 36.9  III 26.1  IV 14.1  Unstaged 3.0 Romano comorbidity  0 71.3  1 23.2  2 5.6 Obstruction (Y/N) 9.0/91.0 Emergent hospitalization (Y/N) 19.8/80.2 

Geographic location: U.S. Dates: 1991-1996 Healthcare setting: All Results:After adjusting for surgeon procedure volume, high hospital procedure volume remained a strong predictor of low postoperative mortality. Surgeonspecific procedure volume was also an important predictor of surgical outcomes although this effect was attenuated after adjusting for hospital volume. (continued on next page) 

General comments:

2-year postoperative massessed. Study claims that it canof care that can accounrates in the various hosRating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 3  Validity: 4  Adaptability: 4  Adequacy of risk a
E-69

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality 

Hospital procedure volume 

30-day mortality rate% 

 Very low 5.8  Low 4.6  Medium 4.1  High 3.7  P-value < 0.0001 Surgeon procedure volume  Very low 5.7  Low 4.6  Medium 4.2  High 3.7  P-value < 0.0001 
E-70

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Schrag, Panageas, Riedel, et al., 2003 #34030 

Quality measure (QM): Ostomy rates Basis of QM: Clinical practice guideline (not specified) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Mortality Intent of QM: Quality improvement (aim of the study was to compare surgeon and hospital procedure volume as predictors of outcomes following colon cancer resection) Definition of denominator/numerator:Denominator: Number of Medicare eligible patients aged 65 or older diagnosed in SEER regions with primary colon cancer in 1991-1996. Exclusion criteria: Excluded were patients enrolled in an HMO; operated on exclusively for intestinal bypass; diagnosis noted exclusively on death certificate or at autopsy; month of death was unknown. Numerator: Number of above patients who had an ostomy. Adjustment was made for sex, race, age, stage of tumor, comorbidity and socioeconomic status, whether hospitalization was emergent and whether obstruction or perforation was present. Data sources: SEER, Medicare claims from HCFA Recommended frequency of data collection: No recommendation 

Study population: N = 24,166 

Characteristic Percent 

Male/female 44.1/55.9 Age  65-69 18.6  70-74 23.6  75-79 23.6  80+ 34.2 Race  White 87.1  Black 6.6  other 6.3 AJCC stage I 19.9  II 36.9  III 26.1  IV 14.1  Unstaged 3.0 Romano comorbidity  0 71.3  1 23.2  2 5.6 Obstruction (Y/N) 9.0/91.0 Emergent hospitalization (Y/N) 19.8/80.2 

Geographic location: U.S. Dates: 1991-1996 Healthcare setting: All Results:Hospital volumes and surgeon volumes were each an important predictor of the ostomy rate. (continued on next page) 

General comments:

2-year postoperative massessed. Study claims that it canof care that can accounrates in the various hosRating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 3  Validity: 4  Adaptability: 4  Adequacy of risk a
E-71

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality 

Hospital procedure volume 

Ostomy rate (%) 

 Very low 19.6  Low 18.5  Medium 16.4  High 15.8  P-value < 0.0001 Surgeon procedure volume  Very low 19.4  Low 18.4  Medium 17.2  high 15.3  P-value < 0.0001 
E-72

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Simons, Ker, Groshen, et al., 1997 #11620 

Quality measure (QM): Rate of sphincter-sparing procedures (SSP) Basis of QM: None Type of QM:(a) Process (b) General Outcome to which the QM is linked: Sphincter preservation Intent of QM: Research (aim of the study was to examine variation in the surgical treatment of rectal cancer to determine if differences exist in management and survival based on hospital type and surgical caseload) Definition of denominator/numerator:Denominator: Number of patients who underwent sphincter-sparing procedures (SSP) or abdominoperineal resection (APR) for rectal adenocarcinoma between 1988 and 1992. Numerator: Number of above patients who underwent SSP. Data sources: University of Southern California Cancer Surveillance Program (CSP), a population based cancer registry. This is part of a national SEER. The CSP provided patient age, gender, date of surgery, type of surgery, tumor stage, hospital type, status and date at last follow up. Recommended frequency of data collection: No recommendation

Study population: N: 2,006 Age: 68 median age (19-102) Race: NR Sex: 55% male Other: 125 hospitals % patients with localized disease: 52% Geographic location: California Dates: 1988-1992 Healthcare setting: inpatient hospital setting Results:Hospitals treating patients with localized disease that perform > 5 rectal cancer surgeries per year performed SSP more frequently (69% of cases) compared to hospitals performing 5 or fewer rectal cases per year (63% of cases). For regional disease, the difference was not statistically significant. Survival plots showed that a significantly higher percentage of patients (localized and regional disease) survived 5 years when operated on at a hospital performing > 5 rectal cancer surgeries a year. 

General comments: N

Rating of quality meastudy (scale of 1-5, wh= ideal): - Importance: 4 - Usability: 4 - Scientific acceptability  Precise specificat Reliability: 3  Validity: 4  Adaptability: 4  Adequacy of risk a
E-73

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Steele, 1994 #35840 

Quality measure: Percentage of stage III colon cancer patients receiving surgery and chemotherapy Basis of QM: Clinical practice guideline It is implicit that the AHCC guidelines be followed but these are not actually cited in the article. Type of QM:(a) Process (b) General Outcome to which the QM is linked: No outcome is exactly linked to the above but is said to be a marker of appropriate cancer diagnosis and treatment. No citation is provided for this statement. Intent of QM: Not specified (aim of the study was assess cancer patient care and outcomes on a national basis) Definition of denominator/numerator:Denominator: Number of patients in the National Cancer Data Base (NCDB) with a new diagnosis of colon cancer in the years 1985-1986 or 1991. Numerator: Number of the above with stage III cancer who underwent surgery and received chemotherapy. Data sources: National Cancer Data Base (NCDB)Recommended frequency of data collection: No recommendation 

Study population: Data given as patients (year) N: 40,384 (85/86); 39,751 (91) Age: Median 71.3 (85/85); 72.0 (91) Race:  Non-Hispanic whites 86.9% (85/85); 85.9% (91)  Hispanic 1.1/1.8  African-American 6.6/8.1  Asian 0.9/1.5  Unknown 4.5/2.7 Sex: Male 48.7% (85/85), 49.6% (91) Tumor stage: 85-86/91  0 3.7/6.5  1 14.4/19.3  2 13.8/26.2  3 12.9/20.3  4 9.2/15.8  Unknown 46.0/11.9 Performance status: NR Geographic location: National sample; note that the sample is not probabilistic and represented different hospitals in different years Dates: 1991 and outcomes for 1985-86 Healthcare setting: 464 hospital in 1985, 474 hospitals in 1986, 937 hospitals in 1991 Results: Treatment trends showed effects of trial data, as a greater percentage of patients were reported as having treatment (including chemotherapy) beyond surgery in 1991 as compared to 1985/86. 

General comments:

Some “treatment” data not stratified by stage oand therefore do not prcount as a QM. Only sQM from this paper. Rating of quality meastudy (scale of 1-5, wh= ideal): - Importance: 4 - Usability: 4 - Scientific acceptability  Precise specificat Reliability: 3  Validity: 3  Adaptability: 3  Adequacy of risk a
E-74

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Temple, Hsieh, Wong, et al., 2004 #29890 

Quality measure (QM): 30-day postoperative mortality rate for stage IV colon cancer Basis of QM: Clinical practice guideline (National Comprehensive Cancer Network; applicable only for stage IV and symptomatic patients) Type of QM:(a) Outcome (b) General Intent of QM: Quality improvement (aim of the study was to evaluate surgical practice patterns for patients 65 years of age and older with stage IV colorectal cancer) Definition of denominator/numerator:Denominator: Number of Medicare-enrolled patients aged 65 and older, initially diagnosed with stage IV colon or rectal cancer in a SEER area during the years 1991-1999. Age, sex, race, ethnicity, comorbidity, year of diagnosis, and socioeconomic status were collected and data were risk-adjusted. Exclusion criteria: Cohort restricted to those with a histologic diagnosis consistent with adenocarcinoma. Diagnoses exclusively on death certificates or at autopsy were excluded, as were those in which the month of diagnosis was unknown. Patients enrolled in HMOs were excluded. Numerator: Number of above patients who died 30 days following surgery. Data sources: SEER, Medicare claims Recommended frequency of data collection: No recommendation 

Study population: N: 9,011 

Pct. 

Primary CDS% 

No primary CDS% 

Age  65-69 21 22 18  70-74 25 26 22  75-79 23 23 24  80-84 18 17 19 

≥85 13 12 18 

Male/female 48/52 48/52 49/51 White 84 85 81 Black 9 9 12 Other 7 7 7 Comorbidity 0 90 90 89 1 5 5 6 2+ 5 4 5 1999 9 9 10 

Geographic location: U.S. Dates: 1991-1999 Healthcare setting: Inpatient hospital Results: The 30-day surgical mortality was significantly greater in the no cancer-directed surgery group among patients who underwent a surgical procedure, when compared with the primary cancer-directed surgery group (26% vs 9%; P = 0.001). 

General comments:

Patient selection was arandomized setting. ThStudy could not ascertaasymptomatic at presepresent with advanced Rating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 3  Validity: 3  Adaptability: 2  Adequacy of risk a
E-75

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Temple, Hsieh, Wong, et al., 2004 #29890 

Quality measure (QM): Metastasectomy rate Basis of QM: Clinical practice guideline (National Comprehensive Cancer Network) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Practice patterns of patients with stage IV cancer Intent of QM: Quality improvement (aim of the study was to evaluate surgical practice patterns for patients 65 years of age and older with stage IV colorectal cancer) Definition of denominator/numerator:Denominator: Number of Medicare-enrolled patients aged 65 and older, initially diagnosed with stage IV colon or rectal cancer in a SEER area during the years 1991-1999. Age, sex, race, ethnicity, comorbidity, year of diagnosis, and socioeconomic status were collected and data were risk-adjusted. Exclusion criteria: Cohort restricted to those with a histologic diagnosis consistent with adenocarcinoma. Diagnoses exclusively on death certificates or at autopsy were excluded, as were those in which the month of diagnosis was unknown. Patients enrolled in HMOs were excluded. Numerator: Number of above patients who received liver resection, pelvic exenteration and pulmonary resection between diagnosis and death. Data sources: SEER data base, Medicare claims data Recommended frequency of data collection: No recommendation 

Study population: N: 9,011 

Pct. 

Primary CDS% 

No primary CDS% 

Age  65-69 21 22 18  70-74 25 26 22  75-79 23 23 24  80-84 18 17 19 

≥85 13 12 18 

Male/female 48/52 48/52 49/51 White 84 85 81 Black 9 9 12 Other 7 7 7 Comorbidity 0 90 90 89 1 5 5 6 2+ 5 4 5 1999 9 9 10 

Geographic location: U.S. Dates: 1991-1999 Healthcare setting: Inpatient hospital Results: Metastasectomy was performed in only 3.9% of the patients in the total cohort of 9,011 having received either synchronous or metachronous pulmonary resection, hepatectomy, or pelvic exenteration. Among those who underwent primary cancer-directed surgery, 5.2% had metastasectomy, the vast majority being liver resections. 

General comments:

Patient selection was arandomized setting. ThStudy could not ascertaasymptomatic at presepresent with advanced Rating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 3 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 3  Validity: 3  Adaptability: 2  Adequacy of risk a
E-76

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Temple, Hsieh, Wong, et al., 2004 #29890 

Quality measure (QM): Cancer-directed surgery (CDS) rate Basis of QM: Basis of QM: Clinical practice 

guideline (National Comprehensive Cancer Network) Type of QM:

(a) Process (b) General Outcome to which the QM is linked: Reduced mortality Intent of QM: Quality improvement (aim of the study was to evaluate surgical practice patterns for patients 65 years of age and older with stage IV colorectal cancer) Definition of denominator/numerator:Denominator: Number of Medicare-enrolled patients aged 65 and older, initially diagnosed with stage IV colon or rectal cancer in a SEER area during the years 1991-1999. Age, sex, race, ethnicity, comorbidity, year of diagnosis, and socioeconomic status were collected and data were risk-adjusted. Exclusion criteria: Cohort restricted to those with a histologic diagnosis consistent with adenocarcinoma. Diagnoses exclusively on death certificates or at autopsy were excluded, as were those in which the month of diagnosis was unknown. Patients enrolled in HMOs were excluded. Numerator: Number of above patients who received CDS subsequent to diagnosis. This included patients having received bowel resection, or evidence of primary tumor resection; surgery of primary sites within the first 4 months of diagnosis. Data sources: SEER database, Medicare claims dataRecommended frequency of data collection: No recommendation 

Study population: N: 9,011

 % Primary 

CDS (%) 

No primary CDS (%) 

Age  65-69 21 22 18  70-74 25 26 22  75-79 23 23 24  80-84 18 17 19 

≥85 13 12 18 

Male/female 48/52 48/52 49/51 White 84 85 81 Black 9 9 12 Other 7 7 7 Comorbidity 0 90 90 89 1 5 5 6 2+ 5 4 5 1999 9 9 10 

Geographic location: U.S. Dates: 1991-1999 Healthcare setting: Inpatient hospital Results: 72% of cohort received CDS within 4 months of diagnosis. CDS rates declined with age. Median survival of CDS patients was 10 months compared to non-CDS patients (3 months). 

General comments:

Patient selection was arandomized setting. ThStudy could not ascertaasymptomatic at presepresent with advanced Rating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 4 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 3  Validity: 3  Adaptability: 2  Adequacy of risk a
E-77

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Tropman, Hatzell, Paskett, et al., 1999 #35610 

Quality measure (QM): Percentage of patients receiving appropriate primary therapy (wide surgical resection and anastomosis) for CRC as defined by the NCI PDQ guidelines Basis of QM: Clinical practice guideline (NCI PDQ Guidelines) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Locoregional recurrence, overall survival. Intent of QM: Not specified (aim of the study was to determine the degree to which colon cancer treatment in rural North and South Carolina conformed to national treatment recommendations) Definition of denominator/numerator:Denominator: Number of patients treated for colon cancer in Wayne County, NC, and its six contiguous counties or in Greenwood County, SC, and its four adjacent counties during the years 1991 and 1996. Cases were identified by the North Carolina Central Cancer Registry and by local tumor registrars in SC. Numerator: Subset of patients above who received primary (surgery) per the NCI PDQ recommendations. Data sources: Patient medical records and records from physician offices in the geographic regions under study Recommended frequency of data collection: No recommendation 

Study population: N: 230 

All stages 1991 1996 

Male (%) 55 58 White 76 75 AA 24 25 Mean age 75 71 Tumor stage I 64 36 II 56 44 III 65 35 

Performance status: NR Geographic location: Wayne County, NC, and its six contiguous counties or in Greenwood County, SC, and its four adjacent counties Dates: 1991, 1996 Healthcare setting: Rural health care facilities Results:A majority (80-95%) of patients received appropriate primary therapy, with the exception of SC stage I cases, who had lower rates of appropriate primary therapy (73% in 1991 and 50% in 1996). 

General comments:

The article compares raPDQ recommendations regions (one in NC, one after (1996) an intervenphysicians. The interveSC region served as a Rating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 4 - Scientific acceptability  Precise specificat Reliability: 4  Validity: 4  Adaptability: 4  Adequacy of risk a
E-78

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Vernon, Hughes, Heckel, et al., 1992 #5670 

Quality measure (QM): Number of colorectal cancer patients that receive surgery as part of primary treatment of colorectal cancer Basis of QM: None stated Type of QM: (a) Structure (b) General Outcome to which the QM is linked: Presumably to improved survival Intent of QM: Not specified (aim of the study was to compare the quality of cre for two types of health plans – fee-for-service [FFS] with third-party coverage and HMO membership – in the diagnosis and treatment of colorectal cancer) Definition of denominator/numerator: Denominator: Number of patients with colorectal cancer diagnosed from 1984 through 1989 at the Kelsey-Seybold Clinic in Houston, TX, of whom 205 used a fee-for-service (FFS) plan and 125 were members of an HMO. Numerator: Number of above patients who received surgery as part of their primary treatment. Data sources: Medical records of patients Recommended frequency of data collection: No recommendation 

Study population: FFS/HMO plans N: 205/125 Age: < 50 to ≥ 60 

Race: NR Sex: 56% male/61% male Tumor stage: NR Performance status: NR Geographic location: Texas

Dates: 1984-89 Healthcare setting: A multispecialty group practice Results: 82% of FFS patients had surgery compared with 81% of HMO patients. Survival was not different for the two groups. 

General comments: The purpose of this papeffectiveness” of two tyHMO) offered by the sadiagnosis and treatmenplans were compared fbe considered quality mother explanations for tduration of symptoms bphysicians who diagnosymptoms at diagnosistreatment, number of trdiagnosis, and survivalbe quality measures. Rating of quality meastudy (scale of 1-5, wh= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 5  Validity: 5  Adaptability: 3  Adequacy of risk a
E-79

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Weaver, Harrison, Eskander, et al., 1991 #6810 

Quality measure (QM): Percentage of patients with CRC who underwent surgical resection Basis of QM: None stated Type of QM:(a) Process (b) General Outcome to which the QM is linked: Mortality Intent of QM: Not specified (aim of the study was to determine if demographic observations were similar in an all-black institution with patients from all socioeconomic levels) Definition of denominator/numerator:Denominator: Number of patients with a diagnosis of colorectal cancer between 1971 and 1982 in the tumor registry databases at a single institution (Meharry). Numerator: Number of the above patients who underwent “surgical intervention.” Data sources: Hospital tumor registry, hospital medical records Recommended frequency of data collection: No recommendation 

Study population: N: 118 Age: mean 68 (29-93) Race: 100% black Sex: 62% female Tumor stage: I 4 Ib 2 II 22 III 36 IV 32 Missing 22 Geographic location: Nashville, TN Dates: 1971-1982 (retrospective review) + 5 years of followup (?1987) Healthcare setting: Single predominantly black hospital (with University affiliation) Results:NA 

General comments:

Mortality was the primathe data for other potenare much weaker, poorguidelines Rating of quality meastudy (scale of 1-5, wh= ideal): - Importance: 5 - Usability: 4 - Scientific acceptability  Precise specificat Reliability: 4  Validity: 4  Adaptability: 3  Adequacy of risk a
E-80

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Whittle, Steinberg, Anderson, et al., 1991 #5450 

Quality measure (QM): Perioperative mortality rate 

Basis of QM: None 

Type of QM:(a) Outcome (b) General Intent of QM: Research (aim of the study was to estimate the perioperative mortality and 1- and 2-year survival rates of elderly Americans undergoing colon resection for colon cancer) Definition of denominator/numerator:Denominator: Number of beneficiaries > 65 yr discharged during 1983-1985 with an ICD-9 code for colon cancer as primary or secondary diagnosis and ICD-9 code for colon resection listed during the same hospitalization. Exclusion criteria: Beneficiaries who first became eligible for Medicare because of end-stage renal disease or disability, members of an HMO, beneficiaries living overseas, non participants in Part 

B. Numerator: Number of above patients who died within 30 days of admission, controlling for age, sex. Data sources: Medicare Part A claims and enrollment 

records Recommended frequency of data collection: No recommendation 

Study population: N: 5,586 

Age % male % female 

66-69 15 12 70-74 30 25 75-84 44 46 85+ 11 17 

Geographic location: U.S. Dates: 1983-1985 Healthcare setting: All Results:Age, but not sex, was significantly associated with 1- and 2-year postoperative mortality. Decreases in survival in older age groups remained statistically significant at 1 yr (p < 0.0001) and 2 yr (p = 0.01) after adjusting for differences in life expectancy among age groups. 

Age % 

male 

% female 

Perioperative mortality 

66-69 15 12 3.35 70-74 30 25 3.65 75-84 44 46 6.67 85+ 11 17 11.1 

General comments:

The observed differencand postoperative outcfactors not considered age and sex and stage presentation, level of coemergency presentatioperioperative mortality rates. Rating of quality meastudy (scale of 1-5, wh= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificat Reliability: 3  Validity: 3  Adaptability: 3  Adequacy of risk a
E-81

Evidence Table 5 – Question 2b (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality Wudel Jr., Chapman, Shyr, et al., 2002 #18640 

Quality measure (QM): Percentage of patients with colon or rectal cancer who had curative resection Basis of QM: None stated Type of QM:(a) Process (b) General Outcome to which the QM is linked: Mortality Intent of QM: Not specified (aim of the study was to determine the outcome of patients with colorectal cancer treated in the same city at 2 nearby medical centers, a university hospital and a city hospital, and to explore disparities in colorectal cancer outcomes 

between black and white patients) Definition of denominator/numerator:

Denominator: Number of patients with a diagnosis of primary colorectal cancer between 1/1/90 and 12/21/99 in the tumor registry databases at each of 2 institutions (university hospital and a city hospital). Excluded patients who were non-white and non-black. Numerator: Number of the above patients who received “curative surgery “defined as surgery performed in the absence of documented distant spread. Data sources: Hospital tumor registries Recommended frequency of data collection: No recommendation 

Study population: 

University hospital 

City hospital 

N 565 77 Black (%) 10.6 53 Male (%) 53.3 62.7 Median age 69 74 Tumor stage I 31.7 29.9 II 21.9 20.8 III 23.7 16.9 IV 22.7 32.5 

Performance status: NR Geographic location: Nashville, TN Dates: 1/1/90 and 12/21/99 (retrospective review) Healthcare setting: University hospital, city hospital Results:There was no significant difference in tumor stage at the time of presentation in the two institutions studied or between racial/ethnic groups. There was no difference in the treatment modalities identified between institutions or racial/ethnic groups. There was a significant difference in overall survival between patients treated at the university hospital (median 5.3 years) and those treated at the city hospital (median 2.1 years). Median survival was 5.7 years for white patients and 3.2 years for black patients treated at the university hospital. White patients treated at the city hospital survived 2.1 years vs. 1.4 years for black patients. 

General comments:

Mortality was the primathe data for other poten“curative” surgery, chemmuch weaker, poorly dguidelines. Rating of quality meastudy (scale of 1-5, wh

= ideal): - Importance: 5 - Usability: 4 - Scientific acceptability  Precise specificat Reliability: 5  Validity: 4  Adaptability: 4  Adequacy of risk a
E-82

Evidence Table 6 – Question 2b (Grey Literature): As appropriate to specific stages of colorectal cancer, what quality-of-care m

what evidence is available for measures of quality of care of treatment of colorectal cancer, including surgical therapy for colon and recta

Study Characteristics of Quality Measure Comments/Quality Scoring 

National Committee for Quality Assurance, 2005 #36580 

Quality measure (QM): Percent of rectal cancer cases that received a sphincter 

preservation procedure at time of surgery Basis of QM: Clinical practice guideline (National Comprehensive Cancer 

Network [NCCN] guidelines) Type of QM:(a) Process (b) Technical Outcome to which the QM is linked: improved QoL Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of patients with rectal cancer surgery over a 12-month period. Numerator: Number of above patients who received a sphincter preservation procedure at time of surgery. Data sources: Cancer registry, claims data Recommended frequency of data collection: No recommendation 

General comments:Not all data may be available. Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-83

Evidence Table 7 – Question 2c (Published Literature): As appropriate to specific stages of colorectal cancer, what quality-of-

available and what evidence is available for measures of quality of care of treatment of colorectal cancer, including appropriate use of ad

adjuvant radiation therapy, including for patients with metastatic but potentially curable (hepatic/pulmonary-resectable) disease?

Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Ayanian, Zaslavsky, Fuchs, et al., 2003 #34680 

Quality measure (QM): Use of adjuvant chemotherapy in patients with stage III colon cancer or stage II or III rectal cancer Basis of QM: Other (adherence to the recommendations of the 1990 NIH Consensus Conference) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Improved survival and decreased local/regional recurrence 

Intent of QM: Not specified (aim of the study was to assess the use of adjuvant chemotherapy and radiation therapy in patients with stage III colon cancer) Definition of denominator/numerator:

Denominator: Number of patients identified from registry records (3 regional registries within the California Cancer Registry) who were 18 years of age and older diagnosed during 1996 and 1997 with stage IIII adenocarcinoma of the colon or stage II or III adenocarcinoma of the rectum, using staging criteria of the AJCC. Patients had undergone surgery and survived for at least 30 days. Numerator: Number of above patients with colon or rectal cancer who received adjuvant chemotherapy. Data sources:California Cancer Registry Physician survey Office records Recommended frequency of data collection: No recommendation. 

Study population: Eligible for chemo: N: 1,956 Age: 18 to > 85 Race: 72.9% white Sex: 50.2% female Tumor stage: Colon stage III 72.7%, Rectal stage III 12.8%, Rectal stage II 14.5% Performance status: 60.7% comorbidity score 0, 20.1% 1, 19.2% ≥ 2 Geographic location: San Francisco/Oakland, San 

Jose/Monterey, and Sacramento, CA areas Dates: 1996-97 Healthcare setting: Hospitals Results:67% of eligible patients received adjuvant chemotherapy. Sensitivity of routinely collected registry data was 87% for chemotherapy. Only 2% of patients had registry data indicating they had received chemo but were reported by their physicians not to have received therapy. On MVA, older patients significantly less likely to receive chemo. In hierarchical models including random effects of individual hospitals, use of chemotherapy varied substantially. Relative to the 67% of patients receiving chemotherapy in the full cohort, the adjusted probability of getting chemo was 79% in hospitals, moderately (one SD) above average, and 51% in hospitals, moderately below average. Most common reasons cited for not giving adjuvant therapy: patient refusal, comorbid illness, lack of clinical indication. 

General comments:Article attempts to estimadjuvant therapies in rouby comparing registry daphysician surveys. Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 4  Validity: 4  Adaptability: 4  Adequacy of risk ad
E-84

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SBeart, Steele Jr., Menck, et al., 1995 #990 

Quality measure (QM): Percentage of patients with stage III colon cancer receiving postoperative chemotherapy Basis of QM: None mentioned; however, consistent with 1990 NIH consensus conference Type of QM:(a) Process (b) General Outcome to which the QM is linked: None specified, but presumably survival Intent of QM: Not specified (aim of the study was to identify current trends in the management of patients with carcinoma of the colon or rectum and to identify changes in patterns of care and survival) Definition of denominator/numerator:Denominator: Number of stage III colon cancer patients seen in 1988 from amongst cases of colon and rectal cancer (up to 25 per program or facility) reported by over 1200 approved cancer programs and 800 other facilities on the Commission on Cancer mailing list that were invited to participate. Numerator: Number of above patients receiving chemotherapy. Data sources: Hospitals sent in up to 25 consecutive cases from their medical records Recommended frequency of data collection: No recommendation 

Study population: N: 39,502 (29,209 with colon cancer; 3599 with stage III) Age: Median 72 years for colon cancer Race: For entire study: African American, 9.1% for colon cancer and 6.5% for rectal cancer; Hispanic, 2.7% for colon cancer and 3.5% for rectal cancer. 1.4% of cases were Asian. The rest were Caucasian.Sex: Male 49.3% (colon) and 57.2% (rectal) Tumor stage: All included (4.7% stage 0, 22.1% stage I, 29.2% stage II, 22.2% stage III, 18.7% stage IV, 3.0% unknown) Performance status: NR Geographic location: Entire U.S. 

Dates: Calendar years 1983 and 1988 Healthcare setting: Hospitals Results:

 Colon Cancer Rectal Cancer 

 1983 1988 1985 1988 Chemotherapy 10.7 11.1 5.9 5.6 Radiation therapy 2.7 10.9 15.3 13.8 Combined chemo and radiation therapy 

1.5 1.6 5.5 10.1 

General comments: Thgenerated prior to 1990 modern recommendationRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 3  Adequacy of risk ad
E-85

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SBeart, Steele Jr., Menck, et al., 1995 #990 

Quality measure (QM): Percentage of patients with stage II or III rectal cancer receiving chemoradiotherapy Basis of QM: None mentioned; however, consistent with 1990 NIH consensus conference Type of QM:(a) Process (b) General Outcome to which the QM is linked: None specified, but presumably survival Intent of QM: Not specified (aim of the study was to identify current trends in the management of patients with carcinoma of the colon or rectum and to identify changes in patterns of care and survival) Definition of denominator/numerator:Denominator: Number of stage II or III rectal cancer patients seen in 1988 from amongst cases of colon and rectal cancer (up to 25 per program or facility) reported by over 1200 approved cancer programs and 800 other facilities on the Commission on Cancer mailing list that were invited to participate. Numerator: Number of above patients receiving chemoradiotherapy. Data sources: Hospitals sent in up to 25 consecutive cases from their medical records Recommended frequency of data collection: No recommendation 

Study population: N: 39,502 (10,293 with rectal cancer; 2051 with stage II or III) Age: median 69.3 years for rectal cancer Race: For entire study: African American, 9.1% for colon cancer and 6.5% for rectal cancer; Hispanic 2.7% for colon cancer and 3.5% for rectal cancer. 1.4% of cases were Asian. The rest were Caucasian. Sex: Male 57.2% (rectal) Tumor stage: All included (4.7% stage 0, 22.1% stage I, 29.2% stage II, 22.2% stage III, 18.7% stage IV, 3.0% unknown) Performance status: NR Geographic location: Entire U.S. 

Dates: Calendar year 1983 and 1988 Healthcare setting: Hospitals Results:

 Colon Cancer Rectal Cancer 

 1983 1988 1985 1988 Chemotherapy 10.7 11.1 5.9 5.6 Radiation therapy 2.7 10.9 15.3 13.8 Combined chemo and radiation therapy 

1.5 1.6 5.5 10.1 

General comments: Thgenerated prior to 1990 modern recommendationRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 3  Adequacy of risk ad
E-86

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SCoburn, Pricolo, and Soderberg, 1994 #2920 

Quality measure (QM): Adjuvant therapy rates Basis of QM: None Type of QM:(a) Process (b) General Outcome to which the QM is linked: None Intent of QM: Not specified (aim of the study was to perform univariate and multivariate analyses of poor indicators in the elderly undergoing colorectal operation, and to compare clinical, pathologic, and therapeutic factors in patients younger than and older than 80 years of age) Definition of denominator/numerator:Denominator: Number of subjects who had received operative treatment for colon and rectal cancer between 1961 and 1987. Numerator: Number of above subjects who received adjuvant therapy (chemo- or radiotherapy). Comparisons were made between patients under and over 80 years. Data sources: Inpatient, outpatient, and office records Recommended frequency of data collection: No recommendation 

Study population: N: 800

< 80 

years 

> 80 

years 

N 177 623 Sex (%F/M) 48/52 56/44 Rectal lesions 36 23 Duke stage  A 9.0 6.0  B1 16.7 12.8  B2 31.2 42.7  C1 4.5 0  C2 21.5 21.8  D 17.1 16.7 

Geographic location: Rhode Island, U.S. Dates: 1961 and 1987 Healthcare setting: Academic institute Results:Adjuvant therapy rate was 5.1% for patients > 80 yrs and 20% for patients < 80 yrs. Long-term survival (at 5 and 8 yr) and rate of recurrence after curative therapy were used to validate the measure. There was a statistically significant difference in survival rates: 48% for patients < 80 vs. 32% for patients > 80. There was no statistically significant difference between the groups with regard to recurrence after curative therapy. 

General comments:Data collected between Other factors that affect more aggressive biologic colon and rectum in the disease and less tendendissemination, greater teright-sided, rarity of adjuolder patients, higher copatients due to comorbid frequency of obstructing Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 2  Adequacy of risk ad
E-87

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SCoia, Wizenberg, Hanlon, et al., 1994 #3140 

Quality measure (QM): Percentage of patients receiving adjuvant radiation therapy who also received adjuvant chemotherapy (either concurrent or 

non-concurrent) for cancer of the sigmoid colon or rectum Basis of QM: Other (1990 NIH consensus 

guidelines) Type of QM:(a) Process (b) General and Technical Outcome to which the QM is linked: Overall survival Intent of QM: Not specified (aim of the study was to determine the national practice standards for the evaluation and treatment of adenocarcinoma of the rectum and sigmoid colon) Definition of denominator/numerator:Denominator: Number of subjects identified from data source who were treated 1998-1999 with radiation therapy for rectal/ and sigmoid adenocarcinoma. Numerator: Number of above subjects receiving chemotherapy in 

addition to radiation therapy for T3N+ disease. 

Data sources: Random sample of all patients in the U.S. with rectal or sigmoid colon cancer identified in a stratified two-stage cluster sampling procedure. First stage: selection of facilities from all facilities in the U.S. based on census data from a facilities survey. Second stage: select patients from sampled facilities.Recommended frequency of data collection: No recommendation 

Study population: N: 408 Age: 64 (median) Race: NR Sex: 61% male Tumor stage: A-C3 (Modified Astler-Coller System); A 0.6%, B1 4.4%, B2 23.5%, B3 5.1%, C1 6.9%, C2 30.1%, C3 6.6%, unknown 22.8% Tumor location: Sigmoid 12.5%, rectum 82.8%, unknown 4.7% Performance status: 98% KPS = 80 or higher Geographic location: U.S. 

Dates: Patients treated 1988-89 Healthcare setting: 73 U.S. institutions Results:General results: Only 37% of patients received chemotherapy concurrent with radiation therapy. Only 55% of patients with disease through the bowel wall and/or involving lymph nodes received chemotherapy. Chemotherapy given to 44.3% of patients overall. Technical results: 95% of patients underwent simulation for radiation therapy. Port films were taken in 98% of patients. 

General comments:This paper has many poprimary QM, which is usradiotherapy. Other QMthose receiving > 40Gy rexperiencing prolonged radiotherapy, and percenscan as part of their evaA major limitation of this surveyed were for patienthe NIH consensus guidTherefore, the quality me

group which it may be inAnother limitation is that rectosigmoid cancers whThis included some patiewho, under usual circumreason to receive radiothbeen a reason not captupatients to receive radiotRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 4 - Scientific acceptability  Precise specificatio Reliability: 4  Validity: 4  Adaptability: 4  Adequacy of risk ad
E-88

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SDominitz, Samsa, Landsman, et al., 1998 #9380 

Quality measure (QM): Percentage of patients with colorectal cancer receiving chemotherapy Basis of QM: None Type of QM:(a) Process (b) General Outcome to which the QM is linked: Survival Intent of QM: Not specified (aim of the study was to assess the influence of race on the treatment and survival of patients with colorectal carcinoma) Definition of denominator/numerator:Denominator: Number of white/black patients with colorectal cancer identified from the VA Patient Treatment File (PTF) who were discharged from VA hospitals with a diagnosis of CRC (ICD-9-CM codes 153.0-153.4, 153.6-154.1) during fiscal year 1989. To exclude prevalent cases, any patient with a diagnosis of CRC (ICD-9-CM codes 153.0-153.4, 153.6-154.1, V10.05-V10.06 ) for hospitalization during FY 1984-1988, or a personal history of CRC (ICD-9-CM codes V10.05-V10.06) during the index hospitalization, was excluded. Male veterans only. No ulcerative colitis or Crohn’s disease. Numerator: Number of above white/black patients treated with chemotherapy, as indicated by specific ICD-9-CM codes in the PTF. Data sources: VA PTF Recommended frequency of data collection: No recommendation

Study population: N: 2,607 whites + 569 blacks Age: 67.1 ± 9.1 (whites); 66.4 ± 9.9 (blacks) Race: See above Sex: 100% male Tumor stage: All stages Performance status: NR Geographic location: VA Medical Centers Dates: 1989 Healthcare setting: VA Hospitals Results:No statistically significant differences in rates of chemotherapy in blacks vs. whites. No difference in 5-year relative survival or overall survival in blacks vs. whites. 

General comments:The article compares ratand chemotherapy amonattempt is made to compa guideline) to determine or whites are optimal. The study is limited becaadministrative database and did not allow for adjuRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 1  Adaptability: 1  Adequacy of risk ad
E-89

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SGovindarajan, Shah, Erkman, et al., 2003 #35310 

Quality measure (QM): Percentage of patients with stage II, III colorectal cancer receiving chemotherapy Basis of QM: None Type of QM:(a) Process (b) General Outcome to which the QM is linked: Not firmly linked to any outcome, but presumably to survival, quality of life, or rates of local recurrence Intent of QM: Not specified (aim of the study was to analyze disease stage, treatment received, and socioeconomic factors to better understand the factors influencing survival differences between African-Americans and Caucasians with colorectal cancer) Definition of denominator/numerator:Denominator: Number of African-Americans (AAs)/Caucasians, 

aged 18 older, with CRC treated between 1984 and 1997 at a state-funded University hospital located in Arkansas, identified through the hospital tumor registry using appropriate tumor registry codes. Numerator: Number of above AAs/Caucasians treated with chemotherapy. Data sources: Tumor registry at Univ. of Arkansas. 

Recommended frequency of data collection: No recommendation 

Study population: N: 617 All figures below for Caucasians/AAs Age: 60.0/61.5 Race: 427 Caucasians/190 AAs Sex: 47.1% male/39.5% male Tumor stage: Stages 0-IV Performance status: NR Other: - Not high school grad: 45%/60% - Low income: 45%/64% - High poverty: 41%/68% Geographic location: Arkansas Dates: 1984-1997 Healthcare setting: University hospital Results:Authors show that significantly higher percentages of Caucasians received chemotherapy, compared with AAs, for patients with all stages of CRC, and for patients with only stages II and III disease. Cancer-specific mortality was higher in AfricanAmericans than in Caucasians for all stages. 

General comments: NoRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 1  Adaptability: 1  Adequacy of risk ad
E-90

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SHyman, Labow, and Vermont Chapter of the American College of Surgeons, 2002 #18980

Quality measure (QM): Adjuvant therapy rate Basis of QM: NIH consensus conference guidelines Type of QM:(a) Process (b) General Outcome to which the QM is linked: Mortality? Intent of QM: Not specified (aim of the study was to assess the feasibility of performing a quality study of the surgical management of colorectal cancer using a voluntary registry) Definition of denominator/numerator:Denominator: Number of colorectal cancer patients reported to registry by 33 surgeons in Vermont Numerator: Number of patients offered adjuvant therapy Data sources: Prospective statewide voluntary registry created by the Vermont chapter of ACS with the Vermont Program for Quality Health Care Recommended frequency of data collection: No recommendation 

Study population: N: 364 (33 surgeons) Age: Mean 68.7 Stage of tumor: Stage I = 24%, stage II = 32%, stage II = 28%, stage IV = 16% Geographic location: Vermont 

Dates: April 1, 1999 to March 31, 2001 Healthcare setting: Mixed Results:All patients with stage III colon cancer and stages II or III rectal cancer were offered adjuvant therapy. 

General comments: Thwith an explicit intention among participating surgpotentially do so), but fobenchmarks. A voluntary registry was Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 2  Validity: 4  Adaptability: 3  Adequacy of risk ad
E-91

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SJessup, McGinnis, Steele Jr., et al., 1996 #35360 

Quality measure (QM): Percentage of patients with stage III colon cancer receiving adjuvant chemotherapy Basis of QM: Clinical practice guideline (NCI-NIH Consensus Development conference) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Survival Intent of QM: Not specified (aim of the study was to report time trends in stage of disease, treatment patterns, and survival for patients with selected cancers [including colon cancer]) Definition of denominator/numerator:Denominator: Number of Cases of stage III colon cancer reported to the National Cancer Database in 1988 and 1993 Numerator: Number of patients who received adjuvant chemotherapy Data sources: National Cancer Database Recommended frequency of data collection: No recommendation 

Study population: N: 9,509 with stage III colon cancer Age: All ages included Race: All races Sex: Both sexes Tumor stage: All stages Performance status: NR Geographic location: United States 

Dates: 1985-1993 Healthcare setting: Hospitals Results:43% of stage III patients received chemotherapy. Overall (not cancer-specific) survival was reported. Those stage II and III patients who received surgery and chemotherapy had a 49% survival rate compared to 44% for those who received surgery alone. 

General comments: NoRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 2  Adaptability: 4  Adequacy of risk ad
E-92

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SKeating, Landrum, Meara, et al., 2005 #36080 

Quality measure (QM): Rate of adjuvant chemotherapy for patients with stage III colon cancer Basis of QM: Clinical practice guideline (not specified, but likely 1990 NCI Consensus Statement) Type of QM:(a) Structure (b) General Outcome to which the QM is linked: Improved survival is implied Intent of QM: To assess quality of care (aim of the study was to examine associations between increases in managed care market share and changes in the quality of care delivered to cancer patients in the fee-for-service sector) Definition of denominator/numerator:Denominator: Number of patients with a first diagnosis of stage III CRC 1993-1999 in a given SEER county who had undergone surgery, were alive and enrolled in fee-forservice Medicare (parts A and B) through month 4. Numerator: Number of patients with stage III CRC in a given county who received adjuvant chemotherapy (The specific CPT, ICD-9, HCPCS, and Revenue Center codes used to identify chemotherapy administration are listed in the manuscript). Data sources:SEER –Medicare dataset 1990 U.S. Census Recommended frequency of data collection: No recommendation 

Study population: N: 48,027 Age: 66-85+ Race: 88 white Sex: 46% male Other: Income, % high school graduates in census 

tract of residence, comorbidity score, market share of managed care Geographic location: SEER geographic regions Dates: 1993-1999 Healthcare setting: Not stated Results:Increased market share of managed care in a given county resulted in no significant change in rates of adjuvant chemotherapy for stage III colon cancer. 

General comments: NoRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 4  Adequacy of risk ad
E-93

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SKeating, Landrum, Meara, et al., 2005 #36080 

Quality measure (QM): Rate of adjuvant chemotherapy and radiation therapy for patients with stage II or III rectal cancer Basis of QM: Clinical practice guideline (not specified, but likely 1990 NCI Consensus Statement) Type of QM:(a) Structure (b) General Outcome to which the QM is linked: Generally, improved survival. Intent of QM: To assess quality of care (aim of the study was to examine associations between increases in managed care market share and changes in the quality of care delivered to cancer patients in the fee-for-service sector) Definition of denominator/numerator:Denominator: Number of patients with a first diagnosis of stage II, III rectal cancer in 1993-1999 in a given SEER county who had undergone surgery, were alive and enrolled in fee-for-service Medicare (parts A and B) through month 4. Numerator: Number of patients with stage II, III rectal cancer in a given county who received adjuvant chemotherapy and radiation therapy. Data sources: SEER-Medicare dataset; 1990 U.S. Census Recommended frequency of data collection: No recommendation 

Study population: N: 48,027 Age: 66-85+ Race: 88% white Sex: 46% male Other: Income, % high school graduates in census 

tract of residence, comorbidity score, market share of managed care Geographic location: SEER geographic regions Dates: 1993-1999 Healthcare setting: Not stated Results:Increased market share of managed care in a given county resulted in significant decrease in % of patients with stage II or III rectal cancer receiving adjuvant chemotherapy and radiotherapy. 

General comments: NoRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 4  Adequacy of risk ad
E-94

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SMahoney, Kuo, Topilow, et al., 2000 #26120 

Quality measure (QM): Percentage of patients with stage III colon cancer receiving adjuvant chemotherapy Basis of QM: Clinical practice guideline (NCI-NIH Consensus Development Conference) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Survival Intent of QM: Not specified (aim of the study was to assess the use of adjuvant chemotherapy in an aging population from the time when the initial recommendation for chemotherapy was made) Definition of denominator/numerator:Denominator: Number of patients with stage III colon cancer in the Tumor Registry of Jersey Shore Medical Center. Numerator: Number of patients receiving postoperative chemotherapy. Data sources: Tumor registry Recommended frequency of data collection: No recommendation 

Study population: N: 69 Age: 40-97 Race: Not reported Sex: 28 men, 41 women Tumor stage: III Performance status: NR Geographic location: New Jersey 

Dates: January 1, 1989-December 30, 1996 Healthcare setting: Jersey Shore Medical Center Results:49% of patients received chemotherapy; men had a 5.8 times greater chance of receiving chemotherapy than women. 

General comments: NoRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 4  Validity: 2  Adaptability: 2  Adequacy of risk ad
E-95

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SNeugut, Fleischauer, Sundararajan, et al., 2002 #18450 

Quality measure (QM): Use of chemotherapy of combined chemoradiotherapy in stage II and III rectal cancer patients among the elderly Basis of QM: Clinical practice guideline (NIH and NCCN) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Survival Intent of QM: Not specified (aim of the study was to investigate the use of treatment with adjuvant 5-FUbased chemotherapy and radiation therapy among patients over 65 years of age with surgically resected stage II or III rectal cancer) Definition of denominator/numerator:Denominator: Number of patients with histologically confirmed primary rectal cancer, diagnosed 1992-1996, stage IIIII, age ≥ 65, s/p surgical tumor resection, eligible for Medicare Parts A and B in the 12 mo before diagnosis and 120 days after diagnosis, and had survived more than 4 mo after diagnosis. Excluded members of HMO in the 12 mo before and 4 mo after their diagnosis. Numerator: Number of patients who received both chemotherapy and radiotherapy. Data sources: SEER-Medicare database Recommended frequency of data collection: No recommendation 

Study population: N: 1,807 Age: 65-85+ Race: 1592 non-Hispanic white Sex: 964 female Tumor stage: II (983). III (824) Performance status: NR Charlson-Deyo comorbidity score: 0 (1098), 1 (299), > 1 (410) Geographic location: SEER geographic regions 

Dates: 1992-96 Healthcare setting: U.S. facilities Results:51% received adjuvant chemotherapy and 48% received radiotherapy overall. 38% received surgery alone; 11% received surgery plus radiation therapy, 14% received surgery plus adjuvant chemotherapy and 37% received surgery with radiation plus adjuvant chemotherapy. Patients with stage III disease were significantly more likely than patients with stage II disease to receive combined 5-FU chemotherapy and radiotherapy. Age inversely associates with receiving combined chemo-radiotherapy. Among cases with stage III cancer, number of nodes and comorbidity score were also significant predictors of receiving combined therapy. 

General comments:Impossible from this datafrequency of adjuvant chuse was “optimal” in the example, omission of chpatient with stage III rectsound decision if the pat

comorbidities. Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio

 Reliability: 3  Validity: 3  Adaptability: 4  Adequacy of risk ad
E-96

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SNewcomb and Carbone, 1993 #3760 

Quality measure (QM): Rate of adjuvant chemotherapy for colorectal cancer Basis of QM: None mentioned, but presumably practice guideline Type of QM:(a) Process (b) General Outcome to which the QM is linked: Not specified, but presumably survival Intent of QM: Not specified (aim of the study was to evaluate the selection of cancer treatment among the elderly) Definition of denominator/numerator:Denominator: Number of women residents of Wisconsin aged 20-74 at the time of diagnosis with a new diagnosis of carcinoma of the large bowel during 1989-1991, and who had a listed telephone number and spoke English, and whose physician consented to their participation, and who agreed to a telephone interview, and were among the 628 consecutive participants interviewed September 1, 1990 through November 30, 1990. Numerator: Number of subjects receiving chemotherapy. Data sources: Registry for staging information and patient interviews (for the treatment information) Recommended frequency of data collection: No recommendation 

Study population: N: 121 with colorectal Age: 20-74 Race: NR Sex: Female Tumor stage: All stages Performance status: NR Geographic location: Wisconsin 

Dates: Sep 1 through Nov 30, 1990 Healthcare setting: None (all participants were outpatients contacted by phone) Results: 41% of patients less than 65 and 26% ≥ 65 received 

chemotherapy. Older women were less likely to be referred to medical or radiation oncologists; they were also less likely to be presented with more than one treatment option. 

General comments:

Relying on recall of patiea treatment is open to biFor some of the patientshave been appropriate, abe included in the denommay not be conclusive. Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 3 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 3  Adequacy of risk ad
E-97

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SOliveria, Yood, Campbell, et al., 2004 #29680 

Quality measure (QM): Referral to medical oncologist for consideration of adjuvant chemotherapy Basis of QM: Clinical practice guideline (1990 NCI guidelines for stage III and IV disease [refs 9, 18, 19])Type of QM:(a) Process (b) General Outcome to which the QM is linked: Survival, quality of life Intent of QM: Not specified (aim of the study was to describe treatment patterns for patients with colorectal cancer and to examine reasons why patients do not receive chemotherapy) Definition of denominator/numerator:Denominator: Number of members of a Massachusetts HMO who were newly diagnosed with CRC from 1/1/97 to 6/30/99 based on ICD-9 codes for CRC and related CPT-4 procedure codes for resection from the HMO’s computerized research database. Numerator: Number of patients above who were referred to a medical oncologist. Data sources: Computerized database of Massachusetts HMO Recommended frequency of data collection: No recommendation 

Study population: N: 217 Age: Median age 72.0 years; range, 38 to > 80 Race: NR Sex: 48% male Tumor stage: I-IV Performance status: NR Geographic location: Massachusetts 

Dates: 1997-99 Healthcare setting: HMO Results:66% of patients had a referral to an oncologist or evidence of chemotherapy within 4 months of the index date. Stratified by stage at diagnosis, authors report percentage of patients who were referred to a medical oncologist and whether they subsequently received chemotherapy. Among patients who did not get referred to a medical oncologist, authors report the reasons for no referral (patient refusal, not recommended by MD, comorbidities/death, or unknown). Multivariable analysis revealed patient age (< 70 vs. ≥ 70) and tumor stage (III vs. I) to be significant predictors of referral. 

General comments:Authors conclude that bofactors influence the rate oncologist. Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 4  Validity: 4  Adaptability: 3  Adequacy of risk ad
E-98

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SPotosky, Harlan, Kaplan, et al., 2002 #19420 

Quality measure (QM): Percentage of patients with 

stage II, III rectal cancer receiving adjuvant chemotherapy (with or without radiotherapy) Basis of QM: Clinical practice guideline (NCI-NIH 

Consensus Development conference) Type of QM: (a) Process (b) General Outcome to which the QM is linked: Survival Intent of QM: Not specified (aim of the study was to assess the dissemination of adjuvant therapies for stages II and III colorectal cancer, and multiple clinical and non-clinical characteristics associated with the use of minimally acceptable adjuvant therapy as a standard of care) Definition of denominator/numerator: Denominator:Number of patients with stage II, III rectal cancer diagnosed in 1990-1991 or 1995 for whom the offer of the receipt of chemotherapy was known. Numerator: Number of patients receiving postoperative chemotherapy with or without radiotherapy. Data sources: SEER program

Recommended frequency of data collection: No 

recommendation 

Study population: N: 1,366 stage II, III rectal cancer Age: All Race: 86% white, 7% African American, 5% HispanicSex: 59% male Tumor stage: II, III Performance status: NR Geographic location: Several cities across U.S. 

Dates: 1990-1991 or 1995 Healthcare setting: Hospitals

Results:Because the data are presented by year and by other demographics, it is difficult to determine the overall % of patients who received chemotherapy.

General comments: NoRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 4  Adequacy of risk ad
E-99

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SPotosky, Harlan, Kaplan, et al., 2002 #19420 

Quality measure (QM): Percentage of patients with stage III colon cancer receiving adjuvant chemotherapy Basis of QM: Clinical practice guideline (NCI-NIH Consensus Development conference) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Survival Intent of QM: Not specified (aim of the study was to assess the dissemination of adjuvant therapies for stages II and III colorectal cancer, and multiple clinical and non-clinical characteristics associated with the use of minimally acceptable adjuvant therapy as a standard of care) Definition of denominator/numerator:Denominator: Number of patients with stage III colon cancer diagnosed in 1990-1991 or 1995 for whom the offer of the receipt of chemotherapy was known. Numerator: Number of patients receiving postoperative chemotherapy. Data sources: SEER program Recommended frequency of data collection: No recommendation 

Study population: N: 779 stage III colon cancer Age: All Race: 85% white, 9% African American, 4% hispanicSex: 47% male Tumor stage: Stage III Performance status: NR Geographic location: Several cities across U.S. 

Dates: 1990-1991 or 1995 Healthcare setting: Hospitals Results:Because the data are presented by year and by other demographics, it is difficult to determine the overall % of patients who received chemotherapy. 

General comments: NoRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 4  Adequacy of risk ad
E-100

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SRetchin and Brown, 1990 #7970 

Quality measure (QM): Palliative chemotherapy 

rates Basis of QM: Clinical practice guideline (not cited, 

but likely standard of care) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Appropriate pre- and postoperative care Intent of QM: Quality improvement (aim of the study was to compare the processes of care for colorectal cancer patients in HMOs with those in fee-for-service [FFS] settings) Definition of denominator/numerator:Denominator: Number of Medicare patients enrolled in HMO or FFS health plans who were hospitalized for colorectal cancer (based on discharge diagnosis or equivalent symptoms, e.g., cancer or mass or tumor of the rectum, sigmoid, cecum or colon; admission for colectomy, hemicolectomy, polypectomy; rectal or gastrointestinal bleeding and abdominal pain, vomiting, obstruction, dehydration, obstipation or severe constipation). FFS patients included in the study were hospitalized and had discharge diagnoses including malignant neoplasms of the colon, rectum, rectosigmoid junction or anus. Patients with carcinoma in situ of either the colon or rectum were also eligible. FFS patients were selected according to number of patients discharged from 7/83 to 3/31/86. HMO patients with an enrollment date of (1/83 to 5/84) to 3/31/86 were included. Exclusion criteria: Individuals enrolled in HMOs with fewer than 2000 enrollees were eliminated. Numerator: Number of patients with advanced disease who received palliative chemotherapy. 

Study population: 

 FFS n = 180 HMO n = 150 

Mean age 70.1 72.3 % male 49.4 58.7 % white 86.7 95.2 

Geographic location: U.S. HMOs were geographically represented Dates: 1983-1986 Healthcare setting: Inpatient and outpatient Results: 36% of HMO patients with advanced disease received palliative chemotherapy compared to 18% of FFS patients. These differences were not statistically significant. 

General comments:Confounding variables wpatients may have been could be another possibPre- or postoperative shRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 1  Adequacy of risk ad
E-101

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Data sources: Medical records, Medicare claims from Medicare Automated Data Retrieval System Recommended frequency of data collection: No recommendation 

Roetzheim, Pal, Gonzalez, et al., 2000 #23910 

Quality measure (QM): Percentage of patients with a colon or rectal cancer diagnosis who received chemotherapy Basis of QM: None stated (see General comments) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Mortality Intent of QM: Not specified (aim of the study was to explore the influence of race/ethnicity and insurance payer on the treatment of and outcomes for colorectal cancer patients in Florida) Definition of denominator/numerator:Denominator: Number of incident cases of colon or rectal cancer occurring in Florida in 1994 as listed in the state tumor registry. Numerator: Number of the above patients who also received chemotherapy within 4 months of initiation of therapy.Data sources: Florida state cancer registry (Florida Cancer Data System = FCDS); Florida Agency for Health Care Administration AHCA discharge abstracts (admissions to all nonfederal acute care hospitals and patient visits to ambulatory surgical centers, freestanding radiation therapy centers and diagnostic imaging centers) Recommended frequency of data collection: No recommendation 

Study population: N: 9,548 

Patient characteristic Percent 

Male 51 Race  White 85  Black 6  Hispanic 8 Diagnosis stage  In situ 6  Local 30  Regional 42  Distant 16  Unstaged 6 Anatomic site  Colon 84  Rectal 16 Comorbidity index  0 71  1 21 

≥ 2 8 

Geographic location: Florida Dates: 1994 Healthcare setting: Multiple Results:21% of the patients in the study received chemotherapy, but no validity testing was done for chemotherapy. Hispanics were less likely than non-Hispanic whites toreceive chemotherapy among non-Medicare patients. Persons with commercial HMO insurance were less 

General comments:Existing guidelines are fostage II or III rectal cancreporting data on all checolon or rectal cancer is driven. Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 4 - Scientific acceptability  Precise specificatio Reliability: 2  Validity: 2  Adaptability: 2  Adequacy of risk ad
E-102

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

likely than those with commercial FFS insurance to receive chemotherapy. Other factors predictive of receiving chemotherapy were younger age, higher education, being married, having rectal cancer, 

advanced tumor stage, and lower level of comorbid illness. Among non-Medicare patients, mortality rates were higher in all models for patients with commercial HMOs compared with patients with commercial FFS insurance. 

Rogers, Ray, and Smalley, 2004 #30130 

Quality measure (QM): Percentage of patients with a diagnosis of stage III colorectal cancer receiving chemotherapy Basis of QM: Not stated Type of QM:(a) Process (b) General Outcome to which the QM is linked: Not stated, but presumably survival or recurrence of CRC cancer Intent of QM: Not specified (aim of the study was to examine the effect of race on CRC outcomes in patients who had identical health care coverage) Definition of denominator/numerator:Denominator: Number of patients enrolled in both Tennessee Medicaid and Medicare and hospitalized with a diagnosis of colorectal cancer. Numerator: Number of above patients who received chemotherapy within 120 days of diagnosis. Data sources: Medical records, administrative files from the Tennessee Medicaid program, Tennessee death certificates, and the National Death Index Recommended frequency of data collection: No recommendation

Study population: N: 969 total Age: 78.9 ± 7.5 years Race: 272 black (28%), 697 white (72%) Sex: 73.9% female (whites), 72.4% female (Blacks) Tumor stage: 75.6% (whites) and 72.1% (blacks) had stage III disease Performance status: Charlson Deyo score of 0 in 64% (whites) and 69% in Blacks Geographic location: Tennessee 

Dates: 1984-1994 Healthcare setting: Patients identified from hospitalizations Results:7.8% of whites and 9.4% of blacks had chemotherapy within 120 days of diagnosis. There was no difference in survival between whites and blacks. 

General comments:Study concluded that whcare, outcomes were eqRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 4  Validity: 4  Adaptability: 3  Adequacy of risk ad
E-103

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SSchrag, Cramer, Bach, et al., 2000 #23710 

Quality measure (QM): Postoperative 

chemotherapy rates Basis of QM: Not stated 

Type of QM:(a) Process (b) General Outcome to which the QM is linked: Mortality Intent of QM: Quality improvement (aim of the study was to determine whether hospital procedure volume predicts short- and long-term survival following primary colon cancer surgery) Definition of denominator/numerator:Denominator: Number of patients with stage III tumors who survived 3 months postoperatively, were enrolled in Medicare Part B, were 65 years and older, were diagnosed as having primary adenocarcinoma of the colon in a SEER area, and underwent surgery followed by at least one claim for chemotherapy. Exclusion criteria: Patients were excluded if diagnoses were exclusively noted on death certificates or at autopsy; month of diagnosis was unknown; or patient was enrolled in an HMO. Also, patients who were operated on exclusively for local resection or creation of an ostomy were excluded. Numerator: Number of patients who received postoperative chemotherapy within 3 months of surgery. Data sources: SEER for patients 65 and older Medicare Claims Part B Recommended frequency of data collection: No recommendation 

Study population: N: 6,423 Geographic location: U.S. 

Dates: 1991-1996 Healthcare setting: All U.S. Results:Overall postoperative chemotherapy rate within 3 months of surgery was 54.8%. Differences were marginally significant for hospitals with low (51.2%), medium (56.6%), high (55.6%), and very high (55.5%) volume. The difference in 5-year mortality for patients operated on at very high- vs. low-volume hospitals was 4.4%. 

General comments: NoRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 3  Adequacy of risk ad
E-104

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SSchrag, Cramer, Bach, et al., 2001 #21910 

Quality measure (QM): Percentage of patients with stage III colon cancer receiving adjuvant chemotherapy Basis of QM: Clinical practice guideline (NCI-NIH Consensus Development Conference) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Survival Intent of QM: Not specified (aim of the study was to assess the extent to which adjuvant therapy is used among the elderly) Definition of denominator/numerator:Denominator:Number of patients aged 65 years and older enrolled in Medicare A and B, diagnosed with primary colon adenocarcinoma stage III, in a SEER area during the years 1991-1996 and not part of an HMO, and who had colon cancer surgery performed within 3 months of primary diagnosis. Numerator: Number of patients who received radiotherapy (and chemotherapy) within 6 months of surgery. Data sources: Linkage of SEER population-based cancer registries with Medicare database of HCFA Recommended frequency of data collection: No recommendation 

Study population: N: 6,262 patients with stage III colon cancer Age: > 65 Race: All races Sex: Both sexes Tumor stage: Stage III Performance status: NR Geographic location: 5 states and 6 U.S. 

metropolitan areas Dates: 1991-1996 Healthcare setting: Hospitals Results:55% of stage III patients received chemotherapy. Age at diagnosis was highly associated with receipt of adjuvant treatment. 

General comments: NoRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 4  Adequacy of risk ad
E-105

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SSchrag, Gelfand, Bach, et al., 2001 #21280 

Quality measure (QM): Percentage of patients with stage II or III rectal cancer receiving chemoradiotherapy Basis of QM: NIH consensus statement Type of QM:(a) Process (b) General Outcome to which the QM is linked: None is specified, but presumably it would be pelvic recurrence Intent of QM: Not specified (aim of the study was to examine the relationship between patient characteristics and the use of adjuvant radiotherapy with and without chemotherapy among patients 65 years of age and older with stage II and II rectal cancer) Definition of denominator/numerator:Denominator:Number of patients aged 65 years and older enrolled in Medicare A and B, diagnosed with primary rectal adenocarcinoma stage II or III, in a SEER area during the years 1991-1996 and not part of an HMO, and who had rectal cancer surgery performed within 6 months of primary diagnosis and who survived more than 6 months and who did not have a secondary cancer within 6 months. Numerator: Number of patients who received radiotherapy (and chemotherapy) within 6 months of surgery. Data sources: Linkage of SEER population-based cancer registries with Medicare database of HCFA Recommended frequency of data collection: No recommendation 

Study population: N: 1,411 Age: > 65 Race: 88% white, 4% black, 8% other Sex: Male 54% Tumor stage: Stage II, III Performance status: NR Geographic location: 5 states and 6 U.S. 

metropolitan areas Dates: 1992-1996 Healthcare setting: Hospitals Results:57% of patients received radiotherapy (42% with chemotherapy and 15% radiation alone). 

General comments:The data for this study wso it may not reflect morfor management. Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 4  Adequacy of risk ad
E-106

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SSchrag, Panageas, Riedel, et al., 2003 #34030 

Quality measure (QM): Adjuvant chemotherapy rateBasis of QM: Clinical practice guideline (not specified) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Mortality Intent of QM: Quality improvement (aim of the study was to compare surgeon and hospital procedure volume as predictors of outcomes following colon cancer resection) Definition of denominator/numerator:Denominator: Number of Medicare eligible patients aged 65 or older diagnosed in SEER regions with primary colon cancer in 1991-1996. Exclusion criteria: Excluded were patients enrolled in an HMO; operated on exclusively for intestinal bypass; diagnosis noted exclusively on death certificate or at autopsy; month of death was unknown. Numerator: Number of patients who received adjuvant chemotherapy. Adjustment was made for sex, race, age, stage of tumor, comorbidity and socioeconomic status, whether hospitalization was emergent and whether obstruction or perforation was present. Data sources: SEER, Medicare claims from HCFA Recommended frequency of data collection: No recommendation 

Study population: N = 24,166 

Characteristic Percent 

Male/female 44.1/55.9 Age  65-69 18.6  70-74 23.6  75-79 23.6  80+ 34.2 Race  White 87.1  Black 6.6  other 6.3 AJCC stage I 19.9  II 36.9  III 26.1  IV 14.1  Unstaged 3.0 Romano comorbidity  0 71.3  1 23.2  2 5.6 Obstruction (Y/N) 9.0/91.0 Emergent hospitalization (Y/N) 19.8/80.2 

Geographic location: U.S. Dates: 1991-1996 Healthcare setting: All Results:Adjuvant chemotherapy rate differences were statistically insignificant. (continued on next page) 

General comments:2-year postoperative moassessed. Study claims that it cannof care that can account rates in the various hospRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 4  Adequacy of risk ad
E-107

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Hospital procedure volume 

Adjuvant chemo% 

 Very low 51  Low 56  Medium 57  High 56  P value NS Surgeon procedure volume  Very low 52  Low 55  Medium 54  High 56  P value NS 

After adjusting for surgeon procedure volume, high hospital procedure volume remained a strong predictor of low postoperative mortality rates. 
E-108

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SSteele, 1994 #35840 

Quality measure: Percentage of stage III colon cancer patients receiving surgery and chemotherapy Basis of QM: Clinical practice guideline It is implicit that the AHCC guidelines be followed but these are not actually cited in the article. There are also guidelines for chemotherapy in stage III colon cancer and chemo/radiation therapy for stage II and III rectal cancer (NCC), but these are not cited by the authors. Type of QM:(a) Process (b) General Outcome to which the QM is linked: No outcome is exactly linked to the above but is said to be a marker of appropriate cancer diagnosis and treatment. No citation is provided for this statement. Intent of QM: Not specified (aim of the study was to assess cancer patient care and outcomes on a national basis) Definition of denominator/numerator:Denominator: Number of patients in the National Cancer Data Base (NCDB) with a new diagnosis of colon cancer in the years 1985-1986 or 1991. Numerator: Number of the above with stage III cancer who underwent surgery and received chemotherapy. Data sources: National Cancer Data Base (NCDB)Recommended frequency of data collection: No recommendation 

Study population: Data given as patients (year) N: 40,384 (85/86); 39,751 (91) Age: Median 71.3 (85/85); 72.0 (91) Race:  Non-Hispanic whites 86.9% (85/85); 85.9% (91)  Hispanic 1.1/1.8  African-American 6.6/8.1  Asian 0.9/1.5  Unknown 4.5/2.7 Sex: Male 48.7% (85/85), 49.6% (91) Tumor stage: 85-86/91  0 3.7/6.5  1 14.4/19.3  2 13.8/26.2  3 12.9/20.3  4 9.2/15.8  Unknown 46.0/11.9 Performance status: NR Geographic location: National sample; note that the 

sample is not probabilistic and represented different hospitals in different years Dates: 1991 and outcomes for 1985-86 Healthcare setting: 464 hospital in 1985, 474 hospitals in 1986, 937 hospitals in 1991 Results: Treatment trends showed effects of trial data, as a greater percentage of patients were reported as having treatment (including chemotherapy) beyond surgery in 1991 as compared to 1985/86. 

General comments:Some “treatment” data anot stratified by stage or and therefore do not procount as a QM. Only suQM from this paper. Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 4 - Usability: 4 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 3  Adequacy of risk ad
E-109

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SSteele Jr., 1994 #35390 

Quality measure (QM): Percentage of patients with stage III colon cancer receiving postoperative chemotherapy Basis of QM: None Type of QM:(a) Process (b) General Outcome to which the QM is linked: None specified, but presumably it would be survival Intent of QM: Not specified (aim of the study was assess cancer patient care and outcomes on a national basis) Definition of denominator/numerator:Denominator: Number of stage III colon cancer patients in 1990 reported to the National Cancer Data Base (NCDB). Numerator: Number of above patients receiving chemotherapy (with or without radiation). Data sources: Registry information reported to National Cancer Data Base Recommended frequency of data collection: No recommendation 

Study population: N: 6,299 patients Age: All Race: All Performance status: NR Geographic location: Entire U.S. 

Dates: 1985, 1988, 1990 Healthcare setting: Hospitals Results:38.9% received chemotherapy and surgery (with or without radiation therapy). 

General comments:This paper addresses si(1995; #990). Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 3  Adequacy of risk ad
E-110

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SSteele Jr., 1994 #35390 

Quality measure (QM): Percentage of patients with stage II or III rectal cancer receiving chemoradiotherapy Basis of QM: None Type of QM:(a) Process (b) General Outcome to which the QM is linked: None specified, but presumably it would be survival Intent of QM: Not specified (aim of the study was assess cancer patient care and outcomes on a national basis) Definition of denominator/numerator:Denominator: Number of patients with stage II, III rectal cancer in 1990 reported to the National Cancer Data Base (NCDB). Numerator: Number of above patients receiving chemoradiotherapy. Data sources: Registry information reported to National Cancer Data Base (NCDB) Recommended frequency of data collection: No recommendation 

Study population: N: 5,495 Age: Median 69.3 years for rectal cancer Race: For entire study: African American, 9.1% for colon cancer and 6.5% for rectal cancer; Hispanic, 2.7% for colon cancer and 3.5% for rectal cancer. 1.4% of cases were Asian. The rest were Caucasian. Sex: Male 57.2% (rectal). Tumor stage: All included (4.7% stage 0, 22.1% stage I, 29.2% stage II, 22.2% stage III, 18.7% stage IV, 3.0% unknown Performance status: NR Geographic location: Entire U.S. 

Dates: Calendar years 1983 and 1988 Healthcare setting: Hospitals Results:Not specifically stated, but can be calculated form Table 19. 43% of stage II or III patients received surgery and radiation (with or without chemotherapy). 

General comments:This paper addresses si(1995; #990). Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 3  Adequacy of risk ad
E-111

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SSteele and Jessup, 1995 #35850 

Quality measure (QM): Percentage of patients with stage III colon cancer receiving postoperative chemotherapy Basis of QM: None Type of QM:(a) Process (b) General Outcome to which the QM is linked: None specified, but presumably it would be survival Intent of QM: Not specified (aim of the study was assess cancer patient care and outcomes on a national basis) Definition of denominator/numerator:Denominator: Number of stage III colon cancer patients in 1992 reported to the National Cancer Data Base (NCDB). Numerator: Number of above patients receiving chemotherapy (with or without radiation). Data sources: Registry information reported to National Cancer Data Base (NCDB) Recommended frequency of data collection: No recommendation 

Study population: N: 9,465 Age: All Race: All Performance status: Not stated Geographic location: Entire U.S. 

Dates: Calendar year 1986/7 and 1992 Healthcare setting: Hospitals Results:44.6% received chemotherapy and surgery (with or without radiation therapy). 

General comments:This paper addresses si(1995; #990), and Steele that it includes data from have data ending in 198Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 3  Adequacy of risk ad
E-112

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SSteele and Jessup, 1995 #35850 

Quality measure (QM): Percentage of patients with stage II or III rectal cancer receiving chemoradiotherapy Basis of QM: None Type of QM:(a) Process (b) General Outcome to which the QM is linked: None specified, but presumably it would be survival Intent of QM: Not specified (aim of the study was assess cancer patient care and outcomes on a national basis) Definition of denominator/numerator:Denominator: Number of stage II, III rectal cancer patients in 1992 reported to the National Cancer Data Base (NCDB). Numerator: Number of above patients receiving chemoradiotherapy. Data sources: Registry information reported to National Cancer Data Base (NCDB) Recommended frequency of data collection: No recommendation 

Study population: N: 9,465 Age: All Race: All Performance status: Not stated Other: Not stated Geographic location: Entire U.S. 

Dates: Calendar year 1986/7 and 1992 Healthcare setting: Hospitals Results:Not specifically stated, but can be calculated form Tables 5-13 that 46% of stage II or III patients received surgery and radiation (with or without chemotherapy). 

General comments:This paper addresses si(1995; #990), and Steele that it includes data from have data ending in 198Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 2  Validity: 3  Adaptability: 5  Adequacy of risk ad
E-113

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SSundararajan, Grann, Jacobson, et al., 2001 #21760 

Quality measure (QM): Percentage of patients with stage III colon cancer receiving adjuvant chemotherapy Basis of QM: Clinical practice guideline (NCI-NIH Consensus Development Conference) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Survival Intent of QM: Not specified (aim of the study was to assess the distribution of adjuvant chemotherapy in the elderly) Definition of denominator/numerator:Denominator: Number of patients aged 65 years and older enrolled in Medicare A and B, diagnosed with primary colon adenocarcinoma stage III, in a SEER area during the years 1992-1996 and not part of an HMO, and who had colon cancer surgery performed within 120 days of primary diagnosis. Numerator: Number of patients who received chemotherapy within 4 months of surgery. Data sources: Linkage of SEER population-based cancer registries with Medicare database of HCFA Recommended frequency of data collection: No recommendation 

Study population: N: 4,998 patients with stage III colon cancer Age: > 65 Race: All races Sex: Both sexes Tumor stage: Stage III Performance status: NR Geographic location: 5 states and 6 U.S. 

metropolitan areas Dates: 1992-1996 Healthcare setting: Hospitals Results:50% of stage III patients received chemotherapy. 

General comments: ThSchrag et al. (2001; #21from 1991-1990. Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 4  Adequacy of risk ad
E-114

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality STemple, Hsieh, Wong, et al., 2004 #29890 

Quality measure (QM): Chemotherapy rate for stage IV colon or rectal cancer Basis of QM: Clinical practice guideline (National Comprehensive Cancer Network Guideline) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Practice patterns of patients with stage IV cancer Intent of QM: Quality improvement (aim of the study was to evaluate surgical practice patterns for patients 65 years of age and older with stage IV colorectal cancer) Definition of denominator/numerator:Denominator: Number of Medicare-enrolled patients aged 65 and older, initially diagnosed with stage IV colon or rectal cancer in a SEER area during the years 1991-1999. Age, sex, race, ethnicity, comorbidity, year of diagnosis, and socioeconomic status were collected and data were risk-adjusted. Exclusion criteria: Cohort restricted to those with a histologic diagnosis consistent with adenocarcinoma. Diagnoses exclusively on death certificates or at autopsy were excluded, as were those in which the month of diagnosis was unknown. Patients enrolled in HMOs were excluded. Numerator: Number of above patients who received chemotherapy within 4 months of diagnosis. Data sources: SEER, Medicare claims Recommended frequency of data collection: No recommendation 

Study population: N: 9,011 Age: 21% 65-69; 25% 70-74; 23% 75-79; 18% 80-84; 13% ≥ 85 Sex: 48% male, 52% female Race: 84% white, 9% black, 7% other Comorbidities: 90% had 0; 5% had 1; 5% had 2+Geographic location: U.S. Dates: 1991-1999 Healthcare setting: inpatient hospital Results: Only 44% of the cohort received chemotherapy within 4 months of diagnosis. Palliative radiotherapy was administered to only 12% of cohort members within the first 4 months of treatment; these were primarily patients with rectal cancer. 

General comments: No

Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 2  Adequacy of risk ad
E-115

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality STropman, Hatzell, Paskett, et al., 1999 #35610 

Quality measure (QM): Percentage of patients receiving appropriate adjuvant chemotherapy for CRC as defined by the NCI PDQ guidelines Basis of QM: Clinical practice guideline (NCI PDQ Guidelines) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Locoregional recurrence, overall survival. Intent of QM: Not specified (aim of the study was to determine the degree to which colon cancer treatment in rural North and South Carolina conformed to national treatment recommendations) Definition of denominator/numerator:Denominator: Number of patients treated for colon cancer in Wayne County, NC, and its six contiguous counties or in Greenwood County, SC, and its four adjacent counties during the years 1991 and 1996. Cases were identified by the North Carolina Central Cancer Registry and by local tumor registrars in SC. Numerator: Number of patients above who received primary and adjuvant therapy per the NCI PDQ recommendations.Data sources: Patient medical records and records from physician offices in the geographic regions under study Recommended frequency of data collection: No recommendation

Study population: N: 230 1991/1996 data Age: Mean 75/71 Race: 76% white/75% white Sex: 55% male/58% male Tumor stage: I-III Performance status: NR Geographic location: Wayne County, NC, and its six contiguous counties or in Greenwood County, SC, and its four adjacent counties Dates: 1991, 1996 Healthcare setting: Rural health care facilities Results:A higher % of patients with stage III cancers received adjuvant chemotherapy compared to patients with stage II disease, a finding in keeping with the stronger PDQ support for adjuvant therapy for stage III disease. 44% of patients in NC with stage III colon cancer received chemotherapy in 1996 compared with 16% of patients with stage II disease. 

General comments:The article compares ratPDQ recommendations regions (one in NC, one after (1996) an interventphysicians. The intervenSC region served as a cRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 4  Validity: 4  Adaptability: 3  Adequacy of risk ad
E-116

Evidence Table 7 – Question 2c (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SWudel Jr., Chapman, Shyr, et al., 2002 #18640 

Quality measure (QM): Percentage of patients with colon or rectal cancer who had chemotherapy Basis of QM: None stated Type of QM:(a) Process (b) General Outcome to which the QM is linked: Mortality Intent of QM: Not specified (aim of the study was to determine the outcome of patients with colorectal cancer treated in the same city at 2 nearby medical centers, a university hospital and a city hospital, and to explore disparities in colorectal cancer outcomes between black and white patients) Definition of denominator/numerator:Denominator: Number of patients with a diagnosis of primary colorectal cancer between 1/1/90 and 12/21/99 in the tumor registry databases at each of 2 institutions (university hospital and a city hospital). Excluded patients who were non-white and non-black. Numerator: Number of the above patients who received chemotherapy. Data sources: Hospital tumor registries Recommended frequency of data collection: No recommendation 

Study population (university hospital = UH, city hospital = CH) N: 565 UH; 77 CH Age: Median 69 UH, 74 CH Race: 10.6% black UH; 53% black CH Sex: 53.3% male UH; 62.7% male CH 

Tumor Stage UH CH 

I 31.7 29.9 II 21.9 20.8 III 23.7 16.9 IV 22.7 32.5 

Performance status: NR Geographic location: Nashville, TN Dates: 1/1/90 and 12/21/99 (retrospective review) Healthcare setting: University hospital, city hospital Results:There was no significant difference in tumor stage at the time of presentation in the two institutions studied or between racial/ethnic groups. There was no difference in the treatment modalities identified between institutions or racial/ethnic groups. There was a significant difference in overall survival between patients treated at the university hospital (median 5.3 years) and those treated at the city hospital (median 2.1 years). Median survival was 5.7 years for white patients and 3.2 years for black patients treated at the university hospital. White patients treated at the city hospital survived 2.1 years vs. 1.4 years for black patients. There was no survival advantage for patients receiving chemotherapy or radiotherapy. 

General comments:Mortality was the primary the data for other potent“curative” surgery, chemmuch weaker, poorly defguidelines. Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 3 - Scientific acceptability  Precise specificatio Reliability: 2  Validity: 2  Adaptability: 3  Adequacy of risk ad
E-117

Evidence Table 8 – Question 2c (Grey Literature): As appropriate to specific stages of colorectal cancer, what quality-of-care m

what evidence is available for measures of quality of care of treatment of colorectal cancer, including appropriate use of adjuvant chemoradiation therapy, including for patients with metastatic but potentially curable (hepatic/pulmonary-resectable) disease? 

Study Characteristics of Quality Measure Comments/Quality Scoring 

Colon Cancer Workgroup, 2003 #36650 

Quality measure (QM): Adjuvant chemotherapy is administered to patients with 

lymph node positive colon cancer Basis of QM: Clinical practice guideline (National Comprehensive Cancer 

Network [NCCN] guidelines) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Improved health status Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of patients having colon cancer resection. Numerator: Number of above patients receiving combination chemotherapy. Data sources: Medical records Recommended frequency of data collection: No recommendation 

General comments:Older patients or those with comorbidities may not reRating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-118

Evidence Table 8 – Question 2c (Grey Literature) – continued Study Characteristics of Quality Measure Comments/Quality Scoring Institute of Medicine report, 2005 #36680 

Quality measure (QM): Adjuvant chemotherapy after surgery for Stage III colon cancer Basis of QM: Clinical practice guideline (National Comprehensive Cancer Network [NCCN] guidelines) Type of QM:(a) Process (b) General Outcome to which the QM is linked:Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of patients with Stage III colon cancer who undergo surgery. Numerator: Number of above patients who receive a full course of adjuvant chemotherapy after surgery. Data sources: Surveillance, Epidemiology and End Results Program (SEER)-Medicare dataset, medical records Recommended frequency of data collection: No recommendation 

General comments:Stage III colon cancer refers to tumors that have sprcolon or rectum into 1 to 4 regional lymph nodes and The current standard for chemotherapy is a 6-month Older patients are less likely to receive recommendedespite evidence that they tolerate it well. Race, maand individual hospitals are also associated with recchemotherapy. Key references cited: Ayanian et al., 2003 Moore and Haller, 1999 NCCN, 2004 Neugut et al., 2002 Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-119

Evidence Table 8 – Question 2c (Grey Literature) – continued Study Characteristics of Quality Measure Comments/Quality Scoring National Committee for Quality Assurance, 2005 #36580 

Quality measure (QM): Percent of late-stage CRC cancer cases (stage ≥ III) 

that received one or more courses of adjuvant chemotherapy within 1 year of initial cancer surgery Basis of QM: Clinical practice guideline (National Comprehensive Cancer Network [NCCN] guidelines) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Improved health status Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of newly diagnosed colon cancer cases (stage ≥ III) that received 

surgical treatment in a 12-month period. Numerator: Number of above who received one or more courses of adjuvant chemotherapy within 1 year of surgery. Data sources: Cancer registry, claims data Recommended frequency of data collection: No recommendation 

General comments: None Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-120

Evidence Table 9 – Question 2d (Published Literature): As appropriate to specific stages of colorectal cancer, what quality-of-

available and what evidence is available for measures of quality of care of treatment of colorectal cancer, including appropriate use of ra

curative or palliative therapy, specifically for rectal cancers? Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Ayanian, Zaslavsky, Fuchs, et al., 2003 #34680 

Quality measure (QM): Use of adjuvant radiation therapy in patients with stage II or III rectal cancer Basis of QM: Other (1990 NIH Consensus Conference) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Decreased local/regional recurrence. Intent of QM: Not specified (aim of the study was to estimate underreporting of adjuvant therapies in routinely collected registry data, assess rates of adjuvant therapies and factors associated with use, and ascertain reasons why eligible patients were not treated) Definition of denominator/numerator:Denominator: Number of subjects identified from registry records (3 regional registries within the California Cancer Registry) who were 18 years of age and older and diagnosed during 1996 and 1997 with stage II or III adenocarcinoma of the rectum, using staging criteria of the AJCC. Patients had undergone surgery and survived for at least 30 days. Numerator: Number of above subjects with rectal cancer who 

received adjuvant radiotherapy. Data sources:

California Cancer Registry Physician survey Office records Recommended frequency of data collection: No recommendation 

Study population: Eligible for radiation therapy: N: 534 Age: 18 to > 85 Race: 73.4% white Sex: 45.9% female Tumor stage: Rectal stage III 46.8%, rectal stage II 53.2% Performance status: 64.8% comorbidity score 0, 19.3% 1, 15.9% ≥ 2 Geographic location: San Francisco/Oakland, San Jose/Monterey, and Sacramento, CA areas Dates: 1996-97 Healthcare setting: Hospitals Results:64% of eligible patients received radiation. Sensitivity of routinely collected registry data was 93% for radiation therapy. Only 2% of patients had registry data indicating they had received radiation therapy but were reported by their physicians not to have received the therapy. Adjusted rates for radiation therapy for rectal cancer were most strongly associated with age. Older patients were less likely (p< .001) to receive radiation therapy than younger patients. Most common reasons cited for not giving adjuvant therapy: patient refusal, comorbid illness, lack of clinical indication. 

General comments:Article attempts to estimadjuvant therapies in rouby comparing registry daphysician surveys. Because the standard of this QM would have beebeen modified to the use chemo/radiotherapy afterectal cancer. Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 4 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 5  Validity: 5  Adaptability: 5  Adequacy of risk ad
E-121

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SBaxter, Rothenberger, Morris, et al., 2005 #44990 

Quality measure (QM): Use of adjuvant radiation therapy in patients with AJCC stage II or III rectal cancer Basis of QM: National Institutes of Health (NIH) Consensus Conference Type of QM:(a) Process (b) General Outcome to which the QM is linked: Decreased local/regional recurrence in patients with rectal cancer. Intent of QM: To evaluate U.S. trends in adjuvant radiation therapy use over a 25-year period, timing of radiation therapy (pre- vs. postoperative) and factors affecting radiation delivery. Definition of denominator/numerator:Denominator: Number of patients ≥ 18 yrs. of age diagnosed with adenocarcinoma of the rectum between January 1976 through December 2000. Exclusion criteria: patients who had in situ or metastatic disease, patients with malignancies other than adenocarcinoma and patients who did not undergo some form of surgical therapy. Numerator: Number of above patients with stage II or III rectal cancer who received adjuvant radiotherapy. Data sources:Surveillance Epidemiology and End Results (SEER) cancer registry Recommended frequency of data collection: No recommendation 

Study population: N: 45,627 

 RT Use No RT Use 

N 14,571 30,429 

Median age 65 70 % male 61 55 % Non AA 94 95 Seer Stage (%) Localized 29 67 Regional 64 29 Unknown 7 4 Grade (%) Well diff. 8 15 Moderate diff. 62 54 Poor diff. 19 10 unknown 11 21 

Geographic location: 14 SEER cancer registry sites across the U.S. Dates: 1976-2000 Healthcare setting: Hospitals Results:

 RT Use No RT Use 

N 14,571 30,429 

Year of Diagnosis % % 1976-1980 15 85 1981-1985 25 75 1986-1990 33 67 1991-1995 38 62 1996-2000 42 58 

Frequency of radiation therapy use for patients with rectal cancer increased substantially over study period: 12 percent underwent RT in 1976 compared to 42 percent in 2000 (p < 0.0001). For patients with regional spread, the percent receiving RT increased from 17% to 65%. Of those with regional spread who 

General comments:Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 5  Adequacy of risk ad
E-122

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

also underwent radical excision, 41% received RT in 1983 compared to 65% in 2000 (p < 0.0001). By 2000, RT use for Stage II patients was 59%; for stage III, 68%. Increased use was almost entirely due to post operative RT use until 1996. Since then, postoperative use rate has declined while preoperative RT use has increased substantially (p < 0.0001). Demographic differences in RT use exist. Patients who underwent RT tended to be younger (65 yrs) compared to those not undergoing RT use (70 yrs; p < 0.0001). They were more likely to be male (61% vs. 55%; p < 0.0001); they were also more likely to have regional spread (64%) than those who did not undergo RT (29%). AAs were more likely to have received RT (35%) than non-AA (32%), p< .0003. AAs were also more likely to have regional disease (47 vs. 42%) than non-AAs, p < 0.00001. 

Patients since 1988 Odds ratio 

for RT use 

Age at diagnosis > 70 1 < 65 - ≤ 70 3.03 ≤ 60 4.95 Male 1 Female .81 Non AA 1 AA .77 AJCC tumor stage II 1 III 1.76 Grade Poorly differentiated 1 Moderate .79 Well .71 Type of surgery Local 1 Radical 1.34 
E-123

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Male sex and younger age were predictors of RT use (p < 0.0001). AAs were less likely to undergo RT (p < 0.003) than non-AAs. Patients with poorly differentiated rectal cancer and those with AJCC stage II were significantly more likely to undergo RT use than those with well- or moderately differentiated rectal cancer or with AJCC stage II cancer (p < 0.0001). The year of diagnosis and geographic location by registry remained important predictors of RT use (p < 0.0001). 

Beart, Steele Jr., Menck, et al., 1995 #990 

Quality measure (QM): Percentage of patients with rectal cancer receiving radiotherapy Basis of QM: None Type of QM:(a) Process (b) General Outcome to which the QM is linked: None is specified, but presumably it would be survival Intent of QM: Not specified (aim of the study was to identify current trends in the management of patients with carcinoma of the colon or rectum and to identify changes in patterns of care and survival) Definition of denominator/numerator:Denominator: Number of subjects with stage II or III rectal cancer seen in 1988 from amongst cases of colon and rectal cancer (up to 25 per program or facility) reported by over 1200 approved cancer programs and 800 other facilities on the Commission on Cancer mailing list that were invited to participate. Numerator: Number of above subjects receiving chemoradiotherapy. Data sources: Hospitals sent in up to 25 consecutive cases from their medical records Recommended frequency of data collection: No 

Study population: N: 39,502 (10,293 with rectal cancer; 2051 with stage II or III) Age: Median 69.3 years for rectal cancer Race: For entire study: African American, 9.1% for colon cancer and 6.5% for rectal cancer; Hispanic, 2.7% for colon cancer and 3.5% for rectal cancer. 1.4% of cases were Asian. The rest were Caucasian.Sex: Male 57.2% (rectal) Tumor stage: All included (4.7% stage 0, 22.1% stage I, 29.2% stage II, 22.2% stage III, 18.7% stage IV, 3.0% unknown) Performance status: NR Geographic location: Entire U.S. 

Dates: Calendar years 1983 and 1988 Healthcare setting: Hospitals Results:

 Colon Cancer Rectal Cancer 

 1983 1988 1985 1988 Chemotherapy 10.7 11.1 5.9 5.6 Radiation therapy 2.7 10.9 15.3 13.8 Combined chemo and RT 1.5 1.6 5.5 10.1 

. 

General comments:The data for this study wso it may not reflect morfor management. Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 5  Adequacy of risk ad
E-124

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

recommendation 

Coburn, Pricolo, and Soderberg, 1994 #2920 

Quality measure (QM): Adjuvant therapy rates Basis of QM: None Type of QM:(a) Process (b) General Outcome to which the QM is linked: None Intent of QM: Not specified (aim of the study was to perform univariate and multivariate analyses of poor indicators in the elderly undergoing colorectal operation, and to compare clinical, pathologic, and therapeutic factors in patients younger than and older than 80 years of age) Definition of denominator/numerator:Denominator: Number of subjects who had received operative treatment for colon and rectal cancer between 1961 and 1987. Numerator: Number of above subjects who received adjuvant therapy (chemo- or radiotherapy). Comparisons were made between patients under and over 80 years. Data sources: Inpatient, outpatient, and office records Recommended frequency of data collection: No recommendation 

Study population: N: 800

< 80 

years 

> 80 

years 

N 177 623 Sex (F%/M%) 48/52 56/44 Rectal lesions 36 23 Duke stage  A 9.0 6.0  B1 16.7 12.8  B2 31.2 42.7  C1 4.5 0  C2 21.5 21.8  D 17.1 16.7 

Geographic location: Rhode Island, U.S. Dates: 1961 and 1987 Healthcare setting: Academic institute Results:Adjuvant therapy rate was 5.1% for patients > 80 yrs and 20% for patients < 80 yrs. Long-term survival (at 5 and 8 yr) and rate of recurrence after curative therapy were used to validate the measure. There was a statistically significant difference in survival rates: 48% for patients < 80 vs. 32% for patients > 80. There was no statistically significant difference between the groups with regard to recurrence after curative therapy. 

General comments:Data collected between Other factors that affect more aggressive biologic colon and rectum in the disease and less tendendissemination, greater teright-sided, rarity of adjuolder patients, higher copatients due to comorbid frequency of obstructing Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 2  Adequacy of risk ad
E-125

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SDemissie, Oluwole, Balasubramanian, et al., 2004 #31380 

Quality measure (QM): Non-receipt of standard radiation treatment Basis of QM: Clinical practice guideline (NIH guidelines) Type of QM:(a) Process (b) General Outcome to which the QM is linked:Intent of QM: Not specified (aim of the study was to examine colorectal cancer treatment differences between races with comparable disease at presentation) Definition of denominator/numerator:Denominator: Number of white or black subjects diagnosed with rectal cancer between Jan 1, 1988 and Dec 31, 1997.Exclusion criteria: Subjects neither white nor black, unknown treatment history, cancer diagnosed at autopsy, in-situ tumor or carcinoid tumor, rare histological types. Numerator: Number of above subjects not receiving radiation therapy (defined as beam radiation, radioactive implants, radioisotopes, or a combination of these). Comparisons were made between blacks and whites.Data sources: SEER database Recommended frequency of data collection: No recommendation 

Study population: 

Charac-teristic 

Male Female 

 White Black White Black n 49,359 4383 47,803 4832 < 84 yr 92.5 95.3 84 92 AJJCC Tumor Stage II 26.84 23.07 24.18 20.99 II 29.54 26.44 32.49 28.04 III 22.66 23.02 23.77 24.71 IV 16.76 22.36 15.70 20.88 Unknown 4.2 5.11 3.87 5.38 

Geographic location: SEER geographic regions Dates: 1988-1997 Healthcare setting: Mixed Results:While there is no significant racial difference in radiation therapy for stages II and IV rectal cancer, notable differences exist for stage I and stage III rectal cancer. For stage III rectal cancer, blacks had a higher risk of non-receipt of radiation therapy. For stage I, black males had a lower likelihood of nonreceipt of radiation therapy. Odds of non-receipt of radiation therapy for rectal cancer for blacks as compared with whites by sex (whites formed the reference group): (continued on next page) 

General comments:The intent-to treat and trare compared, and the fovalid comparison. Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 4  Adequacy of risk ad
E-126

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Radiation therapy 

Male Female 

 unadjusted ad- justed unad- justed ad- justed AJCC stage I 

0.5 0.51 0.93 0.96 

AJCC stage II 

0.67 0.81 1.22 1.37 

AJCC stage III 

1.67 2.04 1.14 1.54 

AJCC stage IV 

0.92 1.03 1.23 1.43 

unstaged 

0.81 1.00 0.86 1.04 

(adjusted for age, tumor grade and histology) Black-to-white disparity in non-receipt of radiation therapy was more prominent when actual treatment received rather than intent-to-treat was considered. Number of patients who refused treatment: Stage I: 32.8% black and 9.2% white males and 19.4% black and 12.8% white females. 
E-127

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SDominitz, Samsa, Landsman, et al., 1998 #9380 

Quality measure (QM): Percentage of patients with colorectal cancer receiving radiation therapy 

Basis of QM: None 

Type of QM:(a) Process (b) General Outcome to which the QM is linked: Survival Intent of QM: Not specified (aim of the study was to assess the influence of race on the treatment and survival of patients with colorectal carcinoma) Definition of denominator/numerator:Denominator: Number of white/black patients with colorectal cancer identified from the VA Patient Treatment File (PTF) who were discharged from VA hospitals with a diagnosis of CRC (ICD-9-CM codes 153.0-153.4, 153.6-154.1) during fiscal year 1989. To exclude prevalent cases, any patient with a diagnosis of CRC (ICD-9-CM codes 153.0-153.4, 153.6-154.1, V10.05-V10.06 ) for hospitalization during FY 1984-1988, or a personal history of CRC (ICD-9-CM codes V10.05-V10.06) during the index hospitalization, was excluded. Male veterans only. No ulcerative colitis or Crohn’s disease. Numerator: Number of above white/black patients treated with radiation therapy, as indicated by specific ICD-9-CM 

codes in the PTF. Data sources: VA PTF 

Recommended frequency of data collection: No recommendation

Study population: N: 2,607 whites + 569 blacks Age: 67.1 ± 9.1 (whites); 66.4 ± 9.9 (blacks) Race: See above Sex: 100% male Tumor stage: All stages Performance status: NR Geographic location: VA Medical Centers Dates: 1989 Healthcare setting: VA Hospitals Results:No statistically significant differences in rates of radiation therapy in blacks vs. whites. No difference in 5-year relative survival or overall survival in blacks vs. whites. 

General comments:The article compares ratand chemotherapy amonattempt is made to compa guideline) to determine or whites are optimal. The study is limited becaadministrative database and did not allow for adjuRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio

 Reliability: 3  Validity: 1  Adaptability: 1  Adequacy of risk ad
E-128

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SGovindarajan, Shah, Erkman, et al., 2003 #35310 

Quality measure (QM): Percentage of patients with stage II, III colorectal cancer receiving radiation therapy Basis of QM: None Type of QM:(a) Process (b) General Outcome to which the QM is linked: Not firmly linked to any outcome, but presumably to survival, quality of life, or rates of local recurrence Intent of QM: Not specified (aim of the study was to analyze disease stage, treatment received, and socioeconomic factors to better understand the factors influencing survival differences between African-Americans and Caucasians with colorectal cancer) Definition of denominator/numerator:Denominator: Number of African-Americans (AAs)/Caucasians, 

aged 18 older, with CRC treated between 1984 and 1997 at a state-funded University hospital located in Arkansas, identified through the hospital tumor registry using appropriate tumor registry codes. Numerator: Number of above AAs/Caucasians treated with radiation therapy. Data sources: Tumor registry at Univ. of Arkansas 

Recommended frequency of data collection: No recommendation 

Study population: N: 617 All figures below for Caucasians/AAs Age: 60.0/61.5 Race: 427 Caucasians/190 AAs Sex: 47.1% male/39.5% male Tumor stage: Stages 0-IV Performance status: NR Other: - Not high school grad: 45%/60% - Low income: 45%/64% - High poverty: 41%/68% Geographic location: Arkansas Dates: 1984-1997 Healthcare setting: University hospital Results:Authors show that significantly higher percentages of Caucasians received radiation therapy, compared with AAs, for patients with all stages of CRC, and for patients with only stages II and III disease. Cancer-specific mortality was higher in AfricanAmericans than in Caucasians for all stages. 

General comments:It is not possible to use tradiation” because they cancer together, but radrectal cancer. Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 1  Adaptability: 1  Adequacy of risk ad
E-129

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SKline, Smith, Coia, et al., 1997 #12070 

Quality measure (QM): Adherence to radiotherapy management treatment guidelines for patients with adenocarcinoma of the rectum and sigmoid colon Basis of QM: Clinical practice guideline (Patterns of Care Study Treatment-Planning Committee consensus guidelines) Type of QM:(a) Process (b) Technical Outcome to which the QM is linked: Not specifically stated, but presumably reduction in complications due to radiation therapy, reduction in local-regional recurrence, or improved survival Intent of QM: Not specified (aim of the study was to conduct a study of the process of treatment planning and treatment of adenocarcinoma of the rectum and sigmoid in the U.S., and to compare survey results to consensus guidelines) Definition of denominator/numerator:Denominator: Number of cases of cancer of rectum or sigmoid colon treated 1989-90 at one of 75 facilities selected from among a random sample of all U.S. facilities administering megavoltage radiation therapy. Numerator: Number of above patients receiving radiotherapy according to treatment guidelines. Data sources: 1989-90 PCS Survey Recommended frequency of data collection: No recommendation 

Study population: N: NR Age: NR Race: NR Sex: NR Tumor stage: NR Performance status: NR Geographic location: U.S. 

Dates: 1989-90 Healthcare setting: Academic, hospital-based, and free-standing radiation therapy centers Results:Adherence to a variety of guideline measures reported in the study. For example: Guideline: “Information from a pelvic CT may be useful in determining field borders.” Result of survey: CT used for radiation therapy planning in only 17-21% of cases. Other radiation therapy quality indicators assessed include: 1. Placement of clips by surgeon when tumor adherent to other pelvic structures. 2. Isodose distribution generated for dose prescription. 3. Beam energy used should be ≥ 4 MV. 

4. At least 3 radiation therapy fields should be used. 5. Fields should be shaped with custom blocks. 6. Wedges or compensators should be used as needed. 7. Port films should be taken. 8. Patients should be simulated with contrast in rectum. 9. A small bowel series should be done if the total dose will exceed 50 Gy. 10. Prone setup used. 

General comments:This paper describes a nmeasures of quality relevrather than a more generadiation was performed indication. Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 5  Validity: 5  Adaptability: 5  Adequacy of risk ad
E-130

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SMerrill, Brown, Potosky, et al., 1999 #27110 

Quality measure (QM): Percentage of CRC patients who had radiotherapy Basis of QM: None Type of QM:(a) Structure (b) General Outcome to which the QM is linked: Presumably survival Intent of QM: Not specified (aim of the study was to compare treatment utilization and long-term survival in HMO and fee-for-service [FFS] settings for Medicare colorectal cancer cases) Definition of denominator/numerator:Denominator: Number of CRC patients in each subgroup of the dataset (see Data sources, below). Numerator: Number of above patients who had adjuvant radiotherapy. Data sources:1. Two tumor registries on colon and rectum cancer patients diagnosed 1985-92 and followed up through 31 December 1994. Both registries are part of SEER.2. Medicare inpatient claims data. 3. Databases of two HMO plans, Kaiser Permanente of Northern California and Group Health Cooperative of Puget Sound. 4. 1990 U.S. Census Recommended frequency of data collection: No recommendation 

Study population: N: 15,352 total Age: ≥ 65 Race: 81.2 to 95.3% white depending on subgroup Sex: 46.8 to 54.3% male, depending on subgroup Tumor stage: 0-IV Performance status: NR Comorbidity Index: 0-2+ Geographic location: San-Francisco-Oakland area, 

Seattle-Puget Sound area Dates: 1985-1992 Healthcare setting: 2 tumor registries participating in the SEER Program augmented with Medicare data along with HMOs KPNC and GHC. Results:The standardized rate of adjuvant radiotherapy use following surgery was significantly higher among HMO cases (44%, 95% CI 37%-50%) than among FFS cases (35%, 95% CI 33-38%). Cancer-specific mortality rates were similar in both settings. 

General comments:A very high-quality study Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 4 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 3  Adequacy of risk ad
E-131

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SMinsky, Coia, Haller, et al., 1998 #9280 

Quality measure (QM): Use of modern radiation therapy techniques; adherence to recommendations of NCI-sponsored RCT in rectal cancer patients. Basis of QM: Other: (use of modern radiation therapy techniques as defined by consensus of experts [Patterns of Care Rectal Cancer Committee]; adherence to RCT recommendations described as above) Type of QM:(a) Process (b) General/Technical Outcome to which the QM is linked: Presumably locoregional recurrence, survival, and reduction in radiation therapy-related complications Intent of QM: Not specified (aim of the study was to determine the U.S. national practice standards for patients with adenocarcinoma of the rectum treated in radiation oncology facilities) Definition of denominator/numerator:Denominator: Number of patients with rectal cancer treated with curative intent from 1992-1994, with no evidence of extrapelvic disease, micro or gross residual disease after surgery, or prior or concurrent malignancies or treatment for rectal cancer, who received radiation therapy as a component of their definitive or adjuvant management, and who were identified from a facilities survey conducted by the PCS. Numerator: Number of above patients whose radiation therapy conformed to recommendations of the expert panel (on various technical dimensions); subset of above patients whose treatment adhered to specific recommendations of the NCI RCTs. Data sources: PCS national survey Recommended frequency of data collection: No recommendation 

Study population: N: 507 Age: 64 Race: NS Sex: 64% male Tumor stage: Tis-3, N0-N3 Performance status: 83% KPS 90-100 Differentiation: 65% moderate Resection margins: 66% negative Geographic location: U.S. 

Dates: 1992-94 Healthcare setting: All U.S. radiation oncology facilities Results:Radiation therapy details: Many factors considered indicative of high-quality radiation therapy were measured in the survey. For example, radiation therapy field arrangement was AP/PA in 11% of patients, 3 or 4 fields in 92%, and unknown in < 0.5%. For full details, see Table 4, p. 2545. Chemotherapy details: Receipt of chemotherapy was documented in the study but the study did not address adherence to treatment guidelines with respect to chemotherapy. Surgery details: Type of surgery performed was documented but no attempt was made to determine if the “correct” surgical procedure was done, or to assess surgical complication rates. 

General comments: NoRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 3 - Usability: 4 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 5  Adequacy of risk ad
E-132

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SMorris, Billingsley, Baxter, et al., 2004 #32200 

Quality measure (QM): Rate of radiation therapy use for stage II, III rectal cancer Basis of QM: Not specified, but appears to be NIH Consensus conference Type of QM:(a) Process (b) General Outcome to which the QM is linked: No explicit outcome link, but presumably to decreased rates of LRR or improved OS Intent of QM: Aim of the study was to investigate racial variation in the performance of sphinctersparing procedures for patients with stage II and III rectal cancer who underwent surgery, and to study race and delivery of any surgical treatment, neoadjuvant therapy, and radiation therapy to these patients, with implications for quality of care. Definition of denominator/numerator:Denominator: Number of blacks/whites diagnosed with rectal cancer and entered into the SEER database 1988-1999. Exclusion criteria: Prior diagnosis of CRC, prior diagnosis of cancer requiring surgery or radiotherapy in the pelvis, or < age 35. Numerator: Number of above patients who received various treatments for CRC including surgery (also type of operation) and radiation (timing of RT also evaluated).Data sources: SEER registry Recommended frequency of data collection: No recommendation 

Study population: N = 52,864 

 White Black 

n 44010 3851 Age (%) 35-64 35 49 65-79 46 40 80+ 20 12 Male (%) 55 52 Tumor stage (%) insitu 9 8 I 3 28 II 21 20 III 19 18 IV 14 17 unstaged 8 10 

Geographic location: SEER regions Dates: 1988-1999 Healthcare setting: U.S. healthcare facilities Results:

N = 18,927 

Radiation Treatment rate (%) 

Adjusted OR (95% CI) 

 Any None White 47 53 Black 44 56 

1.30 

(1.15-

1.47) 

N = 18,560 

Neoadjuvant Radiation Treatment Rate (%) 

Adjusted OR (95% CI) 

White 7 93 Black 7 93 

1.04 (.87-1.36) 

General comments: NoRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 4 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 5  Adequacy of risk ad
E-133

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SNeugut, Fleischauer, Sundararajan, et al., 2002 #18450 

Quality measure (QM): Use of radiotherapy in stage II and III rectal cancer patients among the elderly Basis of QM: Clinical practice guideline (NIH and NCCN) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Rates of LRC, DM, and OS Intent of QM: Not specified (aim of the study was to investigate the use of treatment with adjuvant 5-FUbased chemotherapy and radiation therapy among patients over 65 years of age with surgically resected stage II or III rectal cancer) Definition of denominator/numerator:Denominator: Denominator: Number of patients with histologically confirmed primary rectal cancer, diagnosed 1992-1996, stage IIIII, age ≥ 65, s/p surgical tumor resection, eligible for Medicare Parts A and B in the 12 mo before diagnosis and 120 days after diagnosis, and had survived more than 4 mo after diagnosis. Excluded members of HMO in the 12 mo before and 4 mo after their diagnosis. Numerator: Number of above patients who received radiation therapy. Data sources: SEER-Medicare database Recommended frequency of data collection: No recommendation 

Study population: N: 1,807 Age: 65-85+ Race: 1592 non-Hispanic white Sex: 964 female Tumor stage: II (983). III (824) Performance status: NR Charlson-Deyo comorbidity score: 0 (1098), 1 (299), > 1 (410) Geographic location: SEER geographic regions 

Dates: 1992-96 Healthcare setting: U.S. facilities Results:48% of cohort received radiation therapy (11% received postoperative radiation therapy alone, 37% received chemoradiotherapy). Age was associated with the overall use of radiation. 

General comments:Impossible from this datafrequency of adjuvant uspopulation under study. chemotherapy in an eldecancer may be a medicapatient has significant coRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 4 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 5  Adequacy of risk ad
E-134

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SNewcomb and Carbone, 1993 #3760 

Quality measure (QM): Rate of adjuvant radiation therapy for colorectal cancer Basis of QM: None mentioned, but presumably practice guideline Type of QM:(a) Process (b) General Outcome to which the QM is linked: Not specified, but presumably survival Intent of QM: Not specified (aim of the study was to evaluate the selection of cancer treatment among the elderly) Definition of denominator/numerator:Denominator: Number of women residents of Wisconsin aged 20-74 at the time of diagnosis with a new diagnosis of carcinoma of the large bowel during 1989-1991, and who had a listed telephone number and spoke English, and whose physician consented to their participation, and who agreed to a telephone interview, and were among the 628 consecutive participants interviewed September 1, 1990 through November 30, 1990. Numerator: Number of subjects receiving radiation therapy. Data sources: Registry for staging information and patient interviews (for the treatment information) Recommended frequency of data collection: No recommendation 

Study population: N: 121 with colorectal Age: 20-74 Race: NR Sex: Female Tumor stage: All stages Performance status: NR Geographic location: Wisconsin 

Dates: Sep 1 through Nov 30, 1990 Healthcare setting: None (all participants were outpatients contacted by phone) Results: 14% of patients < 65 years of age and 18% ≥ 65 received radiation therapy. 

General comments:Relying on recall of patiea treatment is open to biThe data on radiation trestage or location of large cancer and locally advantypically be radiated. Theconclusive. Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 3 - Scientific acceptability  Precise specificatio Reliability: 4  Validity: 4  Adaptability: 5  Adequacy of risk ad
E-135

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SO’Connell, Maggard, Liu, et al., 2004 #45210 

Quality measure (QM) Radiation therapy rate Basis of QM: NIH Consensus Conference Guidelines 

for adjuvant therapy for colorectal cancer Type of QM: (a) Process (b) General Intent of QM: Not specified. Aim of study was to compare rectal cancer patient outcomes between young and older populations. Definition of denominator/numerator:Denominator: Patients in the SEER database diagnosed with rectal cancer between 1991 and 1999. Tumors identified as “rectum” in location were selected. Specific histologies were chosen to include only adenocarcinomas. Exclusion criteria: Patients with tumors identified as rectosigmoid. Mucinous, signet ring cell, carcinoid, sarcoma and lumphoma histologies were excluded along with those tumors classified as benign or in situ.Numerator: Number of above patients who received t radiation therapy subsequent to diagnosis. Data sources: SEER databse Frequency of data collection: No recommendation 

Study population: N = 11,778 

 20-40 yrs 

(young) N = 466 

60-80 yrs (old) N = 11,312 

P-value 

Age 34.1± 4.5 70± 5.5 Male % 54.9 60 .03 Race White 62.6 82.3 <.001 Black 12.7 6.1 <.001 Hispanic 8.8 3.7 <.001 Asian 7.5 6.4 NS other 8.4 1.5 <.001 AJCC stage I 17 23.7 <.001 II 15.5 17.8 NS III 27 20 <.001 IV 17.4 13.6 .02 Grade Well 7.5 9.4 NS Moderate 50.4 61.2 <.001 Poorly 24.3 14 <.001 Anaplastic 1.7 .7 .008 Unknown 16.1 14.7 NS 

Geographic location: U.S. Dates: 1991- 1999 Healthcare setting: Hospital setting Results:More young patients received radiation therapy both overall and stage-for-stage. Overall, 57.4 percent of young patients and 39.7 of older patients received radiation (p < 0.0001). Similar results were found by stage where a greater share of younger patients received radiation compared to older patients. Median survival was 24 months for the young group 

General comments:The SEER registry mainmeasures to prevent codone of the best populatiohowever, miscoding and present. The registry dorelevant data, such as fafactors, etc. A third limittumor stage or grade wapatients. Another limitatsubstantially smaller sampatients compared with tpatients, leading to overeRating of quality measstudy (scale of 1-5, whe= ideal): Importance: 2 Usability: 5 Scientific acceptability  Precise specifications Reliability: 3  Validity: 3  Adaptability: 3  Adequacy of risk adju
E-136

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

and 27 months for the older group. Univariate analysis revealed overall 5-year survival was similar for the young group compared with the older (63.2 vs. 62.1%, p = NS). Five-year, stage-specific survival also was similar for all stages. Multivariate Cox regression analysis revealed that young, rectal cancer patients had a lower hazard of dying (hazard ratio, .693, p = 0.0004) when controlling for tumor stage, patient demographics, tumor characteristics and stage and treatment. 
E-137

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SRoetzheim, Pal, Gonzalez, et al., 2000 #23910 

Quality measure (QM): Percentage of patients with a colon or rectal cancer diagnosis who received radiation therapy Basis of QM: None stated (see General comments) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Mortality Intent of QM: Not specified (aim of the study was to explore the influence of race/ethnicity and insurance payer on the treatment of and outcomes for colorectal cancer patients in Florida) Definition of denominator/numerator:Denominator: Number of incident cases of colon or rectal cancer occurring in Florida in 1994 as listed in the state tumor registry. Numerator: Number of the above patients who also received radiation therapy within 4 months of initiation of therapy (stratified by colon or rectal cancer). Data sources: Florida state cancer registry (Florida Cancer Data System = FCDS); Florida Agency for Health Care Administration AHCA discharge abstracts (admissions to all nonfederal acute care hospitals and patient visits to ambulatory surgical centers, freestanding radiation therapy centers and diagnostic imaging centers) Recommended frequency of data collection: No recommendation 

Study population:N: 9,548 

Patient characteristic Percent 

Male 51 Race  White 85  Black 6  Hispanic 8 Diagnosis stage  In situ 6  Local 30  Regional 42  Distant 16  Unstaged 6 Anatomic site  Colon 84  Rectal 16 Comorbidity index  0 71  1 21 

≥ 2 8 

Geographic location: Florida Dates: 1994 Healthcare setting: Multiple Results: 26% of patients received radiation therapy but no validity testing was done for radiation therapy. There were no racial differences in the use of radiation therapy. Among Medicare patients, those having HMO insurance types were less likely to receive radiation therapy, an effect that was primarily restricted to patients with colon cancer. Among NonMedicare patients, there were no insurance-related differences in the receipt of radiation therapy. Other predictors of using radiation therapy included younger age, lower levels of income, non-urban residence, being married, having rectal cancer, and having lower 

General comments:Existing guidelines are fostage II or III rectal cancreporting data on all radicolon or rectal cancer is driven. Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 4 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 3  Adequacy of risk ad
E-138

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

levels of comorbidity. 

Rogers, Ray, and Smalley, 2004 #30130 

Quality measure (QM): Percentage of patients with a diagnosis of stage III colorectal cancer receiving radiation therapy for rectal cancer Basis of QM: Not stated Type of QM:(a) Process (b) General Outcome to which the QM is linked: Not stated, but presumably survival or recurrence of colorectal cancer Intent of QM: Not specified (aim of the study was to examine the effect of race on CRC outcomes in patients who had identical health care coverage) Definition of denominator/numerator:Denominator: Number of patients enrolled in both Tennessee Medicaid and Medicare and hospitalized with a diagnosis of colorectal cancer. Numerator: Number of above patients who received radiation therapy within 180 days of diagnosis. Data sources: Medical records, administrative files from the Tennessee Medicaid program, Tennessee death certificates, and the National Death Index Recommended frequency of data collection: No recommendation 

Study population: N: 969 total (19.3% rectal cancer) Age: 78.9 ± 7.5 years Race: 272 black (28%), 697 white (72%) Sex: 73.9% female (whites), 72.4% female (Blacks) Tumor stage: 75.6% (whites) and 72.1% (blacks) had stage III disease Performance status: Charlson Deyo score of 0 in 64% (whites) and 69% in Blacks Geographic location: Tennessee 

Dates: 1984-1994 Healthcare setting: Patients identified from hospitalizations Results:4.3% of whites and 12.9% of blacks received radiation therapy (p = 0.001). There was no difference in survival between whites and blacks. 

General comments:Study concluded that whcare, outcomes were eqRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 5  Validity: 5  Adaptability: 3  Adequacy of risk ad
E-139

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SSchrag, Gelfand, Bach, et al., 2001 #21280 

Quality measure (QM): Percentage of patients with stage II or III rectal cancer receiving chemoradiotherapy Basis of QM: NIH consensus statement Type of QM:(a) Process (b) General Outcome to which the QM is linked: None is specified, but presumably it would be pelvic recurrence Intent of QM: Not specified (aim of the study was to examine the relationship between patient characteristics and the use of adjuvant radiotherapy with and without chemotherapy among patients 65 years of age and older with stage II and II rectal cancer) Definition of denominator/numerator:Denominator:Number of patients aged 65 years and older enrolled in Medicare A and B, diagnosed with primary rectal adenocarcinoma stage II or III, in a SEER area during the years 1991-1996 and not part of an HMO, and who had rectal cancer surgery performed within 6 months of primary diagnosis and who survived more than 6 months and who did not have a secondary cancer within 6 months. Numerator: Number of patients who received radiotherapy (and chemotherapy) within 6 months of surgery. Data sources: Linkage of SEER population-based cancer registries with Medicare database of HCFA Recommended frequency of data collection: No recommendation 

Study population: N: 1,411 Age: > 65 Race: 88% white, 4% black, 8% other Sex: Male 54% Tumor stage: Stage II, III Performance status: NR Geographic location: 5 states and 6 U.S. 

metropolitan areas Dates: 1992-1996 Healthcare setting: Hospitals Results:57% of patients received radiotherapy (42% with chemotherapy and 15% radiation alone). 

General comments:The data for this study wso it may not reflect morfor management. Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 4  Adequacy of risk ad
E-140

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SSteele Jr., 1994 #35390 

Quality measure (QM): Percentage of patients with stage II or III rectal cancer receiving chemoradiotherapy Basis of QM: None Type of QM:(a) Process (b) General Outcome to which the QM is linked: None specified, but presumably it would be survival Intent of QM: Not specified (aim of the study was to assess cancer patient care and outcomes on a national basis) Definition of denominator/numerator:Denominator: Number of patients with stage II, III rectal cancer in 1990 reported to the National Cancer Data Base (NCDB). Numerator: Number of above patients receiving chemoradiotherapy. Data sources: Registry information reported to National Cancer Data Base (NCDB) Recommended frequency of data collection: No recommendation 

Study population: N: 5,495 Age: Median 69.3 years for rectal cancer Race: For entire study: African American, 9.1% for colon cancer and 6.5% for rectal cancer; Hispanic, 2.7% for colon cancer and 3.5% for rectal cancer. 1.4% of cases were Asian. The rest were Caucasian. Sex: Male 57.2% (rectal). Tumor stage: All included (4.7% stage 0, 22.1% stage I, 29.2% stage II, 22.2% stage III, 18.7% stage IV, 3.0% unknown Performance status: NR Geographic location: Entire U.S. 

Dates: Calendar years 1983 and 1988 Healthcare setting: Hospitals Results:Not specifically stated, but can be calculated form Table 19. 43% of stage II or III patients received surgery and radiation (with or without chemotherapy). 

General comments:This paper addresses si(1995; #990). Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 3  Adequacy of risk ad
E-141

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SSteele, 1994, #35840 

Quality measure (QM): Percentage of stage II and 

III rectal cancer patients who underwent radiation therapy Basis of QM: Clinical practice guideline 

It is implicit that the AHCC guidelines be followed but these are not actually cited in the article. There are also guidelines for chemotherapy in stage III colon cancer and chemo/radiation therapy for stage II and III rectal cancer (NCC), but these are not cited by the authors. Type of QM:(a) Process (b) General Outcome to which the QM is linked: No outcome is exactly linked to the above but is said to be a marker of appropriate cancer diagnosis and treatment. No citation is provided for this statement. Intent of QM: Not specified (aim of the study was to assess cancer patient care and outcomes on a national basis) Definition of denominator/numerator:Denominator: Number of patients in the National Cancer Daba Base (NCDB) with a new diagnosis of rectal cancer (rectosigmoid junction, rectum and anal canal) in the years 1985-1986 or 1991. Numerator: Number of the above who underwent radiation therapy. Data sources: National Cancer Data Base (NCDB)Recommended frequency of data collection: No recommendation 

Study population: Data given as patients (year)N: 18,418 (85/86); 17,348 (91) Age: Median 72.7 (85/85); 73.0 (91) Race:  Non-hispanic whites 88.0% (85/85) 86.9% (91)  Hispanics 1.6/2.3  African-Americans 4.7/6.3  Asian 1.2/2.2  Unknown 4.5/2.3 Sex: Male 55.3% (85/85) 55.7% (91) Tumor stage: 85-86/91  0 5.8/5.0  1 15.2/18.6  2 10.5/17.4  3 12.2/20.0  4 5.0/7.8  Unknown 51.3/31.2 Performance status: NR Geographic location: National sample; note that the 

sample is not probabilistic and represented different hospitals in different years Dates: 1991 and outcomes for 1985-86 Healthcare setting: 464 hospital in 1985, 474 hospitals in 1986, 937 hospitals in 1991 Results:Treatment trends showed effects of trial data, as a greater percentage of patients were reported as having treatment (including chemotherapy) beyond surgery in 1991 as compared to 1985/86. Hispanics and African American patients were noted to have more advanced stage cancer (rectal). 

General comments: Some “treatment” data anot stratified by stage or and therefore do not procount as a QM. Only suQM from this paper. Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 4 - Usability: 4 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 3  Adaptability: 3  Adequacy of risk ad
E-142

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SSteele and Jessup, 1995 #35850 

Quality measure (QM): Percentage of patients with stage II or III rectal cancer receiving chemoradiotherapy Basis of QM: None Type of QM:(a) Process (b) General Outcome to which the QM is linked: None specified, but presumably it would be survival Intent of QM: Not specified (aim of the study was assess cancer patient care and outcomes on a national basis) Definition of denominator/numerator:Denominator: Number of stage II, III rectal cancer patients in 1992 reported to the National Cancer Data Base (NCDB). Numerator: Number of above patients receiving chemoradiotherapy. Data sources: Registry information reported to National Cancer Data Base (NCDB) Recommended frequency of data collection: No recommendation 

Study population: N: 9,465 Age: All Race: All Performance status: Not stated Other: Not stated Geographic location: Entire U.S. 

Dates: Calendar year 1986/7 and 1992 Healthcare setting: Hospitals Results:Not specifically stated, but can be calculated form Tables 5-13 that 46% of stage II or III patients received surgery and radiation (with or without chemotherapy). 

General comments:This paper addresses si(1995; #990), and Steele that it includes data from have data ending in 198Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 2  Validity: 3  Adaptability: 5  Adequacy of risk ad
E-143

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality STemple, Hsieh, Wong, et al., 2004 #29890 

Quality measure (QM): Radiotherapy rate for stage IV colon or rectal cancer Basis of QM: Clinical practice guideline (National Comprehensive Cancer Network Guideline) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Practice patterns of patients with stage IV cancer Intent of QM: Quality improvement (aim of the study was to evaluate surgical practice patterns for patients 65 years of age and older with stage IV colorectal cancer) Definition of denominator/numerator:Denominator: Number of Medicare-enrolled patients aged 65 and older, initially diagnosed with stage IV colon or rectal cancer in a SEER area during the years 1991-1999. Age, sex, race, ethnicity, comorbidity, year of diagnosis, and socioeconomic status were collected and data were risk-adjusted. Exclusion criteria: Cohort restricted to those with a histologic diagnosis consistent with adenocarcinoma. Diagnoses exclusively on death certificates or at autopsy were excluded, as were those in which the month of diagnosis was unknown. Patients enrolled in HMOs were excluded. Numerator: Number of patients who received radiotherapy within 4 months of diagnosis. Data sources: SEER data base, Medicare claims Recommended frequency of data collection: No recommendation 

Study population: N = 9,011 

Age: 21% 65-69; 25% 70-74; 23% 75-79; 18% 80-84; 13% ≥ 85 Sex: 48% male, 52% female Race: 84% white, 9% black, 7% other Comorbidities: 90% had 0; 5% had 1; 5% had 2+Geographic location: U.S. 

Dates: 1991-1999 Healthcare setting: Five states and six U.S. metropolitan areas. Results: Only 12% of patients received palliative radiotherapy within 4 months of diagnosis (majority with rectal cancer). Patients who underwent primary cancerdirected surgery were more likely to receive radiation therapy (12% vs. 15%, p = 0.01) during their life span when compared with those in the no cancer-directed surgery group. 

General comments: NoRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 3 - Usability: 5 - Scientific acceptability  Precise specificatio

 Reliability: 3  Validity: 3  Adaptability: 4  Adequacy of risk ad
E-144

Evidence Table 9 – Question 2d (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SWudel Jr., Chapman, Shyr, et al., 2002 #18640 

Quality measure (QM): Percentage of patients with colon or rectal cancer who had radiation therapy Basis of QM: None stated Type of QM:(a) Process (b) General Outcome to which the QM is linked: Mortality Intent of QM: Not specified (aim of the study was to determine the outcome of patients with colorectal cancer treated in the same city at 2 nearby medical centers, a university hospital and a city hospital, and to explore disparities in colorectal cancer outcomes between black and white patients) Definition of denominator/numerator:Denominator: Number of patients with a diagnosis of primary colorectal cancer between 1/1/90 and 12/21/99 in the tumor registry databases at each of 2 institutions (university hospital and a city hospital). Excluded patients who were non-white and non-black. Numerator: Number of the above patients who received radiation therapy. Data sources: Hospital tumor registries Recommended frequency of data collection: No recommendation 

Study population (university hospital = UH, city hospital = CH) N: 565 UH; 77 CH Age: Median 69 UH, 74 CH Race: 10.6% black UH; 53% black CH Sex: 53.3% male UH; 62.7% male CH Tumor stage: UH: I 31.7% II 21.9% III 23.7% IV 22.7% CH: I 29.9% II 20.8% III 16.9% IV 32.5% Performance status: NR Geographic location: Nashville, TN 

Dates: 1/1/90 and 12/21/99 (retrospective review) Healthcare setting: University hospital, city hospital Results:There was no difference in the various treatment modalities identified – i.e., curative resection, chemotherapy, and radiation therapy – between institutions or ethnic groups. 

General comments:Mortality was the primary the data for other potent“curative” surgery, chemmuch weaker, poorly defguidelines. Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 3 - Scientific acceptability  Precise specificatio Reliability: 2  Validity: 2  Adaptability: 4  Adequacy of risk ad
E-145

Evidence Table 10 – Question 3 (Published Literature): What quality-of-care measures are available and what evidence is ava

colonoscopic surveillance for colorectal cancer? Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Cooper, Yuan, Chak, et al., 1999 #27430 and Cooper, Yuan, Chak, et al., 2000 #35900 

Quality measure (QM): Percentage of patients with non-metastatic colorectal cancer who had a followup colonoscopy Basis of QM: Clinical practice guideline (see ref 5 [ASCRS guidelines, Dis Colon Rectum 1992]) Type of QM:(a) Process (b) General Outcome to which the QM is linked: No definite link to mortality – possible link to earlier detection of recurrence Intent of QM: Not specified (aim of the study was to understand geographic and patient variations among Medicare beneficiaries in the use of followup testing after surgery for non-metastatic colorectal carcinoma)Definition of denominator/numerator:Denominator: Number of patients with invasive colorectal cancer in a merged SEER-Medicare database which included only those patients > 64 years of age living in a SEER area (9 total) in 1991. Exclusion criteria: Distant metastasis or unstaged cancer, did not undergo resection, not enrolled in Medicare part B, enrolled in an HMO, died within 6 months of diagnosis, or did not have complete data in the merged database. Numerator: Number of the above subjects who underwent colonoscopy during the followup period. Data sources: Merged SEER and Medicare administrative (inpatient and Part B – outpatient) database Recommended frequency of data collection: No recommendation 

Study population: N: 5,716 Age: Mean 74.8 ± 7.0 Race: 6% African-American Sex: 51% F Tumor stage: 50% local, 50% regional Performance status: NR Geographic location: 9 SEER regions (Atlanta, 

Detroit, Seattle-Puget Sound, San FranciscoOakland, Connecticut, Hawaii, Iowa, New Mexico, Utah) Dates: Diagnosis in 1991 and followup through the end of calendar year 1994 Healthcare setting: Non-HMO (Medicare) Results:One or more procedures of interest (colonoscopy, sigmoidoscopy, abdominal CT, pelvic CT, barium enema, abdominal ultrasound, chest X-ray, liver enzymes, CEA) were performed in 87.5%of patients. Of these patients, 52.4% had colonoscopy and 35.3% had CEA. There was a significant variation in rates of testing across the 9 SEER areas. Older patients were less likely than younger patients to undergo followup colonoscopy; there were no differences by race or sex.

General comments:Two papers report on saRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 4 - Usability: 4 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 2  Adaptability: 2  Adequacy of risk ad
E-146

Evidence Table 10 – Question 3 (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SKeating, Landrum, Meara, et al., 2005 #36080 

Quality measure (QM): Rate of complete colonoscopy pre- and perioperatively Basis of QM: Clinical practice guideline (not cited, but appears to be 1990 NCI Consensus Statement) Type of QM:(a) Structure (b) General Outcome to which the QM is linked: Not stated Intent of QM: Not specified (aim of the study was to examine associations between increases in managed care market share and changes in the quality of care delivered to cancer patients in the fee-for-service sector; breast and colorectal cancer studied) Definition of denominator/numerator:Denominator: Number of patients with a first diagnosis of stage III CRC in 1993-1999 in a given SEER county who had undergone surgery, were alive and enrolled in fee-forservice Medicare (parts A and B) through month 4. Numerator: Number of above patients with stage I,II III CRC in a given county who received complete pre- or perioperative colonoscopy. Data sources: SEER-Medicare dataset; 1990 U.S. Census Recommended frequency of data collection: No recommendation 

Study population: N: 48,027 Age: 66-85+ Race: 88 white Sex: 46% male Tumor stage: NR Performance status: NR Other: Income, % high school graduates in census 

tract of residence, comorbidity score, market share of managed care Geographic location: SEER geographic regions Dates: 1993-1999 Healthcare setting: Not stated Results:92.5% of patients with stage II or III CRC underwent complete pre- or perioperative colonoscopy. Increased market share of managed care in a given county resulted in significant increase in the % of patients with complete colonoscopy pre- or perioperatively. 

General comments: NoRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 5  Adaptability: 5  Adequacy of risk ad
E-147

Evidence Table 10 – Question 3 (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SKeating, Landrum, Meara, et al., 2005 #36080 

Quality measure (QM): Rate of colonoscopy 7 to 18 months after diagnosis Basis of QM: Clinical practice guideline (not cited) Type of QM:(a) Structure (b) General Outcome to which the QM is linked: Generally, reduced rate of recurrence Intent of QM: Not specified (aim of the study was to examine associations between increases in managed care market share and changes in the quality of care delivered to cancer patients in the fee-for-service sector; breast and colorectal cancer studied) Definition of denominator/numerator:Denominator: Number of patients with a first diagnosis of stage I, II, or III CRC in 1993-1999 in a given SEER county who had undergone surgery, were alive and enrolled in fee-for-service Medicare (parts A and B) through month 4. Numerator: Number of above patients who received colonoscopy 7 to 18 months after diagnosis. Data sources: SEER-Medicare dataset; 1990 U.S. Census Recommended frequency of data collection: No recommendation 

Study population: N: 48,027 Age: 66-85+ Race: 88 white Sex: 46% male Tumor stage: NR Performance status: NR Other: Income, % high school graduates in census 

tract of residence, comorbidity score, market share of managed care Geographic location: SEER geographic regions Dates: 1993-1999 Healthcare setting: Not stated Results:Number of patients with stage I, II, III CRC with colonoscopy 7 to 18 months after diagnosis was 53%.Increased market share of managed care in a given county resulted in no significant change in colonoscopy during months 7 to 18 after diagnosis. 

General comments: NoRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 5 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 5  Adaptability: 5  Adequacy of risk ad
E-148

Evidence Table 10 – Question 3 (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SLaFata, Johnson, BenMenachem, et al., 2001 #22140 

Quality measure (QM): Percentage of patients with colorectal cancer who received a posttreatment colonoscopy Basis of QM: Clinical practice guideline (ASCRS, Dis Col Rectum 1992; Desch [ASCO] 1999; NCCN, Oncology 1996; Winawer [AGA, ACG, et al.], Gastroenterology 1997) Type of QM:(a) Process; (b) General Outcome to which the QM is linked: No definite link to mortality – possible link to earlier detection of recurrence Intent of QM: Not specified (aim of the study was to examine differences in receipt of colorectal cancer surveillance care by race and income) Definition of denominator/numerator:Denominator: Number of patients diagnosed with colorectal cancer between 1/1/1990 and 12/31/1995 (identified in the tumor registry maintained by a large multispecialty group in Southeast Michigan) who were also enrolled in the affiliated HMO at the time of diagnosis, aged ≥40 at time of diagnosis, treated with curative intent (as determined by ICD-9 and CPT procedure codes for endoscopic or surgical resection). Exclusion criteria: Ulcerative colitis, Crohn’s disease, familial polyposis or IBD (by diagnostic encounter ICD-9 codes). Numerator: Number of above patients who received a colonoscopy posttreatment (defined as 2 months to 8 years postresection with curative intent) Data sources: Tumor registry of a large multispecialty practice, health plan membership files, HMO administrative and claims databases (including encounter, hospitalization, pharmacy) state vital 

Study population: N: 251 Age: Mean 65 Race: 157 white, 94 minority Sex: 62% male Tumor stage: I 28%, II 36%, III 36% Performance status: NR Geographic location: Southeast Michigan 

Dates: 1/1/1990-12/31/1995 for diagnosis plus 18 months of followup to 12/31/1997 Healthcare setting: HMO Results:Whites were more likely than non-whites to receive a colon exam. The likelihood of receiving a colon exam also increased with median household income. 

General comments: NoRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 4 - Usability: 4 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 4  Adaptability: 3  Adequacy of risk ad
E-149

Evidence Table 10 – Question 3 (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

statistics, SSA Death Index Recommended frequency of data collection: No recommendation 

Retchin and Brown, 1990 #7970 

Quality measure (QM): Percentage of colorectal cancer patients having a surveillance colonoscopy (or sigmoidoscopy) after surgery Basis of QM: Other (physician advisory panels were selected to develop quality-of-care criteria for the outpatient detection, surgical management, and postoperative followup of colorectal cancer) Type of QM:(a) Process (b) General Outcome to which the QM is linked: None stated Intent of QM: Not specified (aim of the study was to compare the quality of care received by Medicare patients with colorectal cancer in HMOs vs. fee-forservice [FFS] plans) Definition of denominator/numerator:Denominator: Number of Medicare patients with colorectal cancer from either an HMO or FFS plan who had undergone surgery. Numerator: Number of above who had a colonoscopy or sigmoidoscopy within 6 months after surgery. Data sources: Medicare claims files and HMO plan data Recommended frequency of data collection: No recommendation 

Study population: N: 119 HMO and 137 FFS patients Age: 72.3 years for HMO, 70.1 years for FFS Race: 95.2% white in HMO, 86.7% white in FFS Sex: 41.3% female in HMO, 50.6% female in FFS Tumor stage: All Performance status: NR Geographic location: “4 geographic areas” Dates: January 1983-March 1986. Healthcare setting: Hospitals/HMOs Results:22.7% of HMO patients and 22.6% of FFS patients had a colonoscopy in the first 6 months after surgery. 

General comments:

The data for this paper cThis paper has a numbemeasures that physician for study, but none of theare debatable as to whe

determinants of quality. includes: questions ask(statement of the presenfamily history, statement physical exam (abdominfecal occult blood test), lchemistries, CEA, LFTs)

endoscopy), procedures (CEA, hemoccult, abdomcolonoscopy, chest X-racolonoscopy seems to hand so was included herThe time from symptoms cancer was also evaluatthe HMO patients. Rating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 4 - Usability: 5 - Scientific acceptability  Precise specificatio Reliability: 5  Validity: 4  Adaptability: 5  Adequacy of risk ad
E-150

Evidence Table 10 – Question 3 (Published Literature) – continued Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality SRulyak, Mandelson, Brentnall, et al., 2004 #31170 

Quality measure (QM): Percentage of patients with local or regional CRC who had colon surveillance after diagnosis Basis of QM: Clinical practice guideline (Fleischer [AGA/ASGE], JAMA 1989; Desch [ASCO], J Clin Onc 1999; NCCN 1996; Can Soc of Surg Oncol and the Can Soc of Colon and Rectal Surgeons, Can J Surg 1997; ACS/AGA, CA Cancer Journal for Clinicians 1997; Winawer, Gastroenterol 1997; Winawer, Gastroenterol 2003; ASCO, J Clin Oncol 2000) Type of QM:(a) Process (b) General Outcome to which the QM is linked: No definite link to mortality – possible link to earlier detection of recurrence Intent of QM: Not specified (aim of the study was to investigate the utilization of endoscopic surveillance procedures in a population-based cohort of patients with colon or rectal cancer) Definition of denominator/numerator:Denominator: Number of patients who were members of GHC and diagnosed with colon or rectal cancer 1993-1999. Exclusion criteria: Stage IV disease, unknown stage, not treated with resection (surgical or endoscopic), s/p total proctocolectomy, survival < 6 months, IBD. Numerator: Number of the above patients who had colonoscopy or flexible sigmoidoscopy + barium enema (FSBE) after diagnosis. Data sources: Administrative claims data from HMO, HMO pathology database, SEER database for Seattle Recommended frequency of data collection: No recommendation 

Study population: N: 1,002 Age: < 50 8.7% 50-59 13.3% 60-69 24.4% 70-79 3506% ≥ 80 18.0% Race: 92.7% white, 3.5% African-American, 3.8% other Sex: 50.3% M Tumor stage:  0 8.7%  I 32.5%  II 33.8%  III 25.0% Performance status: NR Geographic location: Washington state 

Dates: Diagnosis between 1/1993-12/1999, followup at 18 months and 5 years Healthcare setting: HMO Results:Colon exams were performed in 61% of patients within 18 months of diagnosis, and in 80% within 5 years of diagnosis. Patients over 80 years of age and those with rectal cancer were less likely to undergo surveillance. Patients of higher socioeconomic status, or who were married, were more likely to undergo colon surveillance. There was a trend toward lower utilization of colon surveillance among African-American patients. 

General comments:Indication for colonoscopinvestigation of symptomRating of quality measstudy (scale of 1-5, whe= ideal): - Importance: 4 - Usability: 4 - Scientific acceptability  Precise specificatio Reliability: 3  Validity: 4  Adaptability: 3  Adequacy of risk ad
E-151

Evidence Table 11 – Question 3 (Grey Literature): What quality-of-care measures are available and what evidence is available 

colonoscopic surveillance for colorectal cancer? Study Characteristics of Quality Measure Comments/Quality Scoring 

Institute of Medicine report, 2005 #36680 

Quality measure (QM): Followup colonoscopy after treatment for Stage I to Stage III colorectal cancer Basis of QM: Clinical practice guideline (National Comprehensive Cancer Network [NCCN] guidelines) Type of QM:(a) Process (b) General Outcome to which the QM is linked: reduction in CRC mortality rate Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of stage I to stage III colorectal cancer cases. Numerator: Number of above cases with a colonoscopy within 1 year of surgery. Data sources: Surveillance, Epidemiology, and End Results Program (SEER)-Medicare dataset, special studies of medical records Recommended frequency of data collection: No recommendation 

General comments:Local practice patterns and patient-related factors afprocedures. Key references cited: Cooper et al., 2000 Fisher et al., 2003 NCCN, 2004 Rulyak et al., 2004 Winawer et al., 2003 Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-152

Evidence Table 11 – Question 3 (Grey Literature) – continued Study Characteristics of Quality Measure Comments/Quality Scoring National Committee for Quality Assurance, 2005 #36580 

Quality measure (QM): Percent of colon cancer cases (all stages) that received a followup colonoscopy within 36 months of surgical treatment Basis of QM: Clinical practice guideline (National Comprehensive Cancer Network [NCCN] guidelines) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Improved health status Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of patients with colon cancer surgery in a 12-month period. Numerator: Number of above patients who received a colonoscopy exam within 36 months of surgical treatment. Data sources: Cancer registry, claims data Recommended frequency of data collection: No recommendation 

General comments: None Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-153

Evidence Table 11 – Question 3 (Grey Literature) – continued Study Characteristics of Quality Measure Comments/Quality Scoring National Committee for Quality Assurance, 2005 #36580 

Quality measure (QM): Percentage of rectal cancer cases (all stages) that received a postsurgical endoscopic exam within 12 months postsurgery Basis of QM: Clinical practice guideline (National Comprehensive Cancer Network [NCCN] guidelines) Type of QM:(a) Process (b) General Outcome to which the QM is linked: Improved health status Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of patients with rectal cancer surgery (all stages) over a 12-month period. Numerator: Number of above who received an endoscopic exam within 12 months postsurgical treatment. Data sources: Cancer registry, claims data Recommended frequency of data collection: No recommendation 

General comments:Long-term followup may not be feasible. Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-154

Evidence Table 12 – Question 4 (Published Literature): What measures are available and what evidence is available for meas

adequacy and completeness of documentation of pathology, operative, and chemotherapy reports? Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Wei, Miller, Woosley, et al., 2004 #31910

Quality measure (QM): Adherence to individual Association of Directors of Anatomic and Surgical Pathology (ADASP) recommended items in pathology reports Basis of QM: Clinical practice guideline (Association of Directors of Anatomic and Surgical Pathology [ADASP] recommendations) Type of QM:(a) Process (b) Technical Outcome to which the QM is linked:Intent of QM: Quality improvement (aims of the study were to describe the completeness of pathology reporting for colon carcinoma, evaluate 

potential variation in reporting by laboratory type or hospital case volume, and identify areas for improvement using the recommendations of the ADASP) Definition of denominator/numerator:

Denominator: Number of subjects with surgically resected colon carcinoma of tumor stages T2-T4 with available surgical pathology reports Numerator: (a) Number of above subjects with reports that described how specimens were received 

(b) Number of above subjects with reports that 

described how specimen was identified 

(c) Number of above subjects with reports that described part included (d) Number of above subjects with reports that described tumor site (e) Number of above subjects with reports that described proximity to nearest margin (f) Number of above subjects with reports that described macroscopic subtype (g) Number of above subjects with reports that described tumor dimensions

Study population: N: 438 reports  20 Community hospital labs: 264 reports  5 Teaching hospital labs: 115 reports  6 Contract hospital labs: 59 reports 6 Low-volume institutions: 31 reports 9 Medium-volume institutions 126 reports 10 High-volume institutions: 222 reports Geographic location: North Carolina Dates: 1997-2000 Healthcare setting: Pathology laboratories: contract, teaching hospitals, community hospital Results: (Global scores were not reported) Low-volume hospital labs were less likely to report some items compared with high-volume labs 

Hospital 

Macro- Volume scopic description High (n = 222) Medium (n = 126) Low (n = 31) How specimen was received 

76.0 67.5 45.2 

Specimen identified 73.9 76.2 32.3 Part included 100 100 100 Tumor site 100 100 93.6 Proximity to nearest margin 

96.4 91.3 93.6 

(Data table continued on next page)

General comments:Colon and rectal carcinomseparated. Labs may be reporting. When quality iessential to understand if used in the albs being ev

elements of the protocol cthe entire protocol, and itleverage points could be Rating of quality measustudy (scale of 1-5, wherideal): - Importance: 5 - Usability: 5 - Scientific acceptability (f Precise specification Reliability: 5  Validity: 5  Adaptability: 5  Adequacy of risk adj
E-155

Evidence Table 12 – Question 4 (Published Literature) – continued 

Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

(h) Number of above subjects with reports that described macroscopic depth of penetration (i) Number of above subjects with reports that described appearance of serosa adjacent to the tumor (j) Number of above subjects with reports that described status of residual bowel (k) Number of above subjects with reports that described histologic type (l) Number of above subjects with reports that described histologic grade (m) Number of above subjects with reports that described depth of infiltration (n) Number of above subjects with reports that described lymph node metastases (o) Number of above subjects with reports that described involvement of margins Data sources: Pathology reports Frequency of data collection: No 

recommendation 

Hospital 

Macro- Volumescopic description High (n = 222) Medium (n = 126) Low (n = 31) Macroscopic subtype 

99.6 100 100 

Tumor dimensions 94.6 96.0 87.1 Macroscopic depth of penetration 

83.8 89.6 64.5 

Appearance of serosa adjacent to tumor 

53.9 57.9 51.6 

Appearance of residual bowel 

72.1 78.6 71.0 

Histologic information Histologic type 100 100 100 Histologic grade 97.7 99.2 96.8 Depth of infiltration 97.3 98.4 96.8 Lymph node metastases 99.1 99.2 100 Involvement of margins 97.3 93.7 64.5 

Community labs were less likely to report several items compared with contract or medical schoolaffiliated labs. (continued on next page)
E-156

Evidence Table 12 – Question 4 (Published Literature) – continued 

Study Characteristics of Quality Measure Testing of Quality Measure Comments/Quality S

Lab Type 

Macroscopic description 

Teaching (n = 115) 

Contract (n = 59) 

Community (n = 264) 

How specimen received 

68.7 54.2 71.5 

Specimen identified 83.5 72.9 65.9 Part included 100 100 100 Tumor site 100 98.3 99.2 Proximity to nearest margin 

97.4 88.1 93.2 

Macroscopic subtype 

100 96.6 99.6 

Tumor dimensions 97.4 91.5 93.2 Macroscopic depth of penetration 

90.4 66.1 81.4 

Appearance of serosa adjacent to tumor 

57.4 39.0 48.5 

Appearance of residual bowel 

76.5 67.8 73.1 

Histologic information Histologic type 100 100 100 Histologic grade 98.3 96.6 98.1 Depth of infiltration 99.1 100 97.0 Lymph node metastases 100 100 98.9 Involvement of margins 95.7 96.6 92.4 
E-157

Evidence Table 13 – Question 4 (Grey Literature): What measures are available and what evidence is available for measures t

and completeness of documentation of pathology, operative, and chemotherapy reports? Study Characteristics of Quality Measure Comments/Quality Scoring 

Colon Cancer Workgroup, 2003 #36650 

Quality measure (QM): Pathology reporting of the surgical specimen is 

concordant with the guidelines of the College of American Pathologists (CAP), including reporting of number of nodes resected, number of nodes positive, tumor characteristics (grade, depth of invasion), mucosal and radial margins Basis of QM: Clinical practice guideline (National Comprehensive Cancer Network [NCCN] guidelines) Type of QM:(a) Process (b) Technical Outcome to which the QM is linked: Pathology staging (and subsequent treatment planning and survival) Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of patients having colon cancer surgery. Numerator: Number of above patients with pathology reports in concordance with CAP guidelines. Data sources: Pathology, surgical reports Recommended frequency of data collection: No recommendation 

General comments: None Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-158

Evidence Table 13 – Question 4 (Grey Literature) – continued Study Characteristics of Quality Measure Comments/Quality Scoring Institute of Medicine, 2005 #36680 

Quality measure (QM): Pathology reports on colorectal cancer surgical 

specimens that include College of American Pathologists (CAP) data elements as 

required by the American College of Surgeons’ Commission on Cancer Basis of QM: Clinical practice guideline (CAP, Commission on Cancer) Type of QM:(a) Structure (b) General Outcome to which the QM is linked: Pathology staging (and subsequent treatment planning and survival) Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of pathology reports on colorectal cancer surgical specimens. Numerator: Number of above reports that include CAP data elements as required by the Commission on Cancer. Data sources: CAP, Commission on Cancer, baseline special studies 

Recommended frequency of data collection: No recommendation 

General comments:Measurement goal should be 100 percent. Findings aggregate and individually by pathology laboratory. Key references cited: Compton (CAP), 2004 Commission on Cancer, 2003 Compton, 2003 Stocchi et al., 2001 Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-159

Evidence Table 13 – Question 4 (Grey Literature) – continued Study Characteristics of Quality Measure Comments/Quality Scoring Institute of Medicine, 2005 #36680 

Quality measure (QM): Colorectal cancer cases in which pathologic staging preceded chemotherapy and radiation treatment Basis of QM: Clinical practice guideline (American College of Radiology, American Society of Clinical Oncology, Commission on Cancer, National Comprehensive Cancer Network) Type of QM:(a) Process (b) General Outcome to which the QM is linked:Intent of QM: Quality improvement Definition of denominator/numerator:Denominator: Number of new colorectal cancer cases with chemotherapy or radiation treatment. Numerator: Number of above cases with medical chart documentation of pathologic stage before chemotherapy or radiation is initiated. Data sources: Medical records Recommended frequency of data collection: No recommendation 

General comments: None Key references cited: Compton (CAP), 2004 NCCN, 2004 American College of Radiology, 1999 Commission on Cancer, 2003 Greene et al., 2002 Rating of quality measure as presented (scale of moderate, 5 = ideal): - Importance: 5 - Usability: 5 - Scientific acceptability (two criteria):  Precise specifications: 4  Adaptability: 4 
E-160

References Cited in the Evidence Tables 

Agarwal N, Leighton L, Mandile MA, et al. Outcomes of 

surgery for colorectal cancer in patients age 80 years and 

older. Am J Gastroenterol 1990;85(9):1096-101. 

Ayanian JZ, Zaslavsky AM, Fuchs CS, et al. Use of 

adjuvant chemotherapy and radiation therapy for colorectal 

cancer in a population-based cohort. J Clin Oncol 

2003;21(7):1293-300. 

Baig N, Myers RE, Turner BJ, et al. Physician-reported 

reasons for limited follow-up of patients with a positive 

fecal occult blood test screening result. Am J Gastroenterol 

2003;98(9):2078-81. 

Baxter N, Virnig D, Rothenberger D, et al. Lymph Node 

Evaluation in Colorectal Cancer Patients: A Population 

Based Study. J Natl Cancer Inst 2005;97(3):219-25. 

Baxter NN, Rothenberger DA, Morris AM, et al. Adjuvant 

radiation for rectal cancer: do we measure up to the 

standard of care? An epidemiologic analysis of trends over 

25 years in the United States. Dis Colon Rectum 

2005;48(1):9-15. 

Beart RW, Steele GD Jr, Menck HR, et al. Management 

and survival of patients with adenocarcinoma of the colon 

and rectum: a national survey of the Commission on 

Cancer. J Am Coll Surg 1995;181(3):225-36. 

Chiaverini L, Fulton JP, Darcy DM. Progress in the control 

of colorectal cancer in Rhode Island, 1987-2000. Med 

Health R I 2002;85(7):226-8. 

Coburn MC, Pricolo VE, Soderberg CH. Factors affecting 

prognosis and management of carcinoma of the colon and 

rectum in patients more than eighty years of age. J Am Coll 

Surg 1994;179(1):65-9. 

Coia L, Wizenberg M, Hanlon A, et al. Evaluation and 

treatment of patients receiving radiation for cancer of the 

rectum or sigmoid colon in the United States: results of the 

1988-1989 Patterns of Care Study process survey. J Clin 

Oncol 1994;12(5):954-9. 

Colon Cancer Workgroup. Quality of Cancer Care 

Evaluation and Improvement: Commission on Cancer, 

American College of Surgeons, American Cancer Society 

and Tenet Health Care System, December, 2003. 

Electronically received February 7, 2005. 

Cooper GS, Yuan Z, Chak A, et al. Geographic and patient 

variation among Medicare beneficiaries in the use of 

follow-up testing after surgery for nonmetastatic colorectal 

carcinoma. Cancer 1999;85(10):2124-31. 

Cooper GS, Yuan Z, Chak A, et al. Patterns of endoscopic 

follow-up after surgery for nonmetastatic colorectal cancer. 

Gastrointest Endosc 2000;52(1):33-8. 

Cooper GS, Yuan Z, Landefeld CS, et al. Surgery for 

colorectal cancer: Race-related differences in rates and 

survival among Medicare beneficiaries. Am J Public Health 

1996;86(4):582-6. 

Demissie K, Oluwole OO, Balasubramanian BA, et al. 

Racial differences in the treatment of colorectal cancer: a 

comparison of surgical and radiation therapy between 

Whites and Blacks. Ann Epidemiol 2004;14(3):215-21. 

Dimick JB, Cowan JA Jr, Upchurch GR Jr, et al. Hospital 

volume and surgical outcomes for elderly patients with 

colorectal cancer in the United States. J Surg Res 

2003;114(1):50-6. 

Dominitz JA, Samsa GP, Landsman P, et al. Race, 

treatment, and survival among colorectal carcinoma 

patients in an equal-access medical system. Cancer 

1998;82(12):2312-20. 

Galvis CO, Raab SS, D'Amico F, et al. Pathologists' 

assistants practice: a measurement of performance. Am J 

Clin Pathol 2001;116(6):816-22. 

Govindarajan R, Shah RV, Erkman LG, et al. Racial 

differences in the outcome of patients with colorectal 

carcinoma. Cancer 2003;97(2):493-8. 

Harewood GC, Sharma VK, de Garmo P. Impact of 

colonoscopy preparation quality on detection of suspected 

colonic neoplasia. Gastrointest Endosc 2003;58(1):76-9. 

Harmon J, Tang D, Gordon T, et al. Hospital Volume Can 

Serve as a Surrogate for Surgeon Volume for Achieving 

Excellent Outcomes in Colorectal Resection. Ann Surg 

1999;230(3):404-13. 

Haseman JH, Lemmel GT, Rahmani EY, et al. Failure of 

colonoscopy to detect colorectal cancer: evaluation of 47 

cases in 20 hospitals. Gastrointest Endosc 1997;45(6):451-

5. 

Hodgson DC, Zhang W, Zaslavsky AM, et al. Relation of 

hospital volume to colostomy rates and survival for patients 

with rectal cancer. J Natl Cancer Inst 2003;95(10):708-16. 

Hyman N, Labow SB, Vermont Chapter of the American 

College of Surgeons. The Vermont colorectal cancer 

project: self-portrait. Arch Surg 2002;137(4):413-6. 
E-161

Institute of Medicine. Asessing the Quality of Cancer Care: 

An Approach to Measurement in Georgia. Committee on 

Assessing Improvements in Cancer Care in Georgia. 

National Cancer Policy Board. The National Academies 

Press: Washington, DC; 2005. 

Jessup JM, McGinnis LS, Steele GD Jr, et al. The National 

Cancer Data Base. Report on colon cancer. Cancer. 

1996;78(4):918-26. 

Keating N, Landrum M, Meara E, et al. Do Increases in the 

Market Share of Managed Care Influence Quality of Care 

in the Fee-For-Service Sector? J Natl Cancer Inst 

2005;97(4):257-64. 

Kline RW, Smith AR, Coia LR, et al. Treatment planning 

for adenocarcinoma of the rectum and sigmoid: a patterns 

of care study. PCS Committee. Int J Radiat Oncol Biol 

Phys 1997;37(2):305-11. 

Ko CY, Chang JT, Chaudhry S, et al. Are high-volume 

surgeons and hospitals the most important predictors of inhospital outcome for colon cancer resection? Surgery 

2002;132(2):268-73. 

LaFata EJ, Johnson C, Ben-Menachem T, et al. 

Sociodemographic differences in the receipt of colorectal 

cancer surveillance care following treatment with curative 

intent. Med Care 2001;39(4):361-72. 

Levin B, Hess K, Johnson C. Screening for colorectal 

cancer. A comparison of 3 fecal occult blood tests. Arch 

Intern Med 1997;157(9):970-6. 

Mahoney T, Kuo YH, Topilow A, et al. Stage III colon 

cancers: why adjuvant chemotherapy is not offered to 

elderly patients. Arch Surg 2000;135(2):182-5. 

Mandel JS, Church TR, Bond JH, et al. The effect of fecal 

occult-blood screening on the incidence of colorectal 

cancer. N Engl J Med 2000;343(22):1603-7. 

Marble K, Banerjee S, Greenwald L. Colorectal carcinoma 

in young patients. J Surg Oncol 1992;51(3):179-82. 

Merrill RM, Brown ML, Potosky AL, et al. Survival and 

treatment for colorectal cancer Medicare patients in two 

group/staff health maintenance organizations and the feefor-service setting. Med Care Res Rev 1999;56(2):177-96. 

Minnesota Gastroenterology PA. 2004 Colonoscopy 

Quality Assurance Initiative. Electronically received: 

January 26, 2005. 

Minsky BD, Coia L, Haller DG, et al. Radiation therapy for 

rectosigmoid and rectal cancer: results of the 1992-1994 

Patterns of Care process survey. J Clin Oncol 

1998;16(7):2542-7. 

Morris AM, Billingsley KG, Baxter NN, et al. Racial 

disparities in rectal cancer treatment: a population-based 

analysis. Arch Surg 2004;139(2):151-5; discussion 156. 

Myers RE, Turner B, Weinberg D, et al. Impact of a 

physician-oriented intervention on follow-up in colorectal 

cancer screening. Prev Med 2004;38(4):375-81. 

National Committee for Quality Assurance (NCQA). Final 

List of Colon Cancer Measures Based on Domain Expert 

Review. Measures developed by Kaiser Permanente, 

Internountain Health Care and NCQA. Electronically 

delivered February 8, 2005. 

National Quality Measures Clearinghouse (NQMC). Intraprocedure colonosopy complication rate: percentage of 

patients who developed one or more intrao-procedure 

complications. Available at: 

http://www.qualitymeasures.ahrq.gov/summary/summary.a

spx?doc_id=4868. Accessed: March 21, 2005. 

Nelson DB, McQuaid KR, Bond JH, et al. Procedural 

success and complications of large-scale screening 

colonoscopy. Gastrointest Endosc 2002;55(3):307-14. 

Neugut AI, Fleischauer AT, Sundararajan V, et al. Use of 

adjuvant chemotherapy and radiation therapy for rectal 

cancer among the elderly: a population-based study. J Clin 

Oncol 2002;20(11):2643-50. 

Newcomb PA, Carbone PP. Cancer treatment and age: 

patient perspectives. J Natl Cancer Inst 1993;85(19):1580-

4. 

O'Connell JB, Maggard MA, Liu JH, et al. Are survival 

rates different for young and older patients with rectal 

cancer? Dis Colon Rectum 2004;47(12):2064-9. 

O'Connell JB, Maggard MA, Liu JH, et al. Do young colon 

cancer patients have worse outcomes? World J Surg 

2004;28(6):558-62. 

Oliveria SA, Yood MU, Campbell UB, et al. Treatment and 

referral patterns for colorectal cancer. Med Care. 

2004;42(9):901-6. 

Potosky AL, Harlan LC, Kaplan RS, et al. Age, sex, and 

racial differences in the use of standard adjuvant therapy 

for colorectal cancer. J Clin Oncol 2002;20(5):1192-202. 

Purves H, Pietrobon R, Hervey S, et al. Relationship 

between surgeon caseload and sphincter preservation in 

patients with rectal cancer. Dis Colon Rectum 

2005;48(2):195-202; discussion 202-4. 

Rabeneck L, Davila JA, Thompson M, et al. Outcomes in 

elderly patients following surgery for colorectal cancer in 

the veterans affairs health care system. Aliment Pharmacol 

Ther 2004;20(10):1115-24. 
E-162

Read TE, Myerson RJ, Fleshman JW, et al. Surgeon 

specialty is associated with outcome in rectal cancer 

treatment. Dis Colon Rectum 2002;45(7):904-14. 

Retchin SM, Brown B. Management of colorectal cancer in 

Medicare health maintenance organizations. J Gen Intern 

Med 1990;5(2):110-4. 

Roetzheim RG, Pal N, Gonzalez EC, et al. Effects of health 

insurance and race on colorectal cancer treatments and 

outcomes. Am J Public Health 2000;90(11):1746-54. 

Rogers SO, Ray WA, Smalley WE. A population-based 

study of survival among elderly persons diagnosed with 

colorectal cancer: does race matter if all are insured? 

(United States). Cancer Causes Control 2004;15(2):193-9. 

Rulyak SJ, Mandelson MT, Brentnall TA, et al. Clinical 

and sociodemographic factors associated with colon 

surveillance among patients with a history of colorectal 

cancer. Gastrointest Endosc 2004;59(2):239-47. 

Schrag D, Cramer LD, Bach PB, et al. Age and adjuvant 

chemotherapy use after surgery for stage III colon cancer. J 

Natl Cancer Inst 2001;93(11):850-7. 

Schrag D, Cramer LD, Bach PB, et al. Influence of hospital 

procedure volume on outcomes following surgery for colon 

cancer. JAMA 2000;284(23):3028-35. 

Schrag D, Gelfand SE, Bach PB, et al. Who gets adjuvant 

treatment for stage II and III rectal cancer? Insight from 

surveillance, epidemiology, and end results--Medicare. J 

Clin Oncol 2001;19(17):3712-8. 

Schrag D, Panageas KS, Riedel E, et al. Hospital and 

surgeon procedure volume as predictors of outcome 

following rectal cancer resection. Ann Surg 

2002;236(5):583-92. 

Schrag D, Panageas KS, Riedel E, et al. Surgeon volume 

compared to hospital volume as a predictor of outcome 

following primary colon cancer resection. J Surg Oncol 

2003;83(2):68-78; discussion 78-9. 

Shehadeh I, Rebala S, Kumar R, et al. Retrospective 

analysis of missed advanced adenomas on surveillance 

colonoscopy. Am J Gastroenterol 2002;97(5):1143-7. 

Shields HM, Weiner MS, Henry DR, et al. Factors that 

influence the decision to do an adequate evaluation of a 

patient with a positive stool for occult blood. Am J 

Gastroenterol 2001;96(1):196-203. 

Simons AJ, Ker R, Groshen S, et al. Variations in treatment 

of rectal cancer: the influence of hospital type and caseload. 

Dis Colon Rectum 1997;40(6):641-6. 

Steele GD Jr. The National Cancer Data Base report on 

colorectal cancer. Cancer 1994;74(7):1979-89. 

Steele G. National Cancer Database: Annual review of 

patient care, 1994. Atlanta, Ga. American Cancer Society. 

1994;24-42. 

Steele G, Jessup J. National Cancer Database: Annual 

review of patient care, 1995. Atlanta, Ga. American Cancer 

Society. 1995;66-83. 

Sundararajan V, Grann VR, Jacobson JS, et al. Variations 

in the use of adjuvant chemotherapy for node-positive 

colon cancer in the elderly: a population-based study. 

Cancer J 2001;7(3):213-8. 

Temple LK, Hsieh L, Wong WD, et al. Use of surgery 

among elderly patients with stage IV colorectal cancer. J 

Clin Oncol 2004;22(17):3475-84. 

Tropman S, Hatzell T, Paskett E, et al. Colon cancer 

treatment in rural North and South Carolina. Cancer Detect 

Prev 1999;23(5):428-34. 

Ure T, Dehghan K, Vernava AM 3rd, et al. Colonoscopy in 

the elderly. Low risk, high yield. Surg Endosc 

1995;9(5):505-8. 

Vernon SW, Hughes JI, Heckel VM, et al. Quality of care 

for colorectal cancer in a fee-for-service and health 

maintenance organization practice.[erratum appears in 

Cancer 1994 Dec 15;74(12):3249]. Cancer 

1992;69(10):2418-25. 

Weaver P, Harrison B, Eskander G, et al. Colon cancer in 

blacks: a disease with a worsening prognosis. J Natl Med 

Assoc 1991;83(2):133-6. 

Wei JT, Miller EA, Woosley JT, et al. Quality of colon 

carcinoma pathology reporting: a process of care study. 

Cancer 2004;100(6):1262-7. 

Whittle J, Steinberg EP, Anderson GF, et al. Results of 

colectomy in elderly patients with colon cancer, based on 

Medicare claims data. Am J Surg 1992;163(6):572-6. 

Wudel LJ Jr, Chapman WC, Shyr Y, et al. Disparate 

outcomes in patients with colorectal cancer: effect of race 

on long-term survival. Arch Surg 2002;137(5):550-4; 

discussion 554-6. 
F-1 

Appendix F: Criteria Used To Assign Ratings to 

Quality Measures

(1) Assigning the rating for the Importance of a measure: 

We accomplished this in two steps. 

Step 1: We calculated the average of the scores assigned for each of the two criteria, Importance 

and Usability, for a measure for a given study. 

Step 2: We assigned the highest average score allotted to the measure amongst all the studies 

that utilized it as the “I” rating for the measure. 

(2) Assigning the rating for the Scientific Acceptability of a measure: 

We accomplished this in two steps. 

Step 1: We calculated the average of the scores assigned for each of the five criteria for 

scientific acceptability (precise specifications, validity, reliability, adaptability, adequacy of risk 

adjustment) for a measure for a given study. 

Step 2: The highest average score for scientific acceptability, amongst all the studies that 

utilized it, was assigned as the “S” rating for the measure. 

(3) Assigning the rating for the Testing of a measure: 

We accomplished this in a multi-step process. This rating was based on (a) the number of 

studies that utilized this measure and (b) the extent to which these studies were scientifically 

acceptable (their “S” rating). Table 1 below summarizes the criteria used to combine the two 

concepts. 

Table 1. Assigning the “T” rating 

 Considering all the studies utilizing the quality measure, if: Then the assigned T rating was: 

The highest average scientific acceptability score of the measure in all the studies was 1 

or 2 

1 

The highest average scientific acceptability score of the measure was 3, and there was 

only one study with that score 

2 

The highest average scientific acceptability score of the measure in two or more studies 

was 3 

3 

The highest average scientific acceptability score of the measure was 4, and there was 

only one study with that score 

4 

The highest average scientific acceptability score of the measure was 4, and there were 

two or more studies with that score, OR the highest scientific acceptability score of the 

measure in at least one study was 5 

5 

In the following example, the quality measure “percentage of patients with stage III colon cancer 

receiving chemotherapy”, was assigned a hypothetical quality rating of I5S4T4 because of the 

scores attached to the measure in the two studies that the measure was abstracted from (Table 2): 
F-2 

Table 2. Example of Quality Measure Rating Algorithm* Percentage of patients with stage III colon cancer receiving chemotherapy 

important usable 

Importance 

rating 

or “I” 

average 

precise 

specs reliable valid adaptable 

risk 

adjusted

Scientific 

acceptability or “S” rating

average 

Study 1 5 5 5 5 5 5 5 1 4 Study 2 4 4 4 4 4 3 3 3 3 

* The data in this table are for illustrative purposes only and do not reflect actual ratings. In this example, the overall Importance rating for “percent of patients with stage III colon cancer receiving chemrating, would be a “5” because “5” represents the highest average score for the measure across both studies (stuinvolved. The rating for Scientific Acceptability for this measure, or the “S” rating, would be a “4” because, agaihighest average score across both studies involved. The Testing, or “T” rating, would be a “4” because “4” was the highest average Scientific Acceptability score, anstudy that attained a Scientific Acceptability score of ”4” (the other study had a Scientific Acceptability rating of