Federal Coordinating Council for
 

Comparative Effectiveness Research
 

Report to the President and the Congress
 

June 30, 2009
 

1 
 

 

 

 

 

 

 

 

 

 

 

 

 

TABLE OF CONTENTS 

Executive Summary 3 

1. Introduction 9 

2. Vision and Council Objectives 15 

3. Comparative Effectiveness Research (CER) Definition and Criteria 16
 

4. Importance of Priority Populations and Sub-Group Analysis 17 

5. Strategic Framework for CER 25
 

6. Current CER Inventory and CER Data Infrastructure 28 

7. Priority-Setting Process 42 

8. Priority Recommendations for Office of Secretary CER Funds 43 

9. Longer-Term Outlook and Next Steps 48 

Appendices 

A. Summary of Major Themes from Public Comments and Listening Sessions 52 

B. Summary of the Council’s Meetings and Deliberations 59 

C. Preliminary Data Infrastructure and CER by Condition 64 

D. Council Membership and Staff Support 69 

E. ARRA Statute Relating to Council and Comparative Effectiveness Research 70 

2 
 

 

 

 

 

 

 

 

 

 

EXECUTIVE SUMMARY 

Across the United States, clinicians and patients confront important health care decisions without 

adequate information. What is the best pain management regimen for disabling arthritis in an elderly 

African-American woman with heart disease? For neurologically impaired children with special health 

care needs, what care coordination approach is most effective at preventing hospital readmissions? 

What treatments are most beneficial for patients with depression who have other medical illnesses? 

Can physicians tailor therapy to specific groups of patients using their history or special diagnostic 

tests? What interventions work best to prevent obesity or tobacco use? Unfortunately, the answer to 

these types of comparative, patient-centered questions in health care is often, “We don’t really know.” 

Thousands of health care decisions are made daily; patient-centered comparative effectiveness research 

focuses on filling gaps in evidence needed by clinicians and patients to make informed decisions. 

Physicians and other clinicians see patients every day with common ailments, and they sometimes are 

unsure of the best treatment because limited or no evidence comparing treatment options for the 

condition exists. As a result, patients seen by different clinicians may get different treatments and 

unknowingly be receiving less effective care. Patients and their caregivers search in vain on the 

Internet or elsewhere for evidence to help guide their decisions. They often fail to find this 

information either because it does not exist or because it has never been collected and synthesized to 

inform patients and/or their caregivers in patient-friendly language. When they do find information, it 

may be informed by marketing objectives, not the best evidence.1 

Due to astonishing achievements in biomedical science, clinicians and patients often have a plethora of 

choices when making decisions about diagnosis, treatment, and prevention, but it is frequently unclear 

which therapeutic choice works best for whom, when, and in what circumstances. The purpose of 

comparative effectiveness research (CER) is to provide information that helps clinicians and patients 

choose which option best fits an individual patient's needs and preferences. It also can inform the 

health choices of those Americans who cannot or choose not to access the health care system.2 

Clinicians and patients need to know not only that a treatment works on average but also which 

interventions work best for specific types of patients (e.g. the elderly, racial and ethnic minorities). 

Policy makers and public health professionals need to know what approaches work to address the 

prevention needs of those Americans who do not access health care. This information is essential to 

translating new discoveries into better health outcomes for Americans, accelerating the application of 

beneficial innovations, and delivering the right treatment to the right patient at the right time. 

Examples of successful CER include summaries of evidence from the Agency for Healthcare Research 

and Quality (AHRQ) on numerous conditions, such as prostate cancer and osteoporosis, as well as the 

National Institutes of Health (NIH) diabetes prevention trial that demonstrated lifestyle change was 

superior to metformin and placebo in preventing onset of type 2 diabetes. Additionally, the Veterans 

Affairs (VA) COURAGE trial demonstrated that patients treated with optimal medical therapy alone 

did just as well as patients who received percutaneous coronary intervention plus medical therapy in 

preventing heart attack and death. These exemplars show the power of CER to inform patient and 

clinician decisions and improve health outcomes. 

Patients increasingly and appropriately want to take responsibility for their care. Therefore we have a 

responsibility to provide comparative information to enable informed decision-making. This patient1 Lee TH, Brennan TA. N Engl J Med. 2002;346:529-531. 2 Green LA, et al. N Engl J Med. 2001; 344:2021-5. 

3 
 

 

 

 

 

centered, pragmatic, “real world” research is a fundamental requirement for improving care for all 

Americans. 

Comparative effectiveness differs from efficacy research because it is ultimately applicable to realworld needs and decisions faced by patients, clinicians, and other decision makers. In efficacy 

research, such as a drug trial for the U.S. Food and Drug Administration (FDA) approval, the question 

is typically whether the treatment is efficacious under ideal, rather than real-world, settings. The 

results of such studies are therefore not necessarily generalizable to any given patient or situation. But 

what patients and clinicians often need to know in practice is which treatment is the best choice for a 

particular patient. In this way, comparative effectiveness is much more patient-centered. Comparative 

effectiveness has even been called patient-centered health research or patient-centered outcomes 

research to illustrate its focus on patient needs. 

The American Recovery and Reinvestment Act (ARRA) provided $1.1 billion for comparative 

effectiveness research. The Act allocated $400 million to the Office of the Secretary in the U.S. 

Department of Health and Human Services (HHS), $400 million to the National Institutes of Health 

(NIH), and $300 million to the HHS Agency for Healthcare Research and Quality. It also established 

the Federal Coordinating Council for Comparative Effectiveness Research (the Council) to foster 

optimum coordination of CER conducted or supported by Federal departments and agencies. 

Furthermore, the legislation indicated that “the Council shall submit to the President and the Congress 

a report containing information describing current Federal activities on comparative effectiveness 

research and recommendations for such research conducted or supported from funds made available 

for allotment by the Secretary for comparative effectiveness research in this Act” by June 30, 2009. 

Transparent, Open Process Seeking Public Input 

From the outset, the Council recognized the importance of establishing a transparent, collaborative 

process for making recommendations and sought the input of the American people on this important 

topic. The Council held three public listening sessions, two in the District of Columbia and one in 

Chicago. The Council also received comments for two months on its public Web site. Importantly, 

the open process allowed the Council to hear from hundreds of diverse stakeholders who represent 

views across the spectrum. Many patients expressed their need for this type of research; one of the 

most emotional and moving testimonies came from the mother of a child with a seizure disorder in 

Chicago who had struggled to find the best treatment for her child. A physician from the American 

Board of Orthopedics summarized many physicians’ testimony by saying, “developing high quality, 

objective information will improve informed patient choice, shared decision-making, and the clinical 

effectiveness of physician treatment recommendations.” The Council heard repeatedly at the listening 

sessions that the Federal Government must use this investment to lay the foundation for informing 

decisions and improving the quality of health care. In addition, the Council posted interim working 

documents for feedback, including the definition of CER, the prioritization criteria, and the strategic 

framework, and modified these based on the feedback. Comments from the listening sessions and via 

the Web site significantly influenced Council discussion and decisions. Indeed, this entire report is 

influenced by the public input—and Appendix A elaborates on the key themes that ran through the 

public comments. 

Vision 

The Council’s vision for the investment in comparative effectiveness research focuses on laying the 

foundation for this type of research to develop and prosper so it can inform decisions by patients and 

clinicians. This research is critical to transforming our health care system to deliver higher quality and 

more value to all Americans. The Council specifically focused on recommendations for use of the 

4 
 

 

 

 

Office of Secretary (OS) funds to fill high priority gaps that were less likely to be funded by other 

organizations and therefore represent unique opportunities for these funds. 

Early in the process, the Council set the following objectives consistent with ARRA: 

1.	 Develop a definition, establish prioritization criteria, create a strategic framework, and identify 

priorities that lay the foundation for CER. 

2.	 Foster optimum coordination of comparative effectiveness research conducted or supported by 

relevant Federal departments. 

3.	 Formulate recommendations for investing the $400 million appropriated to the HHS Office of 

Secretary as part of this Report to Congress. 

Definition and Criteria 

The Council first established a definition, building on previous definitions, for comparative 

effectiveness research: 

Comparative effectiveness research is the conduct and synthesis of research comparing the 

benefits and harms of different interventions and strategies to prevent, diagnose, treat and 

monitor health conditions in “real world” settings. The purpose of this research is to improve 

health outcomes by developing and disseminating evidence-based information to patients, 

clinicians, and other decision-makers, responding to their expressed needs, about which 

interventions are most effective for which patients under specific circumstances. 

 To provide this information, comparative effectiveness research must assess a 

comprehensive array of health-related outcomes for diverse patient populations and subgroups. 

 Defined interventions compared may include medications, procedures, medical and 

assistive devices and technologies, diagnostic testing, behavioral change, and delivery 

system strategies. 

 This research necessitates the development, expansion, and use of a variety of data 

sources and methods to assess comparative effectiveness and actively disseminate the 

results. 

The Council needed explicit criteria to make recommendations for priorities. Therefore, the Council’s 

second step was to establish minimum threshold criteria that must be met and prioritization criteria. 

Minimum Threshold Criteria (i.e. must meet these to be considered): 

•	 Included within statutory limits of Recovery Act and the Council’s definition of CER 

•	 Potential to inform decision-making by patients, clinicians, or other stakeholders 

•	 Responsiveness to expressed needs of patients, clinicians, or other stakeholders 

•	 Feasibility of research topic (including time necessary for research) 

The prioritization criteria for scientifically meritorious research and investments are: 

•	 Potential impact (based on prevalence of condition, burden of disease, variability in outcomes, 

costs, potential for increased patient benefit or decreased harm) 

5 
 

 

 

 

•	 Potential to evaluate comparative effectiveness in diverse populations and patient sub-groups 

and engage communities in research 

•	 Uncertainty within the clinical and public health communities regarding management decisions 

and variability in practice 

•	 Addresses need or gap unlikely to be addressed through other organizations 

•	 Potential for multiplicative effect (e.g. lays foundation for future CER such as data 

infrastructure and methods development and training, or generates additional investment 

outside government) 

Importance of Priority Populations and Patient Sub-Groups 

One important consideration for comparative effectiveness research is addressing the needs of priority 

populations and sub-groups, i.e., those often underrepresented in research. The priority populations 

specifically include, but are not limited to, racial and ethnic minorities, persons with disabilities, 

children, the elderly, and patients with multiple chronic conditions. These groups have been 

traditionally under-represented in medical research. 

In addition, comparative effectiveness should complement the trend in medicine to develop 

personalized medicine—the ability to customize a drug and dose based on individual patient and 

disease characteristics. One of the advantages of large comparative effectiveness studies is the power 

to investigate effects at the sub-group level that often cannot be determined in a randomized trial. This 

power needs to be harnessed so personalized medicine and comparative effectiveness complement 

each other. 

Strategic Framework 

After completing the draft definition and criteria for prioritization of potential CER investments, the 

Council recognized the need to develop a strategic framework for CER activity and investments to 

categorize current activity, identify gaps, and inform decisions on high-priority recommendations. 

This framework represents a comprehensive, coordinated approach to CER priorities. It is intended to 

support immediate decisions for investment in CER priorities and to provide a comprehensive 

foundation for longer-term strategic decisions on CER priorities and the related infrastructure. At the 

framework’s core is responsiveness to expressed needs for comparative effectiveness research to 

inform health care decision-making by patients, clinicians, and others in the clinical and public health 

communities. 

Types of CER investments and activities can be grouped into four major categories: 

•	 Research (e.g., comparing medicines for a specific condition or discharge process A to 

discharge process B for readmissions) 

•	 Human and Scientific Capital (e.g., training new researchers to conduct CER, developing 

CER methodology) 

•	 CER Data Infrastructure (e.g., developing a distributed practice-based data network, 

longitudinal linked administrative or Electronic Health Record (EHR) databases, or patient 

registries) 

•	 Dissemination and Translation of CER (e.g., building tools and methods to disseminate 

CER findings to clinicians and patients and translate CER into practice) 

6 
 

 

 

 

Furthermore, investments or activities related to a specific theme can cut across one or more categories 

and may include research, human and scientific capital, CER data infrastructure, and/or translation and 

adoption. These themes could include: 

• Conditions (e.g., cancer, heart failure) 

• Patient populations (e.g., elderly, minorities, children, persons with disabilities) 

• Type of intervention (e.g., devices, behavioral change, delivery system) 

Together, these activities and themes make up the “CER Strategic Framework” (Figure A) 

Figure A 

 

 

Strategic Framework 

Research Human & Dissemination and Scientific Capital CER Data Infrastructure Translation of CER for CER 

Priority Populations 

Priority Conditions 

Types of Interventions 

CrossCutting 

Priority 

Themes 

Specific investments can 

be within a single category 

or be cross-cutting in one 

of the priority themes 

CER Inventory and Priority-Setting Process 

The Council also conducted an inventory of CER and data infrastructure to help identify gaps in the 

current CER landscape. Maintaining that inventory and ongoing evaluation of government and private 

sector (where possible) CER investments and programs across these activities and themes is critical to 

this framework’s value for decision-making. The first draft Federal Government inventory of CER 

and data infrastructure is included in this report, but it is critical to note that evaluation of current 

activities and the identification of gaps in order to inform priority-setting must be iterative and 

continue in the future. 

As noted above, the Council’s priority-setting process was informed by public input, and that input had 

a substantial influence on how the Council formulated its framework and priorities for CER. CER is 

an important mechanism to improve health and continued public input is vital for agenda setting. 

Priority Recommendations 

In developing its recommendations for how to invest the OS ARRA funding of $400 million, the 

Council sought to respond to patient and physician needs for CER, to balance achieving near-term 

results with building longer-term opportunities, and to capture the unique value that the Secretary’s 

ARRA funds could play in filling gaps and building the foundation for future CER. The Council 

recommended that, among the four major activities and three cross-cutting themes in the CER 

7 
 

 

 

 

 

 

 

framework, the primary investment for this funding should be data infrastructure. Data infrastructure 

could include linking current data sources to enable answering CER questions, development of 

distributed electronic data networks and patient registries, and partnerships with the private sector. 

Secondary areas of investment are dissemination and translation of CER findings, priority populations, 

and priority types of interventions. The priority populations identified that could be the focus of crosscutting themes were racial and ethnic minorities, persons with disabilities, persons with multiple 

chronic conditions (including co-existing mental illness), the elderly, and children. CER will be an 

important tool to inform decisions for these populations and reduce health disparities. High-priority 

interventions for OS to consider supporting include medical and assistive devices, procedures/surgery, 

behavioral change, prevention, and delivery systems. For example, behavioral change and prevention 

have the potential to decrease obesity, decrease smoking rates, increase adherence to medical therapies, 

and improve many other factors that determine health. Delivery system interventions, such as 

comparing different discharge and transitions of care processes on hospital readmissions, communitybased care models, or testing the effect of different medical home models on health have substantial 

potential to drive better health outcomes for patients. 

The OS funds may also play a supporting role in research and human and scientific capital. Because 

the Council anticipates that AHRQ, NIH, and VA will likely continue to play a major role in these 

essential activities for the CER enterprise, OS funding would likely only fill gaps in these areas. 

Longer-Term Outlook and Next Steps 

This report and an Institute of Medicine report funded by the Department will inform the prioritysetting process for CER-related funding. The most immediate next step will be the development of a 

specific plan, to be submitted by July 30, 2009, from the Secretary of Health and Human Services for 

the combined $1.1 billion of ARRA CER funding. In addition, an annual report from the Council is 

required under the ARRA legislation. 

It will be important for this funding both to accomplish short-term successes and to build the 

foundation for future CER. The CER activity and investments should be coordinated across the 

Federal Government and avoid duplicative effort. In addition, the funding should complement and link 

to activities and funding in the private sector to maximize the benefits to the American people. 

Clinicians, patients, and other stakeholders greatly need comparative effectiveness research to inform 

health care decisions. One private citizen unaffiliated with any health care group summarized, “It is 

more important than ever to engage in robust research on what treatments work and what do not. 

Doing so empowers doctors and patients, and helps make our practice of medicine more evidencebased.” 

This is a unique opportunity to invest in the fundamental building blocks for transformation of health 

care in the United States to improve the quality and value of health care for all Americans. Physicians 

and patients deserve the best patient-centered evidence on what works, so Americans can have the 

highest quality care and achieve the best possible outcomes. 

8 
 

 

 

 

I. INTRODUCTION 

The American Recovery and Reinvestment Act of 2009 (ARRA), Pub. L. 111-5, made available 

to the Department of Health and Human Services $1.1 billion for comparative effectiveness 

research (CER). Of this amount, $300 million was allocated to the Agency for Healthcare 

Research and Quality (AHRQ), $400 million to the National Institutes of Health (NIH), and $400 

million was allocated to the Office of the Secretary (OS) for disbursement. 

These and all Federal agencies distributing ARRA funds must do so in accordance with all 

nondiscrimination and equal opportunity statutes, regulations, and Executive Orders that apply to 

the distribution of funds under the Recovery Act. Agencies that grant funds also must ensure 

that their recipients comply with Title VI of the Civil Rights Act of 1964 (prohibiting race, color, 

and national origin discrimination), Section 504 of the Rehabilitation Act of 1973 (prohibiting 

disability discrimination), Title IX of the Education Amendments of 1972 (prohibiting sex 

discrimination in education and training programs), the Age Discrimination Act of 1975 

(prohibiting age discrimination in the provision of services), and a variety of program-specific 

statues with nondiscrimination requirements.3 

ARRA provides further guidance on how funds appropriated to the Office of the Secretary are to 

be allocated: 

... the funding appropriated in this paragraph shall be used to accelerate the development 

and dissemination of research assessing the comparative effectiveness of health care 

treatments and strategies, through efforts that: (1) conduct, support, or synthesize 

research that compares the clinical outcomes, effectiveness, and appropriateness of items, 

services, and procedures that are used to prevent, diagnose, or treat diseases, disorders, 

and other health conditions; and (2) encourage the development and use of clinical 

registries, clinical data networks, and other forms of electronic health data that can be 

used to generate or obtain outcomes data. 

Section 804 of ARRA authorizes the establishment of the Federal Coordinating Council for 

Comparative Effectiveness Research (the Council). The Council is composed of senior Federal 

officials with responsibility for health-related programs. Most of the members are physicians 

and many have research expertise. The members represent not only the Department of Health 

and Human Services but also the Department of Veterans Affairs and the Department of 

Defense. Members of the Council come from a broad range of backgrounds, including the 

Office of Minority Health, the Office on Disability, community health centers, mental health, 

HIV and other infectious diseases, prevention, and others. The Council’s purpose is to 

coordinate comparative effectiveness research and related health services research across the 

Federal Government with the intent of reducing duplication and encouraging the complementary 

use of resources. The Council is also charged with advising the President and Congress on 

strategies to address the infrastructure needs for CER within the Federal Government and 

organizational expenditures for CER by relevant Federal Departments and agencies. 

3 Memorandum from the Acting Assistant Attorney General for Civil Rights. 4 March 2009 

9 
 

 

 

 

 

 

 

 

The 15-member Council was announced by HHS via website on March 19, 2009, and has been 

meeting regularly since then.4 One of the Council’s responsibilities is to submit to the President 

and Congress an initial report describing current Federal activities on comparative effectiveness 

research and recommendations for CER conducted or otherwise supported from the $400 million 

made available for CER to be allocated by the Secretary. This report meets that requirement. 

Rationale for Comparative Effectiveness Research 

When patients ask clinicians about the evidence supporting one treatment choice, diagnostic 

plan, or prevention modality over another, the answer too often is that the evidence is unclear. 

Even when evidence exists, it is often from a trial that may not apply to the specific patient 

and/or situation under consideration, such as an elderly African-American woman with multiple 

comorbidities. When specific evidence is lacking, clinicians have to rely on their clinical 

experience to make the best treatment decisions possible. Nevertheless, these decisions can 

result in less than optimal, and sometimes inappropriate, treatment choices. 

Due largely to government and scientific leadership accompanied by astonishing achievements 

in biomedical science, clinicians and patients often have a plethora of choices when making 

decisions about diagnosis, treatment, and prevention. Total investment in health services 

research, which includes CER, accounts for only 1.5 percent of medical research expenditures.5 

The Recovery Act greatly increased funding for CER and the prominence and important of such 

research. The purpose of CER is to provide information that helps clinicians and patients choose 

which option best fits an individual patient's needs and preferences. The amazing biomedical 

discoveries made in the United States to date can now support CER to routinely compare 

commonly used therapies or test which interventions work best for particular patients. This 

information is essential to translate new discoveries into better health outcomes for Americans.6 

We must generate this knowledge to be able to deliver the right treatment to the right patient at 

the right time. Patients increasingly and appropriately want to take responsibility for their care; 

therefore, we have an obligation to provide the comparative information that enables informed 

decisions. 

No standardized Federal definition of comparative effectiveness research existed prior to the 

Council’s definition. However, several government entities had developed individual definitions 

for CER. For example, the Congressional Budget Office has described comparative 

effectiveness research as “rigorous evaluation of the impact of different options that are available 

for treating a given medical condition for a particular set of patients.” The Institute of Medicine 

refers to comparative effectiveness as “the extent to which a specific intervention, procedure, 

regimen, or service does what it is intended to do when it is used under real world 

circumstances.” The Council’s definition builds on these concepts and highlights key aspects of 

the ARRA CER provisions. The Council defined CER broadly, asserting that it is patientcentered, “real world” research that can help patients, clinicians, and other decision makers 

4 See Appendix D for Council membership. 

5 Moses III H, Dorsey EK, Matheson DHM, et al. Financial Anatomy of Biomedical Research. JAMA 2005; 

294:1333-42 6 Dougherty, D, Conway PH. The “3 T’s” Roadmap to Transform U.S. Health Care: The “How” of High Quality 

Care. JAMA. 2008 May 21;299(19):2319-21 

10 
 

 

 

 

 

assess the relative benefits and harms of strategies to prevent, diagnose, treat, manage, or 

monitor health conditions and the systems in which they are made.7 This definition will form the 

foundation of the common Federal definition. 

The Department of Health and Human Services’ ARRA appropriation for CER is a significant 

investment. CER and activities that support CER have been undertaken by a wide range of 

stakeholders both inside and outside the public sector. However, despite diverse activities across 

the Federal Government,8

 funds exclusively appropriated for CER have until now been funded 

under authorized by section 1013 of the Medicare Prescription Drug, Improvement, and 

Modernization Act of 2003 for the Agency for Healthcare Research and Quality, which the 

Agency makes available for projects through its Effective Health Care Program. Since 2005, 

Congress has appropriated a total of $125 million for the program, including $50 million for 

comparative effectiveness in FY 2009. 

The ARRA funding reflects the heightened interest in CER among the nation’s clinicians, 

patients, policy makers and researchers and broader recognition of its potential to improve 

outcomes that matter to patients, including morbidity, mortality, and quality of life. CER has the 

ability to assess these very patient-centered outcomes in a comprehensive way. Furthermore, 

patients increasingly play an active role in their health care and expect to be active participants in 

decisions about their health care. These interests are rooted in the strong desire for better 

evidence upon which to make clinical and other health-related decisions at a time of heightened 

focus on the quality and variability of care delivered. 

A health system guided by better information about “what works” would have benefits for all 

who have a stake in the nation’s health system. Consumers and patients would develop more 

confidence that the increasingly complex array of treatments and interventions could be tailored 

to meet their individual needs; health professionals would have more certainty that their clinical 

decisions were evidence-based and serving patients well. Consequences of the lack of such 

information include wide geographic variations in treatments typically received for specific 

conditions and, with these variations, sizeable differences in related health care spending not 

accompanied by proportional differences in outcomes. 

Noted medical author Dr. Atul Gawande recently summarized this issue, “In situations where the 

right thing to do is well established, physicians from high- and low-cost cities make the same 

decisions. But in cases where the science is more unclear, some physicians pursue the maximum 

possible amount of testing and procedures; some pursue the minimum. And what kind of doctor 

they are depends on where they came from. In case after uncertain case, more was not 

necessarily better... We will need to do in-depth research on what makes the best systems 

successful... and disseminate what we learn. Congress has provided vital funding for research 

that compares the effectiveness of different treatments, and this should help reduce uncertainty 

about which treatments are best. But we also need to fund research that compares the 

effectiveness of different systems of care—to reduce our uncertainty about which systems work 

best for communities. These are empirical, not ideological, questions.”9 This variation in care 

7 See Chapter 3 for the Council’s definition of CER.
 8 See Chapter 6 for a comprehensive listing of CER activities across the Federal Government.
 9 Atul Gawande. “The Cost Conundrum.” The New Yorker. June 1, 2009.
 

11 
 

 

 

 

 

 

 

documented by Wennberg10, Fisher11 and others, means that Americans in one part of the 

country who are seeing particular clinicians may get vastly different care with potentially worse 

outcomes than Americans somewhere else. The health system can no longer produce highly 

variable results and tolerate low quality and inefficiency. The care delivered should be based on 

evidence and best practices, not on which physician a patient was referred to or where a patient 

lives. The Council believes that bringing to bear careful research across the continuum of care, 

from prevention, to diagnosis, to treatment, to delivery systems, will yield improved care for 

both individuals and for populations. 

Current Comparative Effectiveness Research Landscape 

In order to inform recommendations for comparative effectiveness research, the Council 

conducted an inventory of current CER activity. Section 6 summarizes CER activity in the 

Department of Health and Human Services, the Department of Veterans Affairs and the 

Department of Defense. Several examples of these activities are discussed below. 

AHRQ has an established CER program as described above. As an example, an AHRQ 

Comparative Effectiveness Review in 2008 examined treatments for localized prostate cancer. 

There are a number of treatment options available for prostate cancer, each with its own potential 

for risks and benefits, so it is important that men understand what is known about the 

effectiveness of these treatments. Key findings from the report included: 

•	 There is a lack of comparative studies across major modalities of treatment (e.g. surgery, 

radiation, watchful waiting).
 

•	 There were no randomized trials evaluating cryotherapy, laparascopic or robotic 

prostatectomy, primary androgen deprivation therapy, high-intensity focused ultrasound 

(HIFU), proton beam therapy, and intensity modulated radiation therapy (IMRT). While 

these therapies have become increasingly of interest for men considering treatments for 

prostate cancer, it is impossible to evaluate whether these therapies are more or less effective 

than other options. 

•	 Of men who had surgery, those undergoing a radical prostatectomy were less likely to 

experience urinary incontinence and other complications if the operation was done by an 

experienced surgeon in a hospital that does many of the procedures.
 

NIH has funded numerous comparative trials with huge implications for the practice of medicine. 

For example, the Diabetes Prevention Program was a major multicenter trial to evaluate the 

comparative effectiveness of intensive lifestyle changes (diet and exercise), a pill for diabetes 

(Metformin), or a placebo in preventing the onset of type 2 diabetes in adults with pre-diabetes. 

This landmark trial found that while both lifestyle changes and Metformin reduced the risk of 

developing diabetes compared to a placebo, lifestyle changes were significantly more effective 

than Metformin. This effect was seen in men and women, and in all ethnic groups. With the 

increasing incidence of pre-diabetes in this country, the results of this trial were critical in 

informing patients and physicians about prevention strategies for diabetes. Similarly, the BARI 

10 Wennberg J, Gittelsohn A. Small area variations in health care delivery. Science. 1973; 182:1102-8. 11Fisher ES, Wennberg J. Health Care Quality, Geographic Variations, and the Challenge of Supply-Sensitive Care 

Perspectives in Biology and Medicine. 2003; 46(1): 69-79 

12 
 

 

 

 

 

 

2D trial compared optimal medical management with revascularization for preventing premature 

death in Type 2 diabetes and found medical management to deliver equivalent outcomes.12 

VA also has a very strong history of conducting CER. For example, the COURAGE trial, 

published in 2007, compared the effectiveness of percutaneous coronary intervention (PCI, or 

angioplasty) plus optimal medical therapy with optimal medical therapy alone in the prevention 

of heart attack or death in veteran patients with stable heart disease. The results showed that 

patients treated with optimal medical therapy alone did just as well as patients who received PCI 

plus medical therapy. This trial can inform patients and clinicians about the most efficient use of 

PCI in patients with stable angina. 

In addition to Federal activities, state level, private sector, and non-profit sector CER efforts are 

currently underway across the country.13,14 For example, 14 states participate in the Drug 

Effectiveness Review Project (DERP), based at the Center for Evidence-Based Policy (EPC) at 

the Oregon Health & Science University. The project is a collaboration between the Oregon 

EPC and the Oregon Center for Evidence-Based Policy. Together, they produce evidence-based 

reviews of the comparative effectiveness and safety of drugs in many drug classes, and then 

make this information publicly available. 

Large insurers and health organizations such as Aetna, CIGNA, UnitedHealthcare, and Humana 

have developed the capacity to conduct evidence reviews in-house. These payers may also 

commission external studies from entities such as the Blue Cross and Blue Shield Association 

Technology Evaluation Center, which has been conducting evidence-based technology 

assessments for more than thirty years. Pharmaceutical, biotechnology, and medical device 

companies may sponsor studies that share some of the attributes of CER. In the non-profit 

sector, organizations synthesize and publicize CER, rather than generating new evidence. For 

example, Consumers Union relies on DERP reports to provide information for its Best Buy 

Drugs Web site. 

Although there are a number of institutions, both public and private, involved in CER, a number 

of challenges remain unaddressed. Much of the CER underway is fragmented, and not aligned 

with a common set of priorities or definition of what constitutes CER. Databases and patient 

registries that are invaluable for comparative effectiveness analysis are similarly fragmented and 

often limited in numbers of patients or of variable or unknown data quality. Some resources, 

such as privately maintained claims databases and Medicare claims data, are difficult for 

researchers to access due to licensing and cost issues. Furthermore, there are a number of gaps 

in the content of the research being conducted. Studies often do not include participants of subgroups, such as racial minorities or people with disabilities, and generally focus on therapeutics 

at the expense of other types of interventions (e.g., devices or the delivery system). Many 

effective interventions for improving health are likely to involve prevention and community 

12 BARI 2D study group et al. N Engl J Med. 2009; 360(24):2570-2. 

13 Academy Health. A First look at the Volume and Cost of Comparative Effectiveness Research in the United
 

States. Available at: http://www.academyhealth.org/files/FileDownloads/AH_Monograph_09FINAL7.pdf. 

Accessed June 17, 2009.

14 The following paragraphs draw on information contained in an environmental scan prepared by the Lewin Group 

for the Federal Coordinating Council on Comparative Effectiveness Research.
 

13 
 

intervention, but these areas are currently understudied. CER should identify interventions that 

yield the most health improvement and represent the best value wherever and however the 

interventions are delivered. 

The OS ARRA funds are a unique opportunity to address some of these gaps. The following box 

summarizes gaps in CER landscape: 

Major Gaps in CER Landscape 

• Coordination across the CER framework 

–	 Substantial CER assets exist across the Federal Government, but coordination is 

necessary to capture their full value 

• Research 

- Many comparative, patient-centered research questions remain unanswered 

• Human and Scientific Capital 

- CER methods development needed 

- Limited trained researchers for conducting CER 

• CER Data Infrastructure 

- Fragmented data 

- Data sources limited in terms of clinical robustness of data and longitudinal data 

capture 

- Data capture and feedback loop at point of care often lacking 

• Dissemination and Translation of CER 

- Suboptimal dissemination and translation of CER findings to patients and clinicians 

- Limited linkages between CER findings and directly improving patient outcomes 

• Priority populations 

- Limited information on many priority populations and sub-groups 

• Priority Interventions 

- Less information on certain comparative interventions such as behavioral change, 

procedures, devices, delivery system strategies, and prevention 

Opportunity Provided by ARRA Funds 

Within this context of national and international activity, the ARRA CER funds offer an 

extraordinary opportunity to complement ongoing research in the public and private sectors by 

establishing a solid infrastructure for future CER. Such investments could include development 

of data and methods, training of researchers who could accelerate the conduct of future studies, 

and rapid dissemination of results to patients and clinicians. For example, enhancing existing 

data resources and learning better how to maximize their utility could expand the types of 

questions addressed as well as identify high-impact opportunities for research. In addition, 

ARRA’s investment in CER coincides with expected increases in the adoption of health 

information technology to improve health care quality and safety. That technology also offers 

the promise of including care delivery in the conduct of research (what some have termed a 

“learning health care system”) and offering a platform for rapid dissemination of results to the 

14 
 

 

 

 

 

 

point of care to inform physician and patient decisions.15 The field of CER is not entirely new, 

but increased availability of clinical electronic data resulting from diffusion of information 

technology demands improved methods and a cadre of researchers ready to take advantage of 

these expanding data resources. 

As CER becomes a more integrated resource for health care decision-making, we must assure 

public trust by ensuring the privacy and security of health information and by maintaining access 

to appropriate care options. CER should not be used as a sole criterion for denying or awarding 

care or as justification for making care choices based on cost without consideration of 

effectiveness, safety, and convenience for an individual patient. CER has the potential to offer 

tremendous benefits to Americans so long as we apply its conclusions appropriately and protect 

the individual health information that informs it. 

The Council believes that there is much to be learned about how research results can be 

incorporated into the everyday practice of medicine and inform consumer health care choices. 

The Council’s hope is that ARRA funding has the potential to form a firm base for the Federal 

Government’s future investments in CER and lay the foundation for a productive CER enterprise 

that improves care for all Americans. 

II. VISION AND COUNCIL OBJECTIVES 

Comparative effectiveness research has the potential to catalyze a patient-centered 

transformation of the U.S. health care system. By equipping patients and clinicians with the 

information needed to make joint medical decisions, and by optimizing the system in which the 

patient/clinician team makes these decisions, CER can improve the quality, safety, and value of 

care delivered while increasing patient satisfaction.16 By passing ARRA, Congress recognized 

this vision and the need for CER, and also highlighted the need for an unbiased, cross-functional 

Council to “foster optimum coordination” of the Federal Government’s CER efforts. 

Given the Council’s distinct role and the unprecedented resources available to the Secretary, the 

Council has a unique opportunity to begin working toward this vision for CER. The Council 

sees the following as potential accomplishments at the end of the ARRA funding period: 

1.	 Establishment of a process for CER priority-setting that maximizes the value of Federal 

investments in CER through responsiveness to patient and other stakeholder needs, 

transparency, and effective coordination. 

2.	 Development of a robust, foundational infrastructure for CER. 

3.	 Implementation of a strategy to support rapid, systematic dissemination of CER results to 

empower patients, clinicians, and other stakeholders to make more informed decisions 

and increase the quality of care. 

15 Conway PH, Clancy C. Transformation of Health Care at the Front Line. JAMA. 2009 Feb 18;301(7):763-5. 16 Naik AD, Peterson LA. The Neglected Purpose of Comparative Effectiveness Research. NEJM. 2009 May 7; 

360(19):1929-31. 

15 
 

To accomplish this vision, the Council outlined three specific, near-term objectives that build on 

those established in ARRA: 

1.	 Develop a definition, establish prioritization criteria, create a strategic framework, and 

identify priorities that lay the foundation for CER. 

2.	 Foster optimum coordination of comparative effectiveness research conducted or
 

supported by relevant Federal departments. 

3.	 Formulate recommendations for investing the $400 million appropriated to the HHS 

Office of Secretary as part of this Report to Congress. 

III. COMPARATIVE EFFECTIVENESS RESEARCH DEFINITION AND CRITERIA 

One of the first activities of the Council was to build on previous definitions of comparative 

effectiveness research, including IOM, CBO, and others, to develop a definition of comparative 

effectiveness research for the Council. After much discussion and sharing with the public for 

feedback, the Council established the following definition. 

Definition 

Comparative effectiveness research is the conduct and synthesis of research comparing the 

benefits and harms of different interventions and strategies to prevent, diagnose, treat and 

monitor health conditions in “real world” settings. The purpose of this research is to improve 

health outcomes by developing and disseminating evidence-based information to patients, 

clinicians, and other decision-makers, responding to their expressed needs, about which 

interventions are most effective for which patients under specific circumstances. 

 To provide this information, comparative effectiveness research must assess a 

comprehensive array of health-related outcomes for diverse patient populations and subgroups. 

 Defined interventions compared may include medications, procedures, medical and 

assistive devices and technologies, diagnostic testing, behavioral change, and delivery 

system strategies. 

 This research necessitates the development, expansion, and use of a variety of data 

sources and methods to assess comparative effectiveness and actively disseminate the 

results. 

The definition above is not meant to exclude randomized trials; however, these trials would need 

comparator arms other than placebo and be representative of populations seen in “real world” 

practice. 

Once a definition was established, the Council drafted threshold criteria for consideration and 

prioritization criteria for comparative effectiveness research and related investment. These 

criteria were posted on a public Web site, feedback was received, and modifications were made. 

The following are the current Council criteria. 

16 
 

 

 

Prioritization Criteria for Comparative Effectiveness Research Related Investments 

Minimum Threshold Criteria (i.e. must meet these to be considered): 

• Included within statutory limits of Recovery Act and FCC definition of CER 

• Potential to inform decision-making by patients, clinicians, or other stakeholders 

• Responsiveness to expressed needs of patients, clinicians, or other stakeholders 

• Feasibility of research topic (including time necessary for research) 

The prioritization criteria for scientifically meritorious research and investments are: 

•	 Potential impact (based on prevalence of condition, burden of disease, variability in 

outcomes, costs, potential for increased patient benefit or decreased harm) 

•	 Potential to evaluate comparative effectiveness in diverse populations and patient subgroups and engage communities in research 

•	 Uncertainty within the clinical and public health communities regarding management 

decisions and variability in practice 

•	 Addresses need or gap unlikely to be addressed through other organizations 

•	 Potential for multiplicative effect (e.g. lays foundation for future CER such as data 

infrastructure and methods development and training, or generates additional investment 

outside government) 

This definition and criteria guided the Council as it considered potential priority 

recommendations for the OS funds and will guide AHRQ and NIH in allocating their CER funds. 

IV. IMPORTANCE OF PRIORITY POPULATIONS AND SUB-GROUP ANALYSIS 

As the United States has grown in its diversity, there has remained a persistent underrepresentation of women, the elderly, persons with disabilities, and racial and ethnic minorities in 

clinical and other research studies. While the NIH has a policy of inclusion of women and racial 

and ethnic minorities in all NIH-funded clinical trials,17 the majority of research conducted in the 

U.S. does not require the inclusion of these and other priority populations. The lack of adequate 

representation of important patient populations in many research studies presents a major 

challenge in applying the results of these studies to important populations and sub-groups. In 

recognition of this fact, the ARRA legislation notes that “research conducted with funds 

appropriated shall be consistent with Departmental policies related to the inclusion of women 

and minorities.” This criterion is critically important for ensuring that information gained from 

comparative effectiveness research improves the quality of care for all Americans. 

17 http://grants.nih.gov/grants/funding/women_min/guidelines_amended_10_2001.htm 

17 
 

 

 

 

Indeed, focused attention is needed on priority populations,18 including racial and ethnic 

minorities, individuals with disabilities, children, persons with multiple chronic conditions, and 

the elderly, not only because of their under-representation in current research but also because of 

the increased disease burden and health disparities faced by these sub-groups. 

The following sections highlight some of the challenges facing our health system as it relates to 

priority populations. Disparities in health care and health outcomes for these populations persist, 

affecting an ever-increasing proportion of residents of the United States. Also outlined are some 

of the research challenges that exist for priority populations, followed by recommendations to 

address these issues.

Growth in Priority Populations 

Priority populations not only account for a large proportion of current health services utilization, 

but their numbers are growing; their need for health care services will likewise continue to grow. 

The most recent U.S. Census Bureau data reveal that over 100 million people living in the United 

States belong to a racial or ethnic minority group; this equates to 34 percent of the total U.S. 

population, and these minorities will likely become the majority of the U.S. population within 30 

years.19 Similarly, the number of elderly Americans is growing, with that segment of the 

population expected to increase from 35 million today to 71 million by 2030— or nearly 20 

percent of the overall U.S. population. The population over the age of 85 is projected to grow 

from 5.3 million today to 21 million by 2050. 

Health Disparities 

A number of important reports have highlighted disparate disease prevalence, progression, and 

health outcomes for racial and ethnic minorities, elderly Americans, individuals with disabilities 

people of low socioeconomic status, people with mental illness, and others.20,21 In this context, 

health disparities are defined as significant gaps or differences in the overall rate of disease 

incidence, prevalence, morbidity, mortality, or survival rates in the priority population as 

compared to the health status of the general population.22 For example, African-American 

women are 34 percent more likely to die from breast cancer, even though they are diagnosed 

with the disease 10 percent less frequently than white women; Hispanics in the U.S. are 50 

percent more likely than whites to suffer from diabetes; and the incidence of diabetes among 

18 Priority populations are defined in Sec. 901 of the Healthcare Research Act of 1999, S. 580.
 19 U.S. Census Bureau. Minority Population Tops 100 Million: Press Release. Available at:
 

http://www.census.gov/Press-Release/www/releases/archives/population/010048.html. Accessed April 1, 2009.
 

20 Report of the Secretary’s Task Force on Black and Minority Health. U.S. Department of Health and Human 

Services. 1985. 

21 Brown ER et al. Racial and Ethnic Disparities in Access to Health Insurance and Health Care. UCLA Center for 

Health Policy Research and the Henry J Kaiser Family Foundation. 2000. 

22 Minority Health and Health Disparities Research and Education Act of 2000. Public Law 106-525. November 20, 

2008. 

18 
 

 

 

 

 

 

Native Americans is more than twice that for whites. Elderly Americans also face particular 

health challenges, from greater susceptibility to multiple chronic conditions to a lower likelihood 

of obtaining preventive treatments, including mammograms and immunizations. In addition, 

approximately 42 percent of individuals over the age of 65 report a functional limitation.23 

Persons with Disabilities 

According to the 2007 Institute of Medicine report The Future of Disability in America, from 40 

to 54 million people in the United States have disabilities. These numbers will grow considerably 

in coming decades as baby boomers age and as new medical interventions extend the lives of 

young persons with significant impairments who would once otherwise have died. Although 

rates are lower in children, disability prevalence is rising at younger ages. According to figures 

from the National Health Interview Survey, childhood disability has risen by 350 percent during 

the last 40 years, with the largest increase occurring during the past decade. 

Across the lifespan, disabilities are clinically and functionally heterogeneous, encompassing 

diverse cognitive, sensory, physical, and mental health impairments. Traditionally patients with 

disabilities have been excluded from clinical trials, yet they have the same risk for diseases as 

non-disabled persons. Future clinical trials should exclude persons with disabilities only if there 

are clear and compelling reasons to do so. 

Comparative effectiveness research relating to persons with disabilities is important in a number 

of areas. 

First, research would be beneficial about the most effective interventions to prevent or mitigate 

disability and the disabling effects of chronic diseases. All research including comparative 

effectiveness research relating to disability should include outcome measures that address 

functional abilities, people’s abilities to participate in daily activities, and quality of life. This is 

critical as the world’s population is growing older at a very fast pace and this has serious 

implications due to expected increasing rates of chronic conditions. Moreover, with the 

advances in science and technology, lifespan has increased considerably; this is also true for 

persons with disabilities. 

Second, future comparative effectiveness research should look into community-based models of 

care for persons with disabilities. Following the Supreme Court Olmstead v. L.C. ex rel. Zimring, 

527 U.S. 581 (1999) Decision, traditionally institutionalized individuals with disabilities or those 

at risk of being placed in institutions are increasingly being cared for in their own homes and/or 

communities. Underscored by the Olmstead Decision states now have to consider civil rights 

when developing their programs. Effective care coordination/care management is critical to help 

persons with disabilities live independently in their communities with added years of quality life. 

Care coordination/care management is even more important for those individuals with multiple 

chronic conditions, which are often associated with certain levels of disability. While care 

23 Federal Interagency Forum on Aging-Related Statistics. Older Americans 2008: Key Indicators of Well Being. 

Federal Interagency Forum on Aging-Related Statistics. Washington, DC: U.S. Government Printing Office. March 

19 

2008 
 

 

 

 

 

 

coordination/care management is the current state-of-the-art, it is still considered to be in its 

early stage. This represents a major opportunity for building the infrastructure to support future 

CER studies. In addition, because the definition of care coordination varies according to settings 

and models of care, its effectiveness has not been clearly established, particularly as it relates to 

the role support services play and how better integration of health and support services can lead 

to improved health outcomes for persons with disabilities and reduced health care costs for our 

nation. 

Third, persons with disabilities are at increased risk for developing secondary conditions that are 

associated with their primary disabling condition. For example, without preventive measures, 

individuals with spinal cord injuries may acquire a number of adverse health conditions, 

including cardiovascular disease, genitourinary tract disorders, depression, obesity, and pressure 

sores. Comparative effectiveness studies should determine which interventions are most likely 

to prevent secondary conditions or ameliorate their consequences. 

Fourth, studies should investigate the comparative effectiveness of rehabilitation interventions to 

restore or maintain functioning or minimize its loss. For example, much more research is needed 

to identify effective speech-language, physical, and occupational therapy interventions. This 

research could include a comparison of conventional treatments to newer interventions or a 

comparison of various systems of care. More research is also required about various assistive 

devices, medical equipment, and technologies, including technologies addressing sensory 

deficits, communication impairments, and physical and motor limitations. 

Fifth, comparative effectiveness studies of therapeutic and preventive interventions need to 

address explicitly the needs of children with disabilities and be sensitive to the developmental 

stage of the child. For many children with disabilities and complex health care needs, the 

transitions through adolescence and into adulthood are complicated by the absence of 

comprehensive care programs that fully address their needs. Comparative effectiveness studies 

should examine different care models to determine which ones offer the best care coordination 

and generate the greatest patient and family satisfaction and health outcomes. 

Other underrepresented populations 

Children represent another group that can benefit tremendously from comparative effectiveness 

research. Evidence cannot simply be extrapolated from adults to the pediatric population. There 

is a dearth of information to inform decisions by children and their families, especially since 

outcomes, such as quality of life and functioning, are often more subtle. In addition, 

comparative preventive interventions (e.g. for obesity) will often have the most long-term effects 

if started in the pediatric population. 

At the other end of the continuum, the elderly represent another group for which there exists 

little information about best care practices. As our population ages, knowledge about the best 

and most effective treatments for this group will become essential. Other important areas of 

focus for the elderly include home health care strategies and optimal approaches to delivery of 

care within nursing facilities. 

20 
 

 

 

 

 

 

 

 

Veterans and service members often have many conditions for which CER could be informative. 

They have a number of special considerations in deployment-related illness such as post 

traumatic stress disorder, traumatic brain injury, exposures, infectious diseases, disabilities and 

others. CER provides a vital opportunity to glean additional information necessary for clinicians 

to make informed decisions about particular veterans needs and information to assist veterans in 

their participation in care decisions. 

Finally, research to compare the effectiveness of prevention strategies, treatments, diagnostics, 

and care delivery for patients with multiple chronic conditions is essential. Again, as our 

population ages, patients increasingly have several comorbidities which may impact their 

response to treatment. The majority of clinical research to date excludes such patients, so the 

applicability of “standard” treatments to this population is unclear. A physician advising a 45­

year-old woman with asthma and HIV about treatment for breast cancer simply does not have the 

evidence necessary to factor her comorbidities into her patient’s treatment decision. By utilizing 

varied and robust research methodologies, CER affords the opportunity to target treatments and 

other interventions to improve the quality of life and overall health of this important group of 

patients. 

Personalized Medicine and Patient Sub-groups 

The need to identify and address the needs of emerging patient sub-groups, and indeed the very 

concept of sub-categories of conditions to which medical products are applied, is expected to 

change and grow as our understanding of genomics and molecular medicine increases and 

becomes an integral part of health care. Better understanding of an individual’s genomic and 

other individual biological characteristics will enable us to recognize and respond to human 

variability with a new degree of specificity. Understanding biological differences at the 

molecular level promises a significant leap in our ability to use and develop medical technologies 

more effectively, targeting interventions at more defined groups of individuals with greater 

precision. This potential, sometimes referred to as personalized medicine, has strong bearing on 

comparative effectiveness research.24 Many drugs prescribed in the United States today are 

effective in fewer than 60 percent of treated patients. This is not a fault of the drugs, but reflects 

the variability of metabolism or other factors from person to person.25 

Unfortunately, it remains common medical practice to follow a trial-and-error approach in 

selecting medical interventions for patients to achieve a satisfactory therapeutic outcome. In the 

case of breast cancer, for example, while chemotherapy can be an important positive treatment 

for some patients, we have few tools today to successfully predict which patients will benefit– 

and the result is that many women who are treated with chemotherapy today are receiving 

treatments that may not be effective for their condition. 

Personalized medicine aims to make medical care more precise and effective. Increased 

understanding of our individual genomic profiles and other individual biological characteristics 

24 Willard HW: Organization, Variation and Expression of the Human Genome as a Foundation of Genomic and
 

Personalized Medicine. In Genomic and Personalized Medicine. Volume 1. Edited by Willard HW and Ginsburg
 

GS. London: Academic Press; 2009:4-21. 25 Spear BB, Heath-Chiozzi M, Huff J. Trends Mol Med. 2001 May; 7(5):201-4.
 

21 
 

will enable us both to use more effectively the therapies we have now and to identify significant 

areas where research and development of new products may be needed. Pharmacogenomics, the 

use of genetic information or other biomarkers to assist in accurate medical therapy decisionmaking, is expected to be a hallmark of this approach. 

CER can be an important partner in helping to bring about this new level of medical 

effectiveness, personalization, and innovation. At the same time that CER is being used to 

identify which interventions and strategies work best on average, it can also help to identify 

different responses by different groups of patients. In some cases, different existing therapies 

may be identified as most effective for specific sub-groups. In other cases, CER may help to 

identify significant sub-groups for whom effective therapies do not yet exist. CER may also help 

steer research efforts toward the development of products and strategies for areas of significant 

need. 

Research Challenges 

Multiple research challenges exist for priority populations. Examples include a need for 

increased diversity in research populations, expanded data sources for evidence-based studies in 

diverse populations, enhanced collection of racial and ethnic health data, a better understanding 

of the effectiveness of interventions in the context of comorbidities, and a greater focus on 

implementation research. 

Generalizations that result from comparative effectiveness research that fail to consider subgroups and individual differences may have limited applicability. Currently there are gaps in 

knowledge about whether specific treatment strategies work across different sub-groups under a 

variety of circumstances. Recognizing that there might be variations in the effectiveness of 

specific interventions in the elderly, racial and ethnic minorities, individuals with disabilities, 

and other priority populations is key to designing evidence-based strategies to successfully 

improve the quality of care that is delivered. Infrastructure investments that capture priority 

populations and patient sub-groups will be critical to overcoming these challenges. 

Strategies to Strengthen Comparative Effectiveness Research for Priority Populations 

In light of the aforementioned challenges, comparative effectiveness research presents an 

opportunity to be more inclusive of minorities, the elderly, persons with disabilities, and other 

priority populations. This feature of CER is especially true in the context of conducting specific 

studies that take into account health conditions and linguistic and cultural attributes in order to 

develop the most appropriate and effective interventions. 

Investments in CER can be used to address the needs of priority populations by doing the 

following: 

Evaluating and identifying interventions that are tailored for priority populations. To 

explore which interventions are most effective for addressing the needs of priority populations, 

specific studies are needed to look at interventions that target diseases with a high prevalence in 

racial and ethnic minority communities, the elderly, and individuals with disabilities. These 

22 
 

 

 

studies may need to simultaneously address several diseases/conditions, or assess combinations 

of interventions (e.g., behavioral and physical treatments/interventions) that are most effective in 

promoting desired outcomes for these populations. Studies examining care delivery interventions 

tailored for priority populations are also needed in order to ensure that care is delivered to these 

individuals through effective approaches that are targeted to their needs. To ensure effective 

communication with the priority populations both in conducting the research and implementing 

its results, investigators should ensure that those language and communication services are 

available for those with limited English proficiency or disabilities. 

Creating and enhancing potential databases looking at interventions in priority 

populations. Successfully examining and evaluating a range of interventions that are effective 

for priority populations will require a broad range of potential data sources and infrastructure 

investments. In addition to traditional patient registries and systematic reviews, the inclusion of 

distributed data networks that utilize community-based infrastructure, such as Federally 

Qualified Health Centers, will be an important asset in broadening the tools to evaluate 

effectiveness in various priority populations. CER studies should routinely perform and report 

sub-group analyses to examine possible differences in effectiveness for important racial and 

ethnic groups, and should over-sample such groups whenever there is existing evidence to 

suggest differences in effects or outcomes in any priority population. Standardized reporting and 

analysis of priority population sub-groups will also permit pooling of research results across 

studies to explore sub-group differences. 

In addition, efforts should be made to build capacity and infrastructure within traditionally 

underserved racial/ethnic communities to allow for standardization of data collection and to 

enable the seamless integration of such data with larger databases/systems currently in use by the 

research community. This will allow for more accurate downstream comparisons to pre-existing 

and future majority data sets, producing more comprehensive and reliable CER study results. 

Finally, this infrastructure for CER in priority populations is particularly important for 

developing and implementing Clinical Preventive Services Guidelines and recommendations for 

the U.S. Preventive Services Task Force. According to the IOM, CER data on priority 

populations is often unavailable for developing guidelines, and what information is available is 

often insufficient for making conclusions on how to treat priority populations. 

Increasing the number of community-based studies, including community-based 

participatory research (CBPR) studies. CBPR is defined as a collaborative research approach 

in which communities and researchers are equally involved in the design and conduct of research 

that is conducted in their communities. Successful and effective CBPR studies result in the 

development of research tools, strategies, and interventions that are effective in creating 

sustainable and positive behavior changes and outcomes among priority populations within 

communities. Because CBPR studies are conducted with substantial input from the community, 

interventions are typically tailored to fit the needs and characteristics of the community. 

Furthermore, communities become “owners” of the research, which results in sustainable 

research outcomes. 

23 
 

 

 

 

Increasing cultural competency. Understanding the linguistic, cultural, social, and 

environmental attributes of priority populations is essential in designing interventions and 

promoting strategies that are effective in addressing the needs of these populations. Specifically, 

doing so allows for the development of culturally and linguistically appropriate interventions. 

For example, an obesity/diabetes intervention involving diet and/or physical activity would 

require an understanding and assessment of the populations’ cultural attributes (e.g., food 

preferences), social attributes (e.g., competing family and work demands), and environmental 

attributes (e.g., access to ‘healthy’ foods and safe walkways) that support or inhibit adhering to a 

diet and/or physical activity intervention. 

Building workforce capacity. Racial/ethnic minorities, individuals with disabilities, and 

women are underrepresented in the research and medical communities. The lack of a diverse and 

linguistically competent scientific workforce adds to disparities in research development, service 

delivery, and quality of care. Initial CER investments in workforce capacity could create 

opportunities to engage researchers and providers from diverse backgrounds. For example, 90 

percent of minority physicians educated at Historically Black Medical Colleges live and serve in 

minority communities. Hispanic-Serving Institutions (HSIs) also play a major role in educating 

Hispanics researchers. Approximately 49 percent of all Hispanic students attend an HSI. A 

special focus on priority populations could provide an avenue for engaging Historically Black 

Colleges and Universities and HSIs in the conduct of CER among priority populations. 

Developing and implementing outreach strategies to various racial, ethnic, and health 

disparity populations for participation in research protocols. In order to strengthen CER, 

effective outreach strategies must be developed and implemented that will increase the 

participation of priority populations in clinical research protocols. Developing appropriate 

strategies to reach out to various priority communities requires an understanding of the history of 

these populations in research and the identification and recruitment of trusted community 

members who can champion the research benefits and inform communities about risks. 

Community health workers can be important partners in addressing and advocating for the needs 

and concerns of priority populations. In addition, clinicians and providers will need to be 

educated on the benefits and implications of CER and the utilization of evidence-based 

interventions. 

Dissemination, translation and adoption of research results is one of the biggest challenges 

within comparative effectiveness research, particularly as applied to priority populations, but 

also as applied to the population as a whole. The young science of implementation research 

focuses on the acceleration of translation of evidence into everyday care, and affords an 

opportunity to build a more coordinated approach to improving the quality of health care of 

priority populations. This is not a one-way transfer of knowledge. Racial and ethnic minorities, 

persons with disabilities, children, and the elderly, can offer insights into how best to engage 

their communities. Active listening and thoughtful planning of the dissemination process can 

create better health outcomes for all Americans. 

Making CER investments that are responsive to the needs of priority populations and sub-groups 

is critical to ensuring that the benefits of CER reach those with the greatest needs. Such 

24 
 

 

 

 

 

 

 

investments, however, can also benefit the population as a whole by validating new strategies 

and approaches for comparative research and implementation. 

V. STRATEGIC FRAMEWORK FOR CER 

There are countless opportunities for action and investment in CER. Many Federal, state, and 

private institutions are already involved in CER and have made choices about which of these 

activities and investments to pursue. After completing the draft definition and criteria for 

prioritization of potential CER investments, the Council recognized the need to develop a 

strategic framework for CER activity and investments to categorize current activity, identify 

gaps, and inform decisions on high-priority recommendations. 

This framework represents a comprehensive, coordinated approach to CER priorities. It is 

intended to support immediate decisions for investment in CER priorities and to provide a 

comprehensive foundation for longer-term strategic decisions on CER priorities and the related 

infrastructure. At the framework’s core is responsiveness to expressed needs for comparative 

effectiveness research to inform health care decision-making by patients, clinicians, and others in 

the clinical and public health communities. The framework will be supported by detailed 

inventories of Federal CER activities and research/data infrastructure, and a priority-setting 

approach. This organizing framework fosters consideration of the balance of activities and 

priority themes, focuses on the most pressing needs expressed by patients and clinicians, and 

allows for identifying and addressing gaps in the current landscape of CER. 

CER activities and investments made by the government or other institutions can be grouped into 

four major Core Categories: 

• Research includes activities or investments in primary research or meta-analysis. 

Organizations involved in this group of activities may be funding research, conducting 

research themselves, or helping to establish a common set of research priorities to create 

momentum around the most critical research topics. 

• Human and Scientific Capital includes activities or investments that enhance the 

United States’ capacity for CER by expanding and strengthening relevant research skills 

or by advancing CER approaches and methodologies. Organizations involved in this 

group of activities may be directly involved in training and workforce development, 

developing new CER methods, validating results of CER, or driving consensus on valid 

approaches to CER. 

• CER Data Infrastructure includes activities or investments that develop, build, or 

maintain data infrastructure, systems, or tools. These investments could include the 

creation of new research data sets and repositories, aggregation of existing data sources, 

development of new tools to query and analyze existing data sets, or creation of standards 

for new data collection. 

25 
 

• Dissemination and Translation of CER includes activities or investments that 

disseminate CER findings and put them into practice. Activities and investments range 

from dissemination and distribution of CER information to improving processes and 

outcomes in health care and public health delivery systems through CER translation and 

adoption. 

Table 1 

Example Activities in Each Major Category 

Activity Examples 

Research Comparing outcomes of treatments or care delivery for a 

specific condition 

Human & Scientific Capital Training new researchers to conduct CER or developing 

CER methodology and standards 

CER Data Infrastructure Developing a distributed practice-based data network, linked 

administrative or EHR databases, or patient registries 

Dissemination and Translation of 

CER 

Building tools and methods to disseminate findings and 

translate CER into practice to improve health outcomes for 

patients 

Furthermore, investments or activities focused on a specific priority theme can cut across these 

categories. The potential themes include: 

•Conditions. Organizing investments and activities around a condition or disease state is 

common in research and reflects the organization of medical practice. Focusing on a 

single disease state across all four major categories of activity (e.g., funding primary CER 

in oncology, developing new methodologies for CER in palliative care settings, 

expanding the Surveillance, Epidemiology, and End Result database (SEER), and 

partnering with an academic cancer center to pilot CER implementation strategies) could 

result in significantly improved patient-centered outcomes in that disease area. 

•Patient populations. While clinical research is relevant to the patient population it is 

designed to address, it often provides little information relevant to patient groups not 

typically enrolled in clinical studies. In private-sector-funded trials, this often includes 

the elderly, racial and ethnic minorities, children, and persons with disabilities. The NIH, 

however, already requires that all publicly funded trials include appropriate numbers of 

women and racial and ethnic minorities. Cross-cutting activities and investments that 

facilitate studies responsive to the needs of these populations can ensure that all 

Americans benefit from CER. 

•Type of intervention. Several potential areas of focus emerge from studying 

interventions by type. In defining CER, the Council specifically included the following 

types of interventions: medications, medical and assistive devices, procedures, behavioral 

change, diagnostic testing, and delivery system strategies. Each of these has unique 

opportunities for coordinated investment in data infrastructure, research, building 

26 
 

research capacity, and translation. In addition, one could focus on interventions at a stage 

of the disease (i.e., prevention, diagnosis, treatment, and management). 

Together, these activities and themes make up the CER strategic framework (Figure 1). 

Agencies or organizations that are engaged in CER will often make investments in one group of 

activities or across multiple groups within a cross-cutting theme. The pattern of activity and 

investment for a single organization highlights its strategy. For example, a medical information 

database company may concentrate its CER activities in data infrastructure, whereas the National 

Cancer Institute is involved in multiple types of activities with a focus on cancer. When patterns 

of activity for the most critical agencies and organizations involved in CER are viewed in 

aggregate, the CER framework reveals gaps in CER activities and investments. These gaps are 

potential areas of opportunity and impact for the Secretary’s ARRA funds. As such, the 

framework is useful for determining what investments are appropriate for ARRA funds and for 

future Federal investments in CER, as well as for codifying the ongoing activities of Federal 

agencies involved in comparative effectiveness research. 

Figure 1 

 

 

Strategic Framework Human & Research Dissemination and Scientific Capital CER Data Infrastructure Translation of CER for CER 

Priority Populations 

Priority Conditions 

Types of Interventions 

CrossCutting 

Priority 

Themes 

Specific investments can 

be within a single category 

or be cross-cutting in one 

of the priority themes 

Creating and maintaining an inventory summarizing current and past Federal efforts across the 

CER framework is critical to its value in decision-making. This inventory of Federally-sponsored 

CER activities will also be a critical component of future Council annual reports. This process 

of inventory-taking, gap analysis, and establishing priorities for investment should be iterative. 

The process for developing the inventory and aligning findings from that process with CER 

investment decisions is outlined in Figure 2. 

27 
Figure 2

Using the CER Strategic Framework for Inventory and Investment Decisions

Human & 

Scientific Capital 

for CER

Inventory/evaluation of 

Human & Scientific 

Capital

Training 

CER methods 

development 

Methods for 

patient/consumer 

engagement

Funding based 

on identified 

high-priority gaps

Research Priorities

Inventory/evaluation of existing 

Research

Expressed public and federal 

needs for CER

Explicit Prioritization with 

Public Input

High-priority, feasible, nonduplicative CER topics 

Funding of high-value CER 

portfolio to fill gaps

CER Data & Research 

Infrastructure

(Research data repositories & 

clinical research networks)

Inventory/evaluation of existing 

CER infrastructure 

Evidence generation 

• Clinical research networks

• Registries, surveillance databases, 

research-quality observational 

datasets

Evidence linkages • Claims, other admin databases

• EHRs and distributed data networks 

Funding based 

on identified 

high-priority gaps

Dissemination and 

Translation of CER

Inventory/evaluation of 

existing CER 

translational & 

dissemination activities

Potential Capacity for 

Dissemination and 

Translation through Federal 

delivery systems and 

public-private partnerships

Funding based 

on identified 

high-priority gaps

CER Investment Decisions

Cross-Cutting Investment Opportunities

Priority populations (e.g., priority/underserved populations, multiple chronic conditions)

Priority interventions (e.g., procedures, behavior change, delivery system)

Unlikely to be addressed given roles and capacities in CER

Opportunities to catalyze cross-cutting data, research and collaboration

Enhanced Human 

& Scientific Capital 

for CER

New 

Comparative 

Effectiveness 

Research

Enhanced CER 

Infrastructure 

CER Findings into 

Practice and Public 

Health

Legend:

CER Investm3ent 

Opportunities

Overall, the CER framework is a useful strategic and analytic tool to help organize ongoing CER 

activities of Federal agencies, to facilitate development of a strategy for the Secretary’s ARRA 

investments, and to continually monitor progress in CER across the different dimensions of the 

framework.

VI. CURRENT CER INVENTORY AND CER DATA INFRASTRUCTURE

The following CER inventory and data infrastructure was collected for the first time and on a 

very short timeline. The counts of CER studies are based primarily on electronically accessible 

sources, informed in part by interviews of senior agency staff. Attributes of the research 

reported here (study designs, types of interventions studied, etc.) were determined from study 

summaries or abstracts rather than inspection of full-text reports of these studies. 

As described below, providing a high-confidence estimate of the number of Federally-funded 

CER studies underway for a given fiscal year is not currently feasible. Prospective identification 

of CER studies using keywords or other “tagging” in one or more readily searchable electronic 

databases would enable tracking of completed and ongoing CER. Therefore, this preliminary 

inventory is informed by a convenience sample and should be viewed as a rough estimate of 

what will be an iterative process going forward. 

28
 

 

 

 

 

 

 

 

 

 

Although ARRA is the first coordinated Federal CER effort, several Federal agencies have been 

conducting comparative effectiveness research and maintaining data and infrastructure for CER. 

Most of this activity has been conducted independently within the given agency. The agencies 

most active in CER include AHRQ, NIH, and the Veterans Health Administration (VHA). But 

many other agencies conduct or have resources related to CER to a lesser degree, such as 

comparative effectiveness research studies, related data infrastructure, or the potential to be 

effector arms for research dissemination and translation. Finally, it is important to note that this 

inventory does not include CER conducted by private or not-for-profit organizations. 

CER Inventory 

Table 2 provides information about the numbers of studies for these agencies. There is no 

standard, systematic means of reporting on CER studies and funding across Federal agencies. It 

is not possible at this time to estimate the total number of primary or secondary CER studies 

conducted by the Federal Government. Other than AHRQ, by virtue of its dedicated Effective 

Health Care Program, agencies have limited ability to track CER studies and spending, reflecting 

that CER is a relatively new field of inquiry, has no standard definition, and is not “tagged” or 

readily searchable in biomedical or health services research databases. AHRQ tracks its funding 

and number of studies by fiscal year. Funding for CER studies for AHRQ ranges from 12 million 

to 35 million per fiscal year since FY 2006, with 12-18 studies funded per year. Estimates for the 

number of CER studies and funding for DoD and VHA are approximations per year rather than 

specific numbers for particular years. For example, DoD estimates its funding to be 

approximately $125,000 to $500,000 per year for 5-10 studies per year; the VHA estimates are 

50 million to 70 million per year for 40-50 studies per year. 

As part of its large portfolio of biomedical research, the CER funded by NIH makes that agency 

the single largest sponsor of primary comparative effectiveness research. These studies are 

difficult to identify, however, as they are not “tagged” or otherwise readily searchable as CER in 

such databases as ClinicalTrials.gov or CRISP (Computer Retrieval of Information on Scientific 

Projects, a database of biomedical research funded by NIH). 

For purposes of this pilot inventory, a keyword search of ClinicalTrials.gov yielded an initial set 

of 1,800 NIH-funded trials during the years 2006-2009 that were candidates for CER. 

Subsequently, in cooperation with NIH, a sample set of 463 NIH CER studies for 2008 was 

identified, starting with a new searching process under development by NIH to track CER 

studies and spending.26 

26 NIH recently developed an initial process involving a keyword searching software algorithm based on consensus 

among several experts regarding which studies from among those funded by NIH qualify as CER. NIH applied this 

algorithm to all studies funded by NIH in 2008, which yielded more than 800 studies with a score above a certain 

threshold—tagging them as potential CER. Inspection of all of the records of all of these studies in CRISP by staff 

supporting the Coordinating Council identified the set of 443 that appeared to qualify as CER. This set of 443 does 

not necessarily represent the full set of CER studies funded by NIH in 2008. 

29 
 

 

 

 

 

 

 

 

Table 2: Estimated CER Grant/Study Counts FY 2006 – FY 20091 

Agency CER Grants/Studies FY2006-FY 2009 (YTD) 

AHRQ 144 

DoD 25 

VHA 96 

NIH2 463 

1

As of June 2009, based on review of agency/department websites and 

agency/department generated lists

2

NIH is in process of cataloging CER. This primarily represents FY 2008. 

CER studies conducted or sponsored by VHA and DoD often focus on the particular populations 

they serve. These include CER studies involving patient groups that fall within designated U.S. 

priority populations (e.g., the elderly, racial and ethnic minorities, patients with multiple chronic 

conditions, persons with disabilities). 

The main findings from analyses of Federal CER for fiscal years 2006-2009 include the 

following: 

•	 In this initial compilation, the inventory of CER that could be confirmed independently 

for those agencies that perform or sponsor it was generally comparable to the inventory 

as described in interviews with agency staff. The main exception was NIH, where the 

volume of CER is acknowledged to be large yet remains to be quantified. 

•	 Other than that for AHRQ, agency budgets for CER are not well defined. Agency staff 

typically described rough percentages of total research budgets or approximate ranges of 

annual expenditures on CER, but generally could not cite budget amounts allocated to 

CER (e.g., by Federal fiscal year). 

•	 Excluding AHRQ, which could cite studies in its Effective Health Care program as at 

least a core set of CER, agency staff could not specify the number of CER studies 

conducted per year or other period. Three main factors account for this. First, there has 

not been a standard definition of CER. Second, while agencies may have a sense of 

expenditures or relative emphasis of CER, individual studies are typically not titled, 

given keywords, or otherwise “tagged” in a manner for identification as CER. Third, the 

time frame for CER study counts is not standardized; some agencies provided counts in 

terms of studies underway during a given year, others provided counts of studies initiated 

in a given year. Thus, providing a high-confidence estimate of the number of Federallyfunded CER studies underway for a given fiscal year is currently not feasible. Clear 

identification of CER studies, particularly prospectively, would better enable tracking of 

completed and ongoing CER. 

•	 Combined Federal CER is broadly distributed across study types (i.e., primary versus 

secondary studies). The volume of primary CER sponsored by NIH, particularly 

Randomized Controlled Trials (RCTs) and other trials, accounts for the largest general 

type of CER. 

•	 The greatest concentrations of Federal CER are systematic reviews by AHRQ, RCTs by 

NIH, and RCTs by VHA (Table 3). 

30 
 

 

 

 

 

•	 Most AHRQ CER comprises secondary research (i.e., systematic reviews and other 

syntheses) and VHA supports secondary research through its Evidence-based Synthesis 

Program. Otherwise there is little emphasis on secondary research. Moreover, 

mathematical modeling is infrequently used in Federal CER (Table 3). 

•	 Most primary research is done through RCTs (Table 3). 

•	 Without careful inspection on a trial-by-trial basis, reliable detection of “practical” (or 

“pragmatic”) trials among the primary CER studies is not possible. As a group, the VHA 

trials appear to have more such “practical” characteristics than trials sponsored by other 

agencies. 

•	 Relative to the RCT volume from NIH and VHA, the use of observational analyses, 

including those involving large patient-level databases, is relatively infrequent. 

•	 The locus of research varies by agency. All CER funded by VHA and most by DoD is 

intramural. Most CER funded by AHRQ is extramural. Although NIH conducts some 

intramural primary research, most CER is done extramurally. 

•	 The interventions studied most often in Federal CER are pharmacologic, which account 

for the majority of the interventions studied by AHRQ and NIH. These are followed by 

studies of the health care delivery system, led by VHA, and behavioral interventions 

(which are often compared to pharmacologic interventions), led by NIH and VHA (Table 

4). 

•	 Roughly 86 percent of the CER studies in this sample across agencies focus on at least 

one priority disease/condition. The leading categories among these are depression and 

other mental health disorders, substance abuse, cardiovascular disease, and diabetes 

(Appendix C). 

•	 The distribution of priority diseases/conditions studied by DoD and VHA largely reflects 

the respective populations they serve. For DoD, they are cancer, functional limitations 

and disability, and depression and other mental health disorders. For VHA, they are 

cardiovascular disease, and depression and other mental health disorders (Appendix C). 

31 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table 3: Estimated Types of CER by Agency/Department 

Study Type1 

AHRQ NIH2 DoD VHA Total 

Primary Research 

Randomized Controlled Trial 11% 79% 0% 77% 60% 

Practical/Pragmatic Controlled Trial3 3% 1% 16% 1% 2% 

Other Non-Randomized Controlled Trial 2% 2% 32% 0% 3% 

Observational Study (natural experiment) 1% 2% 0% 4% 2% 

Observational Study 

(Prospective/Registry) 4% 3% 16% 6% 4% 

Observational Study (Retrospective) 9% 5% 6% 4% 6% 

Secondary Research 

Systematic Review 58% 0% 13% 0% 14% 

Meta-Analysis 3% 0% 0% 0% 1% 

Mathematical Model 4% 3% 3% 3% 3% 

Research Training n/a4 0% 13% 0% 1% 

Other Capacity Building n/a4 0% 0% 1% 0% 

Other 2% 2% 0% 3% 2% 

1

 Some studies include more than one study design, totals may not equal 100% due to rounding. 

2 NIH 2008 (based on sample of 443 studies) plus NIH multi-year (based on 30 studies across years).
 3 Rough estimate given no standard definition for pragmatic trial.
 4

AHRQ has been heavily involved in development of human and scientific capital for CER. It provides
 

career development (K) grants for CER as well as a T and R grant for CER capability building. It also 

has funded numerous methodology studies for CER. These will be more fully quantified in the completed 

inventory.
 

Table 4: Estimated Types of Interventions Included in Studies 

Study Intervention Type1 AHRQ NIH2 DoD VHA Total 

Pharmacologic Treatment 35% 68% 24% 10% 34% 

Biologic Treatment 1% 1% 10% 4% 4% 

Alternative Medicine 2% 8% 1% 2% 

Medical Device/Equipment 17% 6% 0% 7% 11% 

Surgical Procedure 11% 3% 9% 9% 

Behavioral Intervention 11% 24% 11% 24% 16% 

Public Health Intervention 2% 1% 17% 3% 3% 

Delivery System 11% 19% 41% 20% 

Other 10% 8% 1% 2% 

32 
 

 

 

 

 

 

 

1

 Some studies include multiple types of interventions and may not total 100% due to rounding 

2 NIH multi-year. Will need to be updated once inventory based on types of NIH interventions is 

complete. 

The involvement of priority populations in CER sponsored by Federal agencies is varied. While 

several studies do not explicitly focus on a priority population, investigators sometimes report on 

analyses of one or more specific sub-groups: 

•	 About half of CER studies across these Federal agencies involve a priority population, 

with nearly 60 percent of VHA studies doing so. Many studies focus on more than one 

population group. In part consistent with their respective missions, the agencies exhibit 

different distributions of emphasis on priority populations. 

•	 Among those studies that do involve priority populations, those involving patients 

requiring chronic care, and those who are elderly are the most common. While no studies 

specifically indicate a focus on low-income groups, such individuals often comprise some 

of the patients studied, including the elderly, those with multiple chronic conditions, and 

minority groups. 

•	 Studies vary as to whether there is sufficient representation of one or more priority 

groups in the study population to enable sub-group analysis, even if the study does not 

focus on a priority population as a principal objective. Particularly at AHRQ, in cases 

where studies do not have as their primary focus a priority population, sufficient numbers 

of members of priority groups may not be present for sub-group analyses, especially in 

the case of systematic reviews. 

•	 Future iterations of the inventory will need to drill down on the representation of priority 

populations in studies. 

CER Data Infrastructure 

Substantial Federal and private sector infrastructures exist that could be used to identify potential 

CER priorities, to support the conduct and improve the productivity of CER, and to enable the 

translation of research findings into actionable information. However, the current infrastructure 

for CER is fragmented, and it is not coordinated or mobilized in a way that would enable 

providing coherent and targeted support for CER. 

Patient-level Databases and Databases to Support Researchers 

Federal agencies support or have access to substantial patient- and person-level databases that 

could support CER. Additional databases in the private sector can also deliver specialized 

content for CER. For example, these Federal and private sector databases can support or enable: 

•	 Analyses preparatory to CER, such as.: 

o	 Disease prevalence and burden to help determine priority areas for comparative 

effectiveness research. 

o	 Utilization and distribution (e.g., geographic) of alternative interventions to help 

identify variations in practice and candidate interventions for CER. 

33 
 

 

 

 

o	 Patient characteristics, socioeconomic attributes, comorbidities, and so forth, to 

determine the availability of certain patient populations for clinical trials, 

registries, and other person-level studies. 

•	 Observational studies and retrospective data analysis (e.g., mining data from natural 

experiments). 

•	 Support for prospective studies, including efficient development of registries and 

objective collection of treatment detail. 

Important considerations for investing in and applying patient/person level databases to CER 

include: 

•	 Potential to link to other databases that enrich the person/patient view, such as databases 

containing socioeconomic characteristics of individuals and mortality information (e.g., 

the Social Security Deathmaster or the CDC National Death Index). 

•	 Potential to link databases that contain clinical information to those with transactional 

information (e.g., linking claims databases that have chemotherapy detail on cancer 

patients to electronic health records or registries for the same patients that have clinical 

data such as cancer stage, histology, and patient status). 

•	 Research readiness of the databases (e.g., requiring minimal time on the part of the 

researcher to learn database attributes and develop special programs for data clean-up and 

access). 

•	 Requirement to maintain security and privacy for any personally identifiable health 

information. 

Appendix C lists some key patient-level databases with potential applications for CER. Among 

the ones available through Federal agencies are the major administrative databases maintained by 

CMS, the medical records databases at VHA, targeted databases maintained by AHRQ and NIH 

focused on service areas (e.g., HCUP on hospital-based care), and the NIH’s SEER cancer 

registry. 

Key private sector databases for CER include large administrative databases with longitudinal 

health care detail on millions of patients, and consolidated databases on EHRs. To the extent 

that these repositories can be linked (for which many have the potential), they can be highly 

valuable assets for CER, particularly because they account for commercially insured populations 

that are not captured in Federal and state databases. 

In supporting research activities, the following Federal data infrastructure assets can speed 

communication among researchers and expedite identification of researchers with special skills: 

•	 AHRQ: the DEcIDE Network, the CERTs (Centers for Education & Research on 

Therapeutics), and group of EPCs (Evidence-Based Practice Centers).
 

•	 NIH Clinical Translational Research Awards (CTSAs) recipients. 

•	 CDC: Evaluation of Genomic Applications in Practice and Prevention (EGAPP)
 

workgroup.
 

•	 HRSA research networks: Pediatric Research in Office Settings (PROS) and Emergency 

Medical Services for Children (EMSC) groups, among others 

34 
 

 

 

 

•	 SAMHSA: National Child Traumatic Stress Network 

•	 VA Research Center of Excellence 

Other databases for supporting researchers include: 

•	 ClinicalTrials.gov (Federally and privately supported clinical trials). 

•	 MEDLINE/PubMed (biomedical journal literature), HSRProj (Health Services Research 

Projects in Progress). 

•	 CRISP (biomedical research funded by NIH, including clinical trials and other studies). 

•	 Disease-oriented databases, surveys and Web sites, including the Longitudinal Studies on 

Aging (NCHS and NIA) and the Cardiovascular Health Study (NHLBI). 

•	 Survey of Mental Health Organizations, General Hospitals Mental Health Services, and 

Managed Care Organizations (SAMHSA). 

•	 Numerous CDC disease and research data assets and sites, including the NCHS
 

surveillance systems, cancer registries, and vaccine registries.
 

None of these databases with actual or potential applications in CER were developed for the 

explicit purpose of comparative effectiveness research. Furthermore, they generally have not 

been organized or indexed to enable searching for CER. For example, careful record-by-record 

inspection of such research study databases as ClinicalTrials.gov and CRISP is required to 

identify CER. In order to assess current gaps and support translation and adoption of CER 

findings efficiently, these databases would require “tagging” of records or related searching 

functions that would enable accurate identification of CER.27 

Dissemination and Translation Infrastructure 

A few agencies, notably AHRQ, VHA, NIH, and SAMHSA, have capacities to translate CER 

into actionable information for practitioners, patients, and other target audiences. The VHA’s 

capabilities for translation and adoption are inherent in its integration of research and patient care 

at VHA treatment centers. Additional agencies also have capabilities for disseminating 

information to segments of consumers and practitioners. All of these agencies have the potential 

to influence adoption of CER findings. 

There are, however, minimal formal mechanisms to disseminate and translate CER from research 

agencies such as AHRQ and NIH into the delivery system side of HHS (e.g., HRSA, IHS, 

SAMHSA, CMS QIO’s). In addition, given the current expansion of CER and the increased 

emphasis on achieving impact from its findings, the current dissemination and translation 

capacity of the relevant agencies involved in CER is likely to be insufficient for achieving CER’s 

potential. 

Some of the key elements that can be leveraged in a comprehensive and articulated CER 

dissemination and translation strategy are outlined below. 

27 In MEDLINE, for example, indexing tags for particular “publication types,” such as Randomized Controlled 

Trial, Clinical Trial, Phase III, Meta-Analysis, and Review, would readily enable searching for journal articles that 

report such studies. 

35 
 

 

 

•	 AHRQ 

o	 CER methods guides, tools, and resources made available via the AHRQ Web site, 

Web conferences, public service announcements, advertising campaigns, online audio 

guides available to public, and other means for informing consumers, clinicians, and 

policymakers. 

o	 The John M. Eisenberg Clinical Decisions and Communications Science Center, 

which focuses on translation of research to various target audiences. 

o	 AHRQ Publications Clearinghouse. 

o	 AHRQ dissemination partnerships, including with health professional societies, 

patient advocate groups, and non-profit organizations focused on particular 

diseases/conditions. 

o	 Effective Health Care Program Stakeholder Group, which helps to identify important 

information gaps, ensure transparency, and provide feedback on reports. 

•	 CDC 

o	 Information to monitor the adoption of CER recommendations and to track the effects 

from changes in clinical practices and policies on the following process and outcomes 

measures: clinical management of specific conditions, including the use of 

medications and other specific services, and intermediate health-related outcomes, 

such as test results; incidence and prevalence of specific conditions; personal 

behaviors, health status, and functioning; and births and deaths. 

o	 Public use data from NCHS surveys available through the CDC/NCHS Web site and 

internal confidential data available for researchers through the NCHS Research Data 

Centers. 

o	 NCHS/CDC reports, including Data Briefs and E-Stats, and other analyses available 

through the Web site, and articles in the peer-reviewed literature. 

o	 Dissemination by and with collaborators, including sponsors of specific data
 

collection and analysis.
 

•	 DoD 

o	 Searchable publication libraries, including the Military Health System Publication 

Search. 

o	 DeployMed Research Link, which informs Service members, researchers, health care 

providers, military leaders, and others about DoD and other Federally funded medical 

research related to deployments since 1990. 

•	 NIH 

o	 Clinical and Translational Science Awards (CTSAs), which are NIH-funded 

academic centers that translate research into practice.
 

o	 Nation Cancer Institute’s Physician Data Query, an online database that summarizes 

study results in prevention, screening and management of cancer in versions 

appropriate for physicians and for patients. 

o	 Research databases, including MEDLINE/PubMed, HSRProj, CRISP, and 

ClinicalTrials.gov.
 

36 
 

 

 

 

 

 

 

o	 Public health campaigns, such as Red Dress (women’s heart health) and Small Steps 

Big Rewards (weight loss). 

o	 NIH Consensus Development Conference program, which summarizes knowledge 

about a variety of clinical and public health interventions. 

•	 VHA 

o	 QUERI (Quality Enhancement Research Initiative) program for enhancing the uptake 

of evidence within VHA. 

o	 Periodic research summaries and issues briefs for senior VHA clinical and policy 

leaders, and related research results disseminated to researchers. 

o	 CME programs for nurses and other health professionals that incorporate recent 

research findings. 

o	 Print and online patient education tools, including the MyHealthE Vet Web site, for 

dissemination to patients. 

o	 Point-of-service decision-support tools and reminders to clinicians within the VHA 

EHR system guiding practice toward the most effective treatment, including a Web 

portal for clinicians to access clinical practice guidelines. 

•	 SAMHSA 

o	 National Registry of Evidence-based Programs and Practices (NREPP) and the 

Technical Assistance Centers can serve as translation vehicles. NREPP is a 

searchable online registry of approximately 140 mental health and substance abuse 

interventions and targeted outcomes; it provides quality of research and “readiness for 

dissemination” ratings. 

o	 The Addiction Technology Transfer Center (ATTC) Network is comprised of 14 

Regional Centers and a national office which facilitates alliances among providers, 

administrators, and recovery and treatment communities, and connects them to the 

latest research and information through activities such as skills training, academic 

education, online and distance education, conferences, workshops, and publications. 

o	 The National Centers for the Application of Prevention Technologies (CAPT) work 

to bring research to practice by assisting States/Jurisdictions and community-based 

organizations in the application of the latest evidence-based knowledge to their 

substance abuse prevention programs, practices, and policies. 

o	 The SAMHSA Health Information Network (SHIN) provides a one-stop, quick 

access point that connects the behavioral health workforce and the general public 

with the latest information on the prevention and treatment of mental and substance 

abuse disorders. 

•	 FDA 

o	 Web site provides news and other information to physicians and consumers on drugs, 

biologics, and devices. 

•	 Office of Public Health and Science (OPHS) 

o	 Comprises 12 core public health offices and the Commissioned Corps, some of whom 

work with population and community-based networks to disseminate health 

information (e.g., Office of Disease Prevention and Health Promotion, Office of 

37 
 

 

 

 

 

 

 

Minority Health (OMH), Office on Women’s Health). OMH, for example, has 

cooperative agreements and other partnerships to disseminate research findings 

(though not CER to date) to minority populations. 

•	 Office of the National Coordinator for Health Information Technology (ONC) 

o	 Efforts to develop and implement a nationwide, interoperable health information 

technology infrastructure could provide a means for incorporating CER into decisionsupport systems for clinicians and other applications in health care. 

•	 HRSA 

o	 Among multiple dissemination vehicles, the AIDS Education and Training Centers 

Program and the Ryan White HIV/AIDS Program support a network of 11 regional 

centers and more than 130 associated sites that conduct targeted, multidisciplinary 

education and training programs for health care providers treating people living with 

HIV/AIDS. 

o	 HRSA’s Maternal and Child Health Bureau disseminates information using 

cooperative agreements with professional organizations and academic institutions, 

and funds grants for continuing education to academic centers across the country, 

specifically for the purpose of translating research into practice. 

There is virtually no capacity to track the impact of CER dissemination, translation, and adoption 

activities. As a result, this limits the ability to measure the impact of CER and to conduct 

research on effective approaches. Claims databases could be one resource for tracking changes 

in practice over time and their impact. 

Human and Scientific Capital 

The future workforce engaged in CER should include experts from a wide array of disciplines, 

including biostatistics, epidemiology, mathematics, economics, and ethics. To date, however, 

there has been little focus on human and scientific capital infrastructure for CER. The principal 

exception is the close affiliation of certain AHRQ activities involving academic centers and other 

organizations, including the DEcIDE network, CERTs, EPCs, the Eisenberg Center, and various 

awards to researchers. AHRQ funding of DEcIDE network members and EPCs supports 

research trainees at those organizations. AHRQ also provides career development (K) grants 

focusing on generation of new scientific evidence and analytic tools that enable the prioritization 

of evidence-based services and goals for patients with multiple comorbidities.28 In addition, 

AHRQ has sponsored other scientific and methodological activities, including development of 

methods guides, training seminars, and related events (e.g., at AcademyHealth and other 

professional conferences), and various workshops and support materials on MEPS, HCUP, and 

other data sets.

 NIH provides significant training opportunities that could incorporate CER, including support 

for medical students interested in research, clinical fellowships, workshops for researchers, 

training grants, and consensus conferences. The CTSA program at NIH provides translational 

development support at academic and other research centers, some of which addresses evidence28 http://grants.nih.gov/grants/guide/notice-files/NOT-HS-08-004.html. 

38 
 

 

 

 

 

based medicine approaches, if not CER in particular. The NIH K30 Clinical Research 

Curriculum Awards support training in design of clinical research projects, hypothesis 

development, biostatistics, epidemiology, disease mechanisms, medical technology, human 

genetics, and the legal, ethical, and regulatory issues related to clinical research.29 

Although DoD has an extensive training and professional education infrastructure, it does not 

focus on CER. 

A small number of training programs at academic centers focus on areas that address 

methodologies and study designs related to CER. Among these are the Clinical Research 

Training (CREST) program at Boston University, which provides training in clinical research 

that includes epidemiology, clinical epidemiology, health services research, biobehavioral 

research, and translational research,30 and the Duke Clinical Research Training Program, which 

provides training in quantitative and methodological principles of clinical research, including 

research design, research management, medical genomics, and statistical analysis.31 

Several agencies draw on the considerable scientific and methodological expertise resident in the 

FDA, but there is little emphasis on comparative effectiveness research at that agency. These 

informal links to scientific expertise could be formalized; also, specific CER expertise could be 

housed in selected agencies with an expectation of a cross-agency role. FDA expertise would be 

of exceptional value in, for example, understanding the respective merits of alternative study 

designs for assessing efficacy vs. effectiveness and for collecting and assessing adverse event 

data, strengths and limitations of using surrogate endpoints and other biomarkers in CER, 

incorporation of genomics and other aspects of personalized medicine into CER. Phase III and 

phase IV studies could also generate evidence on comparative effectiveness, as well as on other 

scientific and methodological aspects of CER. 

Despite the promise of “practical” or “pragmatic” trials for CER, methodological gaps and 

threats to internal and external validity remain. Real world trials must deal with confounders, 

including confounding by indication and presence of comorbidities, selection bias, and other 

factors that impede the assessment of cause and effect. Focused research to improve the validity 

of practical trials and interpretation of their findings could enhance the use of these study 

designs. 

Further development of mathematical modeling approaches and retrospective data analysis 

capabilities would also provide alternative means of analyzing comparative effectiveness, as well 

as generating viable research hypotheses and providing input for designing primary and 

secondary CER. 

Gaps in the Current CER Landscape and Investment Opportunities 

The inventories of CER and CER data and research infrastructure reveal gaps and other 

challenges for achieving the potential of comparative effectiveness research. 

29 http://grants.nih.gov/training/K30.htm. 30 http://www.bumc.bu.edu/clinepi/crest/general-info/ 31 http://crtp.mc.duke.edu/content.asp?page=about 

39 
 

 

 

Coordination across the CER framework: Substantial CER assets exist across the Federal 

Government, but coordination is necessary to capture their full value. Several challenges exist in 

achieving this: 

•	 Prior to this report, there was no standardized Federal definition for CER; aligning 

organizations around this definition will be necessary for identifying, cataloging, and 

disseminating CER in a coordinated manner 

•	 Difficulty in setting national CER priorities. 

•	 Structural barriers that limit collaborations among agencies. 

•	 Limited coordination with private sector CER efforts. This includes lack of integration of 

existing data sets across payers, suboptimal development of CER data infrastructure, an 

inability to track populations and treatments across payers, and suboptimal translation 

and adoption of CER findings. 

•	 Unrealized benefits of stakeholder involvement. Greater involvement of stakeholders 

(e.g., patient advocates, health professionals, researchers, technology manufacturers, 

payers) in CER processes can help to achieve the goals of CER, including more informed 

priority setting, input on certain aspects of study design (e.g., identification of important 

subgroups and patient-centered outcomes), and identification of target audiences for CER 

and strategies to reach them. 

Research: Despite the comparative effectiveness research to date, there are many unanswered 

questions. 

•	 Those who sponsor and design clinical trials continue to face challenges in tradeoffs 

between internal validity of CER for causal effects of interventions on outcomes and 

external validity of CER to heterogeneous patient groups and routine health care settings. 

•	 Increased emphasis on well-conducted pragmatic trials could increase acceptance of CER 

findings. 

•	 May research questions for important clinical health care decisions remain unanswered 

Human and scientific capital: Due in part to the increasing interest in comparative 

effectiveness research, continued investment in human and scientific capital for the field is 

needed. 

•	 Greater investment is needed in developing education and training programs to support 

the development of professional talent, the development of methods for linking and using 

databases for CER, the development of new methodologies for pragmatic trials, effective 

translation and adoption of CER findings into practice, modeling approaches for CER, 

and evaluation of the impact of CER 

•	 More methods work is needed to advance the state-of-the-art for pragmatic trials and to 

provide training for using these study designs. 

•	 Recent growth in training for the related fields of health technology assessment, 

outcomes research, and health economics, among others, has helped to yield a cohort of 

researchers who are well-positioned to become more expert in CER, along with 

40 
 

 

 

 

 

 

 

educational curricula and materials that can be adapted for training in comparative 

effectiveness research. 

CER Data Infrastructure: The scope and scale of CER requires data infrastructure that may 

outstrip current capabilities. 

•	 Current data sources are fragmented and limited in terms of clinical robustness and 

longitudinal data capture. 

•	 An evolving inventory of CER data infrastructure is needed to track the capacity of this 

infrastructure and provide a basis for its further development; this inventory should include 

observational databases, registries, claims and other administrative data, pharmacy and 

laboratory data, adverse events registries, EHR networks, and other health information 

technology. 

•	 In addition to one or more inventories, greater understanding is needed regarding the 

strengths and limitations of these data sources, and areas for their further development. An 

example of a relevant resource is the 2007 Registries for Evaluating Patient Outcomes: A 

User's Guide, produced by the AHRQ’s DEcIDE Research Center. 

•	 Investment in linking such data sources is more likely to be realized by establishing clear 

information policies and technical standards, standardized terminology, improved platform 

capability, novel search algorithms, mechanisms to maintain patient privacy, and controls to 

access data, and by reducing and coordinating data processing times.32 

•	 There are few searchable electronic inventories or related databases of CER and CER 

infrastructure. While sources like ClinicalTrials.gov, CRISP, MEDLINE, and HSRProj 

contain information about completed and ongoing CER, but they are not presently 

configured or linked to serve the needs of CER. 

•	 Absence of an inventory of CER limits the ability to assess the magnitude and nature of 

the current portfolio of completed and ongoing CER, to identify CER on particular 

topics, and to inform priority-setting for CER. 

•	 A comprehensive inventory of CER infrastructure would improve the ability to conduct 

CER and to allocate resources to develop the national capacity to conduct CER. 

CER Dissemination and Translation: Many findings to date from CER have not yet been fully 

integrated into clinical practice or made accessible to patients in easy-to-understand language. 

•	 Certain effective dissemination avenues are in place, including among some of the 

agencies engaged in CER. Except for AHRQ, however, these agencies are not yet 

oriented to CER and do not adequately extend beyond dissemination alone to translation 

and adoption of CER into practice. 

•	 Tools and mechanisms to support clinicians and patients in incorporating available CER 

information are lacking. This information needs to be delivered to the front line of care 

where health decisions are made and results measured. 

32 See, for example: Diamond CC, Mostashari F, Shirky C. Collecting and sharing data for population health: a new 

paradigm. Health Aff (Millwood) 2009;28(2):454-66. 

41 
 

 

 

 

Priority populations and other sub-groups: At present, the agencies have largely separate 

approaches to addressing these groups. A better-coordinated Federal approach is needed to 

address priority populations and priority conditions, including sub-groups with multiple chronic 

conditions. 

•	 Greater attention on designing studies with sufficient power to discern treatment effects 

and other impacts of interventions among patient sub-groups (e.g. accounting for 

heterogeneity of treatment effects) will better serve clinical decision-making, enabling 

more individualized, patient-specific care. 

•	 Improved partnerships with Federal grantees serving priority populations, such as
 

Community Health Centers, will enhance their engagement with CER.
 

•	 Improved access to and utilization of Federally sponsored databases that include priority 

populations can significantly enhance the inclusion of sub-groups into CER. 

Types of interventions: To date, CER has been disproportionately focused on pharmacologic 

treatments rather than the full spectrum of intervention types. This likely derives in part because 

of the relative emphases of the research agendas of agencies that sponsor CER and the focus of 

the private sector is primarily on new drugs and biologics. The emphasis on pharmacologic 

treatments has meant fewer resources for other interventions, including behavioral, procedures, 

prevention, and delivery system interventions, that can have major impacts on health outcomes. 

VII. PRIORITY-SETTING PROCESS 

The Council actively sought public input throughout this process, and this input significantly 

influenced all Council decisions. To help guide the Council’s deliberations on the definition, 

framework, and priorities for comparative effectiveness research, the Council held three listening 

sessions and solicited additional public comments online. The Council heard from over 300 

stakeholders representing health care associations; consumer, community, and advocacy 

organizations; academia and think tanks; patients; providers; hospitals and hospital systems; 

payers; pharmaceutical companies; foundations, public health entities; and private sector 

companies engaged in the health care field. One U.S. Senator also submitted comments. 

Several respondents honed in directly on the reason why investments in CER are important. One 

person, for example, said that CER is crucial to reforming the practice of medicine to increase 

the quality, safety, value, and effectiveness of what providers bring to patients on a daily basis. 

Other respondents addressed a wide range of interrelated issues, including priorities for the 

research agenda, collaboration, infrastructure development, research methodology, transparency, 

care delivery, cost, and knowledge transfer. Many patients expressed their need for this type of 

research; one of the most emotional and moving testimonies came from the mother of a child 

with a seizure disorder in Chicago who had struggled to find the best treatment for her child. A 

physician from the American Board of Orthopedics summarized many physicians’ testimony by 

saying, “developing high quality, objective information will improve informed patient choice, 

shared decision-making, and the clinical effectiveness of physician treatment recommendations.” 

42 
 

 

 

 

 

 

 

The public input has been extremely valuable in informing the Council’s deliberations, and many 

of the major thematic threads that run thought the public comments are reflected in the strategic 

framework, focus, and recommendations for priorities for OS CER funds. Details about what the 

public had to say are contained in Appendix A. 

The Council also conducted a first draft inventory of CER and data infrastructure (outlined 

above) to help identify gaps in the current CER landscape. For the Office of Secretary funding 

recommendations, the Council proceeded through structured deliberations informed by public 

input, developed an inventory of current activities, established prioritization criteria and a 

strategic framework, and discussed the unique role for OS funds to fill gaps and build the 

foundation for future CER. In the future, the Council should continually and actively engage 

stakeholders inside and outside the government, including patients, providers, payers, employers, 

industry, academia, and others. This critical component of the priority-setting process could take 

the form of even more active participation by external stakeholders in the future. 

VIII. PRIORITY RECOMMENDATIONS FOR OFFICE OF SECRETARY CER FUNDS 

Using the strategic framework for CER discussed in Section V, and taking into consideration the 

unique role that OS funds can play in addressing high priority gaps, the Council developed a 

recommended high-level investment strategy for the use of the OS ARRA funds. The strategy 

has three different levels of priority recommendations for OS fund investments in the Core 

Activities and Cross-cutting Priority Themes in the CER framework (Figure 3). 

•	 Primary investment. This area of investment should represent a large portion of the OS 

funds. It best fulfills the full range of prioritization criteria and requires scaled 

investment in order to be successful. The Council recommends that CER Data 

Infrastructure be the primary investment. 

•	 Secondary investments. These areas should also receive significant investment. They 

are as critical to success in CER as the primary focus, but individually may require a 

smaller amount of funding to be successful. The Council recommends that 

Dissemination and Translation of CER, Priority Populations, and Priority Types of 

Intervention be secondary investments. 

•	 Supporting investments. These areas should not be the major focus of OS funding as 

they do not fulfill the prioritization criteria as well as primary and secondary investments, 

but some funding may be necessary to support and enable investments in higher priority 

areas and fill identified gaps. The Council recommends that Human and Scientific 

Capital, Research, and Conditions receive supporting investments. It is important to note 

that these recommendations pertain only to OS funds; AHRQ, NIH, and VA have a 

history of significant investments in Research, Human and Scientific Capital, and 

Conditions. 

43 
2

 

 

 

 

Figure 3 

Recommended High Level OS Investment Priorities 

Human & 

Scientific Capital 

for CER 

Research 

CER Data Infrastructure Dissem ination and 

Translation of CER 

Priority Populations 

Priority Conditions 

Types of Interventions 

Cross-cutting 

Priorit y 

Themes 

Legend 

Primar y inves tment Seco ndar y i nvest ments Supporting investments 

The Council believes that this strategy and distribution of investments will best position the 

Secretary to: 

• Respond to patient and physician demand for CER. 

• Balance achieving near-term results with building longer-term opportunities. 

• Capture the distinctive value of the Secretary’s ARRA funds. 

While it is the responsibility of the Office of the Secretary to operationalize this strategy, the 

Council’s rationale for these recommendations is designed to help guide the Secretary in making 

specific investment decisions. The Council based its rationale for each level of investment in the 

strategy on the prioritization criteria described above, as well as representative examples of 

investment in each area proposed through the public comment process and by Federal agencies. 

Primary investment 

CER data infrastructure development is the most distinctive opportunity for OS ARRA funding. 

It requires a large, up-front infusion of capital to be successful that is unlikely to come from any 

source other than OS ARRA funds, making it ideal for this funding mechanism. It has broad 

potential impact, with the ability for resulting research to address conditions and populations 

captured in the primary data. Given the absence of comprehensive databases and data evaluation 

44 
 

 

tools (See Section VI), there is significant demand from the patient, clinical, and public health 

communities for new, expanded data infrastructure and data access to support decision-making. 

Finally, investments in data infrastructure have the potential to generate significant additional 

investments in two ways. First, some of these investments could take the form of public-private 

partnerships. Second, data infrastructure is a tool that, once developed, will result in new 

research conducted and/or funded by entities such as biomedical research organizations, payers, 

foundations, and health care providers. 

The Council received proposals on a number of potentially promising initiatives related to data 

infrastructure, including but not limited to: 

•	 Building, expanding, and linking longitudinal administrative claims databases. 

•	 Linking administrative data with EHR-based or registry data. 

•	 Expanding high-impact patient registries, (e.g., collaborations with specialty
 

organizations, SEER). 

•	 Distributed data networks populated by EHRs in practice and provider settings. 

•	 Expanding analysis of FDA and private sector data on drug and device trials and safety. 

As the Office of the Secretary identifies specific opportunities in data infrastructure, the Council 

recommends that it consider most carefully those that: 

•	 Expand access to existing resources, especially those currently managed by Federal 

agencies. 

•	 Create scaled platforms by leveraging existing data and capabilities in the private sector. 

•	 Capitalize on linkages between health IT investments and the potential for CER
 

infrastructure to develop evidence to inform decision-making.
 

•	 Ensure that infrastructure is responsive to needs of patients, providers, and other
 

decision-makers—and not driven by what is most feasible.
 

The Council appreciates the relationship and need for coordination between CER and health IT 

(e.g. through a distributed network of EHRs) investments. As the Secretary develops HHS’s full 

portfolio of ARRA investments, it will be critical to consider both CER and health IT 

holistically, not as policy silos, recognizing that success in CER is largely dependent on success 

in health IT and vice versa. 

With all data infrastructure investments, the government will need to ensure data security and 

privacy. Protecting security and privacy is key to maintaining the public’s trust. 

Secondary investments 

Secondary investments include a core area of investment—Dissemination and Translation of 

CER—and two cross-cutting themes—Priority Populations and Types of Intervention. 

Dissemination, translation and adoption of CER is about realizing the benefits that comparative 

effectiveness research has to offer both patients and providers. While the breadth and depth of 

the near-term impact depends on what types of pilot programs the OS supports, the lessons and 

tools for translation developed by those pilots will be relevant to all. 

45 
 

 

 

 

 

The lack of reliable success in disseminating findings from CER in ways that translate into better 

health outcomes highlights the uncertainty and difficulty of this enterprise. However, 

dissemination and translation is essential to improving outcomes for patients and the link 

between evidence production and how best to get this information to physicians and patients in a 

way they understand is critical to capitalizing on the CER investment. Despite important efforts 

by the Federal Government, especially AHRQ, NIH, VA and DoD, the majority of current 

funding goes to building evidence as opposed to ensuring that the existing evidence base is 

utilized in patient care and health systems management. This creates a unique role for OS ARRA 

funding. Investments in dissemination and translation programs also have the potential to 

generate additional investments, especially from providers, if private institutions elect to 

implement similar efforts or partner with the Federal Government on translation efforts. 

There are a wide range of potential dissemination, translation and adoption programs that the OS 

could support, including: 

•	 Investing in dissemination and translation of CER findings throughout the Federal 

delivery system. 

•	 Dissemination and translation through partnerships with provider and/or patient
 

organizations. 

•	 Decision support and shared decision-making tools to provide information to clinicians 

and patients at the point of care. 

•	 Developing standards for communication tools for patients and providers, (e.g., a patientfriendly simple scoring system). 

•	 Partnering with an existing consumer media channel (e.g., Internet search engine or 

health information site) to expand patient access to existing CER data. 

•	 Creating a National Patient Library with a primary focus on providing evidence to 

patients in easy-to-use and understandable formats. 

The Council recommends that the Office of the Secretary consider the following in making 

investments in dissemination and translation: 

•	 Investing in better understanding the most effective methods to disseminate and translate 

research findings to improve patient outcomes. 

•	 Identifying opportunities both to develop tools for translation and to pilot implementation 

of these tools. 

•	 Partnering with provider organizations in Federal agencies, as well as in states and the 

private sector. 

•	 Accounting for potential surrogate decision-makers (e.g., families) and the context for 

decisions in patient-focused tools. 

•	 Ensuring that programs address a specific need articulated by the implementing 

organization or the partner to ensure success and the sustainability of dissemination 

activities. 

•	 Focusing on developing standards for communication. 

•	 Increasing understanding of the most effective methods to disseminate findings to 

clinicians and patients to inform decision-making
 

46 
From an operational perspective, investments in the cross-cutting themes are somewhat distinct 

from investments in the core areas. Whereas funding for a core area might go to a project or 

organization focused on a specific activity, funding for a cross-cutting theme requires multiple 

coordinated investments and activities to be successful. Investments in these themes could cover 

some or all of the four core activities: research, data infrastructure, human and scientific capital, 

and dissemination and translation. These investments could involve a coordinated investment 

across HHS or the Federal Government, or they could be focused in academic centers, integrated 

delivery system organizations, private industry, or other non-governmental entities. 

Collaborative efforts to inform and transform care will be essential to achieving meaningful 

impact across these cross-cutting themes. 

Investments in specific populations, meanwhile, will help ensure that the benefits of CER are 

available to all. It can also focus CER efforts on populations with existing health disparities and 

worse outcomes. CER has the potential in some populations, such as racial and ethnic 

minorities, to fill critical gaps that, historically, efficacy research has left unaddressed. 

The Council identified several populations for whom the Secretary should consider allocating 

CER funds: 

•	 Racial and ethnic minorities 

•	 Persons with disabilities 

•	 Elderly 

•	 Children 

•	 Patients with multiple chronic conditions 

Investment in specific types of interventions in a cross-cutting manner also presents a unique 

opportunity for the nation’s health system. The Council has identified six specific interventions 

for the Secretary to consider that address large and varied populations, resulting in high potential 

impact, are areas of high clinical uncertainty, and are not being adequately addressed by other 

entities. They are: 

•	 Medical and assistive devices (e.g., comparing rehabilitative devices). 

•	 Procedures and surgery (e.g., evaluating surgical options or surgery versus medical 

management). 

•	 Diagnostic Testing (e.g. comparing imaging modalities for evaluating certain types of 

cancer) 

•	 Behavioral change (e.g., developing and assessing smoking cessation programs). 

•	 Delivery system strategies (e.g., testing two different discharge process care models on 

readmission rates or testing two different medical home models on preventing hospital 

admissions and improving quality of life). 

•	 Prevention (e.g., comparing two interventions to prevent or decrease obesity, comparing 

strategies for reaching populations that do not access the health care system with 

prevention efforts). 

47 
 

Furthermore, the Council recommends that the Office of the Secretary consider the following in 

making investments in the cross-cutting themes of priority populations and types of 

interventions: 

•	 Focusing on immediate, specific patient needs that can generate results. 

•	 Concentrating on areas with cross-cutting gaps in research, data infrastructure, scientific 

capital, and/or translation. 

•	 Building on promising systems and practices already in place, both within the
 

government and in the private sector, and measuring results when scaled up and 

disseminated.
 

•	 Strongly encouraging coordination across the government and with entities outside of the 

government. 

Supporting investments 

The Council recommends that the OS reserve some ARRA funding for Research, Human & 

Scientific Capital, and the Conditions cross-cutting theme. Because these investments and topics 

are the major foci of CER activities at NIH and AHRQ, both of which will likely utilize ARRA 

funds administered by those organizations for these purposes, they do not represent distinctive 

investment for OS funds. However, there will likely be targeted investments in these areas that 

could support other OS ARRA efforts, such as training new researchers in CER methods or 

addressing gaps not addressed elsewhere in the Federal Government. 

In making these targeted investments, the Council recommends the Office of the Secretary 

consider: 

•	 Focusing on areas that maximize the value of the Secretary’s investments in other areas. 

•	 Avoiding duplication of efforts with other agencies. 

For all of the above investments, the Council recommends that the Office of the Secretary 

consider the portfolio of investments and where synergies exist to leverage one investment into 

multiple areas. For example, a data infrastructure investment that can also be used for a crosscutting priority theme would be of higher value than an investment that has more limited 

applications. Doing so will help to ensure that the funds allocated to the Office of the Secretary 

for CER will have a significant positive impact on the quality of patient care in the near term, 

and lay the foundations for continued improvements going forward. 

IX. LONGER-TERM OUTLOOK AND NEXT STEPS 

Outlook 

The CER investment strategy recommended in the previous section, if implemented successfully, 

has the potential to further a number of elements of the Council’s vision for improved patient 

care. In the near term: 

48 
 

 

4

 

•	 CER dissemination and translation pilots in Federal health care delivery systems could 

help empower patients and their physicians to make better decisions now about their care 

and serve as models for expansion to private delivery systems. Moreover, a time limited 

investment can support establishment of a systematic strategy for translating the products 

of all relevant research to benefit patients served by Federal programs. 

•	 An increased emphasis on CER for priority populations could ensure that all will benefit 

from comparative effectiveness research. 

•	 Improved access for researchers to existing Federal data sources, and development and 

enhancement of distributed data networks and patient registries outside of the Federal 

Government, could jump-start a new wave of CER in the areas that matter most to 

patients. 

•	 The inventory of Federal activities in CER will help reduce duplicative or uncoordinated 

investments among Federal agencies and help create transparency for patients. 

It will be essential that a continuous cycle of CER priority-setting and evaluation of impact take 

place. The four critical steps in this cycle are inventory of CER and infrastructure; gap analysis; 

priority setting; and evaluation of impact. Figure 4 below depicts this process. 

Figure 4 

Continuous Evaluation of CER Inventory, Gaps, Priority 

Setting, and Impact 

CER priority setting and 

evaluation process 

Initial report 

on CER 

assets and 

capabilities 

Inventory data collection 

What are the government’s 

current assets and 

capabilities in CER? 

Gap analysis 

Where are the gapswe need 

to fill based on the latest 

inventory? 

Prioritization and investment 

How do we invest based on the 

gaps and our prioritization criteria? 

Impact evaluation 

What effect have our
 

investments had?
 

The strategic framework for comparative effectiveness research is intended to lay the 

groundwork for longer-term initiatives, such as innovative public-private partnerships to build 

data infrastructure and conduct CER. The goal of this investment is to generate some near-term 

results and momentum for the future. This strategy allows the government to facilitate the 

building of needed infrastructure, to expand access to existing infrastructure, and to demonstrate 

proof of concept for implementation efforts. These efforts are only a first step, however, to 

49 
 

 

 

 

achieving the vision of improved quality, safety, efficiency, equity, and patient satisfaction 

through improved medical decision-making and an optimized health care delivery system. 

Indeed, a number of clear issues and challenges will remain for Federal CER efforts in the near 

term: 

•	 Listening and Engaging External Stakeholders. While the Council has worked to 

create transparency and responsiveness in this process to date, it is critical that it continue 

to have a patient-centered focus going forward. In the future, the Federal Government 

may want to consider the options of listening and engaging stakeholders with 

representatives including, at minimum, patients, providers, payers, employers, and 

industry representatives, to guide CER or broader patient-centered outcome efforts. 

•	 Continued Coordination. The Council laid the groundwork for coordination, but 

coordination will need to become embedded across the Federal Government. As the 

government makes investments in CER, there must be a mechanism in place to track and 

coordinate these investments and avoid duplication of efforts. 

•	 Building Scientific and Human Capital. To maximize the potential benefit of 

investments in CER, the nation needs more researchers trained in the applicable research 

methods and further development of these methodologies. This presents both a short-term 

and a long-term challenge. 

•	 Maintaining Gains. These investments represent only the beginning of CER efforts. 

New research findings will need to be disseminated and successful translation of 

evidence efforts should be expanded. New databases and data sets need to be maintained 

and kept current, and the catalog of Federal activities and data infrastructure in CER 

needs to become a living document. 

•	 Building Leverage. The Federal Government is now a major funding source of CER, 

but the private sector still represents a majority of the investment in biomedical research. 

The government needs to find innovative ways of partnering with the private sector to 

leverage government investments and help private-sector investment better serve patients. 

•	 Keeping it Current. There are no widely accepted and applied common standards or 

approaches for periodically re-evaluating CER to ensure that previous conclusions still 

hold. A system must be developed to ensure that the conclusions from CER remain valid 

over time. 

These issues reflect both the fact that comparative effectiveness research remains in its infancy 

and that it must be seen as a continuous and iterative process that needs to constantly evolve 

based on the changing needs of the patient. 

Next steps 

There are a number of next steps leading to the Secretary’s integrated strategy and spend plan for 

CER on July 30, 2009, and several requirements for the success in implementation of that 

strategy. Most importantly, it is critical to the success of CER and health care transformation 

that the plan is coordinated across the ARRA CER funding allocated to the Secretary, AHRQ, 

and NIH. Furthermore, the Secretary should develop the plan as part of HHS’s broader portfolio 

of ARRA investments, not as a stand-alone program. 

50 
 

 

 

The following steps are needed to finalize the CER operational strategy prior to the July 30 

deadline mandated by Congress: 

•	 Integrate IOM and Council strategic recommendations and leverage the investments, 

resources, and capacity identified through the initial inventory effort. 

•	 Provide more specific recommendations for a portfolio of initiatives for all of HHS’ 

ARRA funds within the framework outlined in the report. 

•	 Define metrics for evaluating success. 

•	 Coordinate the submission of the CER ARRA spend plans to ensure that they cover the 

gaps in CER outlined in this report, and that the agencies’ investments leverage the 

strengths of each agency and are complementary, not duplicative. 

•	 Maintain transparency and engagement with the public. 

 The Federal Government will need to continue its work to coordinate CER investments and 

assure Americans that these resources are being invested wisely. ARRA required that the 

Council submit an annual report regarding its activities and recommendations concerning the 

infrastructure needs, organizational expenditures, and opportunities for better coordination of 

comparative effectiveness research by relevant Federal departments and agencies. The first 

annual report will likely be in June 2010. 

Comparative effectiveness research is being considered as a key piece of health reform, and the 

Federal Government must demonstrate its capability to coordinate that investment, achieve 

impact, and measure the results. This report outlines the priorities and path forward. Now the 

Federal Government must make progress and deliver results for the American people. 

51 
 

 

 

 

 

APPENDICES 

Appendix A. LISTENING SESSIONS AND PUBLIC COMMENT SUMMARY 

Overview 

In order to help guide the Council’s thinking on the definition, framework, and priorities for 

comparative effectiveness research (CER), the Council held three listening sessions (April 14 in 

Washington, D.C.; May 13 in Chicago; and June 10 in Washington, D.C.) and solicited public 

comments through the hhs.gov/recovery Web site. The response was strong: 

• 92 panelists testified 

• Greater than 300 individuals and organizations submitted comments 

A breakdown of the responses by stakeholder type reveals that over half of the comments came 

from three groups: health care associations; academia and think tanks; and consumer, 

community-based, and advocacy organizations. Additional comments were received from 

patients, providers, payers, hospital and health systems, pharmaceutical companies, foundations, 

public health entities, and private sector companies in the health care field. One U.S. Senator 

also submitted comments. 

More important than the diversity of respondents is what individuals and organizations had to 

say. Respondents provided a wide range of opinions and offered recommendations on 

everything from stakeholder participation to how to prioritize investments in CER to specific 

areas of focus to knowledge transfer and dissemination. 

Without question, the public input has been extremely valuable in informing the Council’s 

deliberations; many of the major thematic threads that run through the public comments are 

reflected in the Council’s strategic framework, focus, and recommendations for priorities for OS 

CER funds. Of particular value to the Council was the opportunity to engage with panelists at 

the listening sessions. This back-and-forth discussion enabled Council members to refine their 

ideas and solicit further feedback. 

Before summarizing the key themes, it is useful to note that several respondents honed in directly 

on the reason why investments in comparative effectiveness research are important—CER 

matters. For example, one respondent talked about the value of and application of CER for 

everyone’s health and health care. Another talked about how funding is crucial to reforming the 

practice of medicine to increase the quality, safety, cost benefits, and real world effectiveness of 

what providers bring to patients on a daily basis. 

One theme that wove through many of the comments was the need for greater collaboration 

among Federal agencies, among organizations at the Federal, state and local levels, and between 

the public and private sectors. One respondent stated that because expertise on comparative 

effectiveness research resides in both public and private entities, every effort should be made to 

encourage public-private collaboration in the design, conduct, analysis, and reporting of CER. 

This discussion about collaboration dovetailed with the question of stakeholder input, including 

the need to ensure that patients have a defined and central role in the CER process. 

52 
 

 

 

 

 

Key Themes 

Some respondents recommended targeted research topics; these ranged from testing the total 

effects of medication on the frail elderly, to testing the efficacy of a diet to treat children with 

seizures, to informing prescribing physicians on a wide range of what does and doesn’t work 

well for patients with various complex conditions. 

Most of the comments, however, fell into several broad categories—prioritizing the agenda, 

infrastructure development, research methodology and conduct, care delivery, knowledge 

transfer, cost, and health disparities and personalized medicine. A number of key themes and 

specific comments are summarized below. 

Prioritizing the Agenda 

A number of respondents tried to step back and look broadly at the question of how to prioritize 

the agenda for comparative effectiveness research and what criteria should guide decisionmaking in this arena. An overarching theme that echoed through many of these comments was 

the need to think big and look system-wide. One respondent stated that CER that is localized to 

a single disease may be less of a priority than questions that cross over diseases. Another talked 

about the need for CER to be undertaken for quality, efficiency, effectiveness, and other 

appropriate dimensions for health care delivery systems along the entire spectrum of systems 

integration, adding that the spectrum should include integrated delivery systems, multi-specialty 

group practices, single-specialty groups, “virtual” groups, and small medical practices. 

A number of recommended areas of focus emerged. Many respondents talked about focusing on 

areas of major clinical significance and the greatest impact on health care delivery, including 

chronic conditions. One respondent specifically noted that CER on chronic diseases should 

focus on all relevant health care services, including medical and surgical procedures, diagnostics, 

and medical devices. Another respondent said that more attention is needed in the areas of postacute and long-term care. Still others talked about the need for comparative effectiveness 

research on emergency care processes, and CER to evaluate regional differences in trauma care. 

A few people talked about studying the role of alternative treatments, including homeopathic 

treatments for chronic and acute disease states. Several respondents also talked about looking at 

conditions with the greatest impact on morbidity, and a few about doing research on conditions 

with the greatest impact on cost. 

A few respondents discussed the need to ensure that the priorities of state and local jurisdictions 

be given consideration in evaluating various CER strategies. For example, one participant noted 

that many jurisdictions have on-going investigative agendas designed to improve program 

effectiveness that can be considerably amplified by Federal support, adding that such efforts 

would extend beyond purely clinical protocols to include the evaluation of public health, 

community-based, and behavioral strategies that may enhance the effectiveness of public 

programs. 

One respondent suggested that significant resources be devoted to population-level interventions 

as well as patient-level effectiveness. Another respondent talked about the need for comparative 

effectiveness priority research areas to include critical cross-cutting research questions and cited 

53 
 

 

 

 

several examples (clinical decision-making, human-technology partnership, team coordination 

and continuity of care). 

In addition, respondents talked about the need for Federal investments in CER to focus on health 

disparities and understudied sub-groups. Many of the respondents who addressed this topic 

talked about under-sampling of minorities in clinical trials and stressed the need for research that 

looks at the impact of various treatments on specific sub-groups, including women, minorities, 

people in rural communities, persons with disabilities, and children. 

Infrastructure Development 

A number of respondents honed in on the need to scale up the capacity to do comparative 

effectiveness research. As one respondent put it: “All healthcare reform proposals are predicated 

on the presumption that a robust and well-developed quality infrastructure exists. However, this 

is not uniformly the case.” 

Infrastructure capacity, as framed by the public comments, incorporates three components: 

human and scientific capital, organizational capacity, and data capacity. 

Regarding human and scientific capital, respondents said that investments are greatly needed to 

enhance the skills, supply, and diversity of the research work force. One respondent pointed 

specifically to a dearth of researchers focused on mental health and substance abuse and 

treatment. Another respondent talked specifically about the need to increase the number of 

Hispanic health professional researchers, and suggested that HHS target Hispanic health 

professional, students, residents, and graduate students interested in serving in their 

communities. 

Regarding organizational capacity, many of the comments focused on building capacity at the 

regional and local level. For example, one respondent talked about the role that health 

improvement collaboratives and chartered value exchanges can play in maintaining patient 

registries and other databases, and about using the information for performance reporting. A 

second respondent talked about the role that more community organizations could play in 

helping to address racial and ethnic health disparities were they to have the appropriate 

infrastructure and capacity. 

The third critical subset of infrastructure development is data. A number of respondents talked 

about the need for both better data and access to data for comparative effectiveness research and 

decision-making. They urged the Council to access as much available data as possible, including 

clinical trials data, electronic health record systems, health care claims systems, administrative 

data, and Federal health data (including data from Medicare and Medicaid and that collected by 

the Veterans Health Administration). Respondents also talked about the need to invest in a 

coordinated effort to link public and private sector databases, as well as the need for standardized 

data available from the point of patient care. 

Several respondents also talked about the value of registries, and the need to link data sets in 

order to provide valuable sources of data to examine appropriate use, effectiveness of care, cost 

of care, value-based health care, and other criteria. Another respondent stressed the need for 

54 
 

 

 

research that involves collaboration in different data environments and research that explores the 

use of different types of electronic health care data. 

Research Methodology and Conduct 

How should CER be undertaken? This is another theme that ran through many of the comments. 

Those who tackled this question addressed key issues that ranged from the enterprise level to 

guidance on study design. At the broader level, one respondent talked about the need for a broad 

Federal CER enterprise that spans treatment, prevention, promotion, and health-determinant 

interventions designed for both people and populations. Another respondent recommended 

adopting value of information principles and tools to prioritize CER investments on those studies 

where there is a greater likelihood that the research will lead to changes in practice. A third 

person spoke about the opportunity to fund research into “the science of CER” to build a 

foundation for this work. 

Others talked about the scope of CER, noting that much of the research is conducted in single 

settings of care. One respondent, for example, noted that this poses a challenge for 

“generalizability,” and suggested that many of the questions that remain unanswered relate to 

uncommon conditions or outcomes that have proven challenging to study. He recommended the 

use of multi-center research networks to address this issue. 

Looking more closely at study design, one respondent noted that CER should continue to use a 

variety of study designs to generate evidence about the comparative effectiveness, comparative 

safety, and cost effectiveness of medical interventions. A second respondent talked about the 

limitations of randomized clinical trials, suggesting that the Council should also consider designs 

that are more common for evaluating comprehensive population-focused interventions, such as 

observational cross-sectional studies, quasi-experimental designs, and time series analyses. 

Another respondent stressed that clinical trial design and CER infrastructures must accommodate 

the goal of addressing health disparities. Another respondent pointed out that comparative 

effectiveness can at times be determined by assessing technology and using quantitative metrics 

rather than via an expensive and sometimes-lengthy clinical trial. A fourth respondent talked 

about the need to include utilization of laboratory services in order to effectively compare 

treatments and outcomes for major chronic disease cost drivers. 

Several respondents also addressed the need for greater transparency throughout the process. 

They talked about the critical importance of transparency for reducing bias and rebuilding trust, 

and they recommended that researchers show results prior to adjustments as well as adjusted 

results. Respondents who tackled the issue of transparency also talked about the need to disclose 

in detail the methods and metrics used in any research. One respondent stressed that patients and 

providers need to know all the inputs that go into a research analysis so that they can weigh the 

costs, safety, and quality issues appropriately in each instance. 

A corollary to transparency is addressing potential conflicts of interest. Respondents talked 

about the need to develop a strong and clear policy for conflicts of interest in both research and 

publishing, and suggested that funding decisions for CER should favor researchers and 

institutions that are focused on the public interest and do not have current conflicts. They also 

talked about the need for 100-percent disclosure and transparency at the outset of all conflicts by 

55 
 

 

 

 

individual researchers and institutions. One respondent specifically said that the ARRA 

expenditures on CER offer an opportunity to move to a platform where research funding is 

completely independent of other sources of funds in order to get to research that is independent, 

unbiased, untainted, and neither methodologically flawed nor influenced by industry. 

Care Delivery 

Several respondents pointed out that care delivery is critical, and that investments in CER are 

needed to look at how the health care delivery system should be organized and the best models 

for delivering care to patients. One respondent recommended that the Council invest in research 

that looks at optimal practice models for delivering patient care along with strategies for using 

information technology and clinical decision support tools to implement research findings into 

clinical practice. Another respondent suggested that CER is needed to look at the organization, 

design, and management of patient care. A third said that CER should focus on medical delivery 

systems and operations, resulting in information that can be leveraged to foster better clinical and 

cost outcomes. 

Much of the discussion on care delivery was focused on people with one or more chronic 

conditions (e.g., diabetes). One respondent, for example, talked about the need for CER studies 

that compare current, more traditional models of chronic care delivery with team-based, patientcenters models that include patient education and self-care. Another respondent emphasized the 

need to focus research on the impact of non-medical services (e.g., providing housing) on costeffective and clinical outcomes for chronically medically ill populations. A third person talked 

about CER around the role of support services (e.g., case management) in the health outcomes of 

people with HIV/AIDS; a fourth, about the need for CER on crisis residential services as an 

alternative to psychiatric hospitalization. Yet another respondent talked about the need to study 

the cost-effectiveness of community health worker interventions. 

One respondent talked about the need to study care models that integrate primary and tertiary 

care. Another respondent suggested that there was a need for research into how to deliver care in 

a way that helps patients get the care they need, adhere to proposed treatment regimes, and 

prevent subsequent untoward effects of chronic diseases. Regarding adherence to treatment 

regimes, one respondent specifically noted that patient compliance is a seldom-accounted-for 

variable in CER, and he talked about the value of electronic verification devices to track 

compliance. Another respondent talked about the need to compare palliative care models to 

understand which processes of care and specific program interventions and models are the most 

effective. 

One respondent noted that much of the literature on the impact of electronic medical records is 

anecdotal, and he expressed concern that people are considering the value of electronic health 

records without understanding the totality of what an effective system does for health care 

delivery. As a result, he urged that research be done to evaluate the comparative effectiveness of 

different types of EHR-mediated interventions. A second respondent likewise talked about the 

needs for research on how health information technology and EHR exchanges can be used to 

create more robust data sources and to help evaluate comparative effectiveness issues across a 

broader range of settings. 

56 
 

 

 

Knowledge Transfer 

A number of respondents pointed out that all the data is meaningless if the information is not 

disseminated effectively. One respondent, for example, stressed that knowledge translation 

research must not be overlooked, while another respondent pointed out that both research and 

dissemination of research findings are essential to realizing the quality improvements and 

returns-on-investment that are integral to the success of comparative effectiveness research. 

While respondents had different recommendations for how to approach knowledge transfer, there 

was a consensus that this work is critical. One respondent noted that the evidence base that is 

developed around clinical comparative effectiveness offers a substantial opportunity to improve 

value in health care if the information is disseminated and applied by physicians and patients. 

Others talked about the need to identify what approaches and incentives to dissemination and 

adoption are most effective (and under what circumstances), and when dissemination should 

target change at the organizational level, the community level, or the individual level. One 

respondent talked about cultural competence and health literacy research, and the need for both 

in order to change behaviors and improve lifestyles. 

One respondent noted that while technology (including electronic health records) is one avenue 

for dissemination, other effective dissemination and translation techniques are also needed. She 

noted that while many strategies have been used to enhance the rate and extent of adoption of 

evidence-based best practices (including clinical guidelines, continuing education for health care 

professionals, patient education tools, and academic detailing), the approaches have not been 

well studied and the results are variable. 

One respondent suggested that an independent body be established to disseminate comparative 

effectiveness research findings; others took the approach that everyone—including providers, 

payers, consumers, and employers—has a role to play in disseminating research results. Another 

respondent suggested creating a national citizens’ advisory board to help HHS better understand 

the perspectives and values of the general public when designing and disseminating CER. 

Another respondent talked about the need not only to provide the evidence base for best disease 

prevention, health promotion, and/or clinical interventions, but also to look at how these findings 

can be implemented in “real-world, complex organizational settings.” 

Cost 

Two distinct opinions about cost emerged: (1) that it should be a factor in comparative 

effectiveness research or (2) that it has no place in the discussion. 

Those opposed to factoring cost into CER expressed concern that too often people put cost into a 

separate silo and make decisions without regard to efficacy, and they suggested that a focus on 

costs could lead to limiting access and benefits. For example, one respondent said that 

comparative effectiveness research should not be focused on looking for cheaper treatments, and 

it should not be the basis for coverage decisions. Another talked about the fear that CER results 

might impact physician reimbursement rates. Several respondents also expressed concern that 

CER could be used to restrict access to care, to deny coverage, or to reduce payments for 

interventions, thus undermining physician/patient decision-making and limiting patient access to 

treatment options. 

57 
 

 

On the flip side, other respondents felt equally strongly that cost was an integral component of 

informed decision-making. For example, one respondent said that information about costs 

enables understanding not only of the direct differences in terms of clinical outcomes but also of 

the value of interventions and whether they represent an efficient use of resources. Another 

respondent suggested that, if costs are not considered, the tradeoff in terms of lost health benefits 

would be too steep. Others stressed that a wide range of stakeholders—including employers, 

policymakers, and state and local public health departments—have said that they need cost 

information to make decisions. 

Health Disparities and Personalized Medicine 

Several respondents spoke about the related topics of the need to address health disparities within 

CER and support for the growth of personalized medicine. Inclusion of and attention to 

underrepresented sub-groups was spoken of as a means to address the problem of disparities in 

care. Others spoke of the importance of fostering the application of personalized medicine. 

Respondents cited the need for more CER in the areas of preventive care, pediatric care and 

children’s health, behavioral health interventions, addiction, mental disorders, and suicide 

prevention. One respondent pointed out that CER is needed to understand the cost and quality 

implications to the overall health system of continuing to under-treat conditions in systems that 

are siloed and distinct from mainstream health and health care. Another respondent specifically 

noted that the aim of personalized medicine and the mapping of the human genome is to achieve 

disease interventions much earlier (ideally at the point of preventing the disease from ever taking 

hold, he said). 

One respondent stressed that CER must be mobilized to improve the health outcomes of various 

racial and ethnic minorities in order to close the gap that exists between the health status of some 

minority populations and other Americans. Others warned about relying on small, narrowly 

focused studies, suggesting that understanding and addressing health disparities requires a 

broader approach; conversely, respondents also cautioned against “one-size-fits-all” approaches 

that could decrease access to treatments. One respondent specifically talked about the need for 

research that examines health intervention outcomes across the lifespan, and for different 

minority and gender groups, in order to understand the effectiveness of interventions within and 

between population groups. 

Several people talked about the need to design studies that appropriately include minority 

populations (see also Prioritizing the Agenda, above). For example, one respondent said that the 

design of studies must reflect the diversity of patient populations, including racial and ethnic 

diversity, and must communicate results in ways that reflect the differences in individual patient 

needs. Another respondent stressed that clinical trial design and CER infrastructures must 

accommodate the goal of addressing health disparities. There was also discussion more broadly 

about the need to build the infrastructure to address health disparities relating to people of color. 

One respondent pointed to the dichotomy between studying populations and the promise of 

personalized medicine, asking: How can CER at a broad population level be balanced with the 

58 
 

 

goals and rapid scientific advancements in the area of personalized and stratified medicine in 

order to encourage the development of targeted therapies for sub-groups? 

One respondent talking about personalized medicine recommended that CER studies include the 

evaluation of approaches to health care delivery and care management that foster the effective 

application of personalized medicine. 

Appendix B: SUMMARY OF THE COUNCIL’S MEETINGS AND DELIBERATIONS 

The following contains a summary of the Council’s deliberations as they unfolded once the 

Council was officially convened. 

April 10, 2009 

The Council was presented with background information on comparative effectiveness research 

and briefed on CER activity at AHRQ, NIH, and VA. The Council also discussed the scope of 

their work and objectives. 

Next, the Council began discussion of the components of the definition of CER and potential 

criteria for prioritization. The Council also discussed how CER and data infrastructure for CER 

might be categorized. Finally, the Council reviewed the timeline and discussed plan for listening 

sessions, including the first listening session on April 14, 2009. 

April 22, 2009 

The Council met to discuss what they had heard at the April 14 listening session. Members 

identified several key themes, including the need to outline a clear, well-delineated definition of 

comparative effectiveness research. They noted that participants had also talked about the need 

to prioritize methodology, and the fact that CER should be inclusive of all components of 

medical care. 

Council members also noted that they had heard, loud and clear, that the Council’s governance 

and processes must be transparent, and that the Council must incorporate input from all 

stakeholders to gain credibility and build trust. 

Other themes that emerged from the listening session include the need to focus on patients and 

outcomes; the importance of incorporating diverse populations and multiple research methods; 

and the need for investments in infrastructure. Regarding the focus on patients and outcomes, 

Council members noted that participants had talked about the importance of considering patient 

input from the start and the fact that the results must be framed and disseminated in ways that are 

relevant to patients and providers. Regarding diverse populations, Council members observed 

that there was discussion about the need to include sub-groups with multiple chronic conditions, 

and the need more broadly to make CER relevant to sub-groups. Members also noted that 

participants had talked about the need to use a multitude of different research methodologies (not 

just randomized clinical trials), and to look at the Department of Veterans Affairs’ experience 

using registries. 

59 
 

 

 

Regarding infrastructure, Council members observed that participants had stressed there was a 

need to expand, improve, and build on existing information and registries, and that perhaps this 

investment could lay the foundation for distributed data networks with the capability to answer 

many future CER questions. Members also noted that there had been discussion about the need 

to make data monitoring easier and more routine. 

Finally, Council members talked about how they could tweak the listening session format to 

allow for a more robust conversation with participants. 

May 1, 2009 

The Council looked at the timetable for its work and the due dates for its key deliverable. The 

Report to Congress is due June 30, and the preliminary timetable builds in time for HHS and 

OMB clearance, comments, and suggested edits. The Council also briefly discussed the 

upcoming second listening session, slated for May 13 in Chicago. 

Next, the Council briefly discussed the process for compiling the CER and data infrastructure 

inventories, and agreed that members would identify primary contacts in their division or agency 

who can work with the contractor to drive that process. 

The Council’s next goal was to arrive at consensus on a draft definition of comparative 

effectiveness research, prioritization criteria, and a categorization framework for CER. Once 

complete, the Council agreed to post the draft language on the hhs.gov/recovery Web site and to 

solicit public feedback. 

To begin that work, the Council tackled the draft definition. There was considerable discussion 

about what the definition of CER should be. Members expressed the belief that the definition 

needed to be inclusive of the multiple stakeholders in the health care arena, including 

communities, and they also looked at what types of interventions should be called out. The 

Council ultimately came to consensus that they wanted a definition that was broad-based and 

inclusive, but that was not so detailed as to inadvertently narrow the scope of comparative 

effectiveness research. 

The Council next turned its attention to the prioritization criteria. Before doing so, however, the 

Council first wrestled with the question of whether the criteria should be focused broadly or 

more narrowly targeted to provide guidance to the Office of the Secretary in allocating its 

Recovery Act funds. The Council generally felt that the criteria should be broad enough to allow 

the Council to make recommendations on overall funding and funding criteria. 

Next, Council members discussed how to prioritize the CER criteria, including whether impact 

should be listed first, with feasibility and scientific merit second. One person spoke out about 

the need to keep the criterion on diverse populations and patient sub-groups within the top five. 

There was also discussion about whether knowledge gap was a criterion, or whether it should 

perhaps be wrapped into the criterion on impact. 

60 
The Council also looked at several potential frameworks for comparative effectiveness research, 

including categorization by type of CER investment, by patient sub-groups, by condition, and by 

type of intervention. The aim of developing a framework was to help categorize current CER 

activity and to identify gaps for potential future investments in CER. Council members also 

discussed CER centers, and agreed that Recovery Act funding could be used to support this 

work. One member suggested that the Council, at a future date, should discuss how to 

coordinate interest in CER centers across agencies. 

Finally, the Council received a presentation on enhancing the inclusion of minority and other 

underserved populations in comparative effectiveness research. As a result, the Council agreed 

to establish a small workgroup co-led by NIH, AHRQ, the HHS Office of Minority Health, and 

the HHS Office on Disability. The workgroup will have two key tasks: (1) to develop 

recommendations for the inclusion of minority and other underrepresented populations in the 

expanded comparative effectiveness research agenda, and (2) to receive input from non-Federal 

groups on targeted actions. 

May 8, 2009 

The Council reviewed a revised definition of comparative effectiveness research and agreed to 

post the definition on the hhs.gov/recovery Web site on or about May 15. 

Next, the Council resumed its discussion of the prioritization criteria. There was considerable 

discussion about whether “scientific rigor and validity” needed to be included in the threshold 

minimal criteria, with some members saying that it was implicit (and something already being 

done) and others expressing concerned about including a yes/no component to the threshold 

minimal criteria. The consensus of the Council was that scientific rigor and validity be included 

as part of a concept statement. 

The Council then looked at a first draft outline of the Report to Congress. It included (1) 

Introduction, (2) Objectives, (3) Definition and Criteria, (4) Framework for CER, (5) Current 

CER and CER data infrastructure, (6) Recommendations for Priorities for OS CER Funds, and 

(7) Longer-term Vision and Opportunities. 

Council members discussed a number of items that they believed needed to either be included or 

called out in the report, including concrete examples of what CER is and why it matters as well 

as a discussion about the full range of CER activities (and not just randomized clinical trials). 

There was also discussion about having a stand-alone section on high level priorities; the need to 

call out the roll of public/private partnerships; including a sub-section on the need for CER data 

to be synthesized and operationalized, along with some mechanisms for achieving this outcome; 

and the need to add language on sub-groups. Members also agreed to add a new section, 

Summary of the Listening Sessions, and to include a high-level Executive Summary. 

Next, the Council began its discussion of CER priorities. To frame their discussion, members 

looked at four categories: primary research, dissemination of results, data infrastructure, and 

cross-cutting coordinated investments. One member asked, “What are the gaps that no one else 

can fill?” The Council agreed to continue its discussion at its next meeting. 

61 
 

 

 

 

 

May 22, 2009 

The Council opened its meeting with a debrief from the May 13 listening session in Chicago. 

Members said they found the meeting both useful and exciting, and cited some themes they had 

heard that particularly struck them. These included the need to study chronic diseases (and to 

include sex, ethnicity, and race in the analysis); the idea of using theoretical models to assess 

how to approach a study (and to ensure the information is useable); the inclusion of mental 

health as a priority area; the importance of CER on pediatric populations; the importance of CER 

on prevention; and the need for training, and for starting to build the pipeline early. 

The Council then briefly addressed next steps on the Report to Congress, including the fact that 

certain members would be assigned to draft specific sections of the report. 

Next, the Council resumed its discussion of CER priorities where it had left off: looking at 

research, dissemination, data infrastructure, and cross-cutting investments. There was general 

consensus that OS funds should focus primarily on the latter three areas (as AHRQ and NIH are 

likely to make CER investments in research); there was also discussion about how to frame the 

priorities, including whether they should be framed around the type of CER investment or 

around types of diseases (e.g., people with multiple chronic illnesses, or people with disabilities 

and chronic illnesses). There was also specific discussion about the need to improve 

dissemination of research results—and a related topic, impacting practice. “If we just talk about 

dissemination,” said one Council member, “we won’t get anywhere. We need to look at the best 

methods for impacting practice.” 

There was also discussion around the question of how the Council should think about structuring 

its Report to Congress. At issue was whether the report should focus primarily on guidance to 

the HHS Secretary on how to allocate the $400 million in OS funds. In addition, the Council 

discussed the research time horizon, and whether ARRA monies could be used to fund projects 

that will have a time horizon longer than two years. One member suggested that one way to 

think about the question was to reframe it and ask, “Can we think about creating research centers 

that will be great resources into the future?” 

Council members also stressed the need for the Council to address in its report the process for its 

deliberations and its recommendations, including making clear that CER investments are 

weighted to public health needs and responsive to the needs of decision makers. Council 

members suggested that some of the discussion about impacting practice might be linked to the 

discussion about data infrastructure investments. 

May 29, 2009 

The Council honed in on the details of the strategic framework for comparative effectiveness 

research, and the fact that it represents a comprehensive, coordinated approach to Federal 

investment in CER priorities that is intended to support immediate decisions for investments in 

CER priorities and to provide a comprehensive basis for longer-term CER investment decisions. 

The Council discussed a framework that includes four major categories of activity (research, 

human and scientific capital, data and research infrastructure, and translation and adoption). The 

framework is designed to allow for investments within a single category or to cross-cut priority 

62 
themes. The Council agreed upon the categories. The Council’s next step will be to determine 

the recommended mix among the major activities for OS funds. 

The Council agreed to post on the hhs.gov/recovery Web site a copy of the broad framework 

diagram as well as a more detailed version to inform the public and to seek feedback on the 

strategic framework. 

Next, the Council looked at some examples of the types of investments that might be made in the 

areas of infrastructure and translation and adoption. The idea of the discussion was to enable 

members to think about what types of projects might address gaps and further the CER 

enterprise. 

The Council also looked briefly at an updated draft outline for the Report to Congress, and then 

members heard a presentation on three possible categories for investments in disability 

comparative effectiveness research. 

June 5, 2009 

The Council discussed the first draft of the Report to Congress. There was consensus that the 

Executive Summary needed to better frame the conversation around the value of CER to inform 

patients, clinicians, and other stakeholders. There was also discussion about setting out, early in 

the body of the report, why CER matters and how it matters to each stakeholder group. In 

addition, the Council agreed to add an additional appendix that contains a summary of its 

meetings and deliberations. 

Next, the Council took up its recommendations for priorities. The discussion revolved around 

four key issues: the balance in spending priorities among the major activities versus cross-cutting 

themes; the distribution of spend priorities across the four major activities; what themes should 

be prioritized (and what the distribution of spend priorities should be across those themes); and 

whether the overall distribution makes sense vis-à-vis the prioritization criteria. 

Regarding the distribution of spend priorities across the four major activities, Council members 

generally agreed that the majority of funding (e.g. 60 percent) should be spent on activities rather 

than themes. At the same time, there were lingering questions about the need to identify research 

gaps, implementation gaps, or both. 

Regarding the distribution of spend priorities across the four major activities, the Council 

supported a breakdown that focuses the bulk of the funding in the areas of infrastructure (e.g. 60 

percent) and translation (e.g. 20 percent). Members noted that there is a unique opportunity with 

ARRA funds to make significant investments in infrastructure. 

Regarding potential priorities, members looked at draft lists of both priority populations and 

types of interventions. On the populations side, one Council member said that all of the 

proposed priority populations share in common that they have not traditionally been enrolled in 

clinical trials. There was also discussion about the need to include veterans as well as people 

with co-occurrence of mental health disorders along with physical comorbidities. On the 

interventions side, there was some discussion about the inclusion of delivery systems, and that 

63 
 

 

 

 

 

 

 

CER on delivery systems offers an opportunity to look at promising practices and how they 

might be scaled up and disseminated. 

Finally, the Council was divided as to whether the bulk of OS funds should be used primarily for 

investments in populations or in interventions—or whether they should be equally important 

priorities. 

June 12, 2009 

The Council debriefed on what was heard in the third listening session. This generated 

enhancement to the common themes and some new information to be incorporated. The Council 

then revised the definition, threshold and prioritization criteria, and strategic framework based on 

the feedback from the session and the feedback received online. The Council then further 

discussed priority recommendations and the Report to Congress. The Council suggested edits 

for the Report prior to it going into clearance the next week. 

Appendix C. PRELIMINARY DATA INFRASTRUCTURE AND CER BY CONDITION 

The following is a preliminary inventory of examples of CER data infrastructure and CER by 

condition. 

Person-Level Health Care Research Databases from First Inventory 

Database Owner Data Type 

No. 

Lives 

Disease 

Area 

Data 

Linkable 

at Patient 

Level 

Data 

Linkable 

to 

External 

Data 

Researcher 

Ready 

US Federal 

Healthcare Cost 

and Utilization 

Project (HCUP) 

AHRQ 

Hospital 

information 

system 

— All Y N Y 

HIV Cost and 

Services 

Utilization Study 

(HCSUS) 

AHRQ 

Surevy & 

records 

abatsraction 

2,864 HIV Y N Y 

AIDS Cost and 

Services 

Utilization Study 

(ACSUS) 

AHRQ 

Hospital 

information 

system 

1,900 AIDS Y N Y 

National Vital 

Statistics CDC 

Surveillance 

program/registry 

data 

— All n/a N N 

64
 
 

 

 

 

 

 

 

 

 

 

 

 

Database Owner Data Type 

No. 

Lives 

Disease 

Area 

Data 

Linkable 

at Patient 

Level 

Data 

Linkable 

to 

External 

Data 

Researcher 

Ready 

National Vital 

Statistics—Natality CDC 

Surveillance 

program/registry 

data 

4 

million All n/a N Y 

National Health 

Interview Survey CDC Survey 87,000 All n/a Y Y 

National Health 

and Nutrition 

Examination 

Survey 

CDC Survey 5,000 All n/a Y Y 

National 

Ambulatory 

Medical Care 

Survey 

CDC 

Survey and 

records 

abstraction 

n/a All N Y Y 

National Hospital 

Ambulatory 

Medical Care 

Survey 

CDC 

Survey and 

records 

abstraction 

n/a All N Y Y 

National Hospital 

Discharge Survey CDC 

Survey and 

records 

abstraction 

n/a All N Y Y 

National Nursing 

Home Survey CDC 

Survey and 

records 

abstraction 

13,507 All N Y Y 

National Home and 

Hospice Care 

Survey 

CDC 

Survey and 

records 

abstraction 

9,416 All N Y Y 

Chronic Condition 

Data Warehouse CMS 

Administrative 

claims database, 

enrollment data, 

health 

assessment data, 

prescription 

drug event data 

45 

million All Y Y Y 

Hospice Standard 

Analytical File 

(Hospice SAF) 

CMS Administrative 

claims database — All Y Y ? 

65
 
 

 

 

 

 

 

 

 

 

 

 

Database Owner Data Type 

No. 

Lives 

Disease 

Area 

Data 

Linkable 

at Patient 

Level 

Data 

Linkable 

to 

External 

Data 

Researcher 

Ready 

Medicaid 

Statistical 

Information 

System Personal 

Summary File 

(MSIS Personal 

Summary File) 

CMS 

Administrative 

claims database, 

EMR/EHR 

system 

— All Y Y Y 

National Claims 

History (NCH) 

100% Nearline File 

CMS Administrative 

claims database — All Y Y ? 

MEDPAR Claims 

Data CMS Administrative claims database — All Y Y Y 

MMA Part D 

Claims Data CMS Pharmacy claims database 

25 

million All Y Y Y 

Sentinel System FDA 

Surveillance 

program/registry 

data 

N/A n/a N Y N 

SEER 

(Surveillance 

Epidemiology and 

End Results) 

NCI 

Surveillance 

program/registry 

data 

11.4 

million Cancer Y N Y 

SEER-Medicare 

database 

NCI, 

CMS 

Administrative 

claims database, 

Surveillance 

program/registry 

data 

3.3 

million Cancer Y Y N 

Cancer Research 

Network (CRN) 

NCI, 

AHRQ 

Administrative 

claims database, 

EMR/EHR 

system 

— Cancer Y Y N 

Computerized 

Patient Record 

System (CPRS) 

VA EMR/EHR 

system 

4.2 

million All Y N N 

Diabetes Epidemiology Cohort VA 

Surveillance 

program/registry 

data 

> 4,800 Diabete 

s 

Y Y Y 

66
 
 

 

 

 

 

 

 

 

 

 

 

 

Database Owner Data Type 

No. 

Lives 

Disease 

Area 

Data 

Linkable 

at Patient 

Level 

Data 

Linkable 

to 

External 

Data 

Researcher 

Ready 

Hepatitis C 

Registry VA 

Surveillance 

program/registry 

data 

>60 K Hepatiti 

s C Y N Y 

Immunological 

Case Registry VA 

Surveillance 

program/registry 

data 

>15 K HIV Y N Y 

Dementia Registry VA 

Surveillance 

program/registry 

data 

>150 K Dementi 

a 

Y N N 

National Surgery 

Quality 

Improvement 

Program 

VA 

Surveillance 

program/registry 

data 

>1 

Million 

All 

major 

surgery 

Y Y Y 

Scientific Registry 

of Transplant 

Recipients (SRTR) 

HRSA 

Transplant 

registry and 

outcomes data 

Organ 

specific Y Y Y 

Pediatric 

Emergency Care 

Applied Research 

Network 

(PECARN) 

CDMCC∗

HRSA 

Emergency 

medical services 

for children 

800,000 

+ 

patients 

Emerge 

ncy 

Services 

to 

Childre 

n 

Y Y Y 

AIDS Drug 

Assistance 

Program (ADAP) 

HRSA Care Program 

Registry Data ___ HIV/AI DS Y Y N 

US Private Sector 

National 

Oncologic PET 

Registry 

(NOPR) 

Academ 

y of 

Molecul 

ar 

Imaging 

Intervention 

program data 

>100,00 

0 Cancer Y Y ? 

Cerner Health 

Facts Database Cerner EMR/EHR system — All Y Y Y 

GE Centricity GE EMR/EHR 

system 

10 

million All Y N Y 

∗ Central Data Management and Coordinating Center 

67 
Database Owner Data Type 

No. 

Lives 

Disease 

Area 

Data 

Linkable 

at Patient 

Level 

Data 

Linkable 

to 

External 

Data 

Researcher 

Ready 

Ingenix Research 

Data Mart (RDM) 

Database 

Ingenix Administrative 

claims database 

>39 

million All Y Y Y 

Premier 

Perspective Data 

Warehouse 

Premier Administrative 

claims database — All Y Y Y 

MarketScan Data 

Warehouse 

Thomso 

nReuters 

EMR/EHR 

system — All Y N N 

International Databases 

General Practice 

Research Database 

(GPRD) 

NHS 

(UK) 

EMR/EHR 

system 

> 3.6 

million All Y Y Y 

NHS Care Records 

Service (CRS) 

NHS 

(UK) 

EMR/EHR 

system 

2 

million All Y N Y 

The Health 

Improvement 

Network (THIN) 

INPS 

and 

EPIC 

(UK) 

EMR/EHR 

system — All Y Y Y 

 

 

 

 

 

 

 

 

Priority Diseases/Conditions in CER 

Priority Diseases/Conditions 

AHRQ 

(n=178) 

NIH 

(n=513) 

DoD 

(n=26) 

VHA 

(n=106) 

Total 

(n=823) 

Arthritis and non-traumatic joint disorders 6% 1% 0% 3% 2% 

Cancer 10% 7% 23% 7% 8% 

Cardiovascular disease, including stroke and 

hypertension 20% 10% 4% 23% 13% 

Dementia, including Alzheimer's Disease 1% 1% 0% 3% 1% 

Depression and other mental health disorders 8% 16% 8% 18% 14% 

Developmental delays, attention-deficit 

hyperactivity disorder, and autism 4% 1% 0% 0% 1% 

Diabetes mellitus 11% 11% 0% 8% 10% 

Functional limitations and disability 8% 4% 15% 7% 5% 

68
 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Infectious diseases including HIV/AIDS 3% 11% 0% 6% 8% 

Obesity 1% 3% 0% 2% 3% 

Peptic ulcer disease and dyspepsia 0% 0% 0% 0% 0% 

Pregnancy, including preterm birth 1% 4% 0% 0% 2% 

Pulmonary disease/asthma 5% 3% 0% 4% 3% 

Substance abuse 2% 19% 0% 9% 14% 

Other 20% 11% 50% 12% 14% *

Studies focusing on patients with more than one priority disease or condition are counted in 

applicable rows..
 

**NIH 2008 plus NIH multi-year sample.
 

Appendix D. COUNCIL LIST AND STAFF SUPPORT 

1. Carolyn Clancy, MD AHRQ 

2. Peter Delaney, PhD, LCSW-C SAMHSA 

3. Ezekiel Emanuel, MD, PhD OMB 

4. Jesse Goodman, MD, MPH FDA 

5. Garth Graham, MD, MPH Office of Minority Health 

6. Anne Haddix, PhD CDC 

7. Deborah Hopson, PhD, RN HRSA 

8. David Hunt, MD ONC 

9. Michael Kilpatrick, MD Dept of Defense 

10. Joel Kupersmith, MD Dept of VA 

11. Michael Marge, Ed.D. Office of Disability 

12. Elizabeth Nabel, MD NIH 

13. James Scanlon, PhD ASPE 

14. Neera Tanden, JD Office of the Secretary 

15. Tom Valuck, MD, MHSA, JD CMS 

Executive Director: Patrick Conway, MD, MSc 

Deputy Executive Director: Cecilia Rivera Casale, PhD 

Alternates to the Council participating: Kelley Brix, Margaret Cary, Rosaly Correa-de-Araujo 

(replaced Michael Marge on Council June 12th), Elisabeth Handley, Lynn Hudson, Michael 

Millman 

Contributors to Council and Report: Kate Goodrich, Lauren Hunt, John Poelman, Daria 

Steigman, Caroline Taplin, Jordan VanLare. 

69 
 

 

 

 

Appendix E. THE AMERICAN RECOVERY AND REINVESTMENT ACT STATUTE 

RELATED TO CER AND COUNCIL 

Appropriations 

For an additional amount for 'Healthcare Research and Quality' to carry out titles III and IX of 

the Public Health Service Act, part A of title XI of the Social Security Act, and section 1013 of 

the Medicare Prescription Drug, Improvement, and Modernization Act of 2003, $700,000,000 

for comparative effectiveness research: Provided, That of the amount appropriated in this 

paragraph, $400,000,000 shall be transferred to the Office of the Director of the National 

Institutes of Health ('Office of the Director') to conduct or support comparative effectiveness 

research under section 301 and title IV of the Public Health Service Act: Provided further, That 

funds transferred to the Office of the Director may be transferred to the Institutes and Centers of 

the National Institutes of Health and to the Common Fund established under section 402A(c)(1) 

of the Public Health Service Act: Provided further, That this transfer authority is in addition to 

any other transfer authority available to the National Institutes of Health: Provided further, That 

within the amount available in this paragraph for the Agency for Healthcare Research and 

Quality, not more than 1 percent shall be made available for additional full-time equivalents. 

In addition, $400,000,000 shall be available for comparative effectiveness research to be 

allocated at the discretion of the Secretary of Health and Human Services ('Secretary'): Provided, 

That the funding appropriated in this paragraph shall be used to accelerate the development and 

dissemination of research assessing the comparative effectiveness of health care treatments and 

strategies, through efforts that: (1) conduct, support, or synthesize research that compares the 

clinical outcomes, effectiveness, and appropriateness of items, services, and procedures that are 

used to prevent, diagnose, or treat diseases, disorders, and other health conditions; and (2) 

encourage the development and use of clinical registries, clinical data networks, and other forms 

of electronic health data that can be used to generate or obtain outcomes data: Provided further, 

That the Secretary shall enter into a contract with the Institute of Medicine, for which no more 

than $1,500,000 shall be made available from funds provided in this paragraph, to produce and 

submit a report to the Congress and the Secretary by not later than June 30, 2009, that includes 

recommendations on the national priorities for comparative effectiveness research to be 

conducted or supported with the funds provided in this paragraph and that considers input from 

stakeholders: Provided further, That the Secretary shall consider any recommendations of the 

Federal Coordinating Council for Comparative Effectiveness Research established by section 

804 of this Act and any recommendations included in the Institute of Medicine report pursuant to 

the preceding proviso in designating activities to receive funds provided in this paragraph and 

may make grants and contracts with appropriate entities, which may include agencies within the 

Department of Health and Human Services and other governmental agencies, as well as private 

sector entities, that have demonstrated experience and capacity to achieve the goals of 

comparative effectiveness research: Provided further, That the Secretary shall publish 

information on grants and contracts awarded with the funds provided under this heading within a 

reasonable time of the obligation of funds for such grants and contracts and shall disseminate 

research findings from such grants and contracts to clinicians, patients, and the general public, as 

appropriate: Provided further, That, to the extent feasible, the Secretary shall ensure that the 

recipients of the funds provided by this paragraph offer an opportunity for public comment on 

70 
 

 

 

 

 

the research: Provided further, That research conducted with funds appropriated under this 

paragraph shall be consistent with Departmental policies relating to the inclusion of women and 

minorities in research: Provided further, That the Secretary shall provide the Committees on 

Appropriations of the House of Representatives and the Senate, the Committee on Energy and 

Commerce and the Committee on Ways and Means of the House of Representatives, and the 

Committee on Health, Education, Labor, and Pensions and the Committee on Finance of the 

Senate with an annual report on the research conducted or supported through the funds provided 

under this heading: Provided further, That the Secretary, jointly with the Directors of the Agency 

for Healthcare Research and Quality and the National Institutes of Health, shall provide the 

Committees on Appropriations of the House of Representatives and the Senate a fiscal year 2009 

operating plan for the funds appropriated under this heading prior to making any Federal 

obligations of such funds in fiscal year 2009, but not later than July 30, 2009, and a fiscal year 

2010 operating plan for such funds prior to making any Federal obligations of such funds in 

fiscal year 2010, but not later than November 1, 2009, that detail the type of research being 

conducted or supported, including the priority conditions addressed; and specify the allocation of 

resources within the Department of Health and Human Services: Provided further, That the 

Secretary, jointly with the Directors of the Agency for Healthcare Research and Quality and the 

National Institutes of Health, shall provide to the Committees on Appropriations of the House of 

Representatives and the Senate a report on the actual obligations, expenditures, and unobligated 

balances for each activity funded under this heading not later than November 1, 2009, and every 

6 months thereafter as long as funding provided under this heading is available for obligation or 

expenditure. 

Sec. 804. Federal Coordinating Council for Comparative Effectiveness Research 

(a) ESTABLISHMENT— There is hereby established a Federal Coordinating Council for 

Comparative Effectiveness Research (in this section referred to as the 'Council'). 

(b) PURPOSE— The Council shall foster optimum coordination of comparative effectiveness 

and related health services research conducted or supported by relevant Federal departments and 

agencies, with the goal of reducing duplicative efforts and encouraging coordinated and 

complementary use of resources. 

(c) DUTIES— The Council shall— 

(1) assist the offices and agencies of the Federal Government, including the Departments of 

Health and Human Services, Veterans Affairs, and Defense, and other Federal departments or 

agencies, to coordinate the conduct or support of comparative effectiveness and related health 

services research; and 

(2) advise the President and Congress on— 

(A) strategies with respect to the infrastructure needs of comparative effectiveness research 

within the Federal Government; and 

(B) organizational expenditures for comparative effectiveness research by relevant Federal 

departments and agencies. 

71 
 

 

(d) MEMBERSHIP— 

(1) NUMBER AND APPOINTMENT— The Council shall be composed of not more than 15 

members, all of whom are senior Federal officers or employees with responsibility for healthrelated programs, appointed by the President, acting through the Secretary of Health and Human 

Services (in this section referred to as the 'Secretary'). Members shall first be appointed to the 

Council not later than 30 days after the date of the enactment of this Act. 

(2) MEMBERS— 

(A) IN GENERAL— The members of the Council shall include one senior officer or employee 

from each of the following agencies: 

(i) The Agency for Healthcare Research and Quality. 

(ii) The Centers for Medicare and Medicaid Services. 

(iii) The National Institutes of Health. 

(iv) The Office of the National Coordinator for Health Information Technology. 

(v) The Food and Drug Administration. 

(vi) The Veterans Health Administration within the Department of Veterans Affairs. 

(vii) The office within the Department of Defense responsible for management of the 

Department of Defense Military Health Care System. 

(B) QUALIFICATIONS— At least half of the members of the Council shall be physicians or 

other experts with clinical expertise. 

(3) CHAIRMAN; VICE CHAIRMAN— The Secretary shall serve as Chairman of the Council 

and shall designate a member to serve as Vice Chairman. 

(e) REPORTS— 

(1) INITIAL REPORT— Not later than June 30, 2009, the Council shall submit to the President 

and the Congress a report containing information describing current Federal activities on 

comparative effectiveness research and recommendations for such research conducted or 

supported from funds made available for allotment by the Secretary for comparative 

effectiveness research in this Act. 

(2) ANNUAL REPORT— The Council shall submit to the President and Congress an annual 

report regarding its activities and recommendations concerning the infrastructure needs, 

organizational expenditures and opportunities for better coordination of comparative 

effectiveness research by relevant Federal departments and agencies. 

(f) STAFFING; SUPPORT— From funds made available for allotment by the Secretary for 

comparative effectiveness research in this Act, the Secretary shall make available not more than 

1 percent to the Council for staff and administrative support. 

(g) RULES OF CONSTRUCTION— 

(1) COVERAGE— Nothing in this section shall be construed to permit the Council to mandate 

coverage, reimbursement, or other policies for any public or private payer. 

72 
 

(2) REPORTS AND RECOMMENDATIONS— None of the reports submitted under this 

section or recommendations made by the Council shall be construed as mandates or clinical 

guidelines for payment, coverage, or treatment. 

Title VIII—Departments of Labor, Health And Human Services, and Education, and Related 

Agencies 

73 

200 INDEPENDENCE AVENUE, S.W.
 

WASHINGTON, DC 20201