Comparative Clinical Effectiveness and Cost-Effectiveness Research: Background, History, and Overview
Released: 2007-10-15T00:00:00

Comparative clinical effectiveness research has been discussed as an avenue for producing information to help health care decision makers, such as patients, providers, and public and private payers, reach informed, evidence-based decisions. Proponents maintain that such information would aid in using limited resources effectively and efficiently, becomes even more necessary as resources become more limited, variation in medical practice patterns persist, and the rate of health care spending continues to rise. A recent Institute of Medicine (IOM) report on the future of drug safety notes "what is newest is not always the best" and that decision makers need information on the comparative risks and benefits of treatments and services.

Diverse governmental and non-governmental organizations have publicly expressed their support or reservations about increased comparative clinical effectiveness research. The House Ways and Means Subcommittee on Health held a hearing on June 12, 2007, that discussed the creation of a public-private body that would oversee comparative clinical effectiveness research. The director of the Congressional Budget Office (CBO) concluded at the hearing that comparative clinical effectiveness research, combined with changes in payment incentives, "offers a promising mechanism for reducing health care costs to a significant degree over the long term while maintaining or improving the health of Americans," while emphasizing that significant cost savings from such research would not been seen for many years. The executive director of the Medicare Payment Advisory Commission (MedPAC) stated that "there is not enough credible, empirically based comparative-effectiveness information available to patients, providers, and payers to make informed treatment decisions." AARP, an advocacy group for persons aged 50 and older, mentioned the need for comparative effectiveness studies for pharmaceuticals as early as 2005 in their Solutions Statement to the White House Conference on Aging. America's Health Insurance Plans (AHIP), a trade association representing health insurance plans, has urged Congress to give the Centers for Medicare and Medicaid Services (CMS) the authority to use comparative effectiveness and cost-effectiveness information in its coverage and reimbursement decisions. Jack Rowe, the former chief executive officer of Aetna, has stated that employers are "dying" to have information on the comparative effectiveness of treatments. On the other hand, some have expressed reservations about funding increased comparative clinical effectiveness research. Scott Gottlieb, a researcher at the American Enterprise Institute for Public Policy Research (AEI), articulated problems he perceived with government-sponsored prescription drug research, including poor study design, lack of access to full study results, and bias in interpreting the results. Robert Goldberg, vice president of the Center for Medicine in the Public Interest, argued that comparative clinical effectiveness research ignores individual differences and is biased towards concluding cheaper drugs are more effective.

Related Legislation in the 110th Congress

A number of bills have been introduced that support comparative clinical effectiveness research, including several in the 110 th Congress. S. 3 , the Medicare Part D price negotiation bill sponsored by Senate Finance Committee Majority Leader Max Baucus, would require the Director of Health and Human Services (HHS) to develop a prioritized list of comparative effectiveness research studies. Representatives Tom Allen (D-ME) and Jo Ann Emerson (R-MO) co-sponsored H.R. 2184 , which would establish a public-private funding mechanism for comparative clinical effectiveness research, overseen by an independent advisory board. The Act would establish a trust fund for the research that would receive $100 million in FY2008, $200 million in FY2009, and $900 million per year for FY2010-FY2012. Similarly, H.R. 3162 , the Children's Health and Medicare Protection (CHAMP) Act of 2007, would establish a public-private funding mechanism for comparative clinical effectiveness research, overseen by an independent commission, and a trust fund that would be appropriated at least $90 million in FY2008, $100 million in FY2009, $110 million in 2010, and no more than $90 million in years thereafter. CBO has stated that the information produced by the comparative effectiveness portion of the CHAMP Act would reduce total spending by public and private purchasers by $0.5 billion over 5 years and $6 billion over 10 years. Direct spending by the federal government was estimated to be reduced by $0.1 billion over 5 years and $1.3 billion over 10 years. Thus, the majority of the savings from the research would be realized by private purchasers rather than by the federal government. The net federal expenditures from the comparative effectiveness portion of the CHAMP Act were estimated to be $0.5 billion over 5 years and $1.1 billion over 10 years. CBO assumed the savings would primarily be realized through changes in physicians' practice patterns and, to a lesser extent, changes in coverage rules. The Healthy Americans Act, S. 334 and H.R. 3163 , includes tax deduction, patent extension, and market exclusivity incentives for pharmaceutical and medical device manufacturers to conduct comparative clinical effectiveness research. The Josephine Butler United States Health Service Act, H.R. 3000 , would create a National Health Board that includes a new institute—the National Institute of Evaluative Clinical Research. Among the Institute's responsibilities would be to identify the most effective methods of prevention, diagnosis, and treatment and assist the National Health Board in establishing clinical practice guidelines. More details about these bills and others in the 109 th and 110 th Congress are included in the Appendix .

To help inform the discussion surrounding comparative clinical effectiveness research, this report provides an overview and discusses past and current comparative clinical effectiveness research and other forms of technology assessment in the United States. This report also briefly discusses the use of technology assessment in the U.S. and other countries, and the potential role of a new comparative effectiveness research entity.

What Is Comparative Effectiveness Research?

Comparative effectiveness research is a term that has been defined by people in many different ways. All agree that comparative effectiveness research compares the effectiveness of two or more health care services or treatments, and is one form of health technology assessment. It compares outcomes resulting from different treatments or services, and provides information about the relative effectiveness of treatments.

Additional specifics about the research and its definition are sources of contention. In particular:

Effectiveness—How should effectiveness be measured? Should the research compare only the effectiveness (the effect in routine clinical practice) or also the efficacy (the effect under optimal conditions) of treatments or services? Costs—Should costs be included in the research? Should the costs be reported separately from the effectiveness results? Or should a cost-effectiveness ratio be the ultimate goal?

Effectiveness: How Should It Be Measured?

Effectiveness Is Complicated to Measure

Measuring the benefit or effectiveness is not a straightforward matter; which factors are included and how they are counted can greatly affect the results. Quantifying benefits and effectiveness often requires assumptions about the population benefitting from the treatment. For example, to what extent will the benefits from the treatment vary across the country and in different settings? More specifically, how should the benefits observed in a clinical trial be extrapolated to the rest of the population? Researchers have debated such questions over the years, resulting in an expert consensus concerning best research methods. These suggested methods include the practice of conducting sensitivity analyses to assess how study results would change if parameters, such as effectiveness, were measured differently.

Effectiveness Differs from Efficacy

A treatment's efficacy is the effect of the treatment under optimal conditions. A treatment's effectiveness is the effect of the treatment in routine clinical practice. For example, randomized clinical trials conducted for Food and Drug Administration (FDA) marketing approval typically aim to assess the relative safety and efficacy of a treatment so as to best determine the sole effect of the treatment, absent any other influential factors. Clinical trials for FDA approval also typically compare the efficacy of an investigational treatment to a placebo, rather than another treatment. Effectiveness research relaxes the strict exclusionary criteria that are typically required in such trials, in order to assess the treatment in the wide range of patients and environments in which the product is actually used.

Efficacy and effectiveness research results may differ because often in clinical practice, patients may have more than one illness, doses may vary, methods of administering the treatment may vary, and patients may simultaneously take treatments for multiple illnesses. Moreover, large segments of the potential patient population are often excluded from efficacy trials in order to achieve a more uniform study population. An often cited example is that many patients with high blood pressure also have diabetes; yet, efficacy clinical trials for high blood pressure treatments may not include patients with diabetes. As a result, any interaction between blood pressure and diabetes medications may not be known until the treatment is approved by the FDA and used in clinical practice. Also, different patients may respond to treatments differently due to physiologic differences, such as different metabolic rates of drugs and safety of anesthesia in surgical options.

Although conducted after FDA approval, post-marketing (also known as phase IV) studies are not necessarily effectiveness studies, and only rarely could be classified as comparative effectiveness studies. Post-marketing studies most often assess any ongoing safety concerns of one drug or device rather than the effectiveness of a product. Moreover, the few studies that compare two or more treatments often only assess the equivalence or superiority of the study sponsor's product, rather than the relative effectiveness of competing products.

Costs: Should They Be Included?

Costs Are Complicated to Measure

Costs are not always easy to define or measure. The total treatment costs may differ, sometimes dramatically, depending upon which perspective (e.g., patient, government payer, private insurer, society) is taken in the analysis, and which costs are included. As with the measurement of effectiveness, researchers have tried to resolve these issues through expert consensus of best research methods and the practice of conducting sensitivity analyses.

Inclusion Depends on Role in Health Care Decision Making

Much of the controversy surrounding whether costs should be included in comparative effectiveness research lies in the questions: when, how, and by whom will the research results be used to make decisions? The issue is most controversial if results that include costs are used to make insurance reimbursement, pricing, or coverage decisions. The inclusion of costs in research tends to not be as controversial when the results are not directly linked to medical and health policy decision making. One reason for the controversy is that policymakers may disagree about the way costs are measured or which costs are included in a research study.

Cost-effectiveness and Cost-benefit: Two Ways to Include Costs

Cost-effectiveness and cost-benefit analysis are two frequently used techniques of incorporating costs in the results of health technology assessments. The techniques compare the costs to the health benefits received from services or treatments. Both methods are used to help determine whether the additional health benefits of a service or treatment can justify the additional costs. Cost-effectiveness and cost-benefit differ in how the health benefits are measured. In cost-benefit analysis, the health benefits are monetarized, and the results are stated either in the form of a ratio or monetary difference between costs and benefits. In cost-effectiveness analyses, the health benefits are commonly measured in non-monetary units, such as life years (i.e., the additional years of life gained) or life years adjusted for quality (i.e., quality-adjusted life years—QALYs), and the end product is usually a ratio of the costs and benefits (e.g., dollars/QALY). Also, cost-effectiveness analysis always compares one or more alternatives, while cost-benefit analysis can be used to assess a single option (i.e., assessing whether the benefits are greater than the costs) or more than one option.

What Research Is Needed?

Past and Current Research Efforts

In order to determine whether and what type of comparative effectiveness research is needed, the scope and scale of current comparative effectiveness research efforts must be understood. A survey of the published medical literature and a review of the historical and current research initiatives provide a limited summary of the scale and scope of the comparative effectiveness research that has been funded and conducted. This section provides such a survey of the literature and reviews health technology research initiatives in the U.S. The section also discusses how comparative effectiveness research has been used in the U.S. and other countries and the potential role of a new comparative effectiveness entity.

Research in the Published Medical Literature

The published medical literature is one source of information that may help assess the extent to which various types of entities are currently conducting comparative clinical effectiveness research. We conducted a search of the published medical literature which included all studies published in PubMed journals that compared the effectiveness of at least two treatments or services between January 2004 and August 2007. The search was not intended to be an exhaustive search of all comparative clinical effectiveness studies, but rather was intended to summarize the information available from one large source of recently completed studies. It did not include studies that compared a treatment or service to a placebo. Studies conducted through initiatives discussed later in this report were excluded from this literature search. Each study was categorized by the type of research entity that conducted the study, which was determined by the affiliation of the study's contact author. The categories of types of research entities were academic, private institute, pharmaceutical companies, and government. A private institute was defined in this search as a for-profit or non-profit research group that was not based at a university or pharmaceutical company. Examples of private institutes include hospitals and private practices not affiliated with universities, such as the Kaiser Permanente Medical Center and the Black Hills Regional Eye Institute. Research conducted at Veterans Affairs (VA) hospitals was categorized as government research. The relative share of studies published by different types of entities is shown in Figure 1 . The kinds of studies published by each type of entity are shown in Table 1 .

The published studies primarily focused on treatments for mental health disorders and cardiovascular disease. Most comparative clinical effectiveness studies in the medical literature were produced by academic researchers. Researchers affiliated with pharmaceutical companies and the government have published relatively few comparative clinical effectiveness studies in the medical literature. Few studies focused on the comparative effectiveness of treatments in sub-populations, which was defined as populations other than white middle-age males (or females for diseases, such as ovarian cancer, that only occur in females). Sub-populations, such as children, elderly, and non-white races, may respond to treatments differently due to physiologic differences, such as different metabolic rates of drugs and safety of anesthesia in surgical options; thus research including sub-populations may help inform clinical practice. Also, few studies included patients with more than one disease (i.e., comorbidities ). Since nearly 60% of hospitalizations have at least one comorbidity (i.e., two diseases) and over 33% have two or more (i.e., at least three diseases), this type of research is generally held to help inform clinical decisions.

Federal Funding of Technology Assessments

Health technology assessment, including comparative clinical effectiveness, cost-effectiveness, and cost-benefit analysis, has been conducted for decades in the United States through both public and private initiatives. Some of these initiatives in the United States are ongoing, while others were terminated because of lack of funding. Further details about the initiatives can be found in the Appendix . The initiatives' timing and scale are summarized in Table 2 .

The Agency for Healthcare Research and Quality (AHRQ) and the National Institutes of Health (NIH) are currently the largest federal funders of extramural health technology assessments. Rather than conducting the research at the agency, these executive-branch agencies within the Department of Health and Human Services (HHS) primarily provide funding for academic and private sector researchers. AHRQ in particular has several ongoing programs for health technology assessments, including the Centers for Education and Research on Therapeutics (CERTs), the Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) Program, Evidence-based Practice Centers (EPCs), and the Research Initiative in Clinical Economics (RICE). These centers and programs conduct technology assessments, comparative effectiveness research, pharmaceutical outcomes research, and economic valuations of health care services and treatments. Although some Institutes at the NIH provide some funding for health technology assessments, unlike AHRQ, the NIH has not organized the research into centers or programs. Each of the AHRQ programs differ in their clinical focus, purpose, and types of technology assessments funded. For example, the EPCs are academic and private sector research centers that have five-year research contracts with AHRQ while funding from RICE is primarily allocated through competitive research grants. Unlike AHRQ and the NIH, the Veterans Health Administration's (VHA) Pharmacy Benefits Management Strategic Healthcare Group (PBMSHG) and the Department of Defense (DOD) PharmacoEconomic Center (PEC) do not out-source their health technology assessments. Rather, the assessments are funded, conducted, and used by the respective agency to make formulary and pricing decisions. Moreover, the research funded by AHRQ and NIH is intended to be a public good and to aid all health care decision makers, while the research from VHA and DOD centers is intended to aid decision makers at the respective agencies.

AHRQ previously sponsored research through its Medical Treatment Effectiveness Program (MEDTEP). Among other projects the program funded the Patient Outcomes Research Teams (PORTs). Funding for this program was terminated in 1995 for many reasons, including criticisms over the quality of the PORT guidelines. The Congressional Office of Technology Assessment (OTA) also previously funded and conducted technology assessments. The OTA was a nonpartisan congressional agency that conducted health and non-health technology assessments for Congress. The agency would use in-house researchers as well as experts outside of the agency. It was disbanded in 1995 partly due to controversy over its technology assessments, and partly for other reasons discussed in the Appendix .

Effect of Research on U.S. Policy and Practice

Health technology assessments can be used for many purposes, including aiding decisions by

insurers for coverage, drug formulary placement, and pricing of technologies; health care providers (e.g., physicians, nurses) for improving clinical practice; and consumers for making informed decisions.

Use of Technology Assessments in the United States

Over the past two decades, many organizations have tried to use existing technology assessments for health care decisions. Some organizations that have used these assessments include the Academy of Managed Care Pharmacy (AMCP), Consumer Reports' Best Buy Drugs project, the DOD PEC, for-profit firms (including consulting firms, private insurers, and pharmaceutical manufacturers), the Centers for Medicare and Medicaid Services (CMS), the Oregon Health Plan, and the VHA PBMSHG. The organizations' timing, target audience, and purpose are summarized in Table 3 .

With the exception of Consumer Reports' Best Buy Drugs, these organizations have used technology assessments for insurers' coverage, formulary, and pricing decisions. The AMCP promulgated guidelines for pharmaceutical companies' submissions for formulary assessments by private health insurers. The guidelines suggest comparisons to other products and a model that predicts the costs and health outcomes with the product in the health insurance plan. Research suggests that many pharmaceutical companies have adopted the guidelines; a survey found that managed care organizations had dossiers for 40% of drugs under review for coverage. Best Buy Drugs is a non-profit project of Consumer Reports that combines comparative effectiveness information with drug pricing information to select their "Best Buy picks" for health care consumers and providers. The DOD PEC monitors drugs' use, cost, and pharmacoeconomics within the Military Health System. The PEC has been credited by the DOD with improving patient safety and decreasing costs. Both the Medicare program and the Oregon Health Plan encountered opposition when the programs tried to incorporate cost-effectiveness analyses into policy decisions. In response, the Oregon Health Plan modified their program so that coverage was not based strictly on cost-effectiveness and, since January 2006, Medicare has explicitly excluded treatment costs from its national coverage determinations. The VHA PBMSHG compares the effectiveness of drugs to produce clinical practice guidelines and drug monographs, and to establish the VA formulary, drug pricing, and contracts. Further details about the initiatives can be found in the Appendix .

Translating Research into Clinical Practice

Depending on many factors, new information about treatments' safety or effectiveness may or may not change physicians' clinical practice. For example, one study suggested that simply the wording of a study's results may significantly influence whether physicians change which drugs they prescribe. Another study found that dissemination of educational materials alone was ineffective in changing physicians' prescribing habits. More active (and costly) methods, such as one-to-one educational outreach, multifaceted interventions, and participatory clinical guideline development were found to be more effective. Other more difficult factors to change, such as the management structure of the private insurer employing the physician, were also found to influence prescribing habits. Researchers and policy makers have tested many methods for changing clinical practice, and optimal strategies have evolved over the years. Overall, changing clinical practice is not a simple or inexpensive process, and requires more than disseminating information and expecting individuals to comb-through research studies and find ways to translate the findings into action.

Use of Technology Assessments by Other Governments

Comparative and cost-effectiveness analysis are given explicit roles in some other countries. In a 2001 survey of 11 OECD member countries that use technology assessment, such as comparative effectiveness or cost-effectiveness, three countries (Belgium, Italy, and the Netherlands) reported that the goal of using technology assesssment was cost containment, three (Belgium, the Netherlands, and Portugal) indicated global budgeting, and five (Australia, the Netherlands, Portugal, Sweden, and the U.K.) reported value-for money. Ten of the countries reported that a federal agency is responsible for either processing or conducting the assessments. Three of the countries (Australia, Belgium, and France) would only appoint consultants with no links to pharmaceutical manufacturers, while three other countries (the Netherlands, Portugal, and Switzerland) would only appoint consultants with no links to the manufacturer of the drug under review. Three of the countries (Japan, Belgium and the U.K.) noted that the assessment may be completed by the payer, while the other countries indicated that the assessment would only be completed by the product manufacturer. Belgium was the only country that reported that technology assessments reduced total drug expenditures. Italy and Portugal noted that it reduced unnecessary drug use, while Australia, Belgium, and Portugal indicated that it improved the cost-effectiveness of drug prescribing. None of the countries reported that lack of co-operation from pharmaceutical companies was an obstacle to obtaining improved results from technology assessment.

Three countries that are often as examples of governments using health technology assessments are the U.K., Australia, and Canada. Use of health technology assessments in these countries is summarized in Table 4 . Further details about how these three countries use technology assessments can be found in the Appendix .

The Potential Contribution of a New Research Entity

The multitude of current comparative effectiveness research entities in the U.S. introduces the question: why would a new entity be needed to conduct more comparative clinical effectiveness research? Many entities are currently conducting comparative effectiveness research, but the results are not centralized nor are they necessarily in a format that is easy for health care decision makers to use. Thus, one possible reason that could be offered in support of a new research entity might be increased efficiency and coordination of research. The entity could also arguably improve researchers' independence and scientific integrity, or spawn the genesis of research not currently being conducted on drugs, other health technologies, or services.

Factors that May Influence Its Success

Many factors might influence whether the aforementioned potential gains from a new entity could be realized. Some of these influential factors might be determined when the entity is established. The ideal structure, funding source, mission, and authority of the entity may depend upon the intended use of the research. For example, if the entity aims to influence insurers' decisions, then input from these stakeholders may help achieve this goal. If the entity aims to change clinical practice, then decision makers may wish to plan how the new research will help achieve this goal. Additionally, if, like the OTA, the audience is Congress, then there may be some benefit to establishing the entity as a congressional agency. There may also be some efficiency gains to having the agency work with existing agencies in the executive branch, such as AHRQ or NIH.

Other factors influential to the entity's success would more likely be determined by the entity's administration, including:

Which treatments would be studied? Would different types of treatment options, such as drugs and surgery, be compared to each other? Would the research methods include systematic reviews, decision models, and observational studies, as well as randomized trials? Who would oversee and review the studies' methods, timing, and clinical endpoints? Which researchers and what expertise would be required to conduct the studies? How would the results be presented and used? How would the political support for the entity be maintained?

The answers to these questions could have repercussions on many interested parties including physicians, patients, payers, manufacturers, researchers, and federal agencies.

For example, the timing of the research could influence the impact of the research, since the amount of information that is known about a treatment differs at different time periods. The issue is that conclusions may need to be modified when more information becomes available in a particular research area. On the one hand, waiting for perfect information on a treatment before conducting any analyses would help curb modifications, but any resulting conclusions may have a limited impact on improving clinical decision making. On the other hand, analyzing areas with imperfect information would be more likely to have a large impact on clinical decision making, but the conclusions may change once more information becomes available. A middle, but imperfect, option would be to re-evaluate conclusions as more information becomes available; while this option would allow all information to be incorporated into decisions in a timely manner, it would also increase the overall costs of technology assessments. A new entity might need to grapple with these types of decisions.

Appendix. Description of U.S. and Non-U.S. Comparative Effectiveness Initiatives and Legislation in the 109th and 110th Congresses

U.S. Initiatives

Academy of Managed Care Pharmacy

In 2001, Academy of Managed Care Pharmacy (AMCP) promulgated guidelines, known as the AMCP Format for Formulary Submissions, for conducting formulary assessments, including economic evaluations. The purpose of the guidelines was to help ensure that any increased utilization of pharmaceuticals and vaccines was based on good scientific evidence and value. The guidelines encouraged pharmaceutical manufacturers to submit a dossier with clinical and economic data from published and unpublished studies, along with an economic model that predicts the costs and health outcomes with the product in the health plan. The dossier also includes a section on the comparative pharmacokinetic and pharmacologic data for other agents commonly used to treat the condition. The dossiers have become an industry standard, and are used by pharmacy and therapeutics (P&T) committees of managed care organizations and health insurers.

A recent survey found that managed care organizations had dossiers for 40% of drugs under review for coverage. Fifty-three percent of the dossiers received included budget-impact models and 39.3% included cost-effectiveness or cost-benefit analyses. Less than half of the economic models were deemed adequate by the P&T committees. Nearly two-thirds of survey respondents indicated that P&T committees modified the economic model on the dossier because pharmaceutical manufacturers did not make models directly applicable to the health plan's population.

Agency for Healthcare Research and Quality

The agency, formerly known as the Agency Health Care Policy and Research (AHCPR), was established in 1989 as an agency within the Department of HHS. The agency's statutory responsibilities included outcomes research and clinical practice guidelines development. The Medical Treatment Effectiveness Program (MEDTEP) was established in 1989 as part of AHCPR. MEDTEP funded effectiveness research, guideline development, database development, and methods of disseminating information.

In 1992, Congress directed the AHCPR to incorporate cost-effectiveness information in its technology assessments and clinical practice guidelines. The use of cost-effectiveness and the development of the clinical practice guidelines generated controversy and criticisms. The criticism included critiques from IOM committees, the Government Accountability Office (GAO), the Physician Payment Review Commission (PPRC), the Congressional Office of Technology Assessment (OTA), and an influential lobbyist group of orthopedic surgeons. The agency was also criticized for its role in the Clinton health care reform plan. In 1995, partially in response to the criticism and concerns, Congress sharply decreased the FY1997 budget for AHCPR by 20%, thereby ending the funding for MEDTEP. From 1995 to 1999, AHCPR received operating funds through annual appropriations.

The agency was reauthorized and renamed the Agency for Healthcare Research and Quality (AHRQ) in 1999, and now has many centers and programs that conduct inter-related research on health care treatments. These include the Centers for Education and Research on Therapeutics (CERTs), the Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) Program, the Evidence-based Practice Centers (EPCs), and the Research Initiative in Clinical Economics (RICE), which conduct technology assessments, comparative effectiveness research, pharmaceutical outcomes research, and economic valuations of health care services and treatments, respectively.

Centers for Education and Research on Therapeutics. AHRQ funds pharmaceutical outcomes research through the CERTs, which is a national demonstration program for education and research on the optimal use of drugs, biologicals, and medical devices. AHRQ was given the responsibility of administering the program in 1997 as part of the Food and Drug Administration Modernization Act (FDAMA; P.L. 105-115 ), and the first centers were funded in 1999. The program is administered as a cooperative agreement by AHRQ in consultation with the FDA. Some of the research is also conducted in partnership with private corporations, such as insurers or pharmaceutical manufacturers. The research compares the health risks, benefits, cost-effectiveness, economic implications, and interactions of treatments. Some of the research also examines the cost-effectiveness of treatments. Currently, 10 of the 11 centers are affiliated with academic institutions. Developing Evidence to Inform Decisions about Effectiveness Program. Section 1013 of the MMA authorizes AHRQ to conduct and support research on outcomes, comparative clinical effectiveness, and appropriateness of pharmaceuticals, devices, and health care services. The section prohibits the Administrator of CMS from using the data produced under the section to withhold coverage of a prescription drug. Although the section authorized $50 million to be appropriated for the research in 2004, AHRQ has been appropriated $15 million each year for carrying out the research. AHRQ created the DEcIDE program to tackle the responsibilities described in section 1013. Like the EPCs, the DEcIDE centers are primarily based at universities. Unlike the EPCs, the DEcIDE centers do not examine the cost-effectiveness of technologies, but rather focus on health outcomes and comparative clinical effectiveness. As of August 2007, the agency has funded 15 projects that evaluate the comparative effectiveness of health care treatments. Evidence-based Practice Centers Program. The purpose of the EPC program is to improve the quality, effectiveness, and appropriateness of health care through technology assessments, evidence reports, and research on the methods for systematic reviews. The reports inform public and private insurers' coverage decisions, and are used to develop quality measures, educational materials, guidelines, and research agendas. Cost-effectiveness analysis has been used as a research tool in some of the reports. Topics for technology assessments are nominated by AHRQ's non-federal partners and assessments tend to be completed in approximately 15 months. Thirteen EPCs were awarded five-year contracts with AHRQ in 2002. Three of the Centers (Duke University, ECRI, and Tufts University-New England Medical Center) specialize in technology assessments for CMS to inform national coverage decisions for the Medicare program and provide information to Medicare carriers. One center (Oregon Health & Science University), supports the work of the U.S. Preventive Services Task Force. Ten of the centers are affiliated with academic institutions and the remainder are private institutions. Ten are based in the U.S., and the other three are based in Canada. As of August 2007, 155 evidence reports had been published by the EPC program. Research Initiative in Clinical Economics. Begun in 2001, RICE funds research on the cost-effectiveness, cost-benefit, and methods for estimating the value of health care interventions. Although focused on cost-effectiveness, this research has not sparked controversy. Unlike other initiatives, this research has not been used for clinical practice guidelines or coverage decisions, and thus has not been explicitly connected to policy recommendations or implementation.

Blue Cross Blue Shield Technology Evaluation Center

The Technology Evaluation Center (TEC) of the BlueCross BlueShield (BCBS) Association has been assessing the relative effectiveness and appropriateness of different technologies since 1985. AHRQ designated and funded the TEC as one of its first Evidence-based Practice Centers (EPCs) in 1997, and renewed the designation for an additional five years in 2002. The TEC relies upon medical and research employees to conduct the evaluations, under the guidance of their Medical Advisory Panel of clinical experts. The Center produced 17 evaluations in 2005, 14 in 2006, and 4 between January and August 2007. The Center's evaluations focus on the relative effectiveness of technologies, particularly with regard to the effect upon health outcomes, such as length of life, quality of life, and functional abilities. The TEC compares the effectiveness of pharmaceuticals, medical devices, and health services. Cost-effectiveness analyses are mentioned by the TEC as a potential type of special technology assessment the Center may undertake; however, since 2005, none of the publicly available assessments have analyzed the cost-effectiveness of technologies. All evaluations use the same criteria, approach for reviewing the evidence, and format for reporting the results. The evaluations are intended to be for informational purposes only and are not characterized as recommendations or guidelines. All completed evaluations are available for free on the center's website, along with the list of technology evaluations in process.

Consumer Reports' Best Buy Drugs Project

Consumer Reports' Best Buy Drugs is a non-profit project of Consumer Reports that is primarily supported by educational grants. The project synthesizes DERP findings in order to provide comparative effectiveness information about drugs to health care consumers and providers, and selects "Best Buy picks" within drug classes; the most influential factor in the selection process is the drug's effectiveness. The summaries include information on effectiveness, safety, and price. The drug prices are national average cash prices. The summaries are available for free on the project's website, and are updated as new information becomes available.

DOD PharmacoEconomic Center

The Department of Defense (DOD) PharmacoEconomic Center (PEC) was established in 1992 in response to rising DOD pharmaceutical expenditures. Its mission is to "improve the clinical, economic, and humanistic outcomes of drug therapy in support of the readiness and managed healthcare missions of the Military Health System." The center performs cost-effectiveness analyses, works with the DOD P&T committee to establish the Tri-Service Drug Formulary list and the National Mail Order Pharmacy formulary list, provides drug treatment guidelines with the VHA, and monitors drugs' use, cost, and pharmacoeconomics within the Military Health System. Some of the evaluations by the PEC are publicly available. The PEC publishes a monthly newsletter called the PEC Update "to educate health care providers and other pharmacy benefit stakeholders about cost-effective drug therapy." PEC analyses have shown that the center has improved patient safety and decreased costs for the Military Health System.

Drug Effectiveness Review Project

In 2001, the Oregon state legislature began commissioning, through the Oregon Medicaid program, the Oregon Health Science University (OHSU) to assess the comparative clinical effectiveness and safety of drugs in clinical practice. The initiative was named the Drug Effectiveness Review Project (DERP). OHSU was a logical location for the initiative since, at the time, it was an AHRQ funded EPC; as such, it completed systematic literature reviews to produce evidence reports and technology assessments. The reviews were viewed as a way to equalize buyers' and sellers' information about heterogeneous products. Credibility and transparency of the research were viewed as crucial to the success of the project, and DERP's conflict of interest policy forbids its reviewers from having financial ties with the companies whose products they are evaluating.

Currently, a team of researchers at OHSU coordinate the reviews with experts in the clinical area. The reviews incorporate a scientific literature review as well as assessments of the evidence on the effectiveness, safety and adverse effects of the drugs nationwide, as well as in sub-populations. Cost information is explicitly excluded from the reviews. The reviews only compare drugs within a class, and do not compare the effectiveness of medical devices or health services. The primary audience for the DERP reviews is the state Medicaid programs, which use them to help inform Medicaid drug coverage decisions. Topics are chosen at biannual public meetings with the state pharmacy directors and medical directors from the funding Medicaid programs; the selection is based upon Medicaid expenditures, the potential usefulness of a review, and availability of data. Once a drug class is chosen, a solicitation for information is sent to all pharmaceutical manufacturers with products in the class of interest. Any information provided to DERP by manufacturers is disclosed on request to any interested party. DERP also draws upon information from the published and unpublished scientific literature. DERP does not conduct new comparative effectiveness studies and only uses existing information. The possible outcomes of a DERP report are (1) no evidence of differences between drugs, (2) some differences under some circumstances, (3) unclear whether drugs differ, and (4) significant differences between drugs. The final reports do not make coverage, payment, or formulary recommendations. Rather, state decision making groups in the funding Medicaid programs use the DERP reports to help reach conclusions about coverage, payment, and formulary status; notably, states may not necessarily reach the same conclusion after reviewing the same DERP report. All DERP reports are available for free on the website. As of August 2007, 13 states (plus the Canadian Agency for Drugs and Technologies in Health) were participating in and financially supporting DERP, which has produced more than 30 reports since October 1, 2003.

ECRI Institute

Some non-profit organizations, such as ECRI Institute, also perform health care technology assessments and comparative effectiveness analyses. The clients of these organizations may range from private insurers to government agencies. For example, ECRI is one of AHRQ's EPCs and provides health technology assessments to the DOD TRICARE program, but also counts the World Health Organization, hospitals and private insurers as clients.

For-Profit Firms

Many for-profit firms also perform technology assessments, including comparative effectiveness, cost-effectiveness, or pharmaceutical outcomes research, for their clients, such as pharmaceutical manufacturers and health insurers. Examples of such firms are Hayes Inc., and United Biosource Corporation (formerly known as MEDTAP). Other firms, such as McKesson's InterQual, produce clinical guidelines and decision support criteria for clients. Many pharmaceutical and private health insurance plans also perform technology assessments in-house. Much of the information produced by such firms may be considered confidential client information and would not be available to the public.

Medicare

As dictated by statute, Medicare pays for medically needed and necessary services provided to elderly and disabled individuals. Fundamental questions concern what constitutes medically reasonable and necessary care and under what circumstances should the care be covered by the program? These determinations happen through two mechanisms: national coverage determinations (NCDs) and local coverage determinations (LCDs). As the terms imply, NCDs are coverage determinations that are made by CMS and applied across the nation, whereas LCDs are coverage determinations made by local contractors for the Medicare program and applied to limited geographic areas.

Medicare has had an uneven history of incorporating the cost-effectiveness of treatments when establishing coverage determinations. In 1989, the Health Care Financing Administration (HCFA), the predecessor to CMS, issued a Federal Register notice proposing that cost-effectiveness information be included as a component of Medicare coverage decisions. This was the first time that HCFA had ever proposed to include cost-effectiveness as a factor in coverage decisions. The 1989 proposal met substantial opposition by professional and industry groups, and as a result was never implemented and instead was withdrawn in 1999. In 2000, Medicare issued new guidelines for determination of coverage that included demonstrable medical benefit and "added value." The medical benefit was to be determined using "evidence-based medicine," which weighs the risks and benefits of a treatment by considering the reliability of the source of the evidence. Evidence could include experimental studies, expert opinion, informal studies, logical reasoning from biological knowledge, or clinical guidelines. The term "added value" was defined as adding greater benefit; if two technologies provided equal benefit, then only the lower cost technology would be covered. Due to opposition to the notion of "added value," the 2000 notice of intent for determination of coverage was withdrawn in 2003 and never implemented.

Since January 2006, CMS has explicitly excluded treatment costs from its NCDs. In a 2006 coverage guidance, CMS stated "cost effectiveness is not a factor CMS considers in making national coverage determinations." On the same day, CMS issued a guidance on technology assessments that included the statement "cost is not a factor in our review or determination to cover a particular technology."

In June 2006, a chapter in the MedPAC report explored the methodological advantages and disadvantages of using cost-effectiveness analysis in Medicare. The chapter notes that the results (and conclusions) of cost-effectiveness studies may vary due to differences in studies' methods, differences in the clinical characteristics of patients, and the timing of the study. Moreover, the report noted that some studies' methods were opaque. The report states that considering the clinical and cost-effectiveness of treatments might increase the return on society's investment in health care, but doing so will be most useful when results are comparable across studies and treatments.

CMS continues to use the "evidence-based medicine" approach to evaluate a treatment's health benefit, and studies from AHRQ's EPCs may be included as evidence. CMS also is gathering information about the effectiveness of treatments through its Coverage with Evidence Development (CED) program. Under CED, CMS may cover a treatment with the condition that providers and patients who use the treatment allow data to be collected about the patient, treatment, and health outcomes. This information would then be evaluated to ensure that the medical care is reasonable and necessary. CED may be required for treatments that (1) are in new drug classes with new mechanisms; (2) may be effective only for subpopulations; (3) may provide clinical benefit for off-label uses; or (4) may have substantial consequences for treating the wrong patients.

In contrast to NCDs, CMS believes their contractors have the authority to use cost and cost-effectiveness for LCDs, specifically with regard to "least costly alternative" policies. Least costly alternative policies state that a Medicare contractor will not pay the additional cost of a more expensive item if a clinically comparable item costs less. Such policies have been proposed for nebulizers used for respiratory conditions such as asthma, emphysema, and chronic bronchitis, and implemented for Lupron and Zoladex, treatments for advanced prostate cancer.

Office of Technology Assessment

Although charged with being a nonpartisan source of information about scientific and technical issues for the legislative branch, the analyses of the Congressional Office of Technology Assessment (OTA) were, at times, controversial. Part of the reason for the controversy was its explicit inclusion of costs and cost-effectiveness in health technology assessments. Another part of the reason for controversy was the work was perceived by some as not timely enough, duplicative of other agencies, and not necessarily useful to public programs. As a result of the controversy, the agency was disbanded in 1995, 23 years after its inception, as part of budget reductions in the 104 th Congress. Prior to its disbandment, the OTA was the smallest Congressional agency, with less than 200 employees and an annual budget of $22 million. The agency released approximately 50 reports each year.

In creating the agency, great care had been taken to ensure and protect the agency's nonpartisanship and scientific integrity. The OTA was governed by the Technology Assessment Board (TAB), which was composed of six Senators and six Representatives with equal representation from each party. The TAB appointed the Director of OTA, for a six-year term, as well as an advisory council of 10 experts, including the Comptroller General and the Director of the Congressional Research Service, to advise the Agency. The TAB also reviewed all proposed studies as well as the final reports prior to release. The agency's studies did not recommend a single policy option, but rather laid out different options and projected the possible consequences of each.

The Chairman of any congressional committee, the TAB, and the Director of OTA had the authority to request any technology assessment from OTA; the assessments generally took 1-2 years to complete. Relevant stakeholders and experts were consulted on the reports to provide a diversity of viewpoints and to help shape and critique interim reports. Once they were approved by TAB, the final reports were publicly released.

Oregon Health Plan

An objective of the Oregon Basic Health Services Act of 1989 was to expand the population covered by Medicaid, to all Oregonians with incomes below 100% of the federal poverty level. The additional costs would be managed by covering fewer services. In order to expand the program in this manner, the Oregon Medicaid program was required to receive a federal waiver of Medicaid statutes. Even though increasing the income limits for Medicaid eligibility in this manner is credited with helping to reduce the percentage of uninsured in Oregon from 18% in 1992 to 11% in 1996, controversy surrounded the Oregon Medicaid program's process of selecting which services to cover. The Oregon Health Plan was the first large-scale public attempt to apply cost-effectiveness analysis to set priorities for medical services. The struggles of the program had ripple effects on the use of cost-effectiveness analysis in other settings.

The Act created the Oregon Health Services Commission and charged it with developing a list that ranked medical services by priority. The program initially used cost-effectiveness analysis to develop the prioritized list. The value of the health benefits was determined by assessing (1) Oregonians' community values of different treatments in town meetings; (2) Oregonians' ratings of the desirability of health states (i.e., health-related quality of life); and (3) medical professionals' judgment of the efficacy of different treatments. The resulting cost-effectiveness ratios were criticized by some as not being reflective of societal values. Critics noted that capping teeth would have been ranked as a higher priority than life-saving surgery for appendicitis. Some researchers concluded that cost should not be considered in determining treatment priorities, while others blamed what they saw as counterintuitive rankings on the methods for measuring benefits. As a result of the controversy, the Oregon health commissioners' re-ranked the treatments, which resulted in rankings that were not necessarily related to treatments' relative costs and benefits, and were perceived to be more subjective. The revised list included 709 disease-treatment combinations, and the state Medicaid budget allowed the program to cover the costs of combinations 1 through 587.

In August 1992, the Department of HHS rejected Oregon's waiver application due to concerns about possible discrimination against disabled people through the use of Oregonians' ratings of the desirability of health states; these concerns were supported by an analysis by the Congressional Office of Technology Assessment (OTA). In November 1992, the Oregon Medicaid program submitted a new list that did not use ratings of health states. As a result, the new list was not based on cost-effectiveness ratios and instead was perceived by some observers to be more the result of pressure from advocacy groups and the commissioners' judgments. The revised plan was approved in March 1993 and implemented in 1994.

The Oregon program's analysis was very different from many other cost-benefit, cost-effectiveness, and comparative effectiveness analyses because treatments for each disease were compared to treatments for entirely different diseases. Most other cost-benefit and cost-effectiveness analyses compare like-to-like—that is, only treatments for the same disease are compared to each other. For example, the costs and benefits of a treatment for colorectal cancer would not typically be compared to a treatment for heart disease, but the Oregon program explicitly compared treatments in this manner.

U.S. Preventive Services Task Force

The U.S. Preventive Services Task Force (USPSTF) was established in 1984 as an independent federal advisory committee, under the U.S. Public Health Service, and given the responsibility of developing clinical practice guidelines for primary care physicians. The USPSTF took what was perceived to be a novel approach at the time by basing the guidelines upon quality and strength of clinical evidence, rather than simply "expert consensus." The task force published its first set of guidelines in 1989, and was reconvened in 1990 and 1998, the latter of which incorporated cost-effectiveness analysis in its reviews and was financially supported by AHRQ. The USPSTF has also sponsored some cost-effectiveness studies to better inform the USPSTF clinical practice guidelines. The guidelines, in general, focus on the prevention of diseases, and compare the preventative methods. They do not explicitly compare different drugs or technologies, but rather they compare screening to taking preventative medications to surgical options for a disease.

Veterans Health Administration

The Pharmacy Benefits Management Strategic Healthcare Group (PBMSHG) was established within the Veterans Health Administration (VHA) in 1995 to improve the health status of veterans by encouraging the appropriate use of medications. To this end, the group compares and publishes analyses of the effectiveness of drugs in the same class, produces clinical practice guidelines, and drug monographs, in addition to establishing the Department of Veterans Affairs' (VA) formulary, drug pricing, and contracts. Twenty-five drug class reviews were available on the group's website. However, the group does not publish the guidelines that are used in the internally generated economic assessments.

Other Governments' Initiatives

Australia

The Pharmaceutical Benefits Scheme (PBS) is the Australian health care system's program of subsidizing the cost of outpatient prescription medicines for all Australian citizens who are residents. The objective of the PBS is "to provide timely access to medicines that Australians need, at a cost individuals and the community can afford." Approximately 80% of all prescription medicines in Australia are subsidized under the PBS and more than 90% of outpatient drugs. Drugs may be sold in Australia if they pass the Australian regulatory review, which is similar to the FDA, but patients pay the full cost of the drugs unless they are listed on the PBS. The PBS has more than 650 drugs (more than 2500 items) on the formulary, which includes at least one drug for most medical conditions for which drug therapy is appropriate. To be included in the formulary, a drug must be evaluated by the Pharmaceutical Benefits Advisory Committee (PBAC), which is an independent panel of experts that recommends to the Minister of Health and Ageing whether a drug should be included. The Pharmaceutical Benefits Pricing Authority (PBPA) then provides advice to the Minister on negotiating an appropriate price for formulary drugs.

The PBAC is required to consider the comparative clinical effectiveness, and comparative cost-effectiveness of a drug relative to the therapy most likely to be replaced in practice, which may be another drug and non-drug therapy. The existence of one drug on the formulary does not preclude the addition of a similar drug to the formulary; the number of drugs available to treat a particular condition is not limited. A positive recommendation from PBAC is a necessary but not a sufficient condition for inclusion on the formulary. In other words, the Minister can only add drugs to the formulary that have received a positive recommendation but not all drugs that receive a positive recommendation are automatically added. The only circumstance in which a Minister has exercised the right to not add a positively recommended drug occurred in 2002 when the Minister elected to not include Viagra. This decision was reportedly made due to concern about the impact of erectile dysfunction treatments on the PBS budget. Accordingly, the Minister simultaneously removed Caverject, the only drug at the time that was included on the PBS for the treatment of impotence.

To make a submission to the PBAC, sponsors (usually pharmaceutical manufacturers) are required to undertake a literature review, identify relevant trials, assess the quality of the trials, and aggregate the trial data. They are also required to perform a cost-minimization or cost-effectiveness analysis (which may include modeled analyses), the selection of which would follow PBAC's guidelines. The sponsor selects the initial price of their drug for the economic analysis and nominates a comparator drug. If PBAC does not agree that the nominated comparator is appropriate, then it may instruct the sponsor to use a different comparator drug.

The submitted trials may evaluate either the efficacy or effectiveness of the drug. Head-to-head randomized clinical trials, while preferred, are not mandatory. If such direct comparisons are not available, sponsors may submit two sets of randomized trials, or even non-randomized trials, that use the same reference drug. However, the guidelines note that non-randomized studies often over-estimate the benefit of an intervention, and "claims about the comparative clinical performance that are based solely on data from such sources will be treated with some scepticism." Applicant sponsors are required to demonstrate that differences between the comparison treatments are statistically significant as well as clinically important.

If a drug does not receive a positive recommendation by the PBAC, then the manufacturer may resubmit the application and provide additional data for the committee's consideration. If no additional data are available, the sponsor may seek an independent review. The independent review may only consider specific issues in dispute and cannot review the PBAC's overall recommendation.

Canada

The Canadian Agency for Drugs and Technologies in Health (CADTH) provides assessments of the effectiveness and efficiency of drugs and health technologies to Canadian health decision makers. It is a non-profit organization that was established by the Canadian government on a trial-basis in 1990; it became a permanent entity in 1993. The agency is funded on an annual basis by the Canadian government.

One of CADTH's programs is the Canadian Expert Drug Advisory Committee (CEDAC), which makes recommendations to participating publicly financed drug insurance plans regarding inclusion of a new drug in formularies. CEDAC is an independent advisory committee that is accountable to the CADTH Board of Directors. Its recommendations are informed by drug evaluations from the Common Drug Review (CDR), which is an intergovernmental body established in 2003 to evaluate new chemical entities (NCEs) and drug combinations. A CDR evaluation includes the safety, clinical efficacy, therapeutic advantages and disadvantages, and the relative cost-effectiveness of a drug; it does not consider the net cost or budgetary impact of the drug.

Another of CADTH's programs is the Canadian Optimal Medication Prescribing and Utilization Service (COMPUS), which was launched in 2004 and is charged with identifying and promoting best clinical practices. COMPUS does not create new guidelines, but rather critiques and rates the evidence of guidelines produced by others. The most influential factors in determining COMPUS's research priorities are variations in practice, affected patient population size, availability of data on outcomes, and approval by Canadian deputy ministers of health.

United Kingdom

The National Institute for Clinical Excellence (NICE) was established in April 1999 and expanded its responsibilities, to include guidance on the prevention of ill health and the promotion of good health, and became the National Institute for Health and Clinical Excellence (NICE) in April 2005. Its mission is to advise health care providers in England and Wales on how to use resources effectively and deliver the highest quality of care to National Health Service (NHS) patients. NICE is an independent organization that is funded by the Department of Health in England (with contributions from Wales, Scotland, and Northern Ireland), and accountable to the British Parliament. It was established as one of several "arm's-length bodies" within the NHS so as to politically insulate the organization and help to produce intellectually honest, high quality guidance. The total CY2007 budget was £31 million for recurring programs, £4 million for non-recurring programs, and a separate £3.5 million for guidance on technologies.

Among other types of guidelines, NICE produces guidance on new and existing technology and treatments based on their clinical and cost-effectiveness, the latter of which is required to be included. The cost, utilization, uncertainty, and variation in use of the product are considered to be critical factors in selecting topics for the guidelines. As of May 2007, NICE had published 119 guidelines for new and existing technologies and 46 for treatments. The guidelines play a large role in drug cost reimbursement by the NHS, and as such are important to pharmaceutical manufacturers. For example, some manufacturers have recently cut the price of their products due to concern about potentially negative NICE guidelines. Investors and Wall Street analysts also tend to be concerned about negative NICE guidelines because of their potential impact on NHS coverage and manufacturers' revenues.

NICE does not fund new primary research. Rather, the analysis is produced by academics who use existing research, gathered through systematic literature reviews, and statistical modeling to answer the questions poised by NICE. These academics are affiliated with either universities or professional organizations (Royal Colleges). Multidiscliplinary committees review the technology and clinical assessments and consult with relevant stakeholders to produce determinations, guidelines, and public health guidance. Determinations can be appealed before NICE releases its final determination guidance directly to the NHS.

The process and methods are publicly available and accessible, and all guidance is subject to public consultation. Draft recommendations are subject to public appeals from stakeholders. The evidence on which the recommendations are made is publicly available, with the exception of companies' confidential information. NHS managers are expected to fund the mandatory implementation of technology appraisals no later than three months after the guidance is issued. It is recognized that the implementation of other types of guidelines may take longer than three months due to their greater scope.

The cost-effectiveness analysis includes only the costs from the perspective of the public decision maker, and the comparison treatment is the most commonly used alternative treatment. The analyses segment the patient populations by the value and clinical benefit added by the drug. As a result, the majority of the guidelines state whether a technology or treatment is effective, and specify the population in which the product is most cost-effective.

NICE does not account for the budget impact or affordability of a new technology. Some researchers disagree with this separation and feel that NICE should prioritize its guidance within a fixed budget, or use some other method that helps contain NHS costs. Researchers and policymakers have also disagreed about the role cost-effectiveness should play in the NICE appraisal process.