Comparative Effectiveness: Agency for Healthcare Research and Quality's Process for Awarding Recovery Act Funds and Disseminating Results
Released: 2012-03-30T00:00:00

Letter

Scope and Methodology

To describe the process and criteria AHRQ used to award its $474 million in Recovery Act CER funds, we reviewed relevant statutes as well as documentation on the process and criteria AHRQ uses to (1) determine the scientific and technical merit of grant applications and contract proposals, and (2) select grant recipients and contractors. We reviewed spending plans, which outline AHRQ’s and HHS’s plans for spending Recovery Act funds. We also reviewed summary statements that describe AHRQ’s process for selecting grantees and contractors to be awarded Recovery Act funds. In addition, we reviewed AHRQ’s Management Operations Manual, the agency’s written guidance that provides policies and procedures for selecting grant recipients. We interviewed AHRQ and other HHS officials to learn about the processes and criteria they used to select the grantees and contractors that received awards funded with the $474 million in Recovery Act CER funds and to coordinate these awards with other HHS agencies that also received Recovery Act CER funds. Because NIH also received Recovery Act CER funds, we conducted interviews with NIH officials to confirm the methods and processes used by AHRQ to coordinate funding opportunity announcements (FOA), contract solicitations, and awards with NIH in an effort to prevent the unnecessary duplication of effort in awarding Recovery Act CER funds. Finally, we obtained data from AHRQ on the number and type of awards made between February 2009 and September 2010 using the $474 million in Recovery Act CER funds. We relied on Recovery Act award data provided by AHRQ and did not audit the reported data. To determine whether AHRQ and the Office of the Secretary’s Recovery Act CER award data were sufficiently reliable for our analyses, we conducted a reliability assessment of the data we used by reviewing existing information about the data, conducting quality control checks, and interviewing agency officials knowledgeable about the data. We determined that the data were sufficiently reliable for the purposes of this report.

To describe AHRQ’s plans to disseminate results from CER funded with Recovery Act funds, we reviewed the Recovery Act and other relevant statutes to determine AHRQ’s responsibilities for disseminating CER. We reviewed agency documents, including AHRQ contractors’ work plans describing specific goals and activities; AHRQ’s general publications, including reports and guides posted on AHRQ’s website and electronic newsletters; and samples of AHRQ’s CER, including original research reports, treatment guides, and slide presentations used for educating clinicians. We also interviewed AHRQ officials and a contractor to understand how the agency plans to disseminate the results of CER funded with Recovery Act funds and to obtain information on plans AHRQ has for assessing the effectiveness of its dissemination efforts.

To describe the steps AHRQ has taken to fulfill its roles and responsibilities related to PCORI under PPACA, we reviewed provisions in PPACA to identify these roles and responsibilities, which include to broadly disseminate CER findings to various audiences; develop a publicly available database to collect evidence and research; and promote the timely incorporation of CER findings into health information technology systems that support clinical decision making. While AHRQ is required to conduct a number of activities under PPACA, we focused our review on those activities that are related to PCORI. We also reviewed AHRQ’s PPACA spending plan for fiscal years 2011 and 2012, which describes the agency’s plans for using funds made available by PPACA, as well as PCORI presentation materials and meeting reports. We also conducted interviews with AHRQ officials to determine the steps the agency has taken to meet its responsibilities related to PCORI under PPACA.

We conducted this performance audit from February 2011 to February 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Background

AHRQ supports CER by awarding grants and contracts to entities in order to conduct CER and perform related activities, such as the dissemination As 1 of 12 agencies within HHS, AHRQ’s overarching of CER results.mission is to improve the quality, safety, efficiency, and effectiveness of health care for all Americans. (For more information on AHRQ’s mission, research, priorities, and budget, see app. I.)

CER and the Recovery Act

The Recovery Act provided a significant amount of funding for AHRQ to conduct CER activities. (See table 1.)

HHS developed departmentwide priorities for CER. (See table 2.) Of the total amount of $474 million in Recovery Act funds available to AHRQ for CER, $174 million of these funds were allocated to AHRQ by the Office of the Secretary and were used to support the HHS departmentwide priorities for CER.

AHRQ developed seven agency-specific CER priority areas to guide its spending of Recovery Act funds. (See table 3.) Of the total amount of $474 million in Recovery Act funds available to AHRQ for CER, $300 million of these funds were appropriated to the agency and, therefore, supported the agency’s seven CER priority areas.

The enactment of PPACA in 2010 gave AHRQ new roles and responsibilities related to disseminating CER and building capacity for research, and appropriated funds for carrying out these activities. Several of these responsibilities relate to work conducted by PCORI. Established in November 2010, PCORI was authorized to help coordinate CER at a national level by developing national priorities for CER and conducting and funding CER activities. PPACA directs AHRQ to broadly disseminate CER findings to physicians, other health care providers, patients, payers, and policymakers; develop a publicly available database to collect evidence and research; promote the timely incorporation of CER findings into health information technology systems that support clinical decision making; and establish a process for receiving feedback about the value of information disseminated by AHRQ.

To fund this work, PPACA established the Patient-Centered Outcomes Research Trust Fund (PCORTF). The act specified that percentages of this trust fund be provided to the Secretary of HHS and AHRQ in each of Specifically, AHRQ received $8 million fiscal years 2011 through 2019. from this trust fund in fiscal year 2011 and will receive $24 million in fiscal year 2012, representing 16 percent of the total amount appropriated to this trust fund in each of these fiscal years. In subsequent fiscal years, AHRQ will continue to receive 16 percent of the total amount appropriated to the trust fund, which will be based on the net revenues from fees on health insurance and self-insured plans, amounts transferred from the Medicare trust funds, and appropriations to PCORTF from the General Fund of the Treasury.

AHRQ’s Standard Competitive Process for Selecting Grant Recipients

A portion of AHRQ’s work, including its work related to CER, is conducted through grants awarded to research centers and academic organizations to fund research ideas developed by a grant applicant. Grant applications are submitted in response to publicly available FOAs, which announce AHRQ’s intention to award research grants. AHRQ has established a standard competitive process that is governed by federal law to select According to AHRQ officials, this multistep process grant recipients.includes: (1) an initial review of received applications; (2) preliminary scoring of applications; (3) review and final scoring of applications at a peer review panel meeting; (4) the development of preliminary funding recommendations; (5) review by a senior leadership team within AHRQ; and (6) a final determination of funding by the agency director.

The trustee of PCORTF is to provide for the transfer from PCORTF of 20 percent of the amounts appropriated or credited to PCORTF for each of fiscal years 2011 through 2019 to the Secretary of HHS. Of the amounts transferred, the Secretary of HHS is to distribute 80 percent to AHRQ and 20 percent to the Secretary of HHS. See 26 U.S.C. § 9511.

To evaluate the grant applications it receives in response to FOAs, AHRQ’s peer reviewers, most of whom are authorities in their respective fields and not government employees, use five standard core criteria to score and rank the applications. These five standard core criteria are (1) the significance in addressing an important problem; (2) the investigators’ ability to carry out the research; (3) the originality or innovation of the project; (4) the development of an adequate research approach or framework; and (5) the scientific environment in which the applicant plans to conduct the research. Each FOA also contains criteria that are specific to the announcement. While these other specific criteria are not individually scored, they are used to evaluate the applications during the peer review panel meetings. (See fig. 1 for an overview of AHRQ’s standard process for selecting grant recipients.)

AHRQ uses a separate competitive process to award contracts, which fund specific activities defined by AHRQ. This process for selecting contract proposals for award is governed by the Federal Acquisition Regulation (FAR) and the Public Health Service Act (PHSA) and implementing regulations.opportunities, which occurs through different types of solicitations, varies depending on the type of contracting mechanism used. AHRQ generally uses three types of contracting mechanisms.

The advertising of available contracting

Stand-alone contracts. This contracting vehicle involves the issuance of new, stand-alone contracts. Proposals are submitted in response to publicly-available solicitations referred to as requests for proposals. A request for proposals details the specific tasks or ideas that an agency needs a contractor to fulfill, such as delivery of a certain service or research of a clearly defined topic.

Task orders. This contracting vehicle involves issuing task orders under an existing, master contract, thereby giving a contractor a new task to perform. Proposals are submitted in response to solicitations called requests for task orders, which are issued to contractors already awarded contracts by the agency.

General Services Administration (GSA)-schedule task orders.

This contracting vehicle involves the use of contracts that have been awarded by GSA for governmentwide use. GSA-schedule task orders are issued under existing master contracts awarded by GSA. These task orders are solicited through a request for quote solicitation that is competed among these contractors.

Upon receiving proposals in response to a solicitation, AHRQ typically evaluates contract proposals for award using standard contracting procedures and criteria, which are governed by the FAR and the PHSA and implementing regulations. A technical review panel of agency officials and external experts evaluates each proposal submitted in response to a solicitation against standard criteria that are tailored to the specific needs of each solicitation. These criteria include (1) demonstrated knowledge and understanding of the contract requirements; (2) the proposed approach to address tasks and subtasks listed in the proposal; (3) the qualifications and experience of key management personnel, such as the project director and project manager; (4) the potential contractor’s ability to meet the project’s milestones; (5) the facility, equipment, and space available to support the project goals and objectives; and (6) the past performance of the potential contractor using information from references or other government customers. Based on this evaluation of each proposal’s scientific and technical merit and cost, the review panel, along identifies with the contracting officer’s technical representative (COTR),the entity they believe should be awarded the contract and forwards the recommendation to the contracting officer, the federal official who has authority to enter into a contract. The contracting officer reviews the recommendation and makes a final award decision.

AHRQ Used Standard Competitive Processes and Criteria and Coordinated within HHS to Make Recovery Act Awards

AHRQ used its standard, competitive review process and criteria to select grant recipients and award 110 Recovery Act-funded CER grants, totaling approximately $311 million. In addition, AHRQ primarily used its standard contracting processes and criteria to select contract proposals and enter into 34 contracts for CER using Recovery Act funding, totaling approximately $161 million. AHRQ also took several steps to coordinate with other HHS agencies when soliciting and awarding CER grants and contracts.

AHRQ Used Its Standard Competitive Review Process and Criteria to Select Grant Recipients and Award 110 CER Grants

Between February 2009 and September 2010, AHRQ used its standard, competitive grant review process to select grant recipients and ultimately award 110 CER grants using approximately $311 million in Recovery Act CER funds. Specifically, AHRQ used its standard process for selecting grantees, which includes an initial review and preliminary scoring of applications; review and final scoring of applications at peer review panel meetings; development of preliminary funding recommendations; review of funding recommendations by a senior leadership team within AHRQ; and a final determination of funding by the agency director.

As part of the standard process AHRQ used to select the recipients of Recovery Act-funded CER grants, peer reviewers used the agency’s standard core criteria to score and subsequently rank the applications for this funding. In addition to its core criteria, AHRQ also used other criteria in its process of selecting recipients of Recovery Act CER grants. These other criteria were specific to each FOA and often varied depending on the CER study requested under that announcement. These other criteria may be used to assess, for example, a grant application in terms of the adequacy of the protection afforded human subjects; the inclusion of the extent to which privacy and certain priority populations in the study;security issues have been addressed; the partnerships that the applicant has with the proposed population; and the degree of responsiveness in addressing the purpose and objective of the FOA. AHRQ officials told us that the agency’s peer reviewers, program officials, and members of the senior leadership team used both standard core criteria and other criteria outlined in the FOAs to determine which grant applications should be recommended to the Director of AHRQ for funding.

AHRQ issued 14 FOAs for Recovery Act CER grant opportunities. Using its grant review process and criteria, AHRQ received 536 grant applications and awarded 110 CER grants between February 2009 and September 2010, totaling approximately $311 million. $311 million in grants AHRQ awarded supported HHS’s departmentwide and AHRQ’s agency-specific CER priorities. (See fig. 2 and app. II for more information on the award of AHRQ’s CER grants with Recovery Act funds.)

The 14 FOAs issued and 110 CER grants awarded addressed projects in three of the four HHS-departmentwide CER priority areas and three of AHRQ’s seven agency-specific CER priority areas. Some priority areas were supported by more than one FOA. AHRQ agency-specific and HHS departmentwide priority areas not supported with grants were supported through contracts.

AHRQ made these awards by September 30, 2010, the end of the period in which the Recovery Act funds were available for obligation. See GAO-11-712R. AHRQ awarded an additional $161 million through contracts and spent approximately $2 million for administrative purposes.

AHRQ supported the remaining four CER priority areas through contracts.

AHRQ supported the one remaining HHS departmentwide CER priority area through contracts.

Percentages do not add up to 100 percent due to rounding.

For 55 of the 110 grants, the Director of AHRQ exercised her discretion to make out-of-order funding decisions. An out-of-order funding decision occurs when grant applications are funded out of rank order; that is, they are not funded in accordance with the rank order of most-meritorious to least-meritorious overall impact scores calculated during the peer review process. Out-of-order funding decisions can be made for a number of reasons, including the need to address important agency research priorities, avoid duplication, or meet specific requirements in the original FOA that the more meritorious applicants cannot meet. Recommendations for out-of-order funding decisions are made by the program official or senior leadership team, but the final decision to award grants is made by the Director of AHRQ.

AHRQ Primarily Used Its Standard Competitive Contracting Process and Criteria to Award 34 CER Contracts

Between February 2009 and September 2010, AHRQ primarily used its standard competitive contracting process and criteria to select contract proposals and enter into 34 contracts using approximately $161 million in Recovery Act CER funds. According to AHRQ officials, a review panel, composed of external experts and AHRQ staff, evaluated all Recovery Act CER contract proposals using the standard criteria that are tailored to the specific needs of each contract solicitation. These criteria include evaluating each proposal’s technical approach, management plan, and key personnel. AHRQ officials reported that a contracting officer used the results of the panel’s evaluation to make a final selection of contractors that presented the best value to meet the needs of work specified in each Recovery Act CER solicitation.

To meet the September 30, 2010, deadline established by the Recovery Act, AHRQ made one change to its standard contracting process. Specifically, for 1 of the 13 contract solicitations AHRQ conducted an initial review of the contract proposals it received in order to determine whether these contract proposals were duplicative or responsive to the solicitation’s requirements.proposals in response to this solicitation. Agency officials explained that because they received a large number of proposals in response to this solicitation, they decided to conduct the initial review to identify which of the received proposals would continue through the agency’s standard, competitive contract review process. Four of the 23 task order contract proposals were found to be nonresponsive or duplicative of another previously funded study and, therefore, were not considered for further review.

The agency received 23 task order contract AHRQ awarded task orders under existing AHRQ master contracts, task orders under existing GSA master contracts, and stand-alone contracts using Recovery Act CER funds. The agency primarily awarded task orders under existing master contracts when awarding contracts with Recovery Act CER funds. Specifically, of the 34 contracts that AHRQ awarded using Recovery Act CER funds, 30 were task orders under either existing AHRQ contracts or existing GSA-schedule contracts, and the remaining 4 were stand-alone contracts. AHRQ awarded multiple task orders within the Evidence Synthesis and Gap Identification priority areas under existing contracts that AHRQ entered into prior to the passage of the Recovery Act. Officials stated that issuing task orders under existing master contracts, which included a GSA-schedule task order, facilitated the quick and efficient award of Recovery Act funds in instances where the agency or GSA had existing master contracts with entities capable of conducting work the agency wanted to support with Recovery Act CER funds. AHRQ officials stated that this approach was faster and more cost-effective than entering into new, stand-alone contracts. AHRQ officials also said that they used stand-alone contracts only in instances where there were existing contracts with entities that could perform the planned work.

AHRQ issued 13 CER contract solicitations between February 2009 and September 2010. Using its standard contracting process and criteria, the agency received 80 contract proposals and entered into 34 contracts totaling almost $161 million. departmentwide and AHRQ’s agency-specific CER priorities. (See fig. 3 and app. III for more information on the award of AHRQ’s CER contracts with Recovery Act funds.)

The 13 contract solicitations issued and 34 CER contracts awarded addressed projects in AHRQ and HHS-departmentwide CER priority areas. Some priority areas were supported by more than one contract solicitation. AHRQ agency-specific and HHS departmentwide priority areas not supported with contracts were supported through grants.

AHRQ made these awards by September 30, 2010, the end of the period in which the Recovery Act funds were available for obligation. See GAO-11-712R. AHRQ awarded an additional $311 million through grants and spent approximately $2 million for administrative purposes.

Percentages do not add up to 100 percent due to rounding.

AHRQ Also Reported Taking Steps to Coordinate Recovery Act CER Awards with Other HHS Agencies to Avoid Unnecessary Duplication

AHRQ officials reported that they used five mechanisms in order to coordinate with other HHS agencies to avoid unnecessary duplication when creating FOAs for grants and solicitations for contracts and when awarding Recovery Act CER funds. Specifically, AHRQ participated in a federal interagency coordination council and an HHS working group; contributed to an HHS spending plan to coordinate the department’s solicitations; participated with NIH in another working group to coordinate both solicitations and CER awards; and queried HHS databases when awarding Recovery Act funds to identify potentially duplicative projects.

Federal Coordinating Council for CER—AHRQ participated on the Federal Coordinating Council for CER (“the Council”), a body created by the Recovery Act to foster coordination for CER across the federal government in an effort to reduce duplication and encourage the coordinated and complementary use of resources. In addition to AHRQ, officials from the Veterans Health Administration (VHA), the Department of Defense, and NIH also served on the Council. According to AHRQ officials, the Council provided a mechanism for coordinating, among other things, the establishment of CER priorities and some Recovery Act CER grant announcements and contract solicitations. The Council was terminated by PPACA in March 2010.

CER Coordination and Implementation Team (CER-CIT)—In addition to the Council, AHRQ officials participated in the CER-CIT, a departmentwide effort to coordinate investments in CER supported with Recovery Act funds. Organized by HHS, the CER-CIT served as a centralized forum for HHS officials to assess FOAs for grants and solicitations for contracts. AHRQ officials stated that the CER-CIT’s process helped ensure that the FOAs and solicitations ultimately posted by AHRQ for grants and contracts were not duplicative of FOAs and solicitations posted by other entities within HHS. For example, during the CER-CIT’s review of two proposed AHRQ CER FOAs, reviewers identified aspects of the proposed announcements that were potentially duplicative of other proposed or existing projects.

HHS Intra-Agency Spending Plan—AHRQ contributed to an intra- agency spending plan developed by HHS that describes how all HHS agencies anticipated using the funding they received under the Recovery Act for CER. AHRQ contributed to this intra-agency spending plan by developing an agency-specific spending plan that described AHRQ’s research priorities and how the agency anticipated using its $300 million in Recovery Act CER funds to support these priorities. HHS incorporated AHRQ’s spending plan into the department’s intra-agency spending plan. According to AHRQ officials, officials from the HHS Office of the Secretary, who were responsible for coordinating this effort, reviewed the spending plans it received from AHRQ and other agencies, and this helped ensure that AHRQ’s Recovery Act CER solicitations were not unnecessarily duplicative of other CER efforts within HHS.

AHRQ-NIH Working Group—In addition to the departmentwide working group that was primarily focused on coordination of FOAs and contract solicitations, AHRQ and NIH formed a working group to coordinate the award of Recovery Act funds to avoid unnecessarily funding duplicative projects.making awards, they checked with NIH through this working group to ensure that the two agencies were not funding duplicative projects. For example, during one meeting members noted where issues of duplication needed to be further discussed to ensure that studies were complementary and not duplicative. In addition to reviewing awards, AHRQ officials reported that this working group met regularly during the award of the Recovery Act CER funds to share spending plans, share solicitations, and provide updates on each agency’s respective CER activities.

AHRQ officials stated that before

Querying of HHS Databases—AHRQ officials stated that in order to avoid funding unnecessarily duplicative work with Recovery Act funds, they queried HHS databases prior to awarding any Recovery Act CER funds to ensure that other similar projects were not funded elsewhere within HHS. AHRQ officials stated that if, in the process of querying these databases, a duplicative award was identified, AHRQ would contact the appropriate HHS project officer listed in the database to discuss the award in more detail. AHRQ and NIH officials confirmed that this process resulted in the identification of grant proposals for training awards that were potentially duplicative of projects AHRQ had funded. Once identified, it was decided that NIH would not fund these potentially duplicative awards.

AHRQ Plans to Use Its Existing Mechanisms and Develop Additional Strategies to Disseminate CER Results

AHRQ plans to use a range of existing mechanisms, such as written products, training, social media tools, and its website, to disseminate results of CER funded through the Recovery Act. According to AHRQ officials, the agency will determine which specific mechanisms will be used to disseminate CER results by considering the unique characteristics of the research. In addition, AHRQ awarded four contracts using Recovery Act CER funds to develop and implement innovative approaches for disseminating CER results, including Recovery Act- funded CER.

AHRQ Plans to Use a Range of Existing Mechanisms, Such as Written Products, Training, Social Media Tools, and Its Website, to Disseminate Recovery Act CER Results

AHRQ officials stated that the agency plans to use a range of existing mechanisms to disseminate Recovery Act-funded CER results as such results become available. As of December 6, 2011, 30 Recovery Act CER projects were completed or in draft and some dissemination activities had begun. The mechanisms AHRQ plans to use to disseminate Recovery Act-funded CER include written products, training programs, social media tools, learning networks, and AHRQ’s website. The different types of written products that AHRQ develops for CER results and other research include comprehensive research reviews that summarize existing research on a CER topic; original research reports that introduce new CER results; and plain language publications that summarize the findings of research on the benefits and harms of different treatment options and which are tailored to clinicians, consumers, or policymakers. AHRQ’s training programs include web-based conferences that feature presentations by experts accompanied by instructional slides for clinicians. In addition, the agency employs social media tools to disseminate notices of CER results including electronic newsletters, audio podcasts, and Twitter. AHRQ is also drawing on an existing agency learning network for Medicaid medical directors that was created in 2005. This group convenes periodically to discuss ways to advance the health of Medicaid beneficiaries, including how evidence-based research AHRQ’s website findings can be used to improve quality of care.provides access to CER results through search tools and links to its written and social media formats. (See app. IV for a more detailed description of these existing mechanisms.)

AHRQ officials explained that the agency determines which specific mechanisms will be used to disseminate particular CER results by considering the unique characteristics of the research such as the type of research conducted, its potential impact, the strength of the evidence, and the audiences that can best make use of the information. AHRQ then develops a marketing plan that identifies key messages, target audiences, and the mechanisms to be used to disseminate CER to those audiences. For example, AHRQ’s marketing plan for a CER project that examines certain treatments for type 2 diabetes targeted consumers as well as primary care clinicians and certain specialist clinicians and other health professionals. While this CER project was not funded with Recovery Act funds, AHRQ officials confirmed that the process they use to customize the dissemination for this project is the same process the agency will follow for disseminating Recovery Act-funded CER results. To disseminate the CER results of this project to consumers, AHRQ developed a consumer guide and a series of audio podcasts. To reach clinicians, AHRQ developed a clinician’s guide and a webcast program with educational slides. These products were targeted to be distributed through multiple channels including AHRQ’s website, as well as its newsletters and list-servs. Notices about these products were also sent directly to a range of general media news services; consumer health and advocacy publications; and a wide range of key national organizations that included those representing primary care, specialty clinicians, and payers. (See app. V for a more detailed description of this dissemination effort.)

AHRQ Is Developing Additional Strategies to Disseminate CER Results

In addition to its existing mechanisms for disseminating the results of CER, AHRQ is in the process of developing additional dissemination strategies. Specifically, in September 2010, AHRQ awarded four Recovery Act-funded contracts to develop and implement innovative approaches for disseminating CER results, including Recovery Act- funded CER. The specific purpose of each of the four contracts is described below.

Academic Detailing. Academic detailing involves face-to-face educational sessions by trained clinicians, including physicians, nurses, pharmacists, and others, who visit health professionals in their practice settings. The goal of these sessions is to share evidence- based information and facilitate use of that information to improve patient care. AHRQ awarded a contract in the amount of $11,680,060 for the purpose of implementing academic detailing for CER from 2011 through 2013. The plan under this contract calls for academic detailing to 1,300 primary care providers and 200 health care system practice sites. Each provider or site will receive one face-to-face visit every 6 months plus follow-up e-mail communications and supporting materials. The academic detailing will focus on six CER topics over the 3-year period. Between February 2011 and October 2011, work completed under this contract resulted in over 1,562 visits to providers and practice sites. These visits involved the discussion of AHRQ’s CER results related to the treatment of type 2 diabetes.

Continuing Education Modules. AHRQ awarded a contract in the amount of $3,981,168 for the purpose of developing and disseminating 45 accredited online continuing education programs for health care professionals, including physicians, physician assistants, pharmacists, nurses, nurse practitioners, medical assistants, and other health professionals. These programs translate CER results into a variety of formats, for example, videos featuring case studies and journal supplements. As of November 30, 2011, 13 approved continuing education programs were completed, including programs on CER results related to hip fractures, hypertension, prostate cancer, breast cancer, heart disease, and diabetes.

Regional Dissemination and Partnership Offices. AHRQ awarded a contract in the amount of $8,613,876 to create five regional offices for the purpose of establishing partnerships to facilitate dissemination and use of CER results by regional health care organizations, businesses, unions, and consumer groups. Collaborative efforts are expected to result in local and regional meetings, web conferences, training programs, and distribution of CER results to partner organizations’ memberships.

Publicity Center. AHRQ awarded a contract in the amount of $17,999,988 to develop and implement a national strategic communications plan for AHRQ’s CER results. The communications plan calls for the development of national partnerships with consumer, clinician, policymaker, and business audiences; marketing efforts, including the use of social media, focused on disseminating results of research; and creation of new website portals with established sites reaching patients and clinicians. For example, under the contract, partnerships have been established with such organizations as the National Rural Health Association, the National Alliance for Caregiving, the American College of Cardiology, and the American Medical Student Association.

Along with its efforts to develop additional strategies for disseminating CER results, AHRQ is taking steps to evaluate the effectiveness of these strategies. Specifically, using its Recovery Act funds AHRQ awarded a contract in the amount of $2,371,179 for the purpose of evaluating some of AHRQ’s dissemination strategies by collecting data about dissemination. Under the contract, information will be collected about changes over time in the level of awareness, understanding, use, and perceived benefits of CER . This information will be gathered from clinicians, patients, consumers, health system decision makers, purchasers, and policymakers. In addition, this evaluation includes plans to collect process and outcomes data for each of the additional dissemination strategies being developed under Recovery Act-funded contracts including academic detailing, continuing education, regional dissemination, and the national publicity center.

AHRQ officials noted that evaluating the impact of dissemination of its CER results is important but also challenging. They noted that clinician practice behavior often changes slowly and is affected by many variables, thereby making it difficult to directly attribute changes to information AHRQ has disseminated. In addition, once CER results are disseminated to target audiences through, for example, AHRQ’s website or one of its educational programs, it is often not feasible to track secondary dissemination from those audiences to others.

AHRQ Has Begun to Monitor PCORI and Identify Resources That Could Enable It to Fulfill Its PPACA Responsibilities Related to PCORI

While PCORI is in the early stages of development, AHRQ has begun to monitor PCORI’s needs to determine what resources might be needed by AHRQ to fulfill its PPACA responsibilities related to PCORI and identify existing resources that the agency can use to fulfill these responsibilities. These responsibilities include broadly disseminating the research findings published by PCORI; developing a publicly available database to collect government-funded evidence and research from public, private, not-for-profit, and academic sources; promoting the timely incorporation of PCORI-generated CER findings into health information technology systems that support clinical decision making; and establishing a process for receiving feedback about the value of information disseminated by AHRQ.

AHRQ officials report that they are monitoring PCORI’s needs to determine what resources might be needed by AHRQ to fulfill its PPACA responsibilities related to PCORI. The director of AHRQ serves on PCORI’s Board of Governors and another high-level AHRQ official serves on PCORI’s methodology committee, which allows AHRQ to obtain information on the resources the institute might need and when these resources might be needed. In addition, according to AHRQ officials, the agency has shared information with PCORI members about AHRQ’s existing resources at various PCORI meetings.

AHRQ officials reported that they are also in the process of identifying existing resources, including existing capabilities and ongoing projects, that the agency can leverage to fulfill its responsibilities related to PCORI. For example, AHRQ officials are exploring whether contracts the agency awarded to evaluate AHRQ’s CER dissemination efforts could be leveraged to meet the agency’s responsibilities to obtain, on behalf of PCORI, feedback from health care professionals on the CER information disseminated by AHRQ. In addition, AHRQ is currently assessing whether a research database being developed by HHS’s Office of the Assistant Secretary for Planning and Evaluation could be used to, among other things, store and make publicly available CER funded and generated by PCORI.

In addition, AHRQ has developed spending plans for fiscal years 2011 and 2012 that describe how AHRQ will use the funds it receives from PCORTF to fulfill the agency’s responsibilities related to PCORI. These plans describe proposed FOAs and contract solicitations that would expand opportunities for AHRQ to disseminate CER information through a variety of channels to different target audiences, for example, public service announcements targeting consumers and symposia and publications targeting researchers and health care professionals. AHRQ officials stated that the fiscal year 2011 plan has been approved by OMB, and the fiscal year 2012 plan was under review by HHS as of December 2011. AHRQ officials stated that they have issued one FOA for a project described in the fiscal year 2011 spending plan but, as of December 2011, have not made any awards.

Agency Comments

We provided a draft of this report to AHRQ for review and comment. AHRQ provided technical comments, which we incorporated where appropriate.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the Secretary of HHS, interested congressional committees, and others. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staffs have questions about this report, please contact me at (202) 512-7114 or at kohnl@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VI.

Appendix I: The Agency for Healthcare Research and Quality’s (AHRQ) Mission, Research, Priorities, and Budget

AHRQ’s mission is to improve the quality, safety, efficiency, and effectiveness of health care for all Americans. The purpose of AHRQ’s research is to help people make more informed decisions and improve the quality of health care services. AHRQ, formerly known as the Agency for Health Care Policy and Research, is 1 of 12 agencies within the U.S. Department of Health and Human Services (HHS). While the National Institutes of Health focuses on biomedical research to prevent, diagnose, and treat disease and the Centers for Disease Control and Prevention focuses on population health and the role of community-based interventions to improve health, AHRQ’s research focus is on long-term and systemwide improvement of health care quality and effectiveness.

AHRQ conducts work in five broad focus areas. These areas include comparing effectiveness of treatments; quality improvement and patient safety; health information technology; prevention and care management; and health care value.

Comparing the effectiveness of treatments. AHRQ’s comparative provides patients and physicians with effectiveness research (CER)information on which medical treatments work best for a given condition. This includes comparisons of drugs, medical devices, tests, surgeries, or ways to deliver health care in an effort to help patients and their families understand which treatments work best and how their risks compare. Initiatives under this focus area include:

The John M. Eisenberg Center for Clinical Decisions and

The Centers for Education and Research on Therapeutics

The Developing Evidence to Inform Decisions about Effectiveness Quality improvement and patient safety. AHRQ funds and disseminates research that identifies root causes of threats to patient safety, provides information on the scope and impact of medical errors, and examines effective ways to make system-level changes to help prevent errors. Initiatives under this focus area include:

Patient safety culture assessment tools

Health information technology (health IT). AHRQ provides support to give access to and encourage the adoption of health IT. The agency has focused its health IT activities on the following three goals: Improve health care decision making Improve the quality and safety of medication management

Prevention and care management. AHRQ translates evidence- based knowledge into recommendations for clinical preventative services. AHRQ initiatives under this focus area include:

The U.S. Preventative Services Task Force

The Patient-Centered Medical Home

The Practice-Based Research Network

Health care value. AHRQ aims to find greater value in health care by producing the measures, data, tools, evidence, and strategies that health care organizations, systems, insurers, purchasers, and policymakers need to improve the value and affordability of health care. Initiatives under this focus area include:

The Medical Expenditure Panel Survey

The Healthcare Cost and Utilization Project

The annual National Healthcare Quality Report and National

State snapshots

The Consumer Assessment of Healthcare Providers and Systems

The National Guideline Clearinghouse

The National Quality Measures Clearinghouse In addition to the above focus areas, AHRQ also conducts crosscutting activities related to quality, effectiveness, and efficiency. Activities include data collection and measurement; dissemination and translation; and program evaluation. In addition, support is provided for the investigator- initiated and targeted research grants and contracts that focus on health services research in the areas of quality, effectiveness, and efficiency. These activities provide the core infrastructure used by the other focus areas.

AHRQ staff and budget. AHRQ currently employs approximately 300 staff. The agency’s fiscal year 2010 budget was $402.6 million, of which $270.7 million went to research on health costs, quality, and outcomes. The President’s fiscal year 2012 budget request for AHRQ was $366.4 million, a decrease of approximately $31 million from fiscal year 2010. (See table 4 for the funding amounts under AHRQ’s focus areas.)

Appendix II: AHRQ CER Grants Awarded Using Recovery Act CER Funds, by Priority Area

AHRQ and the HHS Office of the Secretary jointly funded one grant. For purposes of our report, we counted this grant under the number of grants awarded for the HHS Office of the Secretary’s Dissemination and Translation priority area (see table 7). However, the amount of Recovery Act CER funds the HHS Office of the Secretary and AHRQ awarded to this grant are reflected under the amount awarded for the HHS Office of the Secretary’s Dissemination and Translation priority area and AHRQ’s Evidence Generation priority area.

AHRQ and the HHS Office of the Secretary jointly funded one grant. For purposes of our report, we counted this grant under the number of grants awarded for the HHS Office of the Secretary’s Dissemination and Translation priority area. However, the amount of Recovery Act CER funds the HHS Office of the Secretary and AHRQ awarded to this grant are reflected under the amount awarded for the HHS Office of the Secretary’s Dissemination and Translation priority area and AHRQ’s Evidence Generation priority area.

Appendix III: AHRQ CER Contracts Awarded Using Recovery Act CER Funds, by Priority Area

AHRQ combined Evidence Synthesis and Gap Identification awards under a single solicitation when announcing the availability of these funds and within awards because, according to agency officials, having a single solicitation for these two priority areas reduced the amount of work related to these awards, thereby expediting the award process. As a result, AHRQ funded both of these priority areas, but advertised projects and made awards for these priority areas under a single solicitation.

Appendix IV: AHRQ Mechanisms That Support Dissemination of CER

Types of mechanisms supporting dissemination Written products

Examples of AHRQ’s dissemination mechanisms

Research reviews and original research reports: These written products draw on completed scientific studies to make comparisons of different health care interventions or summarize original clinical research to explore practical questions about the effectiveness of treatments. Summary treatment guides for clinicians, consumers, policymakers: Short, plain- language guides summarize the findings of research reviews on the benefits and harms of different treatment options. Education modules and presentation slides: These resources are for clinicians pursuing continuing education credits and for faculty who are instructing clinicians.

Webcasts: Researchers and clinicians participate in online programs to discuss research findings.

Conference series: Scientific meetings on state-of-the art concepts in communication, health literacy, and medical decision making. Audio podcasts: The Healthcare 411 audio podcast series shares news and information with consumers that they can use in health care decision making, through 60-second audio news programs and longer format interviews.

Online videos: AHRQHealthTV provides videos for consumers about a range of health topics on AHRQ’s YouTube channel. Twitter updates: Short messages are broadcast that can be accessed by computer or mobile phone.

RSS Feeds: Subscribers receive news and alerts about AHRQ programs through their RSS reader. E-mail updates: Subscribers receive e-mail updates on topics they are interested in. The Medicaid Medical Directors Learning Network: This is one example of a network formed by AHRQ to create an ongoing collaborative relationship to disseminate AHRQ products, tools, and research to help members make policy and practice decisions related to clinical treatment. Impact Case Studies: AHRQ tracks and summarizes how AHRQ-funded research, tools, and products are actually implemented by state governments, medical practices, clinics, and hospitals. Makes summary information available to other potential users. AHRQ’s website ahrq.gov provides access to its written products, training programs, and social media tools, as well as useful search functions and other resources.

Health Care Innovations Exchange: The Exchange offers health professionals and researchers searchable tools to access information about evidence-based innovations suitable for a range of health care settings and populations, as well as opportunities to network with other professionals who have implemented these innovations.

Appendix V: Example of Dissemination of Comparative Effectiveness Research by the Agency for Healthcare Research and Quality

Appendix V: Example of Dissemination of Comparative Effectiveness Research by the Agency for Healthcare Research and Quality Comparative Effectiveness and Safety of Premixed Insulin Analogues in Type 2 Diabetes: A Systematic Review Type 2 diabetes is an increasingly common chronic disease that occurs in people who have too much glucose in their blood. Blood glucose levels are high either because their cells are resistant to insulin (a hormone that helps convert glucose into energy) or because their pancreas does not produce enough insulin. Insulin analogues are used approximately as commonly as human insulin by diabetics who require insulin to regulate blood glucose levels. Created by genetically modifying human proteins, insulin analogues were developed as an alternative to human insulin to provide tighter control of blood sugar levels.

This study summarizes the effectiveness of insulin analogues against traditional human insulin for type 2 diabetics. Researchers compared the effectiveness of three kinds of synthetic insulin against their human insulin counterparts, against each other, and against other antidiabetic medications. The report found that insulin analogues are more effective than human insulin for treating certain diabetes-related symptoms such as high blood sugar after meals. However, it also found that human insulin appears to be more effective than insulin analogues in treating other aspects of diabetes, including lowering blood sugar levels when patients go 8 hours or more without eating, typically overnight.

Consumer guide (for adults)

Clinician’s guide

Webcast and slides (for clinicians)

Audio podcast (for consumers)

Over 90 target organizations, publications, and electronic venues are identified in the marketing plan in the following categories: clinicians, insurers, payers, pharmacy and drugs associations, federal direct and funded medical care programs, consumer-oriented disease organizations, and government. Provider categories targeted include retail and health system pharmacists; family physicians and general internists; pharmacologists; nurse practitioners; physician assistants; and endocrinologists.

General media news services including radio, television news, and major daily newspapers; consumer and advocacy publications; African-American media; and translation to Spanish-only and Hispanic media.

Appendix VI: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, E. Anne Laffoon, Assistant Director; Shaunessye Curry; Mary Giffin; Andrea E. Richardson; Lisa Motley; Krister Friday; and Jessica C. Smith made key contributions to this report.