VA Health Care: VA Spends Millions on Post-Traumatic Stress Disorder Research and Incorporates Research Outcomes into Guidelines and Policy for Post-Traumatic Stress Disorder Services
Published: 2011-01-24T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In addition to providing health care to veterans, the Department of Veterans Affairs (VA) funds research that focuses on health conditions veterans may experience. According to VA, experts estimate that up to 20 percent of Operation Enduring Freedom and Operation Iraqi Freedom veterans have experienced post-traumatic stress disorder (PTSD) and demand for PTSD treatment is increasing. Because of the importance of research in improving the services that veterans receive, GAO was asked to report on VA's funding of PTSD research, and its processes for funding PTSD research proposals, reviewing and incorporating research outcomes into clinical practice guidelines (CPG)--tools that offer clinicians recommendations for clinical services but do not require clinicians to provide one service over another--and determining which PTSD services are required to be made available at VA facilities. To do this work, GAO obtained and summarized VA data on the funding of PTSD research from its medical and prosthetic research appropriation through its intramural research program. GAO also reviewed relevant VA documents, such as those for developing CPGs and those related to VA's 2008 Uniform Mental Health Services in VA Medical Centers and Clinics handbook (Handbook), which defines certain mental health services that must be made available at VA facilities. GAO also interviewed VA officials.
section_title: What GAO Found
paragraphs: Based on VA data GAO obtained and summarized, GAO found that the amount of funding VA provided for intramural PTSD research increased from $9.9 million in fiscal year 2005 to $24.5 million in fiscal year 2009. From fiscal year 2005 through fiscal year 2009, intramural PTSD research funding ranged from 2.5 percent to 4.8 percent of VA's medical and prosthetic research appropriation. In addition, the number of PTSD research studies VA funded through the Merit Review Program and the Cooperative Studies Program (CSP)--VA's two primary funding mechanisms in its intramural research program--increased from 47 in fiscal year 2005 to 96 in fiscal year 2009. According to VA officials, intramural research proposals, including those on PTSD, are funded primarily according to scientific merit in both the Merit Review Program and CSP. Proposals are evaluated by a panel of reviewers and scored based on their scientific merit. Directors of VA's research and development services--offices that focus on different research areas and administer VA's intramural research program--fund proposals based on their scores, typically up to a specified percentile. The number of proposals funded may vary based on budgetary considerations and, for a small number of proposals, responsiveness to VA research priority areas. VA has a process to review and incorporate relevant research outcomes to develop CPGs for a number of topics, including PTSD. VA relies on the policies of a joint VA and Department of Defense (DOD) work group--comprised of VA and DOD officials--to ensure that systematic reviews of relevant research outcomes are conducted when issuing CPGs. In brief, a systematic review is conducted to identify the most methodologically rigorous research studies that are applicable to each clinical question contained in the CPG. A group of subject matter experts then assesses the individual research studies in order to determine the overall quality of evidence available for each particular clinical question, considers the potential benefits and harms of a clinical intervention to determine its net effect, and, based on an assessment of the overall quality of the evidence and the net effect of an intervention, develops recommendations for the CPG. According to VA officials, the decision to require that two PTSD services--cognitive processing therapy and prolonged exposure therapy--be made available at VA facilities by including them in the Handbook was based on a review of research outcomes and the availability of existing resources. Specifically, VA officials told GAO that these two services were strongly recommended in the 2004 PTSD CPG and had greater evidence supporting their effectiveness than other PTSD services. VA also told GAO that prior to the Handbook's 2008 issuance, VA had already begun investing resources in training programs for cognitive processing therapy in 2006 and prolonged exposure therapy in 2007. While VA provided some documentation regarding the decision-making process for PTSD services, VA officials explained that clinical decision-making processes are not typically expected to be documented in a formal manner. VA officials told GAO that they are currently clarifying language in the Handbook but do not plan to revise any requirements relating to PTSD services at this time. VA provided technical comments that GAO incorporated as appropriate.

section_title: Letter
paragraphs: In addition to providing health care to about 5 million veterans each year,  the Department of Veterans Affairs (VA) also funds research that focuses  on the specific health conditions that veterans may experience. One  condition that is examined in VA-funded research is post-traumatic stress  disorder (PTSD), an anxiety disorder that can occur after a person is  exposed to a life-threatening event. Veterans diagnosed with PTSD may  experience problems sleeping, maintaining relationships, and returning to  their previous civilian lives. According to VA, experts estimate that up to  20 percent of Operation Enduring Freedom and Operation Iraqi Freedom  veterans, up to 10 percent of Gulf War veterans, and up to 30 percent of  Vietnam War veterans have experienced PTSD. Consequently, demand for  PTSD treatment continues to grow. VA data show that from fiscal year  2004 through fiscal year 2008, the number of unique veterans receiving  treatment for PTSD increased by 60 percent from over 274,000 to over  442,000. In particular, the number of Operation Iraqi Freedom and  Operation Enduring Freedom veterans who received VA treatment for  PTSD increased from nearly 4,400 in fiscal year 2004 to over 69,000 in  fiscal year 2008.
Veterans diagnosed with PTSD can receive a range of treatments to  manage their symptoms, including individual and group therapy and  medication. Veterans can receive PTSD treatment on an outpatient basis at  VA facilities such as medical centers, community-based outpatient clinics,  and Vet Centers. Veterans can also receive intensive treatment through  VA medical center inpatient settings for acute care needs and through  residential rehabilitation treatment programs for more prolonged  rehabilitative care.
VA has guidance and policies in place related to PTSD services. For  example, in 2004, the joint VA/Department of Defense (DOD) Evidence- Based Practice Work Group issued a CPG for PTSD. The PTSD CPG is an  educational tool for clinicians that provides evidence-based  recommendations for PTSD services based on a review of PTSD research  outcomes. It does not require clinicians to provide one service over  another. However, in a handbook issued in 2008—Uniform Mental Health  Services in VA Medical Centers and Clinics (Handbook)—VA for the first  time required certain mental health services to be made available to  veterans throughout the system. For PTSD, the Handbook requires that  cognitive processing therapy or prolonged exposure therapy, two  evidence-based psychotherapies, be provided or made available at VA  facilities.
Through its intramural research program, VA funds studies on different  topics, such as PTSD. VA intramural studies are conducted by VA  investigators—that is, researchers who conduct VA research studies. VA  intramural research refers to research that is funded by and conducted  within VA. In fiscal year 2009, VA received an appropriation of $510 million  for medical and prosthetic research. This appropriation funds VA’s  intramural research program, including providing funding for necessary  equipment and supplies. VA also receives three medical care  appropriations that support VA’s intramural research by paying some of  the costs associated with this research, for example, the salaries of VA  investigators who are also VA clinicians. According to VA, in fiscal year  2009, VA’s medical care appropriations totaled about $41 billion and VA  provided $433 million of that amount to support all research conducted at  VA facilities.
Because of the growing demand for PTSD services and the importance of  research in improving the health care services that veterans receive, you  asked us to report on VA’s funding of PTSD research, VA’s processes for  reviewing PTSD research proposals, VA’s incorporation of research  outcomes into clinical practice, and VA’s process for determining which  PTSD services it requires VA facilities to provide or make available. This  report will describe    how much funding VA provided for intramural PTSD research from its  medical and prosthetic research appropriations from fiscal year 2005  through fiscal year 2009,    how VA determines which intramural PTSD research studies will be   funded,    how VA reviews PTSD research outcomes and incorporates them into   its PTSD CPG, and    how VA determines which PTSD services it requires VA facilities to   provide or make available.
To describe how much funding VA provided for PTSD research from its  medical and prosthetic research appropriations through its intramural  research program from fiscal year 2005 through fiscal year 2009, we  obtained and summarized data provided by VA’s Office of Research and  Development (ORD), the office that manages the Merit Review Program  and the Cooperative Studies Program (CSP)—the two primary funding  mechanisms within VA’s intramural research program. VA officials define  PTSD research as “the effort to acquire generalizable knowledge about  causes, epidemiology, susceptibility and genetics, resilience,  pathophysiology, prevention and treatment of post-traumatic stress  disorder.” Based on these data, for fiscal years 2005 through 2009, we  calculated totals and trends for (1) overall intramural PTSD research  funding—including PTSD research studies and other PTSD research- related funding, such as career development awards provided to junior VA  investigators to conduct PTSD studies, salaries for VA investigators who  are not VA clinicians, funding for PTSD research conducted within ORD  research centers, and PTSD research meetings, and (2) the number and  type of intramural PTSD research studies funded. According to VA, the  funding data we obtained from VA do not include funds provided to  conduct or support intramural PTSD research from VA’s medical care  appropriations. Because of this, they do not necessarily represent all the  funding VA provided for PTSD research during this time. In addition, to  obtain information about how VA funds research conducted at VA, we  interviewed officials from VA’s Veterans Health Administration’s Office of  Finance, ORD, and Veterans Integrated Service Networks. We also  reviewed VA documents regarding its total medical and prosthetic  research appropriations. We tested the internal consistency and reliability  of the PTSD research data by reviewing the data for obvious outliers and  performing consistency checks. We found the data to be sufficiently  reliable for the purposes of this report.
To describe how VA determines which intramural PTSD research studies  will be funded, we interviewed VA officials responsible for managing VA’s  intramural research program. We also obtained and analyzed VA  documents that describe VA’s policies for submitting, reviewing, and  funding intramural research proposals. In addition, to understand the  process VA uses to evaluate intramural research proposals, we observed a  meeting in which research proposals were evaluated.
To describe how VA reviews PTSD research outcomes and incorporates  them into its PTSD CPG to inform clinical practice at VA, we interviewed  VA officials who are responsible for developing CPGs. We also obtained  and reviewed the PTSD CPG and documents related to CPG development.
To describe how VA determines which PTSD services it requires VA  facilities to provide or make available to veterans, we interviewed VA  officials responsible for determining these requirements. We also obtained  and reviewed documents that describe the PTSD service requirements.
We conducted this performance audit from May 2010 through November  2010 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: VA manages its intramural research program through ORD. According to  ORD’s 2009 to 2014 strategic plan, ORD has 10 research priority areas,  which are topics of research that are considered important to VA. The  research priority areas are the health care needs of veterans who have  served in Operation Enduring Freedom and Operation Iraqi Freedom,  aging-related conditions, mental health care and well-being, chronic  diseases, long-term care and caregiving, deployment-related exposure to  hazardous environmental agents, equity in care, access in rural areas,  women’s health, and personalized medicine. According to VA officials, all  of these research priority areas could include PTSD research.
VA funds intramural research through the following:    VA’s Merit Review Program: This program supports research studies  typically conducted by one VA investigator at one VA facility and is  administered by ORD’s four research and development services, each of  which has a different research focus. (See table 1.) Each research and  development service is responsible for soliciting, reviewing, selecting, and  funding research proposals submitted to the service.
VA’s CSP: This program, which is administered by Clinical Science, funds  larger-scale, multisite clinical trials and epidemiological research studies  on key diseases that impact veterans.
The Merit Review Program has research award funding limits, which are  set by VA. In some cases, intramural research awards may only be funded  for a certain number of years. See table 2 for more information.
In addition to individual studies conducted at VA facilities, VA has several  research centers and programs that conduct or support PTSD research.  For example, the National Center for PTSD focuses on PTSD research. VA  also has Research Enhancement Award Programs, which help support  PTSD research by providing staff and other resources to investigators.  (For more information on VA research centers and research programs that  conduct or support PTSD research, see app. I.) According to a VA official  from the National Center for PTSD, VA does not fund most of the PTSD  research that is being conducted today.
subsections: section_title: Origin and Criteria for Intramural Research Proposals
paragraphs: Intramural research proposals may be service-directed—solicited by ORD  on specific topics—or investigator-initiated— submitted by investigators  to ORD on their own initiative. Investigators submit proposals either in  response to a request for proposals on a specific topic (for service- directed proposals) or to an open request for proposals (for investigator- initiated proposals). For both the Merit Review Program and CSP,  proposals are typically evaluated in two review cycles per year.
To be considered for intramural research funding:    The proposal must be veteran-centric.
The proposal must have received approval from the director of the  medical center and the research and development office of the medical  center where the lead investigator, known as a principal investigator, is  based.
The principal investigator and any coprincipal investigators must  demonstrate a primary professional commitment to VA, as demonstrated  by at least a 5/8 time VA appointment at the time the funding is awarded  and previous VA experience, including experience in research and patient  care.
Research must be conducted primarily on VA premises. The principal  investigator and any coprincipal investigators must have designated  research space within a VA medical center.
section_title: PTSD Research Funding Increased from Fiscal Year 2005 through Fiscal Year 2009
paragraphs: Overall intramural PTSD research funding from VA’s medical and  prosthetic research appropriation increased from $9.9 million in fiscal year  2005 to $24.5 million in fiscal year 2009. The number of intramural PTSD  research studies funded through the Merit Review Program and CSP  increased from 47 in fiscal year 2005 to 96 in fiscal year 2009.
subsections: section_title: VA Intramural PTSD Research Funding Increased from $9.9 Million in Fiscal Year 2005 to $24.5 Million in Fiscal Year 2009
paragraphs: Based on the VA data we obtained and summarized, we found that overall  intramural PTSD research funding from VA’s medical and prosthetic  research appropriation increased from about $9.9 million in fiscal year  2005 to about $24.5 million in fiscal year 2009, or by about 150 percent (see  fig. 1). Overall intramural PTSD research funding included funding for  specific PTSD studies as well as for other PTSD research-related funding,  such as career development awards provided to junior VA investigators to  conduct PTSD studies, salaries for VA investigators who are not VA  clinicians, funding for PTSD research conducted within ORD research  centers, and PTSD research meetings.
Of the $80.2 million provided for PTSD studies from fiscal year 2005  through fiscal year 2009, $51.3 million, or about 64 percent, was for studies  funded through the Merit Review Program. The remaining approximately  $28.9 million, or about 36 percent, was for CSP studies. (See fig. 2.)
From fiscal year 2005 through fiscal year 2009, intramural PTSD research  funding ranged from 2.5 percent to 4.8 percent of VA’s medical and  prosthetic research appropriation. (See table 3 for VA intramural PTSD  research funding and VA’s medical and prosthetic research appropriations  from fiscal year 2005 through fiscal year 2009.) For comparison,  according to a 2009 report prepared by ORD staff for VA’s National  Research Advisory Council, for fiscal year 2009, funding for intramural  traumatic brain injury research was about $14.6 million, 2.9 percent of the  medical and prosthetic research appropriation. Funding for spinal cord  injury research was $27.2 million, 5.3 percent of the medical and  prosthetic research appropriation. Funding for intramural cardiovascular  disease and stroke research was $53.1 million, 10.4 percent of the medical  and prosthetic research appropriation.
section_title: The Number of PTSD Studies Funded through VA’s Intramural Research Program Increased from 47 in Fiscal Year 2005 to 96 in Fiscal Year 2009
paragraphs: Similarly, we found that the number of PTSD studies funded from VA’s  medical and prosthetic research appropriations through VA’s intramural  research program increased from fiscal year 2005 through fiscal year 2009.  (See fig. 3.) Specifically, in fiscal year 2005, 47 intramural PTSD research  studies were funded while in fiscal year 2009, 96 intramural PTSD research  studies were funded. This represented an increase of more than   100 percent. Of all the studies funded each fiscal year, only a small number  were CSP studies.
section_title: VA Intramural PTSD Research Studies Are Funded Primarily According to Scientific Merit
paragraphs: According to VA officials, intramural research proposals, including those  on PTSD, are reviewed and funded in VA’s Merit Review Program and VA’s  CSP primarily according to scientific merit.
VA’s Merit Review Program  Intramural research proposals submitted to VA’s Merit Review Program  are reviewed through a series of steps prior to funding. See figure 4 for an  overview of the submission, review, and funding process for proposals  submitted to the Merit Review Program. (For more detailed information  on this process, see app. II.)
First, investigators submit proposals electronically. Investigators typically  submit Merit Review Program proposals to grants.gov, the government’s  central grant identification and proposal portal, in response to a request  for proposals. Submitted proposals are then transferred to eRA Commons,  an electronic system for grant administration functions, for VA processing  and review.
Second, each proposal is assigned to a merit review panel for evaluation.  Each merit review panel reviews proposals in a specific research topic  area, and is composed of panelists, typically associate-level professors,  who are selected based on their expertise in this area. According to VA  documents, as of 2010, there were a total of 35 merit review panels across  VA’s research and development services. The merit review panels  evaluate each proposal based on its scientific merit. Panelists consider  several criteria in evaluating the overall scientific merit of a proposal. (See  table 4 for criteria used to determine scientific merit.)
Third, the merit review panelists score the proposals to determine their  rank. Each panelist provides a score to each of the proposals reviewed by  the panel. The scores are averaged to create a “priority score” for the  proposal. (See app. II for specific scoring guidelines given to panelists in  all research and development services.) All proposals scored by the merit  review panel are then ranked by priority score among all of the proposal  scores recently assigned by the merit review panel. The rank of the  proposal is used to determine the “percentile” of the proposal.
Finally, research and development service directors determine how many  proposals to fund. All of the proposals scored by all merit review panels in  a research and development service in the review cycle are ranked  together by their percentiles to be considered for funding. According to VA  officials, research and development service directors typically fund up to  the 25th percentile of proposals in a review cycle, beginning with those  with the most scientific merit, although the number of proposals funded  may vary depending on the budget. According to VA, research and  development service directors may also choose to fund a small number of  additional proposals at the margin that respond to research priority areas.  For example, if the fundable range determined by a research and  development service director was up to the 25th percentile, proposals at  the 26th percentile related to research priority areas could also be  considered for funding.
VA’s Cooperative Studies Program  VA intramural research proposals submitted to CSP are reviewed and  scored in a process similar to that of the Merit Review Program prior to  consideration for funding. To help develop the CSP proposal, investigators  are assisted by members of a CSP center, a VA entity that provides  guidance and support for research across multiple sites. (See fig. 5 for an  overview of the process for submitting, reviewing, and funding a CSP  proposal. For more information on ORD’s CSP review process, see   app. III.)
Before submitting a research proposal, investigators submit a letter of  intent, or a preliminary outline of a proposal, to the Director of Clinical  Science to be approved for planning a CSP proposal. Based on the merit  of the letter of intent, as determined by three or more reviewers, the  Clinical Science Director decides whether to fund planning efforts to  develop a CSP proposal.
When the principal investigator receives approval to begin planning  efforts, the Clinical Science Director assigns a CSP center to provide  statistical and methodological guidance to the investigator. The director of  the CSP center designates a project manager and methodologist, such as a  person with expertise in biostatistics, to provide guidance to the principal  investigator. The Clinical Science Director, with recommendations from  the principal investigator, then forms a planning committee of additional  experts to assist in developing a CSP proposal. The planning committee  develops a CSP proposal over the course of two planning meetings.
Once a proposal is developed, the CSP center, on behalf of the principal  investigator, submits a hard copy proposal to the CSP central office for  evaluation by the Cooperative Studies Scientific Merit Review Board.  This board consists of reviewers who have extensive experience in clinical  research and the conduct of clinical trials or epidemiology studies.  Reviewers evaluate CSP proposals based on scientific merit. According to  VA, the scientific merit of a CSP proposal is defined by the importance of  the proposal, its feasibility, the clarity and achievability of its objectives,  the adequacy of the plan of investigation, the correctness of the technical  details, and the adequacy of the safeguards for the welfare of the  patients. Based on these criteria, reviewers discuss the general scientific  merit of the proposal. Reviewers vote on whether to unconditionally  approve, conditionally approve, reject or defer with recommendation for  resubmittal, or reject each proposal. (See table 5 for an overview of  funding recommendations provided by the board.)
After the reviewers vote, they each provide scores for a proposal  recommended for funding based on scientific merit. The scores are then  averaged to provide a priority score for a proposal. Finally, the Clinical  Science Director considers the priority scores of all the proposals in that  review cycle and selects the proposals with the strongest priority scores  for funding. According to VA officials, the number of proposals funded  may vary depending on the budget.
section_title: VA Has Established a Standardized Process to Review All Relevant PTSD Research Outcomes and Incorporate Them into a PTSD CPG
paragraphs: The VA/DOD Evidence-Based Practice Work Group, which is responsible  for developing and updating all of VA’s CPGs, has a standardized and  reproducible process to review all relevant research outcomes when  developing or updating all CPGs, including the PTSD CPG. To develop or  update a CPG, the VA/DOD Evidence-Based Practice Work Group  identifies and assigns a group of VA and DOD clinical leaders and experts  who are knowledgeable in the subject area to work on the CPG.  Generally, the process to develop or update a CPG consists of the  following steps.
First, the assigned group of VA and DOD clinical leaders and experts  identifies “clinical questions” that will be answered in the CPG. According  to VA officials, clinical questions can be either broad or specific. For  example, the 2004 PTSD CPG contained clinical questions regarding  whether early intervention is more effective than later intervention, and  whether certain interventions, such as different psychotherapies, are more  effective than others.
Second, in order to minimize bias, an external contractor conducts a  systematic review of relevant research and selects and summarizes the  most methodologically rigorous research studies that are applicable to  each of the clinical questions.
Third, after receiving summaries of the studies with the highest level of  evidence, the VA and DOD group of clinical leaders and experts rates the  research using an established grading scheme that considers the level of evidence of each research study—the scope and  methodological rigor of an individual study;  the overall quality of evidence—the overall quality of all of the  research that addresses a particular clinical question, considering the  level of evidence of all the studies considered; and  the net effect of an intervention—according to the collective results of  the studies considered, the intervention’s benefits minus the  intervention’s harms.
Finally, the assigned group of VA and DOD clinical leaders and experts  assigns a grade to each evidence-based recommendation based on an  assessment of the overall quality of evidence and the net effect of the  intervention. (See app. IV for a detailed description of the process used to  develop evidence-based VA/DOD CPGs.)
The process for conducting a systematic review of research outcomes to  develop or update a CPG is repeated as often as is deemed necessary by  the VA/DOD Evidence-Based Practice Work Group according to its written  procedures and designated time frames. According to VA/DOD Evidence- Based Practice Work Group documents, routine updates to the CPGs  should ideally occur approximately every 2 years. However, updates to  CPGs often do not occur every 2 years, and VA officials told us that some  CPGs are updated more frequently than others based on availability of  resources and priority areas. Additionally, VA officials reported that a CPG  will be immediately updated if any evidence-based recommendation  contained in it is identified as harmful to patients.
According to VA, the VA/DOD Evidence-Based Practice Work Group  approved an update to the 2004 PTSD CPG on October 25, 2010, and  published the update on VA’s Web site on November 17, 2010. According  to VA officials, the systematic process outlined above was used to review  all relevant research outcomes and make evidence-based  recommendations for PTSD services to both develop and update the PTSD  CPG.
section_title: VA Reviewed Research Outcomes and Used Existing Resources in Determining Which PTSD Services to Require Its Facilities to Provide or Make Available to Veterans
paragraphs: According to VA officials, the decision to require that cognitive processing  therapy and prolonged exposure therapy be made available to veterans  diagnosed with PTSD at VA facilities—as indicated in the Handbook,  which established certain requirements for mental health services within  VA—was based on a review of research outcomes and the availability of  existing resources.
Review of research outcomes. According to VA, agency officials and  qualified subject matter experts reviewed relevant research outcomes and  the quality of the research to determine the most efficacious PTSD  treatments available when determining which PTSD services to include in  the Handbook and make available to veterans. Specifically, VA officials  told us that their decision to include cognitive processing therapy and  prolonged exposure therapy in the Handbook was influenced by the fact  that both of these had been graded as level “A” treatments in the 2004  PTSD CPG (indicating that the intervention is always indicated and  acceptable). Furthermore, VA officials said that these two therapies had  greater evidence supporting their effectiveness than other PTSD services  also graded as level “A” in the 2004 PTSD CPG. In addition, VA officials  added that their decision was validated by the results of a VA- commissioned Institute of Medicine study published in 2008 that reviewed  the evidence for existing PTSD treatments. According to VA, the study  found that cognitive processing therapy and prolonged exposure therapy  were considered efficacious treatments for PTSD. While the Institute of  Medicine report was released after VA had already decided to include  cognitive processing therapy and prolonged exposure therapy in the  Handbook, VA officials explained that the Institute of Medicine report was  the basis for the decision not to include other PTSD services in the  Handbook.
Availability of existing resources. VA officials told us that prior to  issuing the Handbook in 2008, VA had already begun investing  considerable resources to implement national training programs for  cognitive processing therapy and prolonged exposure therapy in 2006 and  2007, respectively. VA officials said that they decided to implement the  national training programs because VA realized the need to create  sufficient capacity so that evidence-based PTSD treatments could be  available to veterans throughout the VA system. VA explained that the  national training programs were rolled out in advance of the Handbook’s  issuance as part of the implementation of VA’s Comprehensive Veterans  Health Administration Strategic Plan for Mental Health Services, which  called for rapid implementation of evidence-based treatments. VA did this  to ensure that it had the capacity to provide cognitive processing therapy  and prolonged exposure therapy to all veterans with PTSD for whom these  treatments were clinically appropriate. VA officials said that they were  able to begin implementing national training programs for cognitive  processing therapy in 2006 and prolonged exposure therapy in 2007  because VA had qualified instructors to administer the programs and  money available to fund them.
Unlike the written and standardized process that the VA/DOD Evidence- Based Practice Work Group established to develop CPGs, VA does not  have a formal written process or framework to explain its decision for  including cognitive processing therapy and prolonged exposure therapy in  the Handbook. VA officials explained that they followed a process when  choosing cognitive processing therapy and prolonged exposure therapy,  but added that clinical decision-making processes are not typically  expected to be documented in a formal manner.
VA officials told us that they plan to assess the implementation of the  Handbook and will update PTSD requirements in it as needed or as new  information or unexpected obstacles arise in the future. VA officials  stated that they are currently clarifying the language regarding some of the  requirements, but do not plan to revise any of the requirements relating to  PTSD services at this time.
section_title: Agency Comments
paragraphs: We provided a draft of this report to VA and received technical comments,  which we incorporated into our report as appropriate.
We are sending a copy of this report to the Secretary of Veterans Affairs.  The report also is available at no charge on the GAO Web site at  http://www.gao.gov.
If you or your staff have any questions about this report, please contact me  at (202) 512-7114 or williamsonr@gao.gov. Contact points for our Offices   of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff members who made key contributions to  this report are listed in appendix V.
section_title: Appendix I: VA Research Centers and Programs That Conduct or Support PTSD Research
paragraphs: In addition to post-traumatic stress disorder (PTSD) research studies that  are conducted by individual Department of Veterans Affairs (VA)  investigators, or researchers, VA also funds a number of research centers  or programs that conduct or support PTSD research. See table 6 for a  description of these VA research centers and programs.
section_title: Appendix II: The Review Process for VA’s Merit Review Program
paragraphs: Research proposals submitted to the Department of Veterans Affairs’ (VA)  Merit Review Program are evaluated in merit review panels that each  review proposals in a specific research topic area. Each merit review  panel is comprised of panelists, typically associate-level professors, who  are selected based on their expertise in the area. Panelists are responsible  for scoring proposals based on scientific merit to provide funding  recommendations. See figure 6 for a detailed description of the Merit  Review Program’s process for reviewing research proposals and table 7 for  the Merit Review Program’s scoring guidelines.
section_title: Appendix III: The Review Process for VA’s Cooperative Studies Program
paragraphs: Research proposals submitted to the Department of Veterans Affairs’ (VA)  Cooperative Studies Program (CSP) are reviewed and scored by the  Cooperative Studies Scientific Merit Review Board. Reviewers on the  board are chosen based on their expertise in clinical or epidemiological  research. They typically serve 4-year terms, and ad hoc members can be  added depending on specific expertise that may be needed to review a  proposal. According to VA, as of October 2010, there were six reviewers  on the board.
During the research proposal review process, the study team—which  includes the lead researcher (referred to as the principal investigator) and  a methodologist, such as a person with expertise in biostatistics—has an  interactive discussion with the board regarding the proposal. Reviewers  evaluate CSP proposals based on scientific merit and provide scores to  reflect their funding recommendations. See figure 7 for a detailed  description of the review process for CSP research proposals.
section_title: Appendix IV: The Process for Developing VA/DOD Evidence-Based Clinical Practice Guidelines
paragraphs: In 1999, the Department of Veterans Affairs (VA) and the Department of  Defense (DOD) formed the VA/DOD Evidence-Based Practice Work Group  to issue joint VA/DOD clinical practice guidelines (CPG)—tools that  provide guidance and evidence-based recommendations to clinicians  regarding the most effective interventions and services for a variety of  health care topics. To develop or update a CPG, the VA/DOD Evidence- Based Practice Work Group has a standardized process to ensure that  systematic reviews of relevant research outcomes are conducted in order  to formulate evidence-based recommendations for prevention,  assessment, and treatment services.
To develop or update a CPG, the VA/DOD Evidence-Based Practice Work  Group identifies two clinical leaders—one from VA and one from DOD— who then help identify not more than 15 to 20 other experts in the subject  area to form a “guideline working group.” A member of the VA/DOD  Evidence-Based Practice Work Group is also selected to be an evidence  chaperone for each CPG to ensure that conformity to prevailing standards  for conducting high-quality systematic reviews is upheld.
To determine the scope of the CPG, the guideline working group, the  evidence chaperone, and a facilitator are responsible for identifying  clinical questions that are to be answered by a systematic review of  relevant research outcomes. According to VA officials, clinical questions  can be both broad and specific. For example, the 2004 post-traumatic  stress disorder CPG contained clinical questions regarding whether early  intervention is more effective than later intervention and whether certain  interventions, such as different psychotherapies, are more effective than  others.
According to VA, in order to answer these clinical questions, an external  evidence center—an entity that conducts systematic reviews of research  on a variety of topics—is contracted to collect and review all relevant  research (including, but not limited to, VA- and DOD-sponsored research)  to assess its applicability to each clinical question under consideration  using explicit and reproducible methods. The evidence center then  focuses its review on the best available research, that is, high-quality,  methodologically rigorous studies that address health issues that impact  VA and DOD populations and consider the effectiveness as well as the  harms and benefits of the intervention at issue. According to VA officials,  the evidence center provides summaries of only the best available  research to the guideline working group for review.
After receiving the summaries, the guideline working group reviews the  research in sequential steps using an established rating scheme developed  by the U.S. Preventive Services Task Force to formulate evidence-based  recommendations. See figure 8 for an overview of the steps that the  guideline working group uses to formulate evidence-based  recommendations.
Level of evidence. First, the guideline working group reviews the  summaries to identify the level of evidence, or the level of methodological  rigor. For example, research studies that have the highest quality are  categorized as “I” (indicating at least one properly done randomized  controlled trial), while research studies of the lowest quality are  categorized as “III” (indicating that the research reflects the opinion of  respected authorities, descriptive studies, case reports, and expert  committees). (See table 8.)
Overall quality of research. After determining the level of evidence of  individual research studies, the guideline working group makes a  determination regarding the overall quality of all of the research that  addresses a particular clinical question. The overall quality takes into  account the number, quality, and size of all of the individual research  studies together as well as the consistency of the results between research  outcomes to determine the collective overall strength of the research.  Based on this review, the guideline working group determines the overall  quality of the evidence to be good, fair, or poor. (See table 9.)
Net effect of the intervention. For interventions that were supported  by studies of “fair” or “good” overall quality, the guideline working group  evaluates the benefits and the potential harms to determine the net effect  of the intervention. The net effect of an intervention takes into account  the benefits of the intervention minus the harms to determine the overall  potential clinical benefit that the intervention may provide to patients. The  net effect of the intervention ranges from “substantial” (meaning the  benefit substantially outweighs the harm) to “zero or negative” (meaning it  has no impact or a negative impact on patients). (See table 10.)
Grade of evidence-based recommendation. In the final step, the  guideline working group uses its assessment of the overall quality of the  evidence and the net effect of the intervention to grade evidence-based  recommendations. (See table 11.)
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Mary Ann Curran, Assistant  Director; Susannah Bloch; Stella Chiang; Martha R. W. Kelly; Melanie  Krause; Lisa Motley; Michelle Paluga; Rebecca Rust; and Suzanne Worth  made key contributions to this report.
section_title: Related GAO Products
paragraphs: VA Health Care: Progress and Challenges in Conducting the National  Vietnam Veterans Longitudinal Study. GAO-10-658T. Washington, D.C.:  May 5, 2010.
VA Health Care: Status of VA’s Approach in Conducting the National  Vietnam Veterans Longitudinal Study. GAO-10-578R. Washington, D.C.:  May 5, 2010.
VA Health Care: Preliminary Findings on VA’s Provision of Health Care  Services to Women Veterans. GAO-09-899T. Washington, D.C.: July 16,  2009.
DOD and VA Health Care: Challenges Encountered by Injured  Servicemembers during Their Recovery Process. GAO-07-589T.  Washington, D.C.: March 5, 2007.
VA Health Care: Spending for Mental Health Strategic Plan Initiatives  Was Substantially Less Than Planned. GAO-07-66. Washington, D.C.:  November 21, 2006.
VA Health Care: Preliminary Information on Resources Allocated for  Mental Health Strategic Plan Initiatives. GAO-06-1119T. Washington,  D.C.: September 28, 2006.
VA Health Care: VA Should Expedite the Implementation of  Recommendations Needed to Improve Post-Traumatic Stress Disorder  Services. GAO-05-287. Washington, D.C.: February 14, 2005.
VA and Defense Health Care: More Information Needed to Determine If  VA Can Meet an Increase in Demand for Post-Traumatic Stress Disorder  Services. GAO-04-1069. Washington, D.C.: September 20, 2004.