VA Health Care: Improvements Needed in Monitoring Antidepressant Use for Major Depressive Disorder and in Increasing Accuracy of Suicide Data
Released: 2014-12-12T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: In 2013, VA estimated that about 1.5 million veterans required mental health care, including services for MDD. MDD is a debilitating mental illness related to reduced quality of life and productivity, and increased risk for suicide. VA also plays a role in suicide prevention. GAO was asked to review how VA tracks veterans prescribed antidepressants and what suicide data VA uses in its prevention efforts.

This report examines (1) VA's data on veterans with MDD, including those prescribed an antidepressant; (2) the extent that veterans with MDD who are prescribed antidepressants receive recommended care and the extent to which VA monitors such care; and (3) the quality of data VA requires VAMCs to collect on veteran suicides. GAO analyzed VA data, interviewed VA officials, and conducted site visits to six VAMCs selected based on geography and population served. From each of these six VAMCs, GAO also reviewed five randomly selected medical records for veterans diagnosed with MDD and prescribed an antidepressant in 2012, as well as all completed BHAP templates. The results cannot be generalized across VA but provide insights.

section_title: What GAO Found
paragraphs: GAO's analysis of Department of Veterans Affairs (VA) data for fiscal years 2009 through 2013 shows that about 10 percent of veterans who received VA health care services were diagnosed with major depressive disorder (MDD). MDD is characterized by depressed mood or loss of interest along with other symptoms for 2 weeks or more that represent a change in the way individuals function from their previous behaviors. Because GAO found diagnostic coding discrepancies in 11 of the 30 veterans' medical records it reviewed from six VA medical centers (VAMC), VA's data may understate the prevalence of MDD among veterans being treated through VA, to the extent that such discrepancies may permeate VA's data. One treatment for MDD is the use of medications such as antidepressants. According to GAO's analysis, 94 percent of veterans diagnosed with MDD were prescribed at least one antidepressant.

VA policy states that antidepressant treatment must be consistent with VA's current clinical practice guideline (CPG); however, GAO's review of 30 veterans' medical records identified deviations from selected MDD CPG recommendations for most veterans reviewed. For example, 26 of the 30 veterans were not assessed using a standardized assessment tool at 4 to 6 weeks after initiation of treatment, as recommended in the CPG. Additionally, 10 veterans did not receive follow up within the time frame recommended in the CPG. GAO found that VA does not have a system-wide process in place to identify and fully assess whether the care provided is consistent with the CPG. As a result, VA does not know the extent to which veterans with MDD who have been prescribed antidepressants are receiving care as recommended in the CPG and whether appropriate actions are taken by VAMCs to mitigate potentially significant risks to veterans.

The demographic and clinical data that VA collects on veteran suicides were not always complete, accurate, or consistent. VA's Behavioral Health Autopsy Program (BHAP) is a quality initiative to improve VA's suicide prevention efforts by identifying information that VA can use to develop policy and procedures to help prevent future suicides. The BHAP templates are a mechanism by which VA collects suicide data from VAMC's review of veteran medical records. GAO's review of 63 BHAP templates at five VAMCs found that 40 of the templates that VAMCs submitted to VA Central Office had incomplete data. Also, GAO found that the BHAP templates VAMCs submitted contained inaccurate data. For example, 6 BHAP templates included a date of death that was incorrect based on information in the veteran's medical record, and 9 BHAP templates included an incorrect number of outpatient VA mental health visits in the last 30 days. Moreover, GAO found that VAMCs submitted inconsistent information because they interpreted VA's guidance on completing the BHAP templates differently. This situation was further exacerbated because BHAP templates prepared by VAMCs are generally not being reviewed at any level within the Department for completeness, accuracy, and consistency. Lack of complete, accurate, and consistent data and poor oversight can inhibit VA's ability to identify, evaluate, and improve ways to better inform its suicide prevention efforts.

section_title: What GAO Recommends
paragraphs: GAO recommends that VA identify and address MDD coding discrepancies; implement processes to review data and assess deviations from recommended care; and implement processes to improve completeness, accuracy, and consistency of veteran suicide data. VA concurred with GAO's recommendations and described its plans to implement them.

section_title: Letter
paragraphs: In 2013, the Department of Veterans Affairs (VA) estimated that about   1.5 million veterans required mental health services, which VA provides in  a variety of settings, including VA medical centers (VAMC), community- based outpatient clinics, and residential treatment programs. Mental  health treatment includes services for depression—a mood disorder that  causes a persistent feeling of sadness and loss of interest. One type of  depression, major depressive disorder (MDD), is a particularly debilitating  mental illness and is associated with reduced quality of life, reduced  productivity, and increased risk for suicide. These negative effects  underscore the importance of timely, evidence-based assessment for and  treatment of MDD, which may include medications, such as  antidepressants, psychotherapy, or a combination of both. Treatment of  veterans with MDD can improve their occupational and social functioning  and their overall well-being.

In addition to providing ongoing care to veterans with MDD, VA plays a  role in suicide risk assessment and prevention among veterans.  According to VA, about one-quarter of the 18 to 22 veterans who die by  suicide each day were receiving care through VA. identified numerous risk factors for suicide among veterans, which include  substance use disorder, physical impairments, previous suicide attempts,  and depression. Additionally, life stressors, such as marital or financial  problems, contribute to a veteran’s risk of suicide.

VA/Department of Defense (DOD) Assessment and Management of Risk for Suicide  Working Group, VA/DOD Clinical Practice Guideline for Assessment and Management of  Patients at Risk for Suicide (June 2013).

Given the debilitating effects of depression, VA’s monitoring of veterans  with MDD is critical to ensuring positive health outcomes. Additionally, the  relatively high veteran suicide rate makes it important that VA effectively  employs data related to suicides in its prevention efforts. You asked us to  review how VA tracks and manages veterans prescribed antidepressants  and what suicide data VA uses in its suicide prevention efforts. In this  report, we examine  1.  the data VA has on veterans with MDD, including the extent to which  they were prescribed antidepressants;  2.  the extent to which veterans with MDD who are prescribed an  antidepressant receive recommended care and the extent to which  VA monitors such care; and  3.  the information VA requires VAMCs to collect on veteran suicides.

To examine the data VA has on veterans with MDD, including the extent  to which veterans were prescribed antidepressants, we analyzed national  VA data from fiscal years 2009 through 2013 and interviewed VA  officials. The data we analyzed included information on veterans’  demographic characteristics, health care services, and medications  provided through VA. We considered a veteran to have MDD if, in at  least one fiscal year included in our review, the veteran had two or more  outpatient encounters or at least one inpatient hospital stay associated  with a diagnosis of MDD. To ensure the reliability of the data we  analyzed, we interviewed VA Central Office officials, reviewed relevant  documentation and veterans’ medical records, and conducted electronic  testing to identify missing data and obvious errors. On the basis of these  activities, we determined that the data we analyzed were sufficiently  reliable for our purposes. We also conducted site visits to 6 of the 150  VAMCs selected for variation in complexity of health care services  offered, geographic location, and number of veterans using mental health  services. These six VAMCs were located in Canandaigua, New York;  Gainesville, Florida; Iowa City, Iowa; Philadelphia, Pennsylvania;  Phoenix, Arizona; and Reno, Nevada. At each site, we interviewed  VAMC staff about the data maintained on veterans diagnosed with MDD.  In addition, we reviewed a random, nongeneralizeable sample of medical  records for five veterans treated at each of the six VAMCs for a total of 30  veterans. We selected veterans for review that were diagnosed with MDD  and had a new treatment episode of an antidepressant in calendar year  2012. Starting with the initial encounter in 2012 that was associated with  the new antidepressant prescription, we reviewed the medical records to  determine if the diagnostic code entered for subsequent encounters was  consistent with a diagnosis of MDD.

To analyze the extent to which VA monitors whether veterans with MDD  who are prescribed an antidepressant receive care as recommended by  VA’s clinical practice guideline (CPG) for MDD, we reviewed VA policy  documents and interviewed VA Central Office officials responsible for  developing and implementing VA mental health policy. We identified the oversight processes VA has in place to monitor the recommendations in  the CPG and assessed whether this oversight provides VA with adequate  information to identify nonconformance with recommended practices,  assess the risk, and address nonconformance, as appropriate. We  selected three CPG recommendations for inclusion in our review that had  among the highest strength of research evidence, were sufficiently  specific to enable us to determine the extent to which VA providers were  following the recommendation, and would not require clinical judgment to  determine the extent to which VA providers were following the  recommendation. To examine how the recommendations we selected  were implemented, we reviewed the 30 medical records from our selected  six VAMCs to assess the extent to which the antidepressant treatment- related care VAMCs provided was consistent with the CPG  recommendations. We also interviewed VAMC officials regarding the care  provided to veterans with MDD prescribed an antidepressant. Results  from our 30 medical record reviews cannot be generalized to the VAMC  visited or other VAMCs.

The Behavioral Health Autopsy Program (BHAP) Post-Mortem Chart Analysis Template  (BHAP template) is a mechanism by which VA Central Office collects veteran suicide data  from VAMCs’ review of veterans’ medical records. One BHAP template had been  completed as of the time of our site visit, but had not been received by VA Central Office;  therefore, we did not include this template in our review. judgment to assess. Results from our review of veteran suicide data can  be generalized to the VAMCs we visited, but cannot be generalized to  other VAMCs. We also interviewed relevant staff of the six Veterans  Integrated Service Networks (VISN), or regional networks of care, for the  sites we visited to obtain information on suicide prevention efforts in that  VISN.

For more information on our scope and methodology, see appendix I.

We conducted this performance audit from November 2013 to November  2014 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: VA provides care to veterans with mental health needs through its 150  VAMCs, which may include both specialty mental health care settings— including mental health clinics—and other settings that may provide  mental health services but focus primarily on other types of care, such as  primary care. VA has implemented a program to co-locate mental health  care providers within primary care settings in an effort to promote  effective treatment of common mental health conditions in the primary  care environment while allowing mental health specialists to focus on  veterans with more severe mental illnesses.
subsections: section_title: Care for Veterans with MDD
paragraphs: According to VA, the prevalence of MDD in primary care settings among  veterans being treated through VA is higher than that among the general  population. MDD is characterized by the presence of depressed mood or  loss of interest or pleasure along with other symptoms for a period of at  least 2 weeks that represent a change in previous functioning. VA has  policies and guidance in place related to treating veterans with MDD. For  example, the Uniform Mental Health Services in VA Medical Centers and  Clinics handbook (Handbook), which defines VA’s minimum clinical  requirements for mental health services, requires that VA facilities provide  evidence-based treatment through the administration of medication, when  indicated, consistent with the MDD CPG. The CPG is guidance intended  by VA to reduce current practice variation between clinicians and provide facilities with a structured framework to help improve patient outcomes.

The MDD CPG provides evidence-based recommendations as guidance  for clinicians who provide care for veterans with MDD. The MDD CPG  includes approximately 200 recommendations to provide information and  assist in decision making for clinicians who provide care for adults with  MDD.assessments of depressive symptoms, such as the nine item Patient  Health Questionnaire (PHQ-9), should be used at the initial assessment  of MDD symptoms, to monitor treatment response at 4-6 weeks after  initiation of treatment, after each change in treatment, and periodically   For example, the CPG recommends that standardized  thereafter until full remission is achieved.assessment is effective 4-6 weeks after initiation of treatment, making  timely follow-up visits an important part of clinicians’ ability to assess  whether the current treatment plan is effective or should be modified.  According to the MDD CPG, veterans with MDD treated with  antidepressants should be closely observed, particularly at the beginning  of treatment and following dosage changes, to maximize veterans’  recovery and to mitigate any negative treatment effects, including  worsening of depressive symptoms. The CPG should not take the place  of the clinician’s clinical judgment.

section_title: VA Suicide Prevention Efforts
paragraphs: Beginning in June 2006, VA implemented several initiatives aimed at  suicide prevention, including appointing a National Suicide Prevention  Coordinator, developing data systems to increase understanding of  suicide among veterans and inform VA suicide prevention programs, and  instituting suicide prevention programs in VAMCs throughout the country.  Additionally, VA Central Office established the Center of Excellence for  Suicide Prevention and the Veterans Crisis Line in 2007. The Center of  Excellence collects VA suicide prevention program data, which provides  information on veteran suicide completions and suicide attempts for  veterans receiving VA care, as well as those veterans not receiving VA  care. VA’s Veterans Crisis Line provides toll-free, confidential support   24 hours per day for veterans, their families, and their friends through  phone, online chat, or text message.Crisis Line fielded approximately 287,000 calls, 54,800 online chats, and  11,300 text messages.

In fiscal year 2013, the Veterans  As part of VAMCs’ suicide prevention programs, the Handbook requires  each VAMC to have a suicide prevention coordinator whose  responsibilities include establishing and maintaining a list of veterans assessed to be at high  risk for suicide; monitoring these high-risk veterans;  responding to referrals from staff and the Veterans Crisis Line;  collaborating with community organizations and partners;  training staff members who have contact with veterans at the VAMC,  community organizations, and partners; and  collecting and reporting information on veterans who die by suicide  and who attempt suicide.

See appendix II for more information on VAMCs’ tracking of veterans at  high risk for suicide.

section_title: VA Suicide Data
paragraphs: VA Central Office uses several mechanisms to collect data on veteran  suicides to help improve its suicide prevention efforts. One such  mechanism includes data submitted by suicide prevention coordinators at  VAMCs on known veterans who die by suicide. Beginning in December  2012, VA Central Office began a national initiative to collect demographic,  clinical, and other related information on veteran suicides as a quality  improvement initiative to improve its suicide prevention efforts by  identifying information that can be used by VA Central Office to develop  policy and procedures to help prevent future veteran deaths. This  initiative, the Behavioral Health Autopsy Program (BHAP), replaced  previous VA Central Office requirements to collect data on completed  suicides. VA Central Office officials explained that they transitioned to  the BHAP initiative to collect more systematic and comprehensive  information about suicides, to incorporate interviews of family members of  those who die by suicide, and to collect more contextual information.

According to VA, the BHAP quality initiative has been adapted from a  traditional psychological autopsy research framework that emphasizes  the importance of information from outside sources as well as from those  within the health care setting. The BHAP initiative is being implemented  by VA in four phases:

Phase 1—Standardized chart reviews: VAMCs’ suicide prevention  coordinators are required to complete standardized chart reviews for  all veterans’ suicides known to VAMC staff and reported on or after   These reviews include specific information on a  October 1, 2012.veteran’s utilization of VA health care services, as well as a veteran’s  mental health diagnoses and risk factors for suicide. VA Central Office  has instructed suicide prevention coordinators to use all available  information, including VA medical records and information from a  veteran’s family members to complete the chart review. These  reviews are submitted to VA Central Office through completion of a  BHAP Post-Mortem Chart Analysis Template (BHAP template) and  VA Central Office has provided suicide prevention coordinators with a  BHAP Guide on how to complete the fields in the BHAP template. VA  Central Office requires VAMCs to submit the BHAP template within   30 days of VAMC staff becoming aware of a veteran’s death by  suicide.

Phase 2—Interviews with family members: In fiscal year 2013, VA  Central Office began conducting interviews with family members of  veterans who have died by suicide to obtain information on suicide  risks, barriers to care, and suggestions for new programs to prevent  suicide.

Phase 3—Clinician questionnaire: This phase, which has not yet  been implemented, will include an interview with the last provider that  saw the veteran prior to his or her death. VA officials stated that there  are no plans to begin this phase within calendar year 2014, and they  have not established a future time table for implementing this phase.

Phase 4—Public record review: This phase, which has also not  been implemented, will be used to locate public records to identify  stressors in the veteran’s life, such as a bankruptcy or divorce.  Officials stated that there are no plans to begin this phase within  calendar year 2014, and they have not established a future time table  for implementing this phase.

Since beginning the BHAP initiative, VA Central Office has internally  issued two interim reports on data and trends from the submitted BHAP  templates as part of Phase 1. The reports include information for veterans  who died by suicide, both with and without a history of VA health care  service utilization. Analyses of data on demographic characteristics, case  information, period of service, and risk and protective factors were  included for all veterans. Data on clinical characteristics and indicators of  increased risk at the time of the veteran’s last contact with a VA provider  were limited to veterans that utilized VA health care services.

In addition to the BHAP initiative, VA also requires VAMCs to collect and  submit data on suicide attempts and completions through the following  mechanisms.

Suicide Prevention Application Network (SPAN): Through SPAN,  VAMCs submit information to VA Central Office on the number of  veterans that completed suicide, the number of suicide attempts, and  indicators of suicide prevention efforts, such as outreach events  conducted each month by suicide prevention coordinators.

Suicide behavior reports: VAMC clinicians must complete a suicide  behavior report when they learn that a veteran attempted or  completed suicide and add that report to the respective veteran’s  medical record. This report includes the date and time of the event,  and other observations related to the suicide attempt or completed  suicide. According to VA policy, information from suicide behavior  reports is used for National Patient Safety reporting requirements and  to populate SPAN.

Root cause analyses: Patient safety managers at VAMCs complete  root cause analyses for suicide attempts and completed suicides  under certain circumstances, such as when the attempt occurs at the  VAMC during an inpatient stay or within 72 hours of being discharged  from inpatient care. Root cause analyses are used to identify the  factors that contributed to adverse events or close calls and any steps  VAMCs could implement to prevent similar events in the future.

See appendix III for how VAMC and VISN officials we interviewed told us  they have utilized data related to suicides and suicide behavior.

section_title: VA Data Show That 10 Percent of Veterans Had MDD and Most Were Prescribed at Least One Antidepressant, but VA Data May Underestimate MDD Prevalence
paragraphs: Data for fiscal years 2009 through 2013 show that about 10 percent of  veterans who received health care services through VA were diagnosed  with MDD, and of those, 94 percent were prescribed an antidepressant.  However, due to diagnostic coding discrepancies we found in a sample of  veterans’ medical records, VA’s data may not accurately reflect the  prevalence of MDD among veterans.
subsections: section_title: VA Data Show About 10 Percent of Veterans Had a Diagnosis of MDD, and Almost All Were Prescribed at Least One Antidepressant
paragraphs: Based on our analysis of VA data from veterans’ medical records and  administrative sources, 532,222 veterans—about 10 percent of veterans  who received health care services through VA—had a diagnosis of  MDD from fiscal years 2009 through 2013. Among those veterans, the  majority (60 percent) were 35 to 64 years of age. Most (86 percent) were   In addition,  not veterans of the recent conflicts in Iraq and Afghanistan.most of these veterans were male (87 percent) and the highest proportion  was white (68 percent) and non-Hispanic (87 percent). See table 1 for a  summary of characteristics of veterans who had a diagnosis of MDD from  fiscal years 2009 through 2013.

We also found that about 499,000 of the 532,222 (94 percent) veterans  who had a diagnosis of MDD from fiscal years 2009 through 2013 were  prescribed at least one antidepressant. Of those veterans, the majority  (about 73 percent) were dispensed a 12-week supply of an  antidepressant at the start of an MDD episode. Fewer veterans (about   58 percent) were dispensed a 6-month supply of an antidepressant over  the course of their treatment. Receiving a 12-week supply of an  antidepressant can be important for addressing depressive symptoms  initially, while continued treatment after remission of depressive  symptoms, such as receiving a 6-month supply of an antidepressant, is  associated with a decreased risk of relapse, according to the CPG.

section_title: VA’s Data May Not Fully Reflect the Extent to Which Veterans Have MDD Due to a Lack of Diagnostic Coding Precision by Clinicians
paragraphs: Based on our review of the documentation in 30 veterans’ medical  records from VA’s medical record system, we found that over one-third  (11) had diagnostic coding discrepancies. Specifically, these 11 veterans  had at least one encounter where the clinician documented a diagnosis of  MDD in the veteran’s medical record, but the clinician did not code the  encounter accordingly. Instead, the clinician coded the encounter as  “depression not otherwise specified,” a less specific code. According to  the fourth edition of the Diagnostic and Statistical Manual of Mental  Disorders, depression not otherwise specified is to be used to code  disorders with depressive features that do not meet criteria for MDD and  other depressive disorders, or to indicate depressive symptoms about  which there is inadequate or contradictory information. VA’s data on the  number of veterans with MDD are based on the diagnostic codes  associated with patient encounters, so the discrepancies we found  indicate that the number of veterans with MDD is most likely not fully  reflected in these data. Accurately identifying the veteran population with  MDD is critical to assessing Department performance in treating veterans  in accordance with the MDD CPG and measuring health outcomes for  these veterans. VA Central Office reviewed the 11 medical records where  we found coding discrepancies and agreed that the encounters were not  coded accurately. According to a VA Central Office official, the  encounters we identified were corrected in the veterans’ medical record.

According to VHA Handbook 1907.03 - Health Information Management  Clinical Coding Program Procedures, VAMCs are required to monitor the  accuracy of coding and provide training as necessary in order to help  ensure accurate coding. VAMC officials from all six sites in our review  said that monthly or quarterly coding audits are conducted at their  facilities and the findings of those audits are reviewed and action is taken  to correct issues with the accuracy and reliability of coding. However, at  five of the six VAMCs in our review, those audits focus on billable  encounters—that is, encounters that are billed to a third party, such as  private health insurance plans—in part because of the potential  opportunity for facilities to collect third-party revenue from these  encounters. Among the 11 veterans’ medical records where we  identified coding discrepancies, all of the discrepancies were associated  with outpatient, nonbillable encounters, the coding of which, according to  a VA Central Office official, is not typically conducted by VAMC medical  coders—staff who are trained specifically in medical coding terminology  and standards and are responsible for coding inpatient admissions and  discharges—or subject to coding audits.

Diagnostic coding in VA’s medical record system for outpatient  encounters is typically performed by clinicians. VISN officials and VA  medical center clinicians we interviewed said that clinicians do not place a  lot of importance on selecting a more precise diagnostic code because it  does not significantly change the patient care that is provided or the type  of treatment prescribed. In addition, in the interest of expediency,  clinicians may select a previously used or frequently used diagnostic code  for depression rather than take the time to search for a more precise  code. For example, within the medical record, clinicians may access a list  of previous or current diagnoses applicable to the veteran (commonly  referred to as the “problem list”) or a list of frequently used diagnostic  codes in the facility. According to VISN and VAMC officials, the problem  list is not typically kept up to date by clinicians and as a result, MDD may  not be listed and readily available for clinicians to select. As a result of our  review, VA Central Office officials reported that they had discovered a  software mapping error in VA’s medical record system where the  selection of MDD as a diagnosis when using a keyword search function  may result in the selection of the depression not otherwise specified  diagnostic code by mistake. Officials stated that they anticipate that the  software error—which applies to all VAMCs—would be fixed by  November 2014. Officials also stated that the solution would apply only to  those encounters coded from that point going forward and would not  retroactively correct any coding discrepancies that may have occurred  before the error was addressed. VA Central Office officials could not tell  us if any of the 11 coding discrepancies that we identified were a result of  this software error.

Officials at most of the six VISNs we spoke with do not conduct reviews of  medical coding done by clinicians. However, as a result of our inquiry,  one VISN we interviewed reported in the late spring of 2014 that it had  extracted data on MDD-related encounters and noticed the high use of  depression not otherwise specified coding for the facilities within its VISN,  as well as all VAMCs nationwide. Officials from this VISN said the lack of  coding specificity has implications for being able to accurately examine  health outcomes related to the treatment of depression and that they are  planning to further analyze encounter data within their VISN to determine  the appropriateness of diagnostic coding based upon medical record  documentation. As of September 2014, the VISN had not reported any  additional steps to address this issue.

section_title: Not All Veterans in Our Review Received CPG-Recommended Care and VA Lacks Mechanisms to Determine Whether This Care Is Provided
paragraphs: Based on the three CPG recommendations we selected, veterans in our  review with MDD who have been prescribed antidepressants did not  always receive care as recommended in the MDD CPG. Additionally, VA  does not know the extent to which veterans with MDD who have been  prescribed antidepressants are receiving care as recommended in the  CPG, and VA Central Office has not developed mechanisms to determine  the extent to which mental health care delivery conforms to the  recommendations in the MDD CPG.
subsections: section_title: Veterans We Reviewed Did Not Always Receive Care As Recommended
paragraphs: We found that almost all of the 30 veterans with MDD who have been  prescribed antidepressants included in our review did not receive care in  accordance with the three MDD CPG recommendations we reviewed. VA  policy states that antidepressant treatment must be consistent with VA’s  current, evidence-based CPG. However, VA Central Office mental  health officials were unable to tell us what it means to provide care that is  consistent with the CPG, because, while a veteran’s treatment should be  informed by the CPG recommendations, determining the extent to which  the treatment is consistent with CPG recommendations would need to be  done on a veteran-by-veteran basis. The CPG is intended to reduce  practice variation and help improve patient outcomes, but without an  understanding of the extent to which veterans are receiving care that is  consistent with the CPG, VA may be unable to ensure that it meets the  intent of the CPG and improves veteran health outcomes.

Through our review of 30 medical records from the six VAMCs we  selected, we found examples of deviations from the CPG  recommendations for almost all veterans in our review.depicts the specific recommendations we reviewed and the number of  veterans that did not receive care consistent with the corresponding CPG  recommendation.

section_title: VA Lacks Mechanisms to Determine the Extent to Which Veterans Are Receiving Care Consistent with CPG Recommendations
paragraphs: VA does not know the extent to which veterans are receiving care  consistent with the MDD CPG. While deviations from recommended  practice may be appropriate in many cases due to clinician discretion, VA  has not fully assessed whether these examples are acceptable deviations  from the CPG. According to the federal internal control standard for risk  assessment, agencies should comprehensively identify risks, assess the  possible effects, if any, and determine what actions should be taken to  mitigate any significant risks. VA Central Office has not developed a  mechanism to fully identify deviations that could impede veterans’  recovery that may result when VAMCs do not provide care consistent with  the MDD CPG. VA Central Office officials explained that the CPG  recommendations are guidelines that clinicians can use to inform and  guide clinical decision making. VA officials told us that VA cannot require  the use of all recommendations in all cases; rather, CPG  recommendations should be applied on a case-by-case basis based on  the needs of the veteran and with clinician judgment. One official also  said it would be difficult to check every CPG recommendation to ensure  that clinicians are providing care consistent with the CPG, but stated that  VA could identify for review those recommendations that may put  veterans’ health at risk if not followed. However, with no mechanism to  assess whether the care provided is consistent with the CPG, VA is  unable to ensure that deviations from recommended care are identified.

While monitoring full compliance with CPG recommendations may be  difficult, there are nevertheless ways to address the issue. In fact, VA  Central Office and some VAMCs have implemented mechanisms to  determine the extent to which veterans are receiving care that is  consistent with some of the CPG recommendations; however, these  mechanisms do not fully assess all deviations that could impede a  veteran’s recovery, as illustrated by the following.

Officials at one VAMC we visited told us that they use a clinical tool to track veterans  being treated for mental health conditions. The mental health tool includes 67,349 unique  patients, and an official explained that they can run queries of the clinical tool—for  example, for veterans participating in substance abuse treatment who did not return for a  drug screen—by pulling both process and outcome variables including diagnostic codes,  lab results, and medication lists. designed to manage the behavioral health needs of veterans through  telephone or in-person visits. As part of the system, clinicians can use  a structured interview—including a PHQ-9—that assesses veterans’  mental health symptoms in a way that is consistent with the CPG  recommendation for follow-up assessment. Although the BHL can be  used to help ensure care is provided consistent with a few of the  recommendations in the CPG, the BHL is not used to monitor all  veterans prescribed antidepressants. Generally, VAMCs use the BHL  to monitor veterans being treated for mental health conditions, such  as MDD, in primary care clinics, and to participate, veterans can be  referred by their primary care clinician or request to participate.

section_title: Demographic and Clinical Data VA Collects on Veteran Suicides Were Not Always Complete or Accurate, and VAMCs Applied Instructions for Gathering Suicide Data Differently
paragraphs: We found that demographic, clinical, and other data submitted to VA  Central Office on veteran suicides were not always completely or  correctly entered into the BHAP Post-Mortem Chart Analysis Templates— a mechanism by which VA Central Office collects veteran suicide data  from VAMCs’ review of veterans’ medical records. (Figure 1 shows the  number of BHAP templates we found with incomplete or inaccurate data.)  Moreover, VAMCs interpreted and applied instructions for completing the  BHAP templates differently. We also found that most VAMCs and VISNs  we reviewed and VA Central Office did not review suicide data for  accuracy.
subsections: section_title: Veteran Suicide Data Are Incomplete
paragraphs: We found that over half of the 63 BHAP templates we examined had  incomplete information.information, or other specific fields were omitted. Moreover, the data were  lacking entirely for certain known veteran suicides. Incomplete data limits  VA Central Office’s ability to identify information that can be used to help  VA Central Office develop policy and procedures to prevent veteran  deaths.

The data either lacked veteran enrollment  Lack of veteran enrollment information. Approximately one-third (23)  of the BHAP templates we reviewed did not indicate whether the veteran  was enrolled in VA health care services, even though the veteran had a  VA medical record.

Eight did not indicate that the veteran had received VA services when  the templates were submitted by three of the VAMCs in our review,  even though these VAMCs provided care to these veterans.

Fifteen BHAP templates submitted by two VAMCs in our review  originally indicated that the veteran was receiving VA care; however,  when we reviewed the submitted BHAP templates we received from  VA Central Office for the same 15 veterans, the BHAP templates did  not indicate that the veteran was being seen in the VA.

VA Central Office used enrollment information when compiling the most  recent BHAP interim report, which is part of VA Central Office’s quality  improvement efforts for its suicide prevention program. Specifically, VA Central Office included clinical data in the BHAP interim report only for  veterans utilizing VA services. Therefore, clinical data for the 23 veterans  we identified would not be included in the interim report. Missing one-third  of the data from its analysis, as was the case in our sample, could have a  detrimental effect on the trends VA Central Office reports and uses to  improve its suicide prevention efforts.

Requested data was omitted. Forty of the 63 BHAP templates we  reviewed included various data fields where no response was provided,  resulting in incomplete data. For example, for 19 templates, VAMC staff  did not enter requested data as to whether the veteran had all or some of  15 active psychiatric symptoms within 12 months prior to the veteran’s  date of death. Also, 9 templates did not include an answer for the number of previous suicide attempts by the veteran. Officials from one  VAMC told us that they left this field blank if the veteran did not have any  previous suicide attempts, rather than entering a “0,” even though the  BHAP Guide states that officials should enter the appropriate number of  previous suicide attempts. Officials at one VAMC told us that fields are  sometimes left blank if the standardized answers available on the BHAP  template are not adequate; that is, the answer for that veteran does not fit  into one of the answers provided on the BHAP template. Officials at two  VAMCs stated that it is sometimes easy to overlook fields in the BHAP  template, resulting in unanswered questions.

Filling in all fields in the BHAP template, rather than leaving the field  blank, is important because some blank fields are counted as “missing” or  “no” in the analysis conducted by VA Central Office for the BHAP interim  reports. This, in turn, could affect the suicide trends reported. For  example, for the number of previous suicide attempts, blank fields are  counted as “missing” in the BHAP interim report, rather than “0” previous  suicide attempts as officials from one VAMC intended. In other cases,  such as for psychiatric symptoms, missing fields are counted as “no,”  meaning that the veteran did not have these symptoms. In at least one  BHAP template, the answer for the psychiatric symptom of isolation was  left blank, and would therefore be counted as negative in the interim  report despite the fact that officials from the one VAMC told us that the  veteran did have this symptom. See figure 2, which provides an excerpt  of the fields from the BHAP template in which VAMCs provided  incomplete data.

Data were lacking entirely for certain known veteran suicides. We  found that VAMCs did not always submit BHAP templates for all veteran  suicides known to the facility, as required by the BHAP Guide. VA Central  Office does not have a process in place to determine whether it is  receiving the BHAP templates for all known veteran suicides. For  example, one VAMC had completed 13 BHAP templates at the time of  our site visit but had not submitted them; however, neither the VAMC nor  VA Central Office were aware that these templates had not been  submitted until after we requested them from VA Central Office. The  suicide prevention coordinator at this VAMC told us that the BHAP  templates were forwarded to another official at the VAMC, rather than  being submitted through VA Central Office’s process, and that the BHAP  templates were never submitted. As a result of our inquiry, the VAMC  submitted these templates to VA Central Office. In another example,  officials at a different VAMC told us that, at the time of our site visit, they  had recently begun completing and submitting BHAP templates,  beginning with veteran suicides occurring in fiscal year 2014. VA Central  Office officials told us that VAMCs can start submitting BHAP templates  at any point, and officials are not requiring the VAMCs to go back and  submit information on all suicides since October 1, 2012. However, this  practice is contrary to VA policy, which states that VAMCs should submit  BHAP templates for all suicides known to the facility and reported on or  after October 1, 2012.

section_title: Veteran Suicide Data Are Inaccurate
paragraphs: Of the 63 BHAP templates we reviewed, we found numerous instances of  inaccurate data submitted on BHAP templates, as illustrated by the  following examples.

Incorrect date of death: Six BHAP templates included a date of  death that was incorrect based on information in the veteran’s medical  record. The difference in the dates of death in the veterans’ medical  records and the dates of death in the BHAP templates ranged from   1 day to 1 year. For example, one BHAP template indicated that the  veteran died in the year after the veteran’s actual date of death.  Another BHAP template appeared to use the date the suicide  behavior report was completed, rather than the veteran’s actual date  of death. The suicide behavior report was completed 69 days after the  veteran’s date of death.in the BHAP template is important because it is used as a point of  reference to calculate other fields, such as the number of mental  health visits in the last 30 days.

The accuracy of the date of death recorded  Incorrect number of mental health visits: Nine BHAP templates  included the incorrect number of outpatient VA mental health visits in  the last 30 days.  For example, one BHAP template indicated that  the veteran had five outpatient mental health visits, including three  non-mental health visits that should not have been included in the  total number of mental health visits for this veteran. Another BHAP  template indicated the veteran had been seen once by a mental  health provider in the last 30 days; however, we found in reviewing  the medical records that this veteran had not been seen by a mental  health provider during this time period. This veteran would be included  in the BHAP interim report as having a mental health visit, and, as a  result, VA’s data would include an inaccurate count of the number of  veterans with mental health visits in the last 30 days.accurate information, VA cannot use this information to determine  whether policies or procedures need to be changed to ensure that   Without  veterans at high risk for suicide are being seen more frequently by a  mental health provider to help prevent suicides in the future.

See figure 3, which provides an excerpt of the fields from the BHAP  template in which VAMCs provided inaccurate data.

section_title: VAMCs Have Interpreted and Applied Instructions for Completing the BHAP Templates Differently
paragraphs: We found several situations where VAMCs interpreted and applied  instructions for completing the BHAP templates differently, as illustrated  in the following examples.

We found inconsistencies in how different VAMCs arrived at answers  provided in the BHAP templates. For example, one VAMC included a  visit to an immunization clinic as the veteran’s final visit, while another  VAMC did not include this type of visit, even though this was the last  time the veteran was seen in person. The BHAP Guide indicates that  the final visit should be the last time the veteran had in-person contact  with any VAMC staff, but the BHAP Guide does not identify the  different types of visits that should be counted. VA Central Office officials stated that a visit to an immunization clinic should be included  as the final visit with the veteran. When VAMCs do not provide  consistent data, VA Central Office will receive and use inconsistent  data in preparing its trend reports, such as BHAP interim reports,  which are intended to be used to improve suicide prevention efforts.

We also found instances in which BHAP templates included  information that did not conform to the instructions in the BHAP Guide  on how to complete the BHAP medical record reviews.

Last contact did not always represent the last time a VAMC  official spoke with the veteran: The BHAP Guide instructions  specify that the last contact recorded in the BHAP template should  be the last recorded interaction with the veteran, which could be in  person, through a phone call, or through email. Five of the 63  BHAP templates we reviewed did not indicate the last time an  official spoke directly to the veteran. One BHAP template counted  a phone call with a veteran’s spouse after the veteran’s death as  the last contact with the veteran. The BHAP template also counted  this phone conversation as an “in-person” interaction. The  remaining four BHAP templates included a date for the last  contact that was prior to the date for the veteran’s final in-person  visit at the VAMC. In these instances, the veterans’ in-person visit  should have been counted as the last contact. From this flawed  information, VA would not be able to determine reliable trends for  the amount of time between the last contact with the veteran and  the veteran’s date of death for reports that it prepares, such as the  BHAP interim report.

Suicide prevention coordinator contact and referral not within  BHAP time period: The BHAP Guide specifies that VAMCs  should indicate in the BHAP template whether there was a suicide  prevention coordinator contact or referral made within 3 months  prior to the veteran’s date of death. In 3 of the 63 cases we  reviewed, we found that the suicide prevention coordinators  checked the box indicating that they saw the veteran or had a  referral within 3 months of the veteran’s death. However, in each  of these cases we found that the contact was made more than   3 months prior to the veteran’s death, so it should not have been  counted. A suicide prevention coordinator from one VAMC said  she was unaware of the time period requirement and a suicide  prevention coordinator at another VAMC stated that time frames  should be added to the BHAP template, rather than just included  in the BHAP Guide. The BHAP interim reports include the number  of veterans that had a suicide prevention coordinator contact or  referral, and by including information on contacts or referrals that  are outside the BHAP Guide time frame, these reports may be at  risk of misreporting trends in this area.

See figure 4, which provides an excerpt of the fields from the BHAP  template in which VAMCs provided inconsistent data.

VA policy and guidance states that the BHAP template should be  completed for all suicides known to the facility, but at the five VAMCs we  visited, these data were not always being reported. However, the policy  and instructions do not explicitly state that veterans not being seen by VA  should be included, and in the absence of this declaration, some VAMCs  interpreted the instructions to mean that only veterans being seen by VA  should be included in the data submitted. Therefore, two VAMCs have  submitted data only for veterans being treated by VA, while the others  include data on all known veteran suicides—whether they have been  treated by VA or not. This further adds to the inconsistencies in the  information that VAMCs submit on the BHAP templates. VA Central  Office officials told us that BHAP templates should be completed for both  veterans utilizing VA health care services, as well as those veterans not  being seen in the VA, and that this requirement has been discussed at  training sessions and during conference calls with suicide prevention  coordinators. For example, during a suicide prevention conference in  November 2013, a VA Central Office official informed participants that the  BHAP template should be completed for all suicides reported through  SPAN, which VA Central Office officials previously told us includes  veterans that were not receiving VA care. The inconsistency in VAMC  officials’ understanding of which veterans should have a completed BHAP  template results in inconsistent data being reported to VA Central Office.  While VA was in the process of updating its suicide prevention  coordinator manual, we brought this issue to VA’s attention. In August  2014, VA made modifications to the manual that indicated that VA is  changing its policy—now requiring that the BHAP template should be  completed only for veterans receiving VA services. However, the  guidance continues to be unclear on whether suicide prevention  coordinators should complete BHAP templates for veterans not receiving  VA care.

section_title: VAMCs, VISNs, and VA Central Office Do Not Review Suicide Data
paragraphs: We found that BHAP templates are not being reviewed by VA officials at  any level for accuracy, completeness, and consistency. Therefore, our  findings at five VAMCs could be symptomatic nationwide and other  VAMCs may also be submitting incomplete, inaccurate, and inconsistent  suicide-related information and VA may not be getting the data it needs  across the Department to make appropriate resource decisions and  develop new policy. VA policy states that it is the VISN’s and VAMC’s  decision whether to conduct reviews of BHAP data prior to submission to  VA Central Office. With few exceptions, VAMCs and VISNs we visited  generally do not conduct data checks on the information submitted in the  BHAP templates. Additionally, VA Central Office does not review the  information for accuracy and completeness in the BHAP templates it  receives. This approach is inconsistent with internal control standards for  the federal government, which state that agencies should have controls  over information processes, including procedures and standards to  ensure the completeness and accuracy of processed data.

Officials at one VAMC told us that VAMC staff compare the BHAP data  and the veteran’s medical record prior to submitting the BHAP template to  VA Central Office to ensure accuracy. In response to our review, another  VAMC implemented a procedure to check the accuracy and  completeness of their BHAP templates prior to submission. The  procedure at this VAMC requires the suicide prevention coordinator and  case manager to independently complete the BHAP template and  compare their responses. The BHAP templates are then reviewed by the  Assistant Mental Health Clinic Director prior to submission.

We also found that VA lacks sufficient controls to ensure the quality of the  existing BHAP data. For example, VA Central Office officials said there  are no automated data checks to ensure the accuracy of data it uses for  the BHAP interim report, such as checking to ensure that the date of last  contact with the veteran that is recorded in the BHAP template is not after  the veteran’s date of death. Although officials removed apparent  duplicates in submitted BHAP templates by matching the veteran’s name  and social security number while compiling the data for the most recent  BHAP interim report, they do not conduct data checks to help identify  some of the incomplete or inaccurate data we found in our review.

section_title: Conclusions
paragraphs: Given the negative effects of MDD, it is important to provide timely,  evidence-based treatment for veterans with MDD, and VA’s ability to  monitor these veterans is critical to ensuring positive outcomes. However,  our findings demonstrate that VA may not be fully aware of the population  of veterans with MDD due to a lack of coding precision by clinicians. This  can limit VA’s ability to assess the Department’s performance in treating  veterans as recommended in the MDD CPG and in measuring health  outcomes for veterans. Additionally, VA does not have mechanisms in  place to ensure that the Department is able to identify deviations from  CPG-recommended care and remedy those that could impede veterans’  recovery. Even if VA did have mechanisms in place, the coding  discrepancies we identified would limit VA’s ability to extract accurate  data on all veterans diagnosed with MDD, therefore hindering VA’s ability  to determine the extent to which veterans are receiving care consistent  with the CPG recommendations for MDD. The CPG recommendations  are meant to improve veteran outcomes by providing maximum relief from  the debilitating symptoms of MDD, and VA cannot ensure that the care  veterans receive is consistent with those recommendations.

The existence of incomplete, inaccurate, and inconsistent information  submitted through VA’s BHAP templates limits the Department’s ability to  accurately evaluate its suicide prevention efforts and identify trends in  veteran suicides through the BHAP initiative. Specifically, data drawn  from incomplete, inaccurate, and inconsistent BHAP templates limit the  Department’s opportunities to learn from past veteran suicides and  ultimately diminish efforts to improve its suicide prevention activities.  VAMCs, VISNs, and VA Central Office generally lack a process to ensure  that the data that are submitted and used by VA Central Office to identify  trends in veteran suicides are complete, accurate, and consistent.  Checking and verifying the data submitted to VA Central Office would  help ensure that changes made to suicide prevention efforts by VAMCs,  VISNs, and VA are based on actual trends in veteran suicides. Without  clear VA Central Office instructions to guide how VAMCs and VISNs  should complete BHAP templates and report suicide data, the validity of  suicide data and the effectiveness of VA’s actions will be hampered.

section_title: Recommendations for Executive Action
paragraphs: We recommend that the Secretary of Veterans Affairs direct the Under  Secretary for Health to take the following six actions:  To more accurately estimate the prevalence of MDD and identify enrolled  veterans with MDD, VA should  identify the extent to which there is imprecise diagnostic coding of  MDD by further examining encounters with a diagnostic code of  depression not otherwise specified, which could be incorporated into  VAMCs’ ongoing review of diagnostic coding accuracy, and determine and address the factor(s) contributing to the imprecise  coding based on the results of those examinations. For example,  feedback and additional training could be provided to clinicians  regarding the importance of diagnostic code accuracy, or VA’s  medical record could be enhanced to facilitate the selection of a more  accurate diagnostic code.

To ensure that veterans are receiving care in accordance with the MDD  CPG, VA should  implement processes to review data on veterans with MDD prescribed  antidepressants to evaluate the level of risk of any deviations from  recommended care and remedy those that could impede veterans’  recovery.

To improve VA’s efforts to inform its suicide prevention activities, VA  should ensure that VAMCs have a process in place to review data on veteran  suicides for completeness, accuracy, and consistency before the data  are submitted to VA Central Office,  clarify guidance on how to complete BHAP templates to ensure that  VAMCs are submitting consistent data on veteran suicides, and  implement processes to review data on veteran suicides submitted by  VAMCs for accuracy and completeness.

section_title: Agency Comments
paragraphs: We provided a draft of this report to VA for comment. In its written  comments, reproduced in appendix IV, VA generally agreed with our  conclusions and concurred with our recommendations. In addition, VA  provided information on its plans for implementing each recommendation,  with estimated completion dates in calendar year 2015.

As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report’s date. At that time, we will send copies of this report to the  appropriate congressional committees; the Secretary of Veterans Affairs;  the VA Under Secretary for Health; and other interested parties. In  addition, the report will be available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff have any questions, please contact me at (202) 512- 7114 or williamsonr@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. Major contributors to this report are listed in appendix V.

section_title: Appendix I: Scope and Methodology
subsections: section_title: Characteristics of Veterans Diagnosed with Major Depressive Disorder (MDD)
paragraphs: To describe the characteristics of veterans diagnosed with MDD from  fiscal years 2009 through 2013, we analyzed Department of Veterans  Affairs (VA) and Department of Defense (DOD) data. (See table 3.) These  data included information on veterans’ demographic characteristics as  well as clinical information on health care services and medications  provided through VA. Veterans were classified as having a diagnosis of  MDD if, in at least one fiscal year included in our review, they had two or  more outpatient encounters or at least one inpatient hospital stay with a  diagnosis of MDD.

Specifically, we examined the following:

Number of veterans diagnosed with MDD. We used a demographic  file provided by VA to determine the number of veterans diagnosed  with MDD.

Characteristics of veterans diagnosed with MDD. We used  demographic files provided by VA and DOD to describe  characteristics of veterans diagnosed with MDD. In particular, the  veteran characteristics we examined included the following:

Age. We created seven categories for veterans’ ages as of  September 30, 2013—the end of fiscal year 2013, which  corresponds to the last date of data we included in our analysis.  These categories are as follows: (a) 18-24, (b) 25-34, (c) 35-44,  (d) 45-54, (e) 55-64, (f) 65-74, and (g) 75 and older.

Era of service. We categorized veterans as either being veterans  of the recent conflicts in Iraq and Afghanistan—Operation Iraqi  Freedom, Operation Enduring Freedom, and Operation New  Dawn—or of other eras of service.

Sex. We categorized veterans as either being female or male.

Race and ethnicity. We created categories to describe veterans’  race and ethnicity (Hispanic and non-Hispanic). These categories  are consistent with the Office of Management and Budget’s 1997  Standards for Maintaining, Collecting, and Presenting Federal  Data on Race and Ethnicity.

Extent to which veterans diagnosed with MDD were prescribed at  least one antidepressant. Using data from the Pharmacy Benefits  Management database, we examined the extent to which VA  providers prescribed at least one antidepressant for veterans  diagnosed with MDD from fiscal years 2009 through 2013. This  includes antidepressants prescribed to treat depression as well as  those prescribed to treat other conditions.

The percentage of veterans with MDD dispensed a 12-week and a  6-month supply of an antidepressant. Using VA data we obtained  from the Medical SAS Inpatient Datasets, Acute Care Dataset;  Outpatient Encounter Files; Fee Basis Outpatient and Inpatient  Services Files; and Pharmacy Benefits Management Database, we  calculated the percentage of veterans with MDD dispensed a 12-week  and a 6-month supply of an antidepressant according to statistical  programming logic provided by VA. These measures are intended to  assess the effectiveness of antidepressant medication management  and are based on performance measures developed by the National  Committee for Quality Assurance. In addition, these measures are  consistent with the VA/DOD Clinical Practice Guideline for  Management of Major Depressive Disorder, which indicates that  continued antidepressant treatment, after acute depressive symptoms  have resolved, decreases the incidence of relapse of MDD.

We selected six VA medical centers (VAMC) at the following locations to  visit: Canandaigua, New York; Gainesville, Florida; Iowa City, Iowa;  Philadelphia, Pennsylvania; Phoenix, Arizona; and Reno, Nevada.  These  VAMCs represent different facility complexity groups, serve populations of  veterans that differ in terms of the extent of use of mental health services,  and are located in different Veterans Integrated Service Networks (VISN),   To gather additional perspectives, for each  or regional networks of care.VAMC we visited, we selected one associated community-based  outpatient clinic to visit. In particular, we visited community-based  outpatient clinics in the following locations: Cedar Rapids, Iowa; Globe,  Arizona; Gloucester, New Jersey; Lecanto, Florida; Fallon, Nevada; and  Rochester, New York. (See table 4.) As part of our site visits, we  reviewed a nongeneralizeable sample of five medical records for each of  these six VAMCs for a total of 30 veterans. We reviewed these medical  records to determine if the diagnostic code entered for all encounters— starting with the initial encounter in 2012 when the veteran was  diagnosed with MDD and prescribed an antidepressant—was consistent  with a diagnosis of MDD.

To select medical records for review, we completed the following steps:

Randomly generated a list of individuals with a new prescription for an  antidepressant in calendar year 2012.

Selected the first five individuals in the list that met the following

Veteran status.

Had a diagnosis of MDD in calendar year 2012. For the purposes  of medical record reviews, we classified a veteran as having a  diagnosis of MDD if, based on how the veteran’s patient care  encounters were coded or on the narrative contained in clinical  notes in the veteran’s medical record, the veteran had (a) at least  two outpatient encounters with a diagnosis of MDD, or (b) at least  one inpatient stay with a diagnosis of MDD.

Had a new treatment episode of antidepressants in calendar year  2012. New treatment episodes were defined as an initiation of  antidepressant treatment following a period during which the  veteran was either (1) not prescribed an antidepressant or (2)  noncompliant with and had not picked up prescriptions for a  previously prescribed antidepressant.

To ensure the reliability of the data we analyzed, we interviewed VA  Central Office officials, reviewed relevant documentation and veterans’  medical records, and conducted electronic testing to identify missing data  and obvious errors. On the basis of these activities, we determined that  the data we analyzed were sufficiently reliable for our purposes. However,  as discussed in the report, we described limitations of the data due to the  coding discrepancies we found.

section_title: VA’s Oversight of the Extent to Which Veterans with MDD Prescribed Antidepressants Are Receiving Care As Recommended in the CPG
paragraphs: To examine the extent to which VAMCs are providing care to veterans  with MDD who are prescribed antidepressants as recommended in the  CPG, we reviewed relevant VA policy documents. On the basis of that  review, we found that VA policy requires all care sites, VAMCs, and  community-based outpatient clinics to provide evidence-based  antidepressant treatment when indicated for depression and that such  care must be consistent with current VA clinical practice guidelines. The  relevant VA clinical practice guideline, the VA/DOD Clinical Practice  Guideline for Management of Major Depressive Disorder, provides  evidence-based recommendations for providers on how to monitor  veterans prescribed antidepressants; these recommendations are based  on a review of depression research outcomes. These recommendations  are based on available research at the time of publication of the guideline  and are intended to provide information to assist providers in treatment  decision-making.monitoring veterans prescribed antidepressants, we judgmentally  selected three recommendations for inclusion in our review. In particular,  we selected recommendations that (1) had among the highest strength of  research evidence, (2) were sufficiently specific to enable us to determine  the extent to which VA providers were following the recommendation, and  (3) would not require clinical judgment to determine the extent to which  VA providers were following the recommendation. The following  recommendations were included in our review:   From the guideline’s recommendations related to

To enhance antidepressant treatment, veterans should be educated  on when to take the medication, possible side effects, risks, and the  expected duration of treatment, among other things;

Standardized assessments of depressive symptoms, such as the  Patient Health Questionnaire-9, should be used to monitor treatment  response at 4-6 weeks after initiation of treatment, after each change  in treatment, and periodically thereafter until full remission is  achieved; and

A plan should be developed that addresses the duration of  antidepressant treatment, among other things.

After selecting these recommendations for our review, we examined the  extent to which veterans were receiving care consistent with these CPG  recommendations at the six VAMCs we visited. To do this, we  interviewed VAMC clinicians to determine whether and how they were  following these recommendations. At each VAMC, officials interviewed  included members of the executive leadership team, primary care and  mental health providers, and pharmacists.

Additionally, as part of our examination of the extent to which VAMCs are  providing care consistent with the selected guideline recommendations,  we reviewed the sample of five veterans’ medical records per VAMC used  as part of our review of MDD coding. For each medical record, we  reviewed documentation contained in the selected veterans’ medical  records to assess the extent to which the antidepressant treatment- related care VA providers rendered was consistent with the selected CPG  recommendations included in our review. Our review commenced with  the encounter during which a VA clinician ordered an antidepressant to  treat depressive symptoms. Our review ended after five follow-up  encounters with a VA clinician, or sooner if the veteran did not have five  follow-up encounters. Our review was limited to encounters during which  the antidepressant treatment was reviewed, including encounters during  which side effects and treatment effect were assessed, but no change  was made to medication orders. We did not include, for example, an  encounter with an orthopedist during which the fact that the veteran had  been prescribed an antidepressant was simply noted. We provided the  VAMCs with the instances where we found the medical record  documentation was not consistent with the selected CPG  recommendations. The VAMCs confirmed our answers or provided  additional support if they believed the care was consistent with the CPG.

To examine VA’s oversight of the care VAMCs provide to veterans with  MDD who are prescribed an antidepressant, we reviewed VA’s oversight  of the Uniform Mental Health Services in VA Medical Centers and Clinics  handbook and CPG requirements and evaluated whether this oversight  provides VA with adequate information to identify nonconformance with  recommended practices, assess the risk of any nonconformance, and  address nonconformance, as appropriate. As part of this review, we  reviewed VA’s oversight in the context of federal standards for internal  control for risk assessment.refers to an agency’s ability to comprehensively identify risks, assess the  possible effect, if any, and determine what actions should be taken to  mitigate significant risks.

The internal control for risk assessment  We then interviewed officials from VA Central Office, including officials  from the Office of Mental Health Services (OMHS), the Office of Mental  Health Operations, and the Office of Analytics and Business Intelligence;  and the six VISNs that oversee the VAMCs we visited who are  responsible for overseeing compliance with VA’s requirements, including  VA’s requirement that all VA facilities provide evidence-based  antidepressant treatment when indicated for depression and that such  care be consistent with current VA clinical practice guidelines. Through  our interviews, we obtained information on the oversight activities  conducted by VA Central Office and the extent to which VA Central Office  followed up with VAMCs to ensure that they corrected problems identified  through these oversight activities. In addition, we obtained and reviewed  relevant documents regarding VA oversight, including internal reports and  VAMCs’ plans to correct problems identified through oversight activities.

section_title: Information VA Requires VAMCs to Collect on Veteran Suicides
paragraphs: To analyze the information VA requires VAMCs to collect on veteran  suicides, we first reviewed VA policies, guidance, and documents related  to VA’s suicide prevention efforts to identify the mechanisms by which VA  collects veteran suicide data from VAMCs. We also interviewed VA  Central Office and other officials responsible for VA’s suicide prevention  program, including officials from OMHS and the Center of Excellence for  Suicide Prevention. We also interviewed VAMC officials and relevant staff  of the six VISNs for the sites we visited to obtain information on suicide  prevention initiatives.

Next, through the site visits to six VAMCs, we obtained documents and  interviewed officials regarding the collection of veteran suicide data. We  obtained all completed templates from the Behavioral Health Autopsy  Program (BHAP) related to VA’s collection of data on veterans that died  by suicide as of the time of our site visit or at the time we requested the   One VAMC had not completed any of  documents for virtual site visits.these BHAP documents because they had not had a veteran die by  suicide since the beginning of the program. Therefore, our analysis  includes a review of documents from five of the six VAMCs we visited.  Through review of the documents, we noted any fields missing data, such  as a field that requires a yes or no answer but neither answer is provided.

Additionally, using professional judgment, we identified fields in the  documents to review based on whether the field related to aspects of VA  treatment—including treatment for mental health conditions—and the  date of the veteran’s death. We identified these fields because they did  not require clinical judgment to assess. Using the parameters in the  corresponding guide for filling out these documents, including time  frames, we compared these fields to information included in the veteran’s  medical record and noted differences between our answers and the  answers provided by the VAMCs in the documents.

To ensure that we received the final, submitted versions, we also  requested these documents from VA Central Office for each of the five  VAMCs. We compared these documents to the documents we received  from the VAMCs. We used the documents from the VAMCs as the  starting point; therefore, we only analyzed the templates for veterans  identified by the VAMCs. that the template for these documents had changed over time. If  additional fields were included in the templates obtained from VA Central  Office, but were not originally included in the templates obtained from the  VAMCs, we did not review these fields. We generally used the answer  from the document obtained from VA Central Office, which is the final  submitted version, unless a field originally had an answer in the template  from the VAMC, but was blank or not answered in the template from VA  Central Office. In those cases, we used the answer from the VAMC  document.

During the course of our review we learned  We provided the VAMCs with the fields where the answers in the VAMC’s  documents did not match our answers based on our review of the medical  record. The VAMCs confirmed our answers or provided additional support  for their original answer. Results from our review of veteran suicide data  can be generalized to the VAMCs we visited, but cannot be generalized  to other VAMCs.

We received additional templates from VA Central Office, but these were not analyzed  because the VAMC had not provided us with templates for these veterans.

We conducted this performance audit from November 2013 to November  2014 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

section_title: Appendix II: Department of Veterans Affairs Medical Centers’ Tracking of Veterans at High Risk for Suicide
paragraphs: The Department of Veterans Affairs’ (VA) Uniform Mental Health Services  in VA Medical Centers and Clinics handbook (Handbook) requires VA  medical centers (VAMC) to have a suicide prevention coordinator whose  responsibilities include establishing and maintaining a list of veterans  assessed to be at high risk for suicide and monitoring these veterans.

The Handbook also requires suicide prevention coordinators to ensure  that providers follow up on missed appointments for high-risk veterans to  ensure patient safety and in order to initiate problem-solving about any  tensions or difficulties in the veteran’s ongoing care.

Whether a veteran is determined to be at high risk for suicide is based on  clinical judgment made after an evaluation of risk factors—such as history  of past suicide attempts or recent discharge from an inpatient mental  health unit—protective factors—such as positive social support, positive  coping skills, and positive therapeutic relationships—and the presence or  absence of warning signs. Indicators that a veteran is at high risk for  suicide include a current verified report or witnessed suicide attempt;  identification of current serious suicidal ideation that requires an  immediate change in the treatment plan, such as hospitalization; and the  presence of any of the following warning signs: threatening to hurt or plan  to kill oneself; looking for specific ways to kill oneself and seeking access  to such means, such as pills or weapons; and talking or writing about  death, dying, or suicide when these actions are out of the ordinary for the  person.

The Handbook requires each VAMC to have a process for establishing a  patient record flag to help ensure that veterans determined to be at high  risk for suicide are provided with follow up for all missed mental health  and substance abuse appointments. The primary purpose of the patient  record flag is to communicate to staff that a veteran is at high risk for  suicide and VA policy states that the presence of a flag should be  considered when making treatment decisions. Suicide prevention  coordinators are responsible for assessing, in conjunction with the  treating clinician, the risk of suicide in individual veterans, ensuring these  veterans have a “High Risk for Suicide” Patient Record Flag on their  medical record, and reviewing the list of high-risk veterans at least every  90 days.

We interviewed suicide prevention coordinators as part of our site visits  with six VAMCs to obtain information on how they track veterans  determined to be at high risk for suicide. At four of the VAMCs we visited,  suicide prevention coordinators used an electronic spreadsheet to track  information on these veterans. For example, the spreadsheets include  information such as whether the veteran has a patient record flag on their  medical record and when the flag needs to be reviewed, the date for the  veteran’s next scheduled follow-up appointment, whether the veteran has  a safety plan, and the veteran’s assigned psychiatrist. Officials from one  VAMC told us that they maintain the list daily, adding and removing  veterans as necessary. Officials stated that the circumstances under  which a veteran would be removed from the spreadsheet varied, but  veterans are generally removed because their patient record flag has  been removed and the officials no longer consider the veteran to be at  high risk for suicide.

The two remaining VAMCs use other mechanisms to track veterans at  high risk for suicide. Officials from one VAMC told us that they use the  Suicide Prevention Application Network (SPAN) to query high-risk  patients at the VAMC. The SPAN database contains veteran information,  demographic characteristics, and information on suicide attempts and  completed suicides, among other things. According to officials, the  information for each veteran in SPAN includes the date the veteran was  assessed as being at high risk, as well as the date that the veteran needs  to be seen for follow up, if applicable. After our site visit, officials told us  they plan to periodically pull a list of all veterans with an active high risk  flag in VA’s medical record for the VAMC and cross-reference that list to  veterans being tracked for high suicide risk by the suicide prevention  coordinator in SPAN to ensure all high-risk veterans are tracked. Officials  from the other VAMC told us that their case managers each have their  own list of veterans that they track and the suicide prevention coordinator  we spoke with stated that he does not keep a master list of all veterans  that are at high risk for suicide.

section_title: Appendix III: Department of Veterans Affairs’ Use of Data Related to Suicides and Suicide Behavior
paragraphs: Veterans Affairs medical centers (VAMC) collect and submit data on  veteran suicides to the Department of Veterans Affairs (VA) Central Office  through the Suicide Prevention Application Network (SPAN) and suicide  behavior reports. VAMCs also collect and report data through root cause  analyses. Additionally, VA Central Office uses the data from SPAN to  prepare reports that are sent to the VAMCs and Veterans Integrated   VA Central Office officials stated that they  Service Networks (VISN).expect VAMCs and VISNs to use these reports and collected data to  improve suicide prevention efforts and program evaluation. Through site  visits at six VAMCs that we conducted as part of our review and through  interviews with corresponding VISN officials, we identified examples of  how some VAMCs and VISNs are utilizing veteran suicide data to  improve their suicide prevention efforts.

VAMC and VISN officials have used SPAN to create initiatives based  on trends in the data. For example,

Officials at one VAMC stated that they use the information  collected in SPAN to provide data for performing statistical  analyses on the outreach conducted, to study suicide attempts  and completions across the VAMC catchment area, to understand  the means by which veterans are dying by suicide, and to study  the use of high-risk flags.

Officials at a VISN explained that through a review of the SPAN  data about a year ago, officials learned that 60 percent of suicides  in the VISN were completed using a gun. After conducting  research on the subject, the VISN began a firearm safety initiative,  which includes notifying veterans by mail that they can receive  four gun locks each upon request, with no questions asked.

VAMC officials have made programmatic changes to their suicide  prevention efforts based on trends in the suicide data they are  collecting and reviewing. For example,

At one VAMC, officials told us that they reviewed suicide behavior  reports and, as a result of trends identified in these reports,  drafted a policy for medication restriction for veterans at risk of  overdosing. Specifically, over a 3-year period, five or six veterans  receiving VA care repeatedly attempted suicide by overdose,  typically when they were intoxicated. VAMC officials created a  work group to draft policy that mitigates risk for medication  overdose among high-risk veterans. At the time of our site visit,  the group was exploring creating a patient record flag that would  be included in the veteran’s medical record for overdose risk  indicating that medication supplies should be restricted for these  veterans and the possibility of using automated pill dispensers to  dispense medications to these veterans.

Through their work reviewing suicide-related information, the  suicide prevention team at another VAMC identified a trend in its  suicide data. In particular, they noted that some veterans were  given a 90-day supply of the same medications that the veteran  recently tried to use to overdose. The suicide prevention team  mentioned this to a clinical pharmacist who had also noticed this  issue. The VAMC is now trying to restrict days of supply for these  types of veterans, but there is no formal policy about this and no  plans to craft such a policy. Additionally, officials from this VAMC  stated that they have added items to the standardized suicide  behavior report template to help them to collect additional useful  information, such as active medications and pain score at the time  of the last visit.

Officials from one VAMC stated that through a review of medical  records and autopsy reports for veterans who died by suicide,  they found that a vast majority of veterans who died by suicide  were not being seen by a mental health provider. In response,  officials provided education to primary care providers. VAMC  officials also noticed that veterans receiving care for pain were  dying by suicide at a high rate. As a result, the VAMC has started  an initiative with the pain clinic, and, as part of this initiative, the  chief of the pain management clinic consults with psychiatry on  veterans at risk for suicide.

Officials at a VISN described changes made in response to the  suicide data in fiscal year 2012, which showed that a percentage of  veterans who completed suicide had no ongoing mental health care.

These veterans mainly received care from VA primary care providers.  To address this, the VISN partnered with the Center of Excellence for  Suicide Prevention and local university psychologists to help VA  primary care providers at community-based outpatient clinics  formulate mental health plans.

section_title: Appendix IV: Comments from the Department of Veterans Affairs

section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
paragraphs: Randall B. Williamson, (202) 512-7114, williamsonr@gao.gov.

section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Marcia A. Mann, Assistant  Director; Emily Binek; Muriel Brown; Stella Chiang; Cathleen Hamann;  Melanie Krause; Daniel Lee; Lisa Opdycke; Sarah Resavy; and   Jennifer Whitworth made key contributions to this report.


Question-summary annotations: questions: question: What does GAO's analysis of VA data for fiscal years 2009 through 2013 show?
answer: GAO's analysis of Department of Veterans Affairs (VA) data for fiscal years 2009 through 2013 shows that about 10 percent of veterans who received VA health care services were diagnosed with major depressive disorder (MDD).
child_questions: question: What is MDD?
answer: MDD is characterized by depressed mood or loss of interest along with other symptoms for 2 weeks or more that represent a change in the way individuals function from their previous behaviors.

question: What might GAO's discovery of diagnostic coding discrepancies indicate?
answer: Because GAO found diagnostic coding discrepancies in 11 of the 30 veterans' medical records it reviewed from six VA medical centers (VAMC), VA's data may understate the prevalence of MDD among veterans being treated through VA, to the extent that such discrepancies may permeate VA's data.

question: How can MDD be treated?
answer: One treatment for MDD is the use of medications such as antidepressants.
child_questions: question: How does GAO's analysis support this claim?
answer: According to GAO's analysis, 94 percent of veterans diagnosed with MDD were prescribed at least one antidepressant.

questions: question: How did treatment for veterans with MDD in GAO's review differ from VA's CPG recommendations?
answer: VA policy states that antidepressant treatment must be consistent with VA's current clinical practice guideline (CPG); however, GAO's review of 30 veterans' medical records identified deviations from selected MDD CPG recommendations for most veterans reviewed.
child_questions: question: What data did GAO find to support this claim?
answer: For example, 26 of the 30 veterans were not assessed using a standardized assessment tool at 4 to 6 weeks after initiation of treatment, as recommended in the CPG. Additionally, 10 veterans did not receive follow up within the time frame recommended in the CPG.

question: Why might treatment for MDD differ from VA recommendations?
answer: GAO found that VA does not have a system-wide process in place to identify and fully assess whether the care provided is consistent with the CPG.
child_questions: question: How could the lack of a system-wide process negatively affect the VA's oversight?
answer: As a result, VA does not know the extent to which veterans with MDD who have been prescribed antidepressants are receiving care as recommended in the CPG and whether appropriate actions are taken by VAMCs to mitigate potentially significant risks to veterans.

questions: question: What issues did GAO find with VA's data on veteran suicides?
answer: The demographic and clinical data that VA collects on veteran suicides were not always complete, accurate, or consistent.
child_questions: question: What steps has the VA taken to amend these issues?
answer: VA's Behavioral Health Autopsy Program (BHAP) is a quality initiative to improve VA's suicide prevention efforts by identifying information that VA can use to develop policy and procedures to help prevent future suicides. The BHAP templates are a mechanism by which VA collects suicide data from VAMC's review of veteran medical records.
child_questions: question: What issues did GAO discover with BHAP template data?
answer: GAO's review of 63 BHAP templates at five VAMCs found that 40 of the templates that VAMCs submitted to VA Central Office had incomplete data. Also, GAO found that the BHAP templates VAMCs submitted contained inaccurate data. For example, 6 BHAP templates included a date of death that was incorrect based on information in the veteran's medical record, and 9 BHAP templates included an incorrect number of outpatient VA mental health visits in the last 30 days.

question: What is a potential cause of inconsistency in VA's BHAP template data?
answer: Moreover, GAO found that VAMCs submitted inconsistent information because they interpreted VA's guidance on completing the BHAP templates differently.

question: What were these data issues exacerbated by?
answer: This situation was further exacerbated because BHAP templates prepared by VAMCs are generally not being reviewed at any level within the Department for completeness, accuracy, and consistency.

question: What problems might VA's lack of consistent data cause?
answer: Lack of complete, accurate, and consistent data and poor oversight can inhibit VA's ability to identify, evaluate, and improve ways to better inform its suicide prevention efforts.

questions: question: What did VA estimate in 2013 regarding veterans with mental illness?
answer: In 2013, VA estimated that about 1.5 million veterans required mental health care, including services for MDD.
child_questions: question: What is MDD?
answer: MDD is a debilitating mental illness related to reduced quality of life and productivity, and increased risk for suicide.

question: What does the VA play a role in?
answer: VA also plays a role in suicide prevention.

question: What was GAO asked to do?
answer: GAO was asked to review how VA tracks veterans prescribed antidepressants and what suicide data VA uses in its prevention efforts.

questions: question: What does this report examine?
answer: This report examines (1) VA's data on veterans with MDD, including those prescribed an antidepressant; (2) the extent that veterans with MDD who are prescribed antidepressants receive recommended care and the extent to which VA monitors such care; and (3) the quality of data VA requires VAMCs to collect on veteran suicides.
child_questions: question: How did GAO collect data for this report?
answer: GAO analyzed VA data, interviewed VA officials, and conducted site visits to six VAMCs selected based on geography and population served. From each of these six VAMCs, GAO also reviewed five randomly selected medical records for veterans diagnosed with MDD and prescribed an antidepressant in 2012, as well as all completed BHAP templates.

question: What do the results of GAO's analysis provide?
answer: The results cannot be generalized across VA but provide insights.