VA Health Care: Improvements Needed to the Monitoring of Antidepressant Use for Major Depressive Disorder and the Accuracy of Suicide Data
Released: 2015-06-10T00:00:00

Letter

Background

VA provides care to veterans with mental health needs in VAMC primary and specialty care clinics. The Uniform Mental Health Services in VA Medical Centers and Clinics handbook (Handbook), which defines VA’s minimum clinical requirements for mental health services, requires that VA facilities provide evidence-based treatment through the administration of medication, when indicated, consistent with the MDD clinical practice guideline (CPG) recommendations. The CPG is guidance intended by VA to reduce current practice variation between clinicians and provide facilities with a structured framework to help improve patient outcomes, but should not take the place of the clinician’s clinical judgment. The MDD CPG includes approximately 200 evidence-based recommendations to provide information and assist in decision making for clinicians who provide care for adults with MDD. CPG recommendations describe, for example, the use of standardized assessments of veterans’ depressive symptoms as part of an evidence-based treatment plan.

The Handbook requires VAMCs to have a suicide prevention coordinator whose responsibilities include establishing and maintaining a list of veterans assessed to be at high risk for suicide; monitoring these veterans; responding to referrals from staff and the Veterans Crisis Line; and collecting and reporting information on veterans who die by suicide and who attempt suicide. intended to improve VA’s suicide prevention efforts by identifying demographic, clinical, and other related information on veteran suicides that VA Central Office can use to develop policy and procedures to help prevent future veteran deaths. VA Central Office officials explained that the BHAP initiative allows them to collect more systematic and comprehensive information about suicides, including information gleaned from interviews of family members of those veterans who die by suicide. VA Central Office has provided suicide prevention coordinators with a BHAP guide on how to complete the fields in the BHAP template.

Veterans in Our Review Often Did Not Receive Recommended Care, and VA Lacks Methods to Track Whether Recommended Care Is Provided

Our recent work, based on the three CPG recommendations we selected, has found almost all of the 30 veterans with MDD in our review who had been prescribed antidepressants received care that deviated from the MDD CPG recommendations. For example, we found that although the CPG recommends that veterans’ depressive symptoms be assessed at 4-6 weeks after initiation of antidepressant treatment using a standard assessment tool to determine the efficacy of the treatment, 26 of the 30 veterans were not assessed using such a tool within this time frame. Additionally, 10 veterans did not receive follow up within the time frame recommended in the CPG. Table 1 below depicts the specific recommendations we reviewed and the number of veterans that did not receive care consistent with the corresponding CPG recommendation.

We also found that VA does not always know the extent to which veterans with MDD who have been prescribed antidepressants are receiving care as recommended in the CPG, and some clinicians at VAMCs we visited described instances in which they generally do not follow the CPG recommendations. For example, officials from two VAMCs we visited explained that they do not routinely use the nine item Patient Health Questionnaire (PHQ-9). According to officials at one of these VAMCs, the standard of care is to conduct a clinical interview and observation. However, the CPG recommendation states that the PHQ-9 combined with a clinical interview should be used to obtain the necessary information about symptoms and symptom severity. It also states that the PHQ-9 improves diagnostic accuracy and aids treatment decisions by quantifying symptom severity. Additionally, we found that VA Central Office has not developed a mechanism to determine the extent to which mental health care delivery in VAMCs conforms to the recommendations in the MDD CPG. While deviations from recommended practice may be appropriate in many cases due to clinician discretion, VA has not fully assessed whether these examples are acceptable deviations from the CPG. VA Central Office and some VAMCs have implemented limited methods to determine the extent to which veterans are receiving care that However, without is consistent with some of the CPG recommendations.a system-wide process in place to identify and fully assess whether the care provided is consistent with the CPG, VA does not know the extent to which veterans with MDD who have been prescribed antidepressants are receiving care as recommended in the CPG and whether appropriate actions are taken by VAMCs to mitigate potentially significant risks to veterans. The CPG is intended to reduce practice variation and help improve patient outcomes, but without an understanding of the extent to which veterans are receiving care that is consistent with the CPG, we concluded that VA may be unable to ensure that it meets the intent of the CPG and improves veteran health outcomes.

To ensure that veterans are receiving care in accordance with the MDD CPG, we recommended that VA implement processes to review data on veterans with MDD who were prescribed antidepressants to evaluate the level of risk of any deviations from recommended care and remedy those that could impede veterans’ recovery. VA concurred with our recommendations and stated that VA would examine associations between treatment practices and indicators of veteran recovery or adverse outcomes. VA Central Office officials reviewed whether a cohort of veterans with MDD received treatment with an antidepressant that was in line with MDD CPG recommendations. However, in choosing CPG recommendations to review, VA officials told us that they chose the recommendations for review based on the ease of obtaining the needed data and because the antidepressant medication coverage measure is nationally recognized, rather than based on a methodical review of all of the CPG recommendations to identify those that may put veterans at risk and could impede veterans’ recovery if not followed, as we recommended in our November 2014 report. Therefore, it is not clear whether the CPG recommendations that VA chose to review were among those that may put veterans at risk and could impede recovery if not followed. Moreover, VA has not indicated whether it has implemented a process that will review CPG recommendations on an ongoing basis to identify deviations that place veterans at risk and impede recovery. This recommendation remains open pending further VA action.

Diagnostic coding discrepancies further complicate VA’s ability to know if veterans with MDD are receiving care consistent with the CPG. Specifically, we found that for 11 of the 30 veterans’ medical records we reviewed the clinician coded the encounter as “depression not otherwise specified,” a less specific code than MDD, even though the clinician documented a diagnosis of MDD in the veteran’s medical record. Therefore, VA’s data may not fully reflect the extent to which veterans have MDD due to a lack of diagnostic coding precision by clinicians, or As a the extent to which such discrepancies may permeate VA data. result, VA’s ability to monitor veterans with MDD and assess its performance in treating veterans as recommended in the MDD CPG and measuring health outcomes for veterans is further limited because VA may not be fully aware of the population of veterans with MDD.

VA’s data on the number of veterans with MDD are based on the diagnostic codes associated with patient encounters.

To address this shortcoming, our report recommended that VA (1) identify the extent to which there is imprecise diagnostic coding of MDD by examining encounters with a diagnostic code of “depression not otherwise specified” and (2) determine and address the factor(s) contributing to imprecise coding. VA concurred with our recommendations and stated that they would examine patterns of diagnostic coding among veterans with new episodes of depression by evaluating diagnosis patterns and treatment settings, as well as conduct chart reviews for a sample of veterans to examine the diagnosis in the veteran’s medical record and the diagnostic code used for the encounter. VA’s review of the accuracy of diagnostic codes is ongoing. Additionally, during the course of our review, VA Central Office officials reported that they had discovered a software mapping error in VA’s medical record system where selection of MDD as a diagnosis when using a keyword search function may mistakenly result in the selection of the “depression not otherwise specified” diagnostic code. While this error has been resolved, according to VA officials the solution would only apply to encounters going forward and would not retroactively correct any previous coding discrepancies. As a result, any such instances would still be coded in VA’s system as “depression not otherwise specified,” even though these veterans were diagnosed with MDD, and therefore data VA collected prior to resolving the software error may still not fully reflect the number of veterans with MDD.

Data VA Collects on Veteran Suicides Were Not Always Complete, Accurate, or Consistent

Our recent work has found that demographic, clinical, and other data submitted to VA Central Office on veteran suicides were not always completely or correctly entered into the BHAP Post-Mortem Chart Analysis templates (BHAP templates)—a mechanism by which VA Central Office collects veteran suicide data from VAMCs’ review of veterans’ medical records. We found that over half of the 63 BHAP templates we examined had incomplete or inaccurate information (see fig. 1).

It is important that VA have complete, accurate, and consistent data because VA Central Office uses this information to compile internal reports as part of VA’s quality improvement efforts for its suicide prevention program, and unreliable data limits VA Central Office’s ability to develop policy and procedures aimed at preventing veteran deaths.

VA Central Office used veteran enrollment information when compiling the BHAP report in March 2014. Specifically, VA Central Office included clinical data in the BHAP report only for veterans utilizing VA services. However, we found that clinical data for 23 of the 63 BHAP templates we reviewed would not be included in the report because of missing data, such as not indicating whether the veteran was enrolled in VA health care services, even though the veteran had a VA medical record.

Additionally, 40 of the 63 BHAP templates we reviewed included various data fields where no response was provided, resulting in incomplete data. For example, for 19 templates, VAMC staff did not enter requested data as to whether the veteran had all or some of 15 active psychiatric symptoms within 12 months prior to the veteran’s date of death. These missing fields are counted as “no” in the report, meaning that the veteran did not have these symptoms. For at least one BHAP template we reviewed, the nonresponse for the question about the psychiatric symptom of isolation would have counted as “no” in the report; however, officials from the responsible VAMC told us that the veteran did have this symptom. Furthermore, we found that VAMCs did not always submit BHAP templates for all veteran suicides known to the facility, as required by the BHAP guide, and VA Central Office does not have a process in place to determine whether it is receiving the BHAP templates for all known veteran suicides. For example, one VAMC had completed 13 BHAP templates at the time of our site visit but had not submitted them; however, neither the VAMC nor VA Central Office were aware that these templates had not been submitted until after we requested them from VA Central Office.

We also found numerous instances of inaccurate data in the 63 BHAP templates we reviewed. For example, we found 6 BHAP templates that included a date of death that was incorrect based on information in the veteran’s medical record. The difference in the dates of death in the veterans’ medical records and the dates of death in the BHAP templates ranged from 1 day to 1 year. The accuracy of the date of death recorded in the BHAP template is important because it is used as a point of reference to calculate other fields, such as the number of mental health visits in the last 30 days. Without accurate information, VA cannot use this information to determine whether policies or procedures need to be changed to ensure that veterans at high risk for suicide are being seen more frequently by a mental health provider to help prevent suicides in the future.

We also found several situations where VAMCs interpreted and applied instructions for completing the BHAP templates differently, resulting in inconsistent data being reported across VAMCs. For example, one VAMC included a visit to an immunization clinic as the veteran’s final visit, while another VAMC did not include this type of visit, even though this was the last time the veteran was seen in person. The BHAP guide indicates that the final visit should be the last time the veteran had in-person contact with any VAMC staff, but the BHAP guide does not identify the different types of visits that should be counted. Additionally, VA policy and guidance states that the BHAP template should be completed for all suicides known to the facility, but at the five VAMCs we visited, these data were not always being reported. VA policy and instructions do not explicitly state that veterans not being seen by VA also should be included, and in the absence of this declaration, some VAMCs interpreted the instructions to mean that only veterans being seen by VA should be included in the data submitted. Therefore, two VAMCs have submitted data only for veterans being treated by VA, while the others include data on all known veteran suicides—whether they have been treated by VA or not. When VAMCs do not provide consistent data, VA Central Office will receive and use inconsistent data in preparing its trend reports, such as BHAP reports, which are intended to be used to improve suicide prevention efforts.

Further, we found that VA did not have an established process for reviewing the accuracy of BHAP templates, and for the sites we covered in our review, BHAP templates were not being reviewed by VA officials at any level for accuracy, completeness, and consistency. Therefore, our findings at five VAMCs could be symptomatic of a nationwide problem, and other VAMCs may also be submitting incomplete, inaccurate, or inconsistent suicide-related information and VA may not be getting the data it needs across the department to make appropriate resource decisions and develop new policy. We also found that VA lacks sufficient controls, such as automated data checks, to ensure the quality of the existing BHAP data. Not reviewing the data is inconsistent with internal control standards for the federal government, which state that agencies should have controls over information processes, including procedures and standards to ensure the completeness and accuracy of processed data.

To improve VA’s efforts to inform its suicide prevention activities, we made three recommendations in our November 2014 report that directed VA to (1) clarify guidance on how to complete the BHAP templates to ensure that VAMCs are submitting consistent data on veteran suicides, (2) ensure that VAMCs have a process in place to review data on veteran suicides for completeness, accuracy, and consistency before the data are submitted to VA Central Office, and (3) implement processes to review data on veteran suicides submitted by VAMCs for accuracy and completeness. VA agreed with our recommendations and, to date, has made some progress in addressing these recommendations.

VA has issued clarifying guidance to suicide prevention coordinators and VA officials reported that suicide prevention coordinators expressed being more comfortable with filling out the BHAP templates. We closed the first recommendation as implemented.

The last two recommendations remain unimplemented pending VA’s completion of planned actions.

To ensure that the BHAP data are accurate, complete, and consistent, VA created a checkbox in the BHAP template to indicate that the data were checked by VAMC leadership. VA’s initial random checks indicate the checkbox is being used and VAMC leadership is reviewing entries resulting in better consistency of the data. VA plans to run monthly reviews to determine compliance.

Additionally, VA created a software program to compare data from the BHAP templates to the data entered into another suicide prevention database maintained by VA. VA officials plan for this review to become part of the quarterly routine review process and information about missing cases will be sent back to the VAMCs for correction on a quarterly basis.

Chairman Coffman, Ranking Member Kuster, and Members of the Subcommittee, this concludes my prepared statement. I would be pleased to respond to any questions that you might have at this time.

GAO Contact and Staff Acknowledgments

If you or your staff have any questions about this testimony, please contact Randall B. Williamson, Director, Health Care at (202) 512-7114 or williamsonr@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. GAO staff who made key contributions to this testimony are Marcia A. Mann, Assistant Director; Emily Binek; Cathleen Hamann; Sarah Resavy; and Jennifer Whitworth.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.