VA HEALTH CARE

Improvements Needed to 

the Monitoring of 

Antidepressant Use for 

Major Depressive Disorder 

and the Accuracy of 

Suicide Data

Statement of Randall B. Williamson

Director, Health Care

Testimony 

Before the Subcommittee on Oversight 

and Investigations, Committee on 

Veterans’ Affairs, House of 

Representatives

For Release on Delivery 

Expected at 10:30 a.m. ET

Wednesday, June 10, 2015

GAO-15-648T

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-15-648T, a testimony 

before the Subcommittee on Oversight and 

Investigations, Committee on Veterans’ 

Affairs, House of Representatives

June 10, 2015

VA HEALTH CARE

Improvements Needed to the Monitoring of 

Antidepressant Use for Major Depressive Disorder 

and the Accuracy of Suicide Data

Why GAO Did This Study

In 2013, VA estimated that about 

1.5 million veterans required mental 

health care, including for MDD. MDD is 

a debilitating mental illness related to 

reduced quality of life and increased 

risk for suicide. VA also plays a role in 

suicide risk assessment and 

prevention.

This testimony addresses the extent to 

which (1) veterans with MDD who are

prescribed an antidepressant receive 

recommended care and (2) VAMCs 

are collecting information on veteran 

suicides as required by VA. The 

testimony is based on GAO’s 

November 2014 report, VA Health 

Care: Improvements Needed in 

Monitoring Antidepressant Use for 

Major Depressive Disorder and in 

Increasing Accuracy of Suicide Data

(GAO-15-55). For that report GAO 

analyzed VA data, interviewed VA 

officials, and conducted site visits to six 

VAMCs selected based on geography 

and population served. GAO also 

reviewed randomly selected medical 

records for five veterans from each of 

the six VAMCs, for veterans diagnosed 

with MDD and prescribed an 

antidepressant in 2012, as well as all 

completed BHAP templates. The 

results cannot be generalized across 

VA. GAO followed up in May 2015 to 

determine the status of GAO’s 

previous recommendations.

What GAO Recommends

GAO recommended that VA implement 

processes to assess deviations from 

recommended care; identify and 

address MDD coding issues; and 

implement processes to improve 

veteran suicide data. VA has made 

progress on these recommendations 

and has fully implemented one.

What GAO Found

Department of Veterans Affairs (VA) policy states that antidepressant treatment 

must be consistent with VA’s current clinical practice guideline (CPG) for major 

depressive disorder (MDD); however, GAO’s recent review of 30 veterans’ 

medical records found that most contained deviations. For example, although 

the CPG recommends that veterans’ depressive symptoms be assessed at 

4-6 weeks after initiation of antidepressant treatment using a standardized 

assessment tool, 26 of the 30 veterans were not assessed in this manner within 

this time frame. Additionally, 10 veterans did not receive follow up within the time 

frame recommended in the CPG. GAO found that VA (1) does not have a 

system-wide process in place to identify and fully assess the extent to which 

veterans with MDD who have been prescribed antidepressants are receiving 

care as recommended in the CPG and (2) does not know whether appropriate 

actions are being taken by VA medical centers (VAMC) to mitigate potentially 

significant risks to veterans. GAO also found that VA’s data may underestimate 

the prevalence of MDD among veterans being treated through VA as a result of 

imprecise coding by clinicians, further complicating VA’s ability to know if 

veterans with MDD are receiving care consistent with the CPG.

GAO’s recent work has found that the demographic and clinical data that VA 

collects on veteran suicides were not always complete, accurate, or consistent. 

VA’s Behavioral Health Autopsy Program (BHAP) is a quality initiative to improve 

VA’s suicide prevention efforts by identifying information that VA can use to 

develop policy to help prevent future suicides. The BHAP templates are a 

mechanism by which VA collects suicide data from VAMCs’ review of veteran 

medical records. GAO’s review of the 63 completed BHAP templates at five 

VAMCs found that (1) over half of the templates that VAMCs submitted to VA 

had incomplete or inaccurate data, and (2) VAMCs submitted inconsistent 

information because they interpreted VA’s guidance differently. Lack of complete, 

accurate, and consistent data—coupled with GAO’s finding of poor oversight of 

the review of BHAP templates—can inhibit VA’s ability to identify, evaluate, and 

improve ways to better inform its suicide prevention efforts.

Number of Behavioral Health Autopsy Program (BHAP) Post-Mortem Chart Analysis 

Templates with Incomplete or Inaccurate Data

View GAO-15-648T. For more information, 

contact Randall B. Williamson at (202) 512-

7114 or williamsonr@gao.gov.
Letter

Page 1 GAO-15-648T

Chairman Coffman, Ranking Member Kuster, and Members of the 

Subcommittee:

I am pleased to be here to discuss our recent work on the monitoring of 

veterans with major depressive disorder (MDD) who were prescribed an 

antidepressant and data collection efforts by the Department of Veterans 

Affairs (VA) on veteran suicides. In 2013, VA estimated that about 

1.5 million veterans required mental health services, including for MDD. 

MDD is a particularly debilitating mental illness and is associated with 

reduced quality of life, reduced productivity, and increased risk for 

suicide.1 These negative effects underscore the importance for veterans 

of timely, evidence-based assessment for and treatment of MDD, which 

may include medications such as antidepressants, psychotherapy, or a 

combination of both. Based on our previous analysis of VA data from 

veterans’ medical records and administrative sources, 532,222 veterans 

had a diagnosis of MDD from fiscal years 2009 through 2013, and among 

those veterans, about 499,000 (94 percent) veterans were prescribed at 

least one antidepressant by a VA provider.2

In addition to providing ongoing care to veterans with MDD, VA plays a 

role in suicide risk assessment and prevention among veterans. 

According to VA in a June 2013 report, about one-quarter of the 18 to 

22 veterans who die by suicide each day were receiving care through 

According to VA, the 

prevalence of MDD among veterans being treated in VA primary care 

settings is higher than that among the general population.

 

1

MDD is characterized by the presence of depressed mood or loss of interest or pleasure 

along with other symptoms for a period of at least 2 weeks that represent a change in 

previous functioning. These symptoms include significant weight loss; insomnia or 

excessive sleeping; psychomotor agitation or retardation; fatigue or loss of energy; 

feelings of worthlessness or excessive or inappropriate guilt; diminished ability to think or 

concentrate, or indecisiveness; and recurrent thoughts of death, recurrent suicidal ideation 

without a specific plan, or a suicide attempt or a specific plan for committing suicide. 

American Psychiatric Association, Diagnostic and Statistical Manual of Mental Disorders, 

5th ed. (Arlington, Va: American Psychiatric Association, 2013). 

2

Veterans were classified as having a diagnosis of MDD if, in at least one fiscal year 

included in our review, they had two or more outpatient encounters or at least one 

inpatient hospital stay with a diagnosis of MDD. The 532,222 veterans diagnosed with

MDD represent about 10 percent of veterans who received health care services through 

VA. This estimate is based on published Congressional Research Service data on the 

number of veterans who received health care services through VA from fiscal years 2009 

through 2013 (roughly 5.5 million).

Letter
Page 2 GAO-15-648T

VA.3

Given the debilitating effects that depression can have on veterans’ 

quality of life, VA’s monitoring of veterans with MDD is critical to ensuring 

that they receive care that is associated with positive health care 

outcomes. Additionally, the relatively high veteran suicide rate makes it 

important that VA use data that it collects related to veteran suicides to 

drive its prevention efforts. Today I will address two areas: the extent to 

which (1) veterans with MDD who are prescribed an antidepressant 

receive recommended care, and (2) VA medical centers (VAMC) are 

collecting information on veteran suicides as required by VA.

Research has identified numerous risk factors for suicide among 

veterans, which include substance use disorder, physical impairments, 

previous suicide attempts, and depression. Additionally, life stressors, 

such as marital or financial problems, contribute to a veteran’s risk of 

suicide.

My statement is based on a GAO report released in November 2014 

examining VA’s monitoring of veterans with MDD who have been 

prescribed an antidepressant and the use of suicide data within VA.4 For 

our work examining the care received by veterans with MDD who are 

prescribed an antidepressant, we reviewed VA policy documents and 

interviewed VA Central Office officials responsible for developing and 

implementing VA mental health policy. We also conducted site visits to six 

VAMCs, which we selected for variation in complexity of health care 

services offered, geographic location, and number of veterans using 

mental health services.5

 

3

VA/Department of Defense (DOD) Assessment and Management of Risk for Suicide 

Working Group, VA/DOD Clinical Practice Guideline for Assessment and Management of 

Patients at Risk for Suicide (June 2013).

We reviewed a random, nongeneralizeable 

sample of medical records for 5 veterans treated at each of the 6 

VAMCs—for a total of 30 veterans—to assess the extent to which the 

antidepressant treatment-related care VAMCs provided was consistent 

with three evidence-based treatment recommendations included in VA 

4

GAO, VA Health Care: Improvements Needed in Monitoring Antidepressant Use for 

Major Depressive Disorder and in Increasing Accuracy of Suicide Data, GAO-15-55

(Washington, D.C.: Nov. 12, 2014).

5

These six VAMCs were located in Canandaigua, New York; Gainesville, Florida; Iowa 

City, Iowa; Philadelphia, Pennsylvania; Phoenix, Arizona; and Reno, Nevada. In contrast 

to the other site visits, which were completed in person, we completed the site visits to the 

VAMCs located in Gainesville, Florida, and Reno, Nevada, through telephone interviews.
Page 3 GAO-15-648T

guidelines.6 Results from our medical record review cannot be 

generalized to the VAMC or across VAMCs. For our work examining the 

data that VA requires VAMCs to collect on veteran suicides, we reviewed 

VA policies, guidance, and documents related to VA’s suicide prevention 

efforts to identify the data collected by VA staff on veteran suicides. We 

also interviewed VA Central Office and VAMC officials responsible for 

VA’s suicide prevention program, obtained documents and interviewed 

officials regarding the collection of veteran suicide data, and compared 

data obtained from VAMCs to information included in the veterans’ 

medical records and information we obtained from VA Central Office.7

The work on which this statement is based was conducted, with updates 

in May 2015, in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives. Further details on our 

scope and methodology are included in our report.

Results from our review of veteran suicide data can be generalized to the 

VAMCs we visited, but cannot be generalized to other VAMCs. In May 

2015, in preparation for this statement, we met with VA officials to discuss 

the status of VA’s implementation of action plans to address the six 

recommendations included in our November 2014 report.

VA provides care to veterans with mental health needs in VAMC primary 

and specialty care clinics. The Uniform Mental Health Services in VA 

Medical Centers and Clinics handbook (Handbook), which defines VA’s 

minimum clinical requirements for mental health services, requires that 

VA facilities provide evidence-based treatment through the administration 

 

6

The 30 veterans we selected were diagnosed with MDD and had a new treatment 

episode of an antidepressant in calendar year 2012. For our review we selected three 

evidence-based treatment recommendations for inclusion in our review that had among 

the highest strength of research evidence, were sufficiently specific to enable us to 

determine the extent to which VA providers were following the recommendation, and 

would not require clinical judgment to determine the extent to which VA providers were 

following the recommendation.

7

We reviewed completed information on veteran suicides from five of the VAMCs included 

in our review. One VAMC reported having no veteran suicides as of the date of our site 

visit; therefore, our analysis of suicide data does not include this VAMC.

Background
Page 4 GAO-15-648T

of medication, when indicated, consistent with the MDD clinical practice 

guideline (CPG) recommendations.8 The CPG is guidance intended by 

VA to reduce current practice variation between clinicians and provide 

facilities with a structured framework to help improve patient outcomes, 

but should not take the place of the clinician’s clinical judgment. The MDD 

CPG includes approximately 200 evidence-based recommendations to 

provide information and assist in decision making for clinicians who 

provide care for adults with MDD. CPG recommendations describe, for 

example, the use of standardized assessments of veterans’ depressive 

symptoms as part of an evidence-based treatment plan.9

In June 2006, VA began implementing several initiatives aimed at suicide 

prevention, including appointing a National Suicide Prevention 

Coordinator, developing data collection systems to increase 

understanding of suicide among veterans and inform VA suicide 

prevention programs, and instituting suicide prevention programs in all 

VAMCs.10 VA Central Office uses several mechanisms to collect data on 

veteran suicides to help improve its suicide prevention efforts, including 

the Behavioral Health Autopsy Program (BHAP).11

 

8

The MDD CPG is formally known as the VA/DOD Clinical Practice Guideline for 

Management of Major Depressive Disorder (May 2009). The MDD CPG was issued by the 

joint VA/DOD Evidence-Based Practice Work Group in 2009. Formed in 1999 and 

composed of VA and DOD officials, the VA/DOD Evidence-Based Practice Work Group 

makes decisions about which CPGs for specific conditions will be developed and 

oversees their development.

The BHAP initiative, 

which began in December 2012, is a quality improvement initiative 

9

According to the MDD CPG, veterans with MDD treated with antidepressants should be 

closely observed, particularly at the beginning of treatment and following dosage changes, 

to maximize veterans’ recovery and to mitigate any negative treatment effects, including 

worsening of depressive symptoms.

10The Handbook requires VAMCs to have a suicide prevention coordinator whose 

responsibilities include establishing and maintaining a list of veterans assessed to be at 

high risk for suicide; monitoring these veterans; responding to referrals from staff and the 

Veterans Crisis Line; and collecting and reporting information on veterans who die by 

suicide and who attempt suicide.

11VA also collects data through the following mechanisms: Suicide Prevention Application 

Network, in which VAMCs submit information on the number of veterans that completed 

suicides, the number of suicide attempts, and indicators of suicide prevention efforts; 

suicide behavior reports, which clinicians must complete when they learn that a veteran 

attempted or completed suicide and add to the veteran’s medical record; and root cause 

analyses that are completed by VAMC patient safety managers for suicide attempts and 

completed suicides under certain circumstances.
Page 5 GAO-15-648T

intended to improve VA’s suicide prevention efforts by identifying 

demographic, clinical, and other related information on veteran suicides 

that VA Central Office can use to develop policy and procedures to help 

prevent future veteran deaths.12 VA Central Office officials explained that 

the BHAP initiative allows them to collect more systematic and 

comprehensive information about suicides, including information gleaned 

from interviews of family members of those veterans who die by suicide. 

VA Central Office has provided suicide prevention coordinators with a 

BHAP guide on how to complete the fields in the BHAP template.

Our recent work, based on the three CPG recommendations we selected, 

has found almost all of the 30 veterans with MDD in our review who had 

been prescribed antidepressants received care that deviated from the 

MDD CPG recommendations. For example, we found that although the 

CPG recommends that veterans’ depressive symptoms be assessed at 

4-6 weeks after initiation of antidepressant treatment using a standard 

assessment tool to determine the efficacy of the treatment, 26 of the 

30 veterans were not assessed using such a tool within this time frame. 

Additionally, 10 veterans did not receive follow up within the time frame 

recommended in the CPG. Table 1 below depicts the specific 

recommendations we reviewed and the number of veterans that did not 

receive care consistent with the corresponding CPG recommendation.

 

12Veteran suicide data for the BHAP initiative are submitted by VAMCs to VA’s Center of 

Excellence for Suicide Prevention. The Center of Excellence was created by VA Central 

Office, and for the purposes of our testimony, we refer to the Center of Excellence as part 

of VA Central Office.

Veterans in Our 

Review Often Did Not 

Receive 

Recommended Care, 

and VA Lacks 

Methods to Track 

Whether 

Recommended Care 

Is Provided
Page 6 GAO-15-648T

Table 1: Number of Veterans in GAO’s Sample Not Receiving Care As 

Recommended in the Clinical Practice Guideline (CPG) for Major Depressive 

Disorder (MDD)

CPG recommendation

Number of veterans not receiving care as 

recommended in the CPG for MDD

To enhance antidepressant 

treatment, veterans should be 

educated on when to take the 

medication, possible side effects, 

risks, and the expected duration of 

treatment, among other things

6 of 30 veterans lacked documentation of patient 

education when the medication was prescribed

Standardized assessments of 

depressive symptoms, such as the 

Patient Health Questionnaire-9, 

should be used to monitor treatment 

at 4-6 weeks after initiation of 

treatment and after each change in 

treatment

26 of 30 veterans were not assessed using a 

standardized assessment tool at 4-6 weeks after 

initiation of treatment

18 of 30 veterans were not assessed using a 

standardized assessment tool at any encounter

10 of 30 veterans did not have a follow-up 

encounter that occurred 4-6 weeks after initiation 

of treatment

a

A plan should be developed that 

addresses the duration of 

antidepressant treatment, among 

other things

b

1 veteran of 30 did not have a planned date for 

follow up and plan for future care documented in 

the veteran’s medical record at the initial 

encounter

Source: GAO analysis of Department of Veterans Affairs (VA) data. | GAO-15-648T

Note: We included 30 veterans in our review. Our review began with the encounter during which a VA 

clinician ordered an antidepressant to treat depressive symptoms (initial encounter) and five follow-up 

encounters with a VA clinician, or sooner if the veteran did not have five follow-up encounters. Our 

review was limited to encounters during which the antidepressant treatment was reviewed, including 

encounters during which side effects and treatment effect were assessed, but no change was made 

to medication orders.

a

Of the 30 veterans included in our review, only 6 were assessed using a standardized assessment at 

the initial encounter where antidepressant medication was prescribed. VA Central Office officials 

explained that they would expect a standardized assessment to be conducted at the start of an 

antidepressant to establish a baseline score.

b

We also found that VA does not always know the extent to which 

veterans with MDD who have been prescribed antidepressants are 

receiving care as recommended in the CPG, and some clinicians at 

VAMCs we visited described instances in which they generally do not 

follow the CPG recommendations. For example, officials from two VAMCs 

we visited explained that they do not routinely use the nine item Patient 

Three veterans did not receive a follow-up encounter at all. Two veterans did not show for scheduled 

appointments that were within the CPG recommended time frame. Five veterans did not have a 

follow-up encounter until after 6 weeks.
Page 7 GAO-15-648T

Health Questionnaire (PHQ-9).13 According to officials at one of these 

VAMCs, the standard of care is to conduct a clinical interview and 

observation. However, the CPG recommendation states that the PHQ-9 

combined with a clinical interview should be used to obtain the necessary 

information about symptoms and symptom severity. It also states that the 

PHQ-9 improves diagnostic accuracy and aids treatment decisions by 

quantifying symptom severity. Additionally, we found that VA Central 

Office has not developed a mechanism to determine the extent to which 

mental health care delivery in VAMCs conforms to the recommendations 

in the MDD CPG. While deviations from recommended practice may be 

appropriate in many cases due to clinician discretion, VA has not fully 

assessed whether these examples are acceptable deviations from the 

CPG. VA Central Office and some VAMCs have implemented limited 

methods to determine the extent to which veterans are receiving care that 

is consistent with some of the CPG recommendations.14

To ensure that veterans are receiving care in accordance with the MDD 

CPG, we recommended that VA implement processes to review data on 

veterans with MDD who were prescribed antidepressants to evaluate the 

level of risk of any deviations from recommended care and remedy those 

that could impede veterans’ recovery. VA concurred with our 

However, without 

a system-wide process in place to identify and fully assess whether the 

care provided is consistent with the CPG, VA does not know the extent to 

which veterans with MDD who have been prescribed antidepressants are 

receiving care as recommended in the CPG and whether appropriate 

actions are taken by VAMCs to mitigate potentially significant risks to 

veterans. The CPG is intended to reduce practice variation and help 

improve patient outcomes, but without an understanding of the extent to 

which veterans are receiving care that is consistent with the CPG, we 

concluded that VA may be unable to ensure that it meets the intent of the 

CPG and improves veteran health outcomes.

 

13The PHQ-9 is a diagnostic tool, which uses the nine MDD diagnosis symptoms as 

criteria to help clinicians make a criteria-based diagnosis of depressive disorders and 

measure depression severity to aid treatment decisions.

14These methods include (1) a psychopharmacology quality improvement initiative that 

began in fiscal year 2014 consisting of a series of prescribing practice metrics such as the 

proportion of veterans with depression prescribed three or more concurrent 

antidepressant medications for 60 or more continuous days, and (2) a software system 

called the Behavioral Health Laboratory that some VAMCs have implemented to help 

ensure that veterans with MDD who are prescribed antidepressants receive care 

consistent with the CPG when the veteran is treated in a primary care clinic.
Page 8 GAO-15-648T

recommendations and stated that VA would examine associations 

between treatment practices and indicators of veteran recovery or 

adverse outcomes. VA Central Office officials reviewed whether a cohort 

of veterans with MDD received treatment with an antidepressant that was 

in line with MDD CPG recommendations.15

Diagnostic coding discrepancies further complicate VA’s ability to know if 

veterans with MDD are receiving care consistent with the CPG. 

Specifically, we found that for 11 of the 30 veterans’ medical records we 

reviewed the clinician coded the encounter as “depression not otherwise 

specified,” a less specific code than MDD, even though the clinician 

documented a diagnosis of MDD in the veteran’s medical record. 

Therefore, VA’s data may not fully reflect the extent to which veterans 

have MDD due to a lack of diagnostic coding precision by clinicians, or 

the extent to which such discrepancies may permeate VA data.

However, in choosing CPG 

recommendations to review, VA officials told us that they chose the 

recommendations for review based on the ease of obtaining the needed 

data and because the antidepressant medication coverage measure is 

nationally recognized, rather than based on a methodical review of all of 

the CPG recommendations to identify those that may put veterans at risk 

and could impede veterans’ recovery if not followed, as we recommended 

in our November 2014 report. Therefore, it is not clear whether the CPG 

recommendations that VA chose to review were among those that may 

put veterans at risk and could impede recovery if not followed. Moreover, 

VA has not indicated whether it has implemented a process that will 

review CPG recommendations on an ongoing basis to identify deviations 

that place veterans at risk and impede recovery. This recommendation 

remains open pending further VA action.

16

 

15To do this, VA reviewed (1) the extent to which veterans received antidepressant 

medication coverage for at least 84 days during a 12-week period; (2) rates of psychiatric 

hospitalization; and (3) the association between the receipt of guideline-consistent care 

and changes in depression symptoms documented using PHQ-9 scores.

As a 

result, VA’s ability to monitor veterans with MDD and assess its 

performance in treating veterans as recommended in the MDD CPG and 

measuring health outcomes for veterans is further limited because VA 

may not be fully aware of the population of veterans with MDD.

16VA’s data on the number of veterans with MDD are based on the diagnostic codes 

associated with patient encounters.
Page 9 GAO-15-648T

To address this shortcoming, our report recommended that VA (1) identify 

the extent to which there is imprecise diagnostic coding of MDD by 

examining encounters with a diagnostic code of “depression not 

otherwise specified” and (2) determine and address the factor(s) 

contributing to imprecise coding. VA concurred with our recommendations 

and stated that they would examine patterns of diagnostic coding among 

veterans with new episodes of depression by evaluating diagnosis 

patterns and treatment settings, as well as conduct chart reviews for a 

sample of veterans to examine the diagnosis in the veteran’s medical 

record and the diagnostic code used for the encounter. VA’s review of the 

accuracy of diagnostic codes is ongoing. Additionally, during the course 

of our review, VA Central Office officials reported that they had 

discovered a software mapping error in VA’s medical record system 

where selection of MDD as a diagnosis when using a keyword search 

function may mistakenly result in the selection of the “depression not 

otherwise specified” diagnostic code. While this error has been resolved, 

according to VA officials the solution would only apply to encounters 

going forward and would not retroactively correct any previous coding 

discrepancies. As a result, any such instances would still be coded in 

VA’s system as “depression not otherwise specified,” even though these 

veterans were diagnosed with MDD, and therefore data VA collected prior 

to resolving the software error may still not fully reflect the number of 

veterans with MDD.

Our recent work has found that demographic, clinical, and other data 

submitted to VA Central Office on veteran suicides were not always 

completely or correctly entered into the BHAP Post-Mortem Chart 

Analysis templates (BHAP templates)—a mechanism by which VA 

Central Office collects veteran suicide data from VAMCs’ review of 

veterans’ medical records. We found that over half of the 63 BHAP 

templates we examined had incomplete or inaccurate information (see 

fig. 1).

Data VA Collects on 

Veteran Suicides 

Were Not Always 

Complete, Accurate, 

or Consistent
Page 10 GAO-15-648T

Figure 1: Number of Behavioral Health Autopsy Program (BHAP) Post-Mortem Chart Analysis Templates with Incomplete or 

Inaccurate Data

It is important that VA have complete, accurate, and consistent data 

because VA Central Office uses this information to compile internal 

reports as part of VA’s quality improvement efforts for its suicide 

prevention program, and unreliable data limits VA Central Office’s ability 

to develop policy and procedures aimed at preventing veteran deaths.17

Additionally, 40 of the 63 BHAP templates we reviewed included various 

data fields where no response was provided, resulting in incomplete data. 

For example, for 19 templates, VAMC staff did not enter requested data 

as to whether the veteran had all or some of 15 active psychiatric 

symptoms within 12 months prior to the veteran’s date of death. These 

missing fields are counted as “no” in the report, meaning that the veteran 

VA Central Office used veteran enrollment information when compiling 

the BHAP report in March 2014. Specifically, VA Central Office included 

clinical data in the BHAP report only for veterans utilizing VA services. 

However, we found that clinical data for 23 of the 63 BHAP templates we 

reviewed would not be included in the report because of missing data, 

such as not indicating whether the veteran was enrolled in VA health care 

services, even though the veteran had a VA medical record.

 

17Department of Veterans Affairs, Behavioral Health Autopsy Program Interim Summary, 

December 1, 2012 – February 27, 2014 (Mar. 13, 2014). This report includes information 

on veterans who die by suicide, both with and without a history of VA health care service 

utilization.
Page 11 GAO-15-648T

did not have these symptoms. For at least one BHAP template we 

reviewed, the nonresponse for the question about the psychiatric 

symptom of isolation would have counted as “no” in the report; however, 

officials from the responsible VAMC told us that the veteran did have this 

symptom. Furthermore, we found that VAMCs did not always submit 

BHAP templates for all veteran suicides known to the facility, as required 

by the BHAP guide, and VA Central Office does not have a process in 

place to determine whether it is receiving the BHAP templates for all 

known veteran suicides. For example, one VAMC had completed 13 

BHAP templates at the time of our site visit but had not submitted them; 

however, neither the VAMC nor VA Central Office were aware that these 

templates had not been submitted until after we requested them from VA 

Central Office.

We also found numerous instances of inaccurate data in the 63 BHAP 

templates we reviewed. For example, we found 6 BHAP templates that 

included a date of death that was incorrect based on information in the 

veteran’s medical record. The difference in the dates of death in the 

veterans’ medical records and the dates of death in the BHAP templates 

ranged from 1 day to 1 year. The accuracy of the date of death recorded 

in the BHAP template is important because it is used as a point of 

reference to calculate other fields, such as the number of mental health 

visits in the last 30 days. Without accurate information, VA cannot use 

this information to determine whether policies or procedures need to be 

changed to ensure that veterans at high risk for suicide are being seen 

more frequently by a mental health provider to help prevent suicides in 

the future.

We also found several situations where VAMCs interpreted and applied 

instructions for completing the BHAP templates differently, resulting in 

inconsistent data being reported across VAMCs. For example, one VAMC 

included a visit to an immunization clinic as the veteran’s final visit, while 

another VAMC did not include this type of visit, even though this was the 

last time the veteran was seen in person. The BHAP guide indicates that 

the final visit should be the last time the veteran had in-person contact 

with any VAMC staff, but the BHAP guide does not identify the different 

types of visits that should be counted. Additionally, VA policy and 

guidance states that the BHAP template should be completed for all 

suicides known to the facility, but at the five VAMCs we visited, these 

data were not always being reported. VA policy and instructions do not 

explicitly state that veterans not being seen by VA also should be 

included, and in the absence of this declaration, some VAMCs interpreted 

the instructions to mean that only veterans being seen by VA should be 
Page 12 GAO-15-648T

included in the data submitted. Therefore, two VAMCs have submitted 

data only for veterans being treated by VA, while the others include data 

on all known veteran suicides—whether they have been treated by VA or 

not. When VAMCs do not provide consistent data, VA Central Office will 

receive and use inconsistent data in preparing its trend reports, such as 

BHAP reports, which are intended to be used to improve suicide 

prevention efforts.

Further, we found that VA did not have an established process for 

reviewing the accuracy of BHAP templates, and for the sites we covered 

in our review, BHAP templates were not being reviewed by VA officials at 

any level for accuracy, completeness, and consistency. Therefore, our 

findings at five VAMCs could be symptomatic of a nationwide problem, 

and other VAMCs may also be submitting incomplete, inaccurate, or 

inconsistent suicide-related information and VA may not be getting the 

data it needs across the department to make appropriate resource 

decisions and develop new policy. We also found that VA lacks sufficient 

controls, such as automated data checks, to ensure the quality of the 

existing BHAP data. Not reviewing the data is inconsistent with internal 

control standards for the federal government, which state that agencies 

should have controls over information processes, including procedures 

and standards to ensure the completeness and accuracy of processed 

data.18

To improve VA’s efforts to inform its suicide prevention activities, we 

made three recommendations in our November 2014 report that directed 

VA to (1) clarify guidance on how to complete the BHAP templates to 

ensure that VAMCs are submitting consistent data on veteran suicides, 

(2) ensure that VAMCs have a process in place to review data on veteran 

suicides for completeness, accuracy, and consistency before the data are 

submitted to VA Central Office, and (3) implement processes to review 

data on veteran suicides submitted by VAMCs for accuracy and 

completeness. VA agreed with our recommendations and, to date, has 

made some progress in addressing these recommendations.

 

18See GAO, Standards for Internal Control in the Federal Government, 

GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999). Internal control is 

synonymous with management control and comprises the plans, methods, and 

procedures used to meet missions, goals, and objectives.
Page 13 GAO-15-648T

• VA has issued clarifying guidance to suicide prevention coordinators 

and VA officials reported that suicide prevention coordinators 

expressed being more comfortable with filling out the BHAP 

templates. We closed the first recommendation as implemented.

• The last two recommendations remain unimplemented pending VA’s 

completion of planned actions.

• To ensure that the BHAP data are accurate, complete, and 

consistent, VA created a checkbox in the BHAP template to 

indicate that the data were checked by VAMC leadership. VA’s 

initial random checks indicate the checkbox is being used and 

VAMC leadership is reviewing entries resulting in better 

consistency of the data. VA plans to run monthly reviews to 

determine compliance.

• Additionally, VA created a software program to compare data from 

the BHAP templates to the data entered into another suicide 

prevention database maintained by VA. VA officials plan for this 

review to become part of the quarterly routine review process and 

information about missing cases will be sent back to the VAMCs 

for correction on a quarterly basis.

Chairman Coffman, Ranking Member Kuster, and Members of the 

Subcommittee, this concludes my prepared statement. I would be 

pleased to respond to any questions that you might have at this time.

If you or your staff have any questions about this testimony, please 

contact Randall B. Williamson, Director, Health Care at (202) 512-7114 or 

williamsonr@gao.gov. Contact points for our Offices of Congressional 

Relations and Public Affairs may be found on the last page of this 

statement. GAO staff who made key contributions to this testimony are 

Marcia A. Mann, Assistant Director; Emily Binek; Cathleen Hamann; 

Sarah Resavy; and Jennifer Whitworth.

GAO Contact and 

Staff 

Acknowledgments

(291293)
This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.