VETERANS CRISIS LINE

Further Efforts Needed to 

Improve Service 

Statement for the Record by Seto J. Bagdoyan, Director, 

Forensic Audits and Investigative Service and Randall B. 

Williamson, Director, Health Care

Statement for the Record for the 

Committee on Veterans' Affairs, House 

of Representatives

For Release on Delivery 

Expected at 10:00 a.m ET

Tuesday, April 4, 2017

GAO-17-545T

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-17-545T, a statement for 

the record for the Committee on Veterans' 

Affairs, House of Representatives

April 4, 2017

VETERANS CRISIS LINE

Further Efforts Needed to Improve Service

What GAO Found

In May 2016, GAO found that the Department of Veterans Affairs (VA) did not 

meet its call response-time goals for the Veterans Crisis Line (VCL), although 

extended call wait times were not common; issues with text messages were also 

found. VA’s goal was to answer 90 percent of VCL calls at the VCL primary 

center within 30 seconds in the period GAO studied. Calls not answered within 

30 seconds were to route to VCL backup call centers; however, for 5 months of 

fiscal year 2015, calls were routed to VCL backup call centers after 60 seconds. 

According to VA officials about 65 to 75 percent of VCL calls were answered at 

the VCL primary center in fiscal year 2015 within either 30 or 60 seconds. GAO’s 

covert testing in July and August 2015 confirmed this. Specifically, 119 covert 

test calls showed that an estimated 73 percent of calls made during this period 

were answered within 30 seconds. GAO also covertly tested the VCL’s text 

messaging services and found that 4 of 14 GAO test text messages did not 

receive responses. GAO recommended that VA regularly test the VCL’s text 

messaging system to identify issues and correct them promptly. In response, VA 

developed and implemented procedures to regularly test the VCL's text 

messaging system, as well as its telephone and online chat systems.

In May 2016, GAO found that VA had taken a number of steps to improve its 

monitoring of the VCL primary center operations. For example, VA had 

established a permanent VCL call center evaluation team and created a 

mechanism for tracking complaints about the performance of the VCL primary 

center from VCL callers or external parties. However, GAO found that VA had

not specified quantifiable or otherwise measurable targets and had not included 

dates for when it would expect the VCL to complete actions covered by each key 

performance indicator. This was inconsistent with guidance provided by the 

Office of Management and Budget. GAO recommended that VA document 

clearly stated and measurable targets and time frames for key performance 

indicators needed to assess VCL performance. Although VA agreed, as of March 

2017, this recommendation remains open.

GAO also found that VA had established an interagency agreement with its 

service partner, the Department of Health and Human Services’ (HHS) 

Substance Abuse and Mental Health Services Administration (SAMHSA), to 

manage the shared operations of the VCL and Lifeline (a public-private network 

that provides free and confidential support to people in suicidal crisis or 

emotional distress). Despite these efforts to coordinate, VA and SAMHSA did not 

collect information needed to assess how often and why callers intending to 

reach the VCL do not follow voice prompts and instead reach Lifeline local crisis 

centers. As a result, VA and SAMHSA did not know the extent to which this 

occurred and could not determine the underlying causes that may need to be 

addressed. GAO recommended that VA and SAMHSA (1) collect information on 

how often and why callers intending to reach the VCL instead reach Lifeline local 

crisis centers and (2) review the information collected and, if necessary, develop 

plans to address the underlying causes. Although VA and SAMHSA agreed, as 

of March 2017, these recommendations remain open.

View GAO-17-545T. For more information, 

contact Seto J. Bagdoyan at (202) 512-6722 

or bagdoyans@gao.gov or Randall B. 

Williamson at (202) 512-7114 or 

williamsonr@gao.gov.

Why GAO Did This Study

VA established the VCL in July 2007 to 

provide support to veterans in 

emotional crisis. Between fiscal years 

2008, its first full year of operation, and 

2015, the number of calls received by 

the VCL increased almost 700 percent, 

exceeding VA’s expectations. As VA 

began to address increasing numbers 

of requests for assistance, reports of 

dissatisfaction with VCL’s service 

periodically appeared in the media.

This statement summarizes GAO’s 

May 2016 report on VA’s oversight of 

the VCL. Specifically, the statement 

describes (1) the extent to which VA 

met response-time goals for VCL calls 

and text messages, (2) how VA 

monitored VCL primary center call 

center operations, and (3) how VA 

worked with VCL service partners to 

help ensure veterans receive highquality service. For the May 2016 

report, GAO visited the VCL’s primary 

center and two backup call centers; 

tested VCL response time through a 

generalizable sample of covert 

telephone calls and a nongeneralizable 

sample of text messages in July and 

August 2015; reviewed internal reports 

and policies and plans; and 

interviewed VA officials. GAO 

contacted VA and SAMHSA for an 

update on actions taken in response to 

the report’s recommendations.

What GAO Recommends

In its May 2016 report, GAO made four 

recommendations to VA and HHS to 

improve the VCL. VA and HHS 

concurred with the recommendations

and while action has been taken on 

one, three remain open as of March 

2017.
Letter

Page 1 GAO-17-545T 

Chairman Roe, Ranking Member Walz, and Members of the Committee:

We are pleased to submit this statement on our May 2016 report 

regarding the Department of Veterans Affairs’ (VA) Veterans Crisis Line 

(VCL).1 Upon returning home from deployments in Afghanistan, Iraq, 

Vietnam, and other locations, many servicemembers struggle with mental 

health issues, including post-traumatic stress disorder, depression, and 

substance abuse. Several of these mental health issues have been 

identified as risk factors for suicide among veterans. As part of the 

continuum of mental health services it provides, VA established the VCL 

in July 2007.2

The VCL supports veterans in emotional crisis and helps implement VA’s 

goal of improving mental health outcomes for servicemembers, veterans, 

and their families through a number of actions—including reducing 

barriers to seeking mental health treatment and expanding access to VA 

services. During the time of our review for the May 2016 report, the VCL 

operated through a VA-operated primary center staffed with VA-employed 

responders and five backup call centers that provided additional 

responders and other services through a backup call coverage contract.3

Veterans can access the VCL by calling a national toll-free number—1-

800-273-TALK (8255). The VCL and the National Suicide Prevention 

Lifeline (Lifeline) share this national number through an interagency 

agreement between the VA and the Substance Abuse and Mental Health 

 1

GAO, Veterans Crisis Line: Additional Testing, Monitoring, and Information Needed to 

Ensure Better Quality Service, GAO-16-373 (Washington, D.C.: May 26, 2016).

2

VA established its crisis line at the VA medical center located in Canandaigua, New York. 

The original name of VA’s crisis line was the National Veterans Suicide Prevention Hotline 

until it was rebranded as the VCL in 2011.

3

For the purposes of this statement, the term “VCL service partners” includes the 

Substance Abuse and Mental Health Services Administration (SAMHSA), the VCL backup 

call coverage contractor, and the backup call centers that this contractor used to provide 

coverage to the VCL at the time of our 2016 review. 

VA has since opened an additional call center in Atlanta.

Letter
Page 2 GAO-17-545T 

Services Administration (SAMHSA).4 In addition to responding to calls, 

the VCL can also be accessed via online chat and text message.

Since it was established, demand for the VCL’s services has exceeded 

VA’s expectations. The VCL received about 534,000 calls in fiscal year 

2015, an almost 700 percent increase from the about 67,000 calls it 

received in fiscal year 2008, its first full year of operation. In response, VA 

steadily increased the VCL’s spending from about $3 million to $30 million 

from fiscal year 2008 through fiscal year 2015, devoting additional staff 

and resources to the VCL over time. As VA endeavored to address 

increasing numbers of requests for assistance, reports of dissatisfaction 

with VCL service periodically appeared in the media, and the VA Office of 

Inspector General was asked to investigate complaints about the VCL’s 

lack of timely response to callers.5 The Inspector General identified gaps 

in the VCL quality-assurance process, including challenges associated 

with supervisory review, tracking of issues, and collection and analysis of 

data from VCL backup call centers. In addition, the Inspector General 

found that in some cases callers did not receive immediate assistance 

from responders.

Our statement discusses (1) the extent to which VA met response-time 

goals for calls, online chats, and text messages received through the 

VCL; (2) how VA monitored the performance of the VCL primary center 

responders and call center operations; and (3) how VA worked with VCL 

service partners—backup call centers and SAMHSA—to help ensure 

veterans receive high-quality service from responders. This statement is 

based on our May 2016 report on VA’s oversight of the VCL as well as 

updates from VA and SAMHSA about efforts to address the report’s 

recommendations.

 4

The VCL is distinct from Lifeline, which operates through a network of private, nonprofit 

providers working independently of one another while maintaining agreed-upon clinical 

standards. SAMHSA is an agency within the U.S. Department of Health and Human 

Services that leads public health efforts to advance the behavioral health of the nation. 

SAMHSA funds a cooperative agreement grant to administer Lifeline with the same entity 

that VA contracts with to provide VCL backup call coverage. Through this interagency 

agreement, VA and SAMHSA set out to establish a seamless crisis management system 

through a collaborative and cooperative relationship between the agencies that provides 

consistent suicide prevention techniques to callers

5

Department of Veterans Affairs, Office of Inspector General, Veterans Crisis Line Caller 

Response and Quality Assurance Concerns (Washington, D.C.: 2016). 
Page 3 GAO-17-545T 

For the May 2016 report, we made covert test telephone calls, text 

messages, and online chats to assess the extent to which VA met its 

response-time goals through the VCL. The test calls included a 

generalizable sample of 119 calls that could be used to describe all 

callers’ wait times when calling the VCL during July and August of 2015. 

We also sent a nongeneralizable sample of 15 test online chats to the 

VCL and 14 test text messages during the same time period. In addition, 

we examined telephone call, online chat, and text message data and 

summary reports from January 2013 through December 2015 that VA 

maintained related to the timeliness of the VCL’s operations.6

We reviewed VCL policies, procedures, and monitoring data and 

interviewed VA officials. We also compared VA’s use of key performance 

indicators to the Office of Management and Budget’s guidance on 

performance goals, which are consistent with the Government 

Performance and Results Modernization Act of 2010.7 We observed call 

centers’ operations and interviewed officials and representatives of the 

VCL primary center and two of the five VCL backup call centers to 

examine the extent to which VA coordinates with the VCL’s service 

partners in ensuring that veterans receive high-quality service from 

responders. Further, we reviewed VA’s contract that provides backup call 

coverage and VA’s interagency agreement with SAMHSA. We also made 

34 covert calls in which we mimicked the experience of veterans who did 

not follow the instructions of a voice prompt to press “1” to reach the VCL. 

Finally, to examine the extent to which VA had plans to improve VCL 

operations, we reviewed VA’s improvement plans and interviewed VA 

officials responsible for planning and implementing those improvements.

More detailed information on our objectives, scope, and methodology for 

this work can be found in our 2016 report.

We conducted the work on which this statement is based in accordance 

with generally accepted government auditing standards. Those standards 

require that we plan and perform the audit to obtain sufficient, appropriate 

 6

We reviewed telephone call data to determine how many calls were answered at the VCL 

primary center; we reviewed online chat data to determine how many online chat requests 

received by the VCL received a response within 1 minute; and we reviewed text message 

data to determine how many text messages sent to the VCL received a response within 2 

minutes. 

7

See Office of Management and Budget, Preparation, Submission, and Execution of the 

Budget—Strategic Plans, Annual Performance Plans, Performance Reviews, and Annual 

Program Performance Reports, Circular No. A-11, pt. 6 (Washington, D.C.: June 2015).
Page 4 GAO-17-545T 

evidence to provide a reasonable basis for our findings and conclusions 

based on our audit objectives. We believe that the evidence obtained 

provides a reasonable basis for our findings and conclusions based on 

our audit objectives. In addition, the related investigative work was 

performed in accordance with the standards prescribed by the Council of 

the Inspectors General on Integrity and Efficiency.

In 2007, VA established the VCL, a 24-hour crisis line staffed by 

responders trained to assist veterans in emotional crisis. Through an 

interagency agreement, VA collaborated with SAMHSA to use a single, 

national toll-free number for crisis calls that serves both Lifeline and the 

VCL.8 Through this interagency agreement, VA and SAMHSA set up a 

cooperative relationship between the agencies that would provide 

consistent suicide-prevention techniques to callers.

The national toll-free number presents callers with choices. Callers are 

greeted by a recorded message that explains the function of the crisis line 

and prompts individuals to press “1” to reach the VCL. Callers who do not 

press “1” by the end of the message are routed to one of Lifeline’s 164 

local crisis centers.9 All callers who press “1” are routed first to the VCL 

primary center. Calls that are not answered at the VCL primary center 

within 30 seconds of the time that the caller presses “1” during the Lifeline 

greeting are automatically routed to one of five VCL backup call centers. 

If a call is not answered by the VCL backup call center that initially 

receives it, the call may be sent to another VCL backup call center.10 VA 

entered into a contract with a firm to oversee the operations of the VCL 

backup call centers.

At the time of our 2016 report, there were a total of 164 Lifeline local crisis 

centers, 5 of which also serve the VCL.11

 8

SAMHSA and the Mental Health Association of New York City launched Lifeline on 

January 1, 2005. Lifeline provides free and confidential emotional support to people in 

suicidal crisis or emotional distress, 24 hours a day, 7 days a week. 

9

The automated greeting also prompts Spanish speakers to press “2” for assistance in 

Spanish. 

10Some VCL backup call centers do not allow calls to be rerouted to another VCL backup 

call center and instead hold the call in a queue awaiting response by that backup call 

center’s responders. 

11VA does not directly contract with any of the VCL backup call centers. 

Background
Page 5 GAO-17-545T 

VA added online chat and text message capabilities to the VCL in fiscal 

years 2009 and 2012, respectively. The number of online chats and text 

messages handled by the VCL generally increased every year, though 

the number of online chats decreased in fiscal year 2015.

In our covert testing of the VCL’s call response time in July and August 

2015, we found that it was uncommon for VCL callers to wait an extended 

period before reaching a responder since all of our calls that reached the 

VCL were answered in less than 4 minutes. However, we also found VA 

did not meet its goal of answering 90 percent of calls to the VCL within 30 

seconds for test calls that we made. Our test calls included a 

generalizable sample of 119 test calls that could be used to describe all 

callers’ wait times when calling the VCL during this period.12 On the basis 

of our test calls, we estimated that during July and August 2015 about 73 

percent of all VCL calls were answered at the VCL primary center within 

30 seconds.13 VA officials told us that, during fiscal year 2015, about 65 to 

75 percent of VCL calls were answered at the VCL primary center and 

 12For these test calls, callers’ wait times refer to the length of time that elapses between 

when callers press “1” and when responders at either the VCL primary center or backup 

call centers answer the calls. 

13In addition, we estimated that during July and August of 2015, 99 percent of all VCL 

calls were answered within 120 seconds and the median call response time was 17 

seconds. These percentage estimates have a margin of error of within plus or minus 9 

percentage points, and the median response times estimates have a relative margin of 

error that is less than 9 percent at the 95 percent confidence level. 

Extended Call Wait 

Times Were 

Uncommon in July 

and August 2015, but 

VA Did Not Meet Its 

Call Response Time 

Goals and Some Text 

Messages Did Not 

Receive Responses

VA Responded to Most 

Calls within 30 Seconds in 

July and August 2015, but 

Did Not Meet Its Goal to 

Answer 90 Percent of 

Calls within 30 Seconds at 

the VCL Primary Center
Page 6 GAO-17-545T 

about 25 to 35 percent of VCL calls were answered at the backup call 

centers. These VA-reported results indicate that about 65 to 75 percent of 

VCL calls were answered within either 30 or 60 seconds.14 These results 

are consistent with our test results for July and August 2015.

During our 2016 review, VA officials told us that VA attempts to maximize 

the percentage of calls answered at the VCL primary center because 

these responders have additional resources—including access to 

veterans’ VA electronic medical records—that are unavailable to VCL 

backup call center responders. All responders—whether at primary or 

backup centers—receive specialized training to assist callers in crisis.15

To improve its performance toward meeting the goal of answering 90 

percent of calls at the VCL primary center within 30 seconds, VA 

implemented two changes in fiscal year 2015—namely, staggered work 

shifts for responders and new call-handling procedures.

Staggered work shifts. VA implemented staggered shifts for responders 

at the VCL primary center on September 6, 2015. Staggered shifts are 

work schedules that allow employees to start and stop their shifts at 

different times as a way to ensure better coverage during peak calling 

periods. Specifically, it helps schedule more employees to work when call 

volume is highest and fewer employees to work when call volume is 

lowest.16 Additionally, staggered shifts help limit disruptions in service as 

responders begin and end their shifts.

By comparing VCL telephone call data from September through 

December of 2014 to that of September through December of 2015, we 

found that VA’s implementation of staggered shifts at the VCL primary 

 14For approximately 5 months of fiscal year 2015, VA allowed calls to ring at the VCL 

primary center for 60 seconds before routing the calls to VCL backup call centers. VA then 

returned to the standard that calls not answered at the VCL primary center within 30 

seconds are then routed to VCL backup call centers. 

15All VCL primary and backup call center responders are required to complete Applied 

Suicide Intervention Skills Training in which they learn to use a suicide intervention model 

to identify persons with thoughts of suicide, seek a shared understanding of reasons for 

dying and living, develop a safe plan based on a review of risk, be prepared to do followup, and become involved in suicide-safer community networks. 

16The International Customer Management Institute includes staggered shifts as a best 

practice among call centers. 

To Help Achieve 

Response-Time Goals, VA 

Implemented Changes at 

the VCL Primary Center
Page 7 GAO-17-545T 

center had mixed results.17 For example, the average percentage of calls 

answered per hour at the VCL primary center from September through 

December 2015—after staggered shifts were implemented—was 75 

percent, slightly less than the average of 79 percent answered during the 

corresponding period in 2014 before staggered shifts were implemented. 

However, the VCL received an average of about 1.3 more calls per hour 

during this period in 2015 than it received during the corresponding period 

in 2014 and, according to VA officials, the VCL primary center employed

fewer responder staff in 2015 than 2014.

New call-handling procedures. VA implemented new call handling 

procedures at the VCL primary center beginning in June 2015 that 

provided responders with specific guidance to more efficiently handle 

“noncore” callers—those callers who were not seeking crisis assistance 

but rather seeking help with other issues, such as help with veterans’ 

benefits questions. For example, if a caller reached the VCL with a 

question about VA disability benefits, the VCL primary center responder 

would verify that the caller was not in crisis and transfer the caller to the 

Veterans Benefits Administration to address the question.

VCL telephone call data provided by VA suggest that the average time 

VCL primary center responders spent handling noncore calls decreased 

by approximately 30 percent over a 5-month period after responder 

training began on these new call-handling procedures.18 We would expect 

that as the average time VCL primary center responders spent handling 

noncore calls decreased, these responders would have more time 

available to answer additional incoming calls.

 17Our analysis compared VCL call data from September 6, 2015, through December 31, 

2015, to VCL call data from September 1, 2014, through December 31, 2014. The 

percentage of calls answered was likely affected by several factors, such as call volume, 

staffing levels, and complexity of calls. Our analysis controlled for day of the week, time of 

day, and holidays, but did not control for all factors that may affect the percentage of calls 

answered. 

18We did not test this aspect of VCL operations with covert test calls. 
Page 8 GAO-17-545T 

To determine the timeliness of the VCL’s responses to online chats and 

text messages, we conducted a covert test in July and August 2015 using 

nongeneralizable samples of 15 online chats and 14 text messages. All 

15 of our test online chats received responses within 60 seconds, 13 of 

which were within 30 seconds. This result was consistent with VA data 

that indicated VCL responders sent responses to over 99 percent of 

online chat requests within 1 minute in fiscal years 2014 and 2015. During 

our 2016 review, VA officials told us that all online chats are expected to 

be answered immediately. Although this was an expectation, we found in 

2016 that VA did not have formal performance standards for how quickly 

responders should answer online chat requests and expected to develop 

them before the end of fiscal year 2016.

However, our tests of text messages revealed a potential area of concern. 

Four of our 14 test text messages did not receive a response from the 

VCL. Of the remaining 10 test text messages, 8 received responses 

within 2 minutes, and 2 received responses within 5 minutes.

As we reported in May 2016, VA officials stated that text messages are 

expected to be answered immediately, but, as with online chats, VA had 

not developed formal performance standards for how quickly responders 

should answer text messages. VA data indicated that VCL responders 

sent responses to 87 percent of text messages within 2 minutes of 

initiation of the conversation in both fiscal years 2014 and 2015. During 

our 2016 review, VA officials said that VA planned to establish 

performance standards for answering text messages before the end of 

fiscal year 2016. VA officials noted and we observed during a site visit 

that some incoming texts were abusive in nature or were not related to a 

crisis situation.19 According to VA officials, in these situations, if this is the 

only text message waiting for a response, a VCL responder will send a 

response immediately. However, if other text messages are awaiting 

responses, VA will triage these text messages and reply to those with 

indications of crisis first. This triage process may have contributed to the 

number of our test texts that did not receive responses within 2 minutes.

The VCL’s text messaging service provider offered several reasons for 

the possible nonresponses that we encountered in our test results. These 

included: (1) incompatibilities between some devices used to send text 

messages to the VCL and the software VA used to process the text 

 19Our test text messages consisted of a simple greeting, such as “Hi” or “Hello.” 

In July and August 2015, 

Most of Our Test Online 

Chats Were Answered 

Within 30 Seconds, but VA 

Did Not Ensure That 

Veterans Received 

Responses through Its 

Text Messaging Service
Page 9 GAO-17-545T 

messages, (2) occasional software malfunctions that freeze the text 

messaging interface at the VCL primary center, (3) inaudible audio 

prompts used to alert VCL primary center responders of incoming text 

messages, (4) attempts by people with bad intentions to disrupt the VCL’s 

text messaging service by overloading the system with a large number of 

texts, and (5) incompatibilities between the web browsers used by the 

VCL primary center and the text messaging software.

At the time of our 2016 review, VA officials told us that they did not 

monitor and test the timeliness and performance of the VCL text 

messaging system, but rather relied solely on the VCL’s text messaging 

service provider for such monitoring and testing. They said that the 

provider had not reported any issues with this system. According to the 

provider, no routine testing of the VCL’s text messaging system was 

conducted. Standards for internal control in the federal government state 

that ongoing monitoring should occur in the course of normal operations, 

be performed continually, and be ingrained in the agency’s operations.20

We concluded that without routinely testing its text messaging system, or 

ensuring that its provider tests the system, VA cannot ensure that it is 

identifying limitations with its text messaging service and resolving them 

to provide consistent, reliable service to veterans.

We recommended that VA regularly test the VCL’s text messaging 

system to identify issues and correct them promptly. In response, VA 

developed and implemented procedures to regularly test the VCL’s text 

messaging system, as well as its telephone and online chat systems. We 

believe this change will allow VA to more reliably and quickly identify and 

correct errors in the text messaging system and therefore help veterans 

reach VCL responders in a timelier manner.

 20See GAO, Standards for Internal Control in the Federal Government, 

GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999). 
Page 10 GAO-17-545T 

As we reported in May 2016, VA had sought to enhance its capabilities 

for overseeing VCL primary center operations through a number of 

activities—including establishing a call center evaluation team, 

implementing revised performance standards for VCL primary center 

responders, implementing silent monitoring of VCL primary center 

responders, and analyzing VCL caller complaints.21

Establishment of a call center evaluation team. In October 2013, VA 

established a permanent VCL call center evaluation team that is 

responsible for monitoring the performance of the VCL primary center.22

As we reported in May 2016, the call center evaluation team analyzes 

VCL data, including information on the number of calls received and the 

number of calls routed to backup call centers from the primary center. VA 

officials told us that they use these data to inform management decisions 

about VCL operations.

Implementation of revised performance standards for VCL primary 

center responders. In October 2015, VA implemented new performance 

standards for all VCL primary center responders that will be used to 

assess their performance in fiscal year 2016. According to VA officials, 

 21VCL also tracks and analyzes complaints about the services of VCL backup call centers 

as a part of this effort. 

22According to VA officials, this team was initially staffed with VA employees detailed from 

other areas of the department in December 2012. Permanent staff for call center 

evaluation were hired in October 2013. 

VA Had Taken Steps 

to Improve Its 

Monitoring of VCL 

Primary Center 

Performance but Had 

Not Established 

Targets and Time 

Frames for VCL Key 

Performance 

Indicators

VA Established a Call 

Center Evaluation Team, 

Implemented Revised 

Responder Performance 

Standards, and Analyzed 

VCL Caller Complaints
Page 11 GAO-17-545T 

these performance standards include several measures of responder 

performance—such as demonstrating crisis-intervention skills, identifying 

callers’ needs, and addressing those needs in an appropriate manner 

using VA approved resources.

Silent monitoring of VCL primary center responders. In February 

2016, VA officials reported that they were beginning silent monitoring of 

all VCL responders using recently developed standard operating 

procedures, standard data collection forms, and standard feedback 

protocols.

Analysis of VCL caller complaints. In October 2014, VA created a 

mechanism for tracking complaints it receives from VCL callers and 

external parties, such as members of Congress and veterans, about the 

performance of the VCL primary and backup call centers. According to 

VA officials, each complaint is investigated to validate its legitimacy and 

determine the cause of any confirmed performance concerns. The results 

and disposition of each complaint are documented in VA’s complaint 

tracking database.

In 2011, VA established key performance indicators to evaluate VCL 

primary center operations; however, in our May 2016 review, we found 

these indicators did not have established measureable targets or time 

frames for their completion.

VCL key performance indicators lacked measurable targets. We 

found that VA’s list of VCL key performance indicators did not include 

information on the targets the department had established to indicate their 

successful achievement. For example, VA included a key performance 

indicator for the percentage of calls answered by the VCL in this list but 

did not include information on what results would indicate success for (1) 

the VCL as a whole, (2) the VCL primary center, or (3) the VCL backup

call centers. As another example, VA had not established targets for the 

percentage of calls abandoned by callers prior to speaking with VCL 

responders. Measureable targets should include a clearly stated minimum 

performance target and a clearly stated ideal performance target.23 These 

targets should be quantifiable or otherwise measurable and indicate how 

 23See GAO, VA Health Care: Additional Guidance, Training, and Oversight Needed to 

Improve Clinical Contract Monitoring, GAO-14-54 (Washington, D.C.: Oct. 2013). 

VCL Key Performance 

Indicators Lacked 

Measureable Targets and 

Time Frames
Page 12 GAO-17-545T 

well or at what level an agency or one of its components aspires to 

perform.24 Such measurable targets are important for ensuring that the 

VCL call center evaluation team can effectively measure VCL 

performance.

VCL key performance indicators lack time frames for their 

completion. We found that VA’s list of VCL key performance indicators 

did not include information on when the department expected the VCL to

complete or meet the action covered by each key performance indicator. 

For example, for VA’s key performance indicator for the percentage of 

calls answered by the VCL, the department had not included a date by 

which it would expect the VCL to complete this action. As another 

example, VA had not established dates by which it would meet targets yet 

to be established for the percentage of calls abandoned by callers prior to 

speaking with VCL responders. Time frames for action are a required 

element of performance indicators and are important to ensure that 

agencies can track their progress and prioritize goals.25

Guidance provided by the Office of Management and Budget states that 

performance goals—similar to VA’s key performance indicators for the 

VCL—should include three elements: (1) a performance indicator, which 

is how the agency will track progress; (2) a target; and (3) a period.26

Without establishing targets and time frames for the successful 

completion of its key performance indicators for the VCL, we concluded 

that VA could not effectively track and publicly report progress or results 

for its key performance indicators for accountability purposes.

 24Consistent with the Government Performance and Results Modernization Act of 2010, 

the Office of Management and Budget states that a performance goal should include a 

tangible, measurable objective or a quantifiable standard, value, or rate. See Office of 

Management and Budget, Preparation, Submission, and Execution of the Budget—

Strategic Plans, Annual Performance Plans, Performance Reviews, and Annual Program 

Performance Reports. 

25Consistent with the Government Performance and Results Modernization Act of 2010, 

the Office of Management and Budget defines a performance goal as a statement of the 

level of performance to be accomplished within a time frame. See Office of Management 

and Budget, Preparation, Submission, and Execution of the Budget—Strategic Plans, 

Annual Performance Plans, Performance Reviews, and Annual Program Performance 

Reports. 

26See Office of Management and Budget, Preparation, Submission, and Execution of the 

Budget—Strategic Plans, Annual Performance Plans, Performance Reviews, and Annual 

Program Performance Reports. 
Page 13 GAO-17-545T 

We recommended that VA document clearly stated and measurable 

targets and time frames for key performance indicators needed to assess 

VCL performance. While VA officials have informed us that they have 

created scorecards that track information related to calls answered, 

staffing, and average handle times, as of March 2017, clearly stated and 

measurable targets and time frames have not yet been developed.

As we reported in May 2016, VA’s backup call coverage contract, 

awarded in October 2012 and in place at the time of our review, did not 

include detailed performance requirements in several key areas for the 

VCL backup call centers. Clear performance requirements for VCL 

backup call centers are important for defining VA’s expectations of these 

service partners. However, VA had taken steps to strengthen the 

performance requirements of this contract by modifying it in March 2015 

and was beginning the process of replacing it with a new contract. 

According to VA officials, the new contract was awarded in April 2016.

October 2012 backup call coverage contract. This contract provided a 

network of Lifeline local crisis centers that could serve as VCL backup call 

VA Was 

Strengthening 

Requirements for 

VCL Backup Call 

Centers, but VA and 

SAMHSA Did Not 

Collect Information to 

Assess How Often 

and Why Callers 

Were Not Reaching 

the VCL

VA Was Enhancing 

Performance 

Requirements for Its 

Backup Call Coverage 

Contractor
Page 14 GAO-17-545T 

centers managed by a contractor.27 This contractor was responsible for 

overseeing and coordinating the services of VCL backup call centers that 

answer overflow calls from the VCL primary center. This contract as 

initially awarded included few details on the performance requirements for 

VCL backup call centers. For example, the contract did not include any 

information on the percentage of VCL calls routed to each VCL backup 

call center that should be answered. Detailed performance requirements 

on these key aspects of VCL backup call center performance are 

necessary for VA to effectively oversee the performance of the contractor 

and the VCL backup call centers. By not specifying performance 

requirements for the contractor on these key performance issues, we 

believe that VA missed the opportunity to validate contractor and VCL 

backup call center performance and mitigate weaknesses in VCL call 

response.

As we reported in May 2016, VA officials told us about several concerns 

with the performance of the backup call centers operating under the 

October 2012 contract based on their own observations and complaints 

reported to the VCL. These concerns included the inconsistency and 

incompleteness of VCL backup call centers’ responses to VCL callers, 

limited or missing documentation from records of VCL calls answered by 

VCL backup call center responders, limited information provided to VA 

that could be used to track VCL backup call center performance, and the 

use of voice answering systems or extended queues for VCL callers 

reaching some VCL backup call centers. For example, VA officials 

reported that some veterans did not receive complete suicide 

assessments when their calls were answered at VCL backup call centers. 

In addition, VA officials noted that they had observed some VCL backup 

call centers failing to follow VCL procedures, such as not calling a veteran 

who may be in crisis when a third-party caller requested that the 

responder contact the veteran. According to VA officials, these issues led 

to additional work for the VCL primary center, including staffing one to two 

responders per shift to review the call records submitted to the VCL 

primary center by backup call centers and to determine whether these 

calls required additional follow-up from the VCL primary center. These 

 27The backup call coverage contract in place at the time of our review was awarded in 

October 2012 with a 1-year base and two 1-year option periods (for a total of 3 years of 

coverage) and was set to expire in September 2015. However, according to VA officials, 

the contract was extended through May 2016 while the department was finalizing a new 

contract. VA officials reported that the new backup call coverage contract was awarded in 

April 2016. 
Page 15 GAO-17-545T 

officials estimated that 25 to 30 percent of backup call center call records 

warranted additional follow-up to the caller from a VCL primary center 

responder, including approximately 5 percent of backup call center call 

records that needed to be completely reworked by a VCL primary center 

responder.

March 2015 backup call coverage contract modification. Given these 

concerns, in March 2015, VA modified the October 2012 backup call 

coverage contract to add more explicit performance requirements for its 

backup call coverage contractor, which likely took effect more quickly 

than if the department had waited for a new contract to be awarded. 

These modified requirements included (1) the establishment of a 24-

hours-a-day, 7-days-a-week contractor-staffed emergency support line 

that VCL backup call centers could use to report problems, (2) a 

prohibition on VCL backup call centers’ use of voice answering systems, 

(3) a prohibition on VCL backup call centers placing VCL callers on hold 

before a responder conducted a risk assessment, (4) documentation of 

each VCL caller’s suicide risk assessment results, and (5) transmission of 

records for all VCL calls to the VCL primary center within 30 minutes of 

the call’s conclusion.

Development of new backup call coverage contract. In July 2015, VA 

began the process of replacing its backup call coverage contract by 

publishing a notice to solicit information from prospective contractors on 

their capability to satisfy the draft contract terms for the new contract; this 

new backup call coverage contract was awarded in April 2016.28 We 

found that these new proposed contract terms included the same 

performance requirement modifications that were made in March 2015, 

as well as additional performance requirements and better data reporting 

from the contractor that could be used to improve VA’s oversight of the 

VCL backup call centers. Specifically, the proposed contract terms added 

performance requirements to address VCL backup call center 

performance—including a requirement for 90 percent of VCL calls 

received by a VCL backup call center to be answered by a backup call 

center responder within 30 seconds and 100 percent to be answered by a 

backup call center responder within 2 minutes. In addition, the proposed 

contract terms included numerous data reporting requirements that could 

allow VA to routinely assess the performance of its VCL backup call 

 28This notice—referred to as a sources sought notice—included a draft performance work 

statement. In April 2016, VA officials reported that this contract was awarded to the 

previous backup call coverage contractor. 
Page 16 GAO-17-545T 

centers and identify patterns of noncompliance with the contract’s 

performance requirements more efficiently and effectively than under the 

prior contract. The proposed terms for the new contract also state that VA 

will initially provide and approve all changes to training documentation 

and supporting materials provided to VCL backup call centers in order to 

promote the contractor’s ability to provide the same level of service that is 

being provided by the VCL primary center.

In May 2016, we found that when callers did not press “1” during the initial 

Lifeline greeting, their calls may take longer to answer than if the caller 

had pressed “1” and been routed to either the VCL primary center or a 

VCL backup call center.29 As previously discussed, VA and SAMHSA 

collaborated to link the toll-free numbers for both Lifeline and the VCL 

through an interagency agreement. The greeting instructs callers to press 

“1” to be connected to the VCL; if callers do not press “1,” they will be 

routed to one of SAMHSA’s 164 Lifeline local crisis centers. To mimic the 

experience of callers who did not press “1” to reach the VCL when 

prompted, we made 34 covert nongeneralizable test calls to the national 

toll-free number that connects callers to both Lifeline and the VCL during 

August 2015 and we did not press “1” to be directed to the VCL.30 For 23 

of these 34 calls, our call was answered in 30 seconds or less. For 11 of 

these calls, we waited more than 30 seconds for a responder to answer—

including 3 calls with wait times of 8, 9, and 18 minutes. Additionally, one 

of our test calls did not go through, and during another test call we were 

asked if we were safe and able to hold.31 VA’s policy prohibits VCL 

responders from placing callers on hold prior to completing a suicide 

assessment; Lifeline has its own policies and procedures.32

 29At the time of our tests, the initial greeting was about 30 seconds long and prompted the 

caller to press “1” to be connected to the VCL at the end of the greeting. If callers did not 

press”1,” the call was routed to one of SAMHSA’s 164 Lifeline local crisis centers based 

on the area code of the callers’ telephone numbers. 

30These 34 calls were a random but nongeneralizable sample. 

31When asked if we were safe and could hold, we terminated this test call. 

32We did not review Lifeline’s policies and procedures as a part of our May 2016 report 

due to our focus on the VCL. We focused our review of Lifeline on those elements of their 

operations that interacted with the VCL or VA, such as the interagency agreement 

between VA and SAMHSA that governs the shared use of a single national toll-free 

number between the VCL and Lifeline. 

VA and SAMHSA Did Not 

Collect Information 

Needed to Assess How 

Often and Why Callers 

Were Not Reaching the 

VCL
Page 17 GAO-17-545T 

According to officials and representatives from VA, SAMHSA, and the 

VCL backup call centers, as well as our experience making test calls 

where we did not press “1,” there are several reasons why a veteran may 

not press “1” to be routed to the VCL, including

• an intentional desire to not connect with VA,

• failure to recognize the prompt to press “1” to be directed to the VCL,

• waiting too long to respond to the prompt to press “1” to be directed to 

the VCL, or

• calling from a rotary telephone that does not allow the caller to press 

“1” when prompted.

As we found in May 2016, VA officials had not estimated the extent to 

which veterans intending to reach the VCL did not press “1” during the 

Lifeline greeting.33 These officials explained that their focus had been on 

ensuring that veterans who did reach the VCL received appropriate 

service from the VCL primary center and backup call centers. In addition, 

SAMHSA officials said that they also did not collect this information.34

These officials reported that SAMHSA did not require the collection of 

demographic information, including veteran status, for a local crisis center 

to participate in the Lifeline network. However, they noted that SAMHSA 

could request through its grantee that administers the Lifeline network 

that local crisis centers conduct a one-time collection of information to 

help determine how often and why veterans reach Lifeline local crisis 

centers. SAMHSA officials explained that they could work with the Lifeline 

grantee to explore optimal ways of collecting this information that would 

be (1) clinically appropriate, (2) a minimal burden to callers and Lifeline’s 

local crisis centers, and (3) in compliance with the Office of Management 

and Budget’s paperwork reduction and information collection policies. The 

interagency agreement between VA and SAMHSA assigns SAMHSA 

responsibilities for monitoring the use of the national toll-free number that 

 33According to SAMHSA officials, in 2014, about 383,000 callers abandoned their calls to 

Lifeline during the initial greeting used to direct callers to either Lifeline local crisis centers 

or the VCL. We did not assess the reasons these calls were abandoned. 

34According to SAMHSA officials, the SAMHSA grantee responsible for administering 

Lifeline conducted a survey in 2014 that captured veteran-related data. However, 

SAMHSA had no involvement with this survey or the data collection activities of the 

Lifeline local crisis centers that provided the information because it was outside the scope 

of SAMHSA’s grant to the organization. Further, HHS stated that the SAMHSA grantee did 

not share the results of the survey with SAMHSA. We did not evaluate the results of this 

survey. 
Page 18 GAO-17-545T 

is used to direct callers to both the VCL and Lifeline. These 

responsibilities include monitoring the use of the line, analyzing trends, 

and providing recommendations about projected needs and technical 

modifications needed to meet these projected needs. Using the 

information collected from the Lifeline local crisis centers on how often 

and why veterans reach Lifeline, as opposed to the VCL, VA and 

SAMHSA officials could then assess whether the extent to which this 

occurs merits further review and action.

Although the results of our test were not generalizable, substantial wait 

times for a few of our covert calls suggested that some callers may 

experience longer wait times to speak with a responder in the Lifeline 

network than they would in the VCL’s network. We concluded that without 

collecting information to examine how often and why veterans do not 

press “1” when prompted to reach the VCL, VA and SAMHSA could not 

determine the extent veterans reach the Lifeline network when intending 

to reach the VCL and may experience longer wait times as a result. In 

addition, limitations in information on how often and why this occurs did 

not allow VA and SAMHSA to determine whether or not they should 

collaborate on plans to address the underlying causes of veterans not 

reaching the VCL. Standards for internal control in the federal 

government state that information should be communicated both 

internally and externally to enable the agency to carry out its 

responsibilities.35 For external communications, management should 

ensure there are adequate means of communicating with, and obtaining 

information from, external stakeholders that may have a significant impact 

on the agency achieving its goals.

We recommended VA and SAMHSA collaborate in taking the following 

two actions: (1) collect information on how often and why callers intending 

to reach the VCL instead reach Lifeline local crisis centers and (2) review 

the information collected and, if necessary, develop plans to address the 

underlying causes. We understand that VA and SAMHSA have been 

coordinating on these issues. However, as of March 2017, both of these 

recommendations remain open.

 35GAO/AIMD-00-21.3.1
Page 19 GAO-17-545T 

Chairman Roe, Ranking Member Walz, and Members of the Committee, 

this concludes our statement for the record.

For questions about this statement, please contact Seto J. Bagdoyan at 

(202) 512-6722 or bagdoyans@gao.gov or Randall B. Williamson at (202) 

512-7114 or williamsonr@gao.gov.

Contact points for our Offices of Congressional Relations and Public 

Affairs may be found on the last page of this statement.

In addition to the contacts listed, GAO staff who made key contributions 

to this statement are Jennie F. Apter, Christie Enders, Cathleen J. 

Hamann, Marcia A. Mann, Vikki Porter, Lisa Rogers, and Julie T. Stewart.

GAO Contact and 

Staff 

Acknowledgments

(101959)
This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
The Government Accountability Office, the audit, evaluation, and investigative 

arm of Congress, exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability of the 

federal government for the American people. GAO examines the use of public 

funds; evaluates federal programs and policies; and provides analyses, 

recommendations, and other assistance to help Congress make informed 

oversight, policy, and funding decisions. GAO’s commitment to good government 

is reflected in its core values of accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through GAO’s website (http://www.gao.gov). Each weekday afternoon, GAO 

posts on its website newly released reports, testimony, and correspondence. To 

have GAO e-mail you a list of newly posted products, go to http://www.gao.gov

and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of production and 

distribution and depends on the number of pages in the publication and whether 

the publication is printed in color or black and white. Pricing and ordering 

information is posted on GAO’s website, http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, MasterCard, 

Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, LinkedIn, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.

Visit GAO on the web at www.gao.gov and read The Watchblog.

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-4400, 

U.S. Government Accountability Office, 441 G Street NW, Room 7125, 

Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

James-Christian Blockwood, Managing Director, spel@gao.gov, (202) 512-4707

U.S. Government Accountability Office, 441 G Street NW, Room 7814, 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Strategic Planning and 

External Liaison

Please Print on Recycled Paper.