VETERANS CRISIS 

LINE

Additional Testing, 

Monitoring, and 

Information Needed to 

Ensure Better Quality 

Service 

Report to Congressional Requesters

May 2016

GAO-16-373

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-16-373, a report to 

congressional requesters

May 2016

VETERANS CRISIS LINE

Additional Testing, Monitoring, and Information 

Needed to Ensure Better Quality Service

Why GAO Did This Study

VA established the VCL in July 2007 to 

provide support to veterans in 

emotional crisis. Between fiscal years 

2008, its first full year of operation, and 

2015, the number of calls received by 

the VCL increased almost 700 percent, 

exceeding VA’s expectations. As VA 

began to address increasing numbers 

of requests for assistance, reports of 

dissatisfaction with VCL’s service 

periodically appeared in the media.

GAO was asked to review VA’s 

administration of the VCL. This report, 

among other issues, examines (1) the 

extent to which VA meets responsetime goals for VCL calls and text 

messages, (2) how VA monitors VCL 

primary center call center operations, 

and (3) how VA works with VCL 

service partners to help ensure 

veterans receive high-quality service. 

GAO visited the VCL’s primary center 

and two backup call centers; tested 

VCL response time through a 

generalizable sample of covert 

telephone calls and a nongeneralizable 

sample of text messages in July and 

August 2015; reviewed internal reports 

and policies and plans; and 

interviewed VA and SAMHSA officials.

What GAO Recommends

GAO recommends that VA regularly 

test VCL’s text messaging system and 

document targets and time frames for 

key performance indicators. GAO also

recommends that VA and SAMHSA 

collect information on how often and 

why callers reach Lifeline when 

intending to reach the VCL, review this 

information, and, if necessary, develop 

plans to address the causes. VA and 

HHS concurred with GAO’s

recommendations and described 

planned actions to address them.

What GAO Found

GAO found that the Department of Veterans Affairs (VA) did not meet its call 

response time goals for the Veterans Crisis Line (VCL), although extended call 

wait times were not common. VA’s goal is to answer 90 percent of VCL calls at 

the VCL primary center within 30 seconds. Currently, calls not answered within 

30 seconds route to VCL backup call centers; however, for 5 months of fiscal 

year 2015, calls were routed to VCL backup call centers after 60 seconds. VA 

officials told GAO that VA data show about 65 to 75 percent of VCL calls were 

answered at the VCL primary center in fiscal year 2015 within either 30 or 60 

seconds. GAO’s covert testing in July and August 2015 confirms VA’s data. 

Specifically, 119 covert test calls show that an estimated 73 percent of calls 

made during this period were answered within 30 seconds. GAO also estimates 

that 99 percent of all VCL calls during this period were answered within 120 

seconds. GAO also covertly tested the VCL’s text messaging services and found 

that 4 of 14 GAO test text messages did not receive responses. VA officials said

they do not monitor or test the timeliness and performance of the VCL text 

message system and instead rely solely on the VCL’s text messaging provider 

for these functions. VA officials told GAO that the provider had not reported any 

issues with the system, but the provider told GAO that routine testing of the VCL 

system is not conducted. Without routinely testing its text messaging system or 

ensuring that its provider does so, VA cannot identify limitations to this service.

While VA has taken a number of steps to improve its monitoring of the VCL 

primary center operations, VA has not developed measurable targets and time 

frames for its key performance indicators, such as the program’s percentage of 

abandoned calls. VA established a permanent VCL call center evaluation team 

and created a mechanism for tracking complaints about the performance of the 

VCL primary center from VCL callers or external parties. However, GAO found 

that VA has not specified quantifiable or otherwise measurable targets and has 

not included dates for when it would expect the VCL to complete actions covered 

by each key performance indicator. This is inconsistent with guidance provided 

by the Office of Management and Budget. As a result, VA cannot ensure that the 

VCL is providing consistent, high-quality services to callers and cannot effectively 

track and publicly report progress or results.

VA established an interagency agreement with its service partner, the

Department of Health and Human Services’ (HHS) Substance Abuse and Mental 

Health Services Administration (SAMHSA), to manage the shared operations of 

the VCL and the National Suicide Prevention Lifeline (Lifeline), which include a 

single national toll-free number used by both. Despite these efforts to coordinate, 

VA and SAMHSA do not collect information needed to assess how often and why 

callers intending to reach the VCL do not follow voice prompts and instead reach 

Lifeline local crisis centers. VA officials told GAO that the type of information that 

would be needed to do so is not collected because VA has focused on 

addressing the concerns of those callers who did reach the VCL. In addition, 

SAMHSA officials said that they do not require Lifeline local crisis centers to 

collect this type of information, noting that it would be possible to collect it. As a 

result, VA and SAMHSA do not know the extent to which this occurs and cannot 

determine the underlying causes that may need to be addressed.

View GAO-16-373. For more information, 

contact Seto J. Bagdoyan at (202) 512-6722 

or bagdoyans@gao.gov or Randall B. 

Williamson at (202) 512-7114 or 

williamsonr@gao.gov.
Page i GAO-16-373 Veterans Crisis Line

Letter 1

Background 5

Extended Call Wait Times Were Uncommon in July and August 

2015, but VA Did Not Meet Its Call Response Time Goals and 

Some Text Messages Did Not Receive Responses 10

VA Has Taken Steps to Improve Its Monitoring of VCL Primary 

Center Performance but Has Not Established Targets and Time 

Frames for VCL Key Performance Indicators 16

VA Is Strengthening Requirements for VCL Backup Call Centers, 

but VA and SAMHSA Do Not Collect Information to Assess 

How Often and Why Callers Are Not Reaching the VCL 21

VA Has Developed and Begun Implementation of Plans to 

Improve VCL Operations 27

Conclusions 29

Recommendations for Executive Action 30

Agency Comments and Our Evaluation 30

Appendix I Scope and Methodology 33

Testing VCL Response Times 33

Analysis of the VCL’s 2013 through 2015 Call Data 36

Test of Lifeline Local Crisis Center Response Times 37

Appendix II Comments from the Department of Veterans Affairs 38

Appendix III Comments from the Department of Health and Human Services 45

Appendix IV GAO Contacts and Staff Acknowledgments 47

Figures

Figure 1: Call Routing Process for the National Suicide Prevention 

Lifeline (Lifeline) and Veterans Crisis Line (VCL) 7

Figure 2: Veterans Crisis Line (VCL) Milestones, Call Volume, and 

Funding Obligations, Fiscal Years 2007 through 2015 8

Figure 3: Veterans Crisis Line (VCL) Online Chat and Text 

Message Volumes, Fiscal Years 2009 through 2015 9

Contents
Page ii GAO-16-373 Veterans Crisis Line

Figure 4: Wait Time Estimates for Calls to the Veterans Crisis Line 

(VCL), July and August 2015 11

Abbreviations

HHS Department of Health and Human Services

Lifeline National Suicide Prevention Lifeline

OMHO Office of Mental Health Operations

SAMHSA Substance Abuse and Mental Health Services 

Administration

VA Department of Veterans Affairs

VCL Veterans Crisis Line

VHA Veterans Health Administration

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-16-373 Veterans Crisis Line

441 G St. N.W.

Washington, DC 20548

May 26, 2016

The Honorable Johnny Isakson

Chairman

Committee on Veterans’ Affairs

United States Senate

The Honorable Jeff Miller

Chairman

Committee on Veterans’ Affairs

House of Representatives

The Honorable Joni K. Ernst

United State Senate

Upon returning home from deployments in Afghanistan, Iraq, Vietnam, 

and other locations, many servicemembers have struggled with mental 

health issues, including post-traumatic stress disorder, depression, and 

substance abuse. Several of these mental health issues have been 

identified as risk factors for suicide among veterans.1 The Department of 

Veterans Affairs (VA) provides an array of health care services, including 

mental health services and other benefits to over 21 million veterans and 

their families. As part of the continuum of services it provides, VA 

established the Veterans Crisis Line (VCL) in July 2007.2 The VCL 

supports veterans in emotional crisis and helps implement VA’s goal of 

improving mental health outcomes for servicemembers, veterans, and 

their families through a number of actions—including reducing barriers to 

seeking mental health treatment and expanding access to VA services.

Veterans can access the VCL by calling a national toll-free number—

1-800-273-TALK (8255)—and following instructions to press “1” to be 

connected with a VCL responder. The VCL and the National Suicide 

 1

Department of Veterans Affairs and Department of Defense, VA/DoD Clinical Practice 

Guideline for Assessment and Management of Patients at Risk for Suicide (Washington, 

D.C.: 2013).

2

VA established its crisis line at the VA medical center located in Canandaigua, New York. 

The original name of VA’s crisis line was the National Veterans Suicide Prevention Hotline 

until it was rebranded as the VCL in 2011.

Letter
Page 2 GAO-16-373 Veterans Crisis Line

Prevention Lifeline (Lifeline)—a Substance Abuse and Mental Health 

Services Administration (SAMHSA) grantee—share this national toll-free 

number through an interagency agreement between VA and SAMHSA.3

Currently, the VCL operates through a VA-operated primary center 

staffed with VA-employed responders and five backup call centers that 

provide additional responders and other services through a backup call 

coverage contract.4 The VCL is distinct from Lifeline, which operates 

through a network of private, nonprofit providers working independently of 

one another while maintaining agreed-upon clinical standards.5 In 

addition to responding to calls, the VCL can also be accessed via online 

chat and text message.

Demand for the VCL’s services has exceeded VA’s expectations. 

Specifically, in its first full year of operation in fiscal year 2008, the VCL 

received about 67,000 calls compared to about 534,000 calls received in 

fiscal year 2015, an almost 700 percent increase. In response, VA 

steadily increased the VCL’s spending from about $3 million to $30 million 

from fiscal year 2008 through fiscal year 2015, devoting additional staff 

and resources to the VCL over time. As VA endeavored to address 

increasing numbers of requests for assistance, reports of dissatisfaction 

with VCL service periodically appeared in the media, and the VA Office of

Inspector General was asked to investigate complaints about the VCL’s 

lack of timely response to callers.6 The Inspector General identified gaps 

in the VCL quality-assurance process, including challenges associated 

with supervisory review, tracking of issues, and collection and analysis of 

 3

SAMHSA is an agency within the U.S. Department of Health and Human Services (HHS) 

that leads public health efforts to advance the behavioral health of the nation. Currently, 

SAMHSA funds a cooperative agreement grant to administer Lifeline with the same entity 

that VA currently contracts with to provide VCL backup call coverage. Through their 

interagency agreement, VA and SAMHSA set out to establish a seamless crisismanagement system through a collaborative and cooperative relationship between the 

agencies that provides consistent suicide-prevention techniques to callers. 

4

For the purposes of this report, the term “VCL service partners” includes SAMHSA, the 

VCL backup call coverage contractor, and the backup call centers that this contractor uses 

to provide coverage to the VCL.

5

According to HHS, Lifeline providers largely do not receive federal funding to support 

their operations other than the use of the common phone number and routing software, 

tools designed more to facilitate access to needed help than to achieve uniformity in 

performance results.

6

Department of Veterans Affairs, Office of Inspector General, Veterans Crisis Line Caller 

Response and Quality Assurance Concerns (Washington, D.C.: 2016). 
Page 3 GAO-16-373 Veterans Crisis Line

data from VCL backup call centers. In addition, the Inspector General 

found that in some cases callers did not receive immediate assistance 

from responders.

In response to accounts of dissatisfaction with VCL services, you asked 

us to assess the effectiveness of VA’s policies and procedures designed 

to provide timely assistance to veterans through the VCL. In this report, 

we examine

1. the extent to which VA meets response-time goals for calls, online 

chats, and text messages received through the VCL;

2. how VA monitors the performance of the VCL primary center 

responders and call center operations;

3. how VA works with VCL service partners—backup call centers and 

SAMHSA—to help ensure veterans receive high-quality service from 

responders; and

4. what plans VA has to improve the VCL operations.

To examine the extent to which VA meets response-time goals for calls, 

online chats, and text messages received through the VCL, we made 

covert test telephone calls, text messages, and online chats to measure 

the VCL’s response times. The test calls included a generalizable sample 

of 119 calls that can be used to describe all callers’ wait times when 

calling the VCL during July and August of 2015.7 We also sent a 

nongeneralizable sample of 15 test online chats to the VCL and 14 test 

text messages during the same period. For these tests, we measured the 

length of time between sending the outgoing message to the VCL and 

receiving a response from a VCL responder. In addition, we examined 

telephone call, online chat, and text message data and summary reports 

from January 2013 through December 2015 that VA maintains related to 

 7

The generalizable sample of test calls was selected using a stratified two-stage random 

cluster design. Percentage estimates from this sample have a margin of error of within 

plus or minus 9 percentage points at the 95 percent confidence level. Estimates of the 

median wait time have a margin of error of within plus or minus 10 percent at the 95 

percent confidence level. The results of these test calls are not generalizable to calls to 

the VCL outside of the July and August 2015 period. See app. I for more details.
Page 4 GAO-16-373 Veterans Crisis Line

the timeliness of the VCL’s operations.8 To determine the validity of these 

data, we interviewed officials responsible for managing them and 

compared our test call response data with VA’s reported telephone data 

for July and August 2015. On the basis of these actions, we found these 

data to be sufficiently reliable for the purposes of describing the quantity 

of requests for services reaching the VCL. We used these data to 

evaluate the timeliness of the VCL’s call response and compared the data 

to the department’s own goals. Finally, we spoke with the VCL text 

messaging service provider to learn about its operations.

To examine how VA monitors the performance of VCL primary center 

responders and call center operations, we reviewed relevant 

documents—including VCL policies and procedures—and interviewed VA 

officials. In addition, we reviewed data and information that resulted from 

VCL monitoring. Finally, we compared VA’s use of key performance 

indicators to the Office of Management and Budget’s guidance on 

performance goals, which are consistent with the Government 

Performance and Results Modernization Act of 2010.9

To examine how VA works with VCL service partners—backup call 

centers and SAMHSA—to help ensure veterans receive high-quality 

service from responders, we conducted site visits to the VCL primary 

center and two of the five VCL backup call centers, where we observed 

call center operations and interviewed VA officials and backup call center 

representatives. We also interviewed SAMHSA officials and 

representatives of the remaining three VCL backup call centers by 

teleconference.10 In addition, we interviewed officials from the Veterans 

Health Administration’s (VHA) Office of Mental Health Operations 

 8

We reviewed telephone call data to determine how many calls were answered at the VCL 

primary center; we reviewed online chat data to determine how many online chat requests 

received by the VCL received a response within 1 minute; and we reviewed text message 

data to determine how many text messages sent to the VCL received a response within 2 

minutes. 

9

See Office of Management and Budget, Preparation, Submission, and Execution of the 

Budget—Strategic Plans, Annual Performance Plans, Performance Reviews, and Annual 

Program Performance Reports, Circular No. A-11, pt. 6 (Washington, D.C.: June 2015).

10The VCL’s primary center is located in Canandaigua, New York, and is operated by VA 

staff. VCL backup call coverage is provided by a contractor who manages the VCL backup 

call centers. Currently, there are five VCL backup call centers, located in Greenville, North 

Carolina; Miami, Florida; Portland, Oregon; Rochester, New York; and St. Louis, Missouri.
Page 5 GAO-16-373 Veterans Crisis Line

(OMHO)—the VA Central Office entity responsible for overseeing the 

VCL—and representatives from the contractor that manages VCL backup 

call coverage.11 Additionally, we reviewed VA’s contract that provides 

backup call coverage and VA’s interagency agreement with SAMHSA. 

Finally, we conducted a nongeneralizable sample of 34 calls in August 

2015 in which we mimicked the experience of veterans who do not follow 

the instructions of a voice prompt to press “1” to reach the VCL.

To examine what plans VA has to improve the VCL operations, we 

reviewed VA’s documented improvement plans and interviewed VA 

officials responsible for planning and implementing those improvements. 

For additional details about the scope and methodology used in this 

report, see appendix I.

We conducted this performance audit from February 2015 to May 2016 in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives. We conducted our related 

investigative work from July through August 2015 in accordance with the 

standards prescribed by the Council of the Inspectors General on Integrity 

and Efficiency.

In 2007, VA established the VCL, a 24-hour crisis line staffed by 

responders trained to assist veterans in emotional crisis. Through an 

interagency agreement, VA collaborated with SAMHSA to use a single, 

national toll-free number for crisis calls that serves both Lifeline and the 

VCL.12 Through this interagency agreement, VA and SAMHSA set out to 

 11Within VA, VHA is the organization responsible for providing health care to veterans at 

medical facilities across the country. VCL leadership reports directly to VHA’s OMHO—the 

entity responsible for coordinating mental health care services throughout VA’s health 

care system. 

12SAMHSA and the Mental Health Association of New York City launched Lifeline on 

January 1, 2005. Lifeline provides free and confidential emotional support to people in 

suicidal crisis or emotional distress, 24 hours a day, 7 days a week.

Background

VCL Structure
Page 6 GAO-16-373 Veterans Crisis Line

establish a seamless crisis-management system through a collaborative 

and cooperative relationship between the agencies that would provide 

consistent suicide-prevention techniques to callers.

The national toll-free number presents callers with choices. Callers are 

greeted by a recorded message that explains the function of the crisis line 

and prompts individuals to press “1” to reach the VCL. Callers who do not 

press “1” by the end of the message are routed to one of Lifeline’s 164 

local crisis centers.13 All callers who press “1” are routed first to the VCL 

primary center. Calls that are not answered at the VCL primary center 

within 30 seconds of the time that the caller presses “1” during the Lifeline 

greeting are automatically routed to one of five VCL backup call centers. 

If a call is not answered by the VCL backup call center that initially 

receives it, the call may be sent to another VCL backup call center.14 VA 

entered into a contract with a firm to oversee the operations of the VCL 

backup call centers. There are a total of 164 Lifeline local crisis centers, 5 

of which also serve the VCL.15 (See fig. 1.)

 13The automated greeting also prompts Spanish speakers to press “2” for assistance in 

Spanish.

14Some VCL backup call centers do not allow calls to be rerouted to another VCL backup 

call center and instead hold the call in a queue awaiting response by that backup call 

center’s responders.

15VA does not directly contract with any of the VCL backup call centers.
Page 7 GAO-16-373 Veterans Crisis Line

Figure 1: Call Routing Process for the National Suicide Prevention Lifeline (Lifeline) and Veterans Crisis Line (VCL)
Page 8 GAO-16-373 Veterans Crisis Line

The number of calls reaching the VCL has increased substantially since 

the VCL’s first full year of operation. Increases in the number of VCL calls 

received have corresponded with increased annual funding obligations for 

the VCL. (See fig. 2.)

Figure 2: Veterans Crisis Line (VCL) Milestones, Call Volume, and Funding Obligations, Fiscal Years 2007 through 2015

Note: Annual obligation totals have been rounded.

a

The Veterans Suicide Prevention Line began operations in July 2007.

b

This office is located within the Veterans Health Administration.

VA added online chat and text message capabilities to the VCL in fiscal 

years 2009 and 2012, respectively. The number of online chats and text 

messages handled by the VCL generally increased every year, though 

the number of online chats decreased in fiscal year 2015. (See fig. 3.)

VCL Workload and 

Funding Obligations
Page 9 GAO-16-373 Veterans Crisis Line

Figure 3: Veterans Crisis Line (VCL) Online Chat and Text Message Volumes, Fiscal Years 2009 through 2015

Note: The VCL started its online chat service in fiscal year 2009 and text message service in fiscal 

year 2012.
Page 10 GAO-16-373 Veterans Crisis Line

To determine how well VA performed against its goal for responding to 

VCL callers, we covertly tested the VCL’s call response time in July and 

August 2015. During this testing we found that it was uncommon for VCL 

callers to wait an extended period before reaching a responder since all 

of our calls that reached the VCL were answered in less than 4 minutes. 

According to VA officials, VA established a goal that the VCL primary 

center would answer 90 percent of calls to the VCL within 30 seconds. 

Our test included a generalizable sample of 119 test calls that can be 

used to describe all callers’ wait times when calling the VCL during this 

period.16 On the basis of our test calls, we estimate that during July and 

August 2015 about 73 percent of all VCL calls were answered at the VCL 

primary center within 30 seconds.17 (See fig. 4.) VA officials told us that, 

during fiscal year 2015, about 65 to 75 percent of VCL calls were 

answered at the VCL primary center and about 25 to 35 percent of VCL 

calls were answered at the backup call centers. These VA-reported 

results indicate that about 65 to 75 percent of VCL calls were answered 

 16For these test calls, callers’ wait times refer to the length of time that elapses between 

when callers press “1” and when responders at either the VCL primary center or backup 

call centers answer the calls. 

17In addition, we estimate that during July and August of 2015 99 percent of all VCL calls 

were answered within 120 seconds and the median call response time was 17 seconds. 

These percentage estimates have a margin of error of within plus or minus 9 percentage 

points, and the median response times estimates have a relative margin of error that is 

less than 9 percent at the 95 percent confidence level. See app. I for more details. 

Extended Call Wait 

Times Were 

Uncommon in July 

and August 2015, but 

VA Did Not Meet Its 

Call Response Time 

Goals and Some Text 

Messages Did Not 

Receive Responses

VA Responded to Most 

Calls within 30 Seconds in 

July and August 2015, but 

Did Not Meet Its Goal to 

Answer 90 Percent of 

Calls within 30 Seconds at 

the VCL Primary Center
Page 11 GAO-16-373 Veterans Crisis Line

within either 30 or 60 seconds.18 These results are consistent with our 

test results for July and August 2015. According to VA officials, VA 

attempts to maximize the percentage of calls answered at the VCL 

primary center because these responders have additional resources—

including access to veterans’ VA electronic medical records—that are 

unavailable to VCL backup call center responders. All responders receive

specialized training to assist callers in crisis.19

Figure 4: Wait Time Estimates for Calls to the Veterans Crisis Line (VCL), July and August 2015

a

Wait times included in this figure are based on the elapsed time between when a GAO test caller 

pressed “1” to be connected to the VCL and when a responder answered the call. Percentage 

 18For approximately 5 months of fiscal year 2015, VA allowed calls to ring at the VCL 

primary center for 60 seconds before routing the calls to VCL backup call centers. VA then 

returned to the standard that calls not answered at the VCL primary center within 30 

seconds are then routed to VCL backup call centers. 

19All VCL primary and backup call center responders are required to complete Applied 

Suicide Intervention Skills Training as part of their training to assist callers in crisis. In this 

training, participants learn to use a suicide intervention model to identify persons with 

thoughts of suicide, seek a shared understanding of reasons for dying and living, develop 

a safe plan based on a review of risk, be prepared to do follow-up, and become involved 

in suicide-safer community networks. Lifeline promotes Applied Suicide Intervention Skills 

Training among its local crisis centers, though not all 164 centers have participated in the 

training. However, HHS stated that all of the centers’ responders adhere to Lifeline’s 

standards for risk assessment.
Page 12 GAO-16-373 Veterans Crisis Line

estimates used in the figure have a margin of error of within plus or minus 9 percentage points at the 

95 percent confidence level.

b

At the time of our test, VA allowed calls to ring at the VCL primary center for 60 seconds before 

routing the calls to VCL backup call centers.

To improve its performance toward meeting the goal of answering 90

percent of calls at the VCL primary center within 30 seconds, VA 

implemented two changes in fiscal year 2015—namely, staggered work 

shifts for responders and new call-handling procedures.20

Staggered work shifts. VA implemented staggered shifts for responders 

at the VCL primary center on September 6, 2015. Staggered shifts are 

work schedules that allow employees to start and stop their shifts at 

different times as a way to ensure better coverage during peak calling 

periods. Specifically, it helps schedule more employees to work when call 

volume is highest and fewer employees to work when call volume is 

lowest.21 Additionally, staggered shifts help limit disruptions in service as 

responders begin and end their shifts. By comparing VCL telephone call 

data from September through December of 2014 to that of September 

through December of 2015, we found that VA’s implementation of 

staggered shifts at the VCL primary center had mixed results.22

• Overall: The average percentage of calls answered per hour at the 

VCL primary center from September through December 2015—after 

staggered shifts were implemented—was 75 percent, slightly less 

than the average of 79 percent answered during the corresponding 

period in 2014 before staggered shifts were implemented. However, 

the VCL received an average of about 1.3 more calls per hour during 

this period in 2015 than it received during the corresponding period in 

2014 and, according to VA officials, the VCL primary center employed 

fewer responder staff in 2015 than 2014.

 20According to VA officials, their long-term goal is to answer 100 percent of calls at the 

VCL primary center.

21The International Customer Management Institute includes staggered shifts as a best 

practice among call centers.

22Our analysis compared VCL call data from September 6, 2015, through December 31, 

2015, to VCL call data from September 1, 2014, through December 31, 2014. The 

percentage of calls answered is likely affected by several factors, such as call volume, 

staffing levels, and complexity of calls. Our analysis controlled for day of the week, time of 

day, and holidays, but did not control for all factors that may affect the percentage of calls 

answered.

To Help Achieve 

Response-Time Goals, VA 

Implemented Changes at 

the VCL Primary Center
Page 13 GAO-16-373 Veterans Crisis Line

• By day of the week: The average percentage of calls answered per 

hour at the VCL primary center increased on Mondays to 89 percent 

and Tuesdays to 83 percent after VA implemented staggered shifts, 

up from 78 percent and 79 percent, respectively, during the 

corresponding period in 2014. These increases suggest that 

staggered shifts may have helped VA answer more calls at the VCL 

primary center on these days because VCL call data from our analysis 

indicate that these days of the week typically experienced the highest 

number of calls prior to implementing staggered shifts, and VA 

officials told us that they used the implementation of staggered shifts 

to schedule more responders on these days. However, the average 

percentage of calls answered per hour at the VCL primary center 

decreased on Saturdays to 61 percent and Sundays to 70 percent 

after VA implemented staggered shifts, down from 78 percent and 80 

percent, respectively, during the corresponding period in 2014.

• By hours of the day: VA answered a higher percentage of calls at the 

VCL primary center during the mid-day and evening hours after the 

implementation of staggered shifts. Specifically, from 11:00 a.m. to 

4:00 p.m. and from 9:00 p.m. to 11:00 p.m. the VCL primary center

answered a higher percentage of calls compared with the 

corresponding periods in 2014. However, VA answered a lower 

percentage of calls at the VCL primary center during overnight 

hours—midnight to 9:00 a.m.—and in the early evening—5:00 p.m. to 

8:00 p.m.—compared to corresponding periods in 2014.

To address staffing limitations and align the number of responders 

available for each staggered shift according to demand, VA officials said 

that VA planned to hire 63 additional responders for the VCL primary 

center in fiscal year 2016 and assign these new responders to weekend 

and evening shifts. This change would likely help improve the mixed 

results we identified in our analysis of VA’s implementation of staggered 

shifts for responders. As of February 2016, VA officials said that 22 

applicants had accepted employment offers and that VA planned to 

extend employment offers to an additional 15 applicants. These officials 

also noted that recent attrition at the VCL primary center was largely due 

to VCL primary center responders being promoted into new positions at 

the VCL primary center or to VCL primary center responders leaving 
Page 14 GAO-16-373 Veterans Crisis Line

because their work with the VCL did not qualify as clinical hours required 

for licensure in their specialties.23

New call-handling procedures. VA implemented new call handling 

procedures at the VCL primary center beginning in June 2015 that 

provided responders with specific guidance to more efficiently handle 

“noncore” callers—those callers who were not seeking crisis assistance 

but rather seeking help with other issues, such as help with veterans’ 

benefits questions. For example, if a caller reached the VCL with a 

question about VA disability benefits, the VCL primary center responder 

would verify that the caller was not in crisis and transfer the caller to the 

Veterans Benefits Administration to address the question. VCL telephone 

call data provided by VA suggest that the average time VCL primary 

center responders spend handling noncore calls decreased by 

approximately 30 percent over a 5-month period after responder training 

began on these new call-handling procedures.24 We would expect that as 

the average time VCL primary center responders spend handling noncore 

calls decreases, these responders should have more time available to 

answer additional incoming calls.

To determine the timeliness of the VCL’s responses to online chats and 

text messages, we conducted a covert test in July and August 2015 using 

nongeneralizable samples of 15 online chats and 14 text messages. All 

15 of our test online chats received responses within 60 seconds, 13 of 

which were within 30 seconds. This result is consistent with VA data that 

indicate VCL responders sent responses to over 99 percent of online chat 

requests within 1 minute in fiscal years 2014 and 2015. VA officials said 

that all online chats are expected to be answered immediately. Although 

this is an expectation, VA does not yet have formal performance 

 23In February 2016, VA officials reported that 20 VCL primary center responders had been 

promoted into new positions within the VCL. These positions were in the VCL’s quality 

management, workforce management, and training units. These officials explained that 

they are working with licensing bodies in New York to discuss the applicability of VCL 

primary center responders’ work to clinical hour requirements for several specialties, 

though the officials did not indicate that the licensing standards had recently changed. 

24We did not test this aspect of VCL operations with covert test calls. 

In July and August 2015, 

Most of Our Test Online 

Chats Were Answered 

Within 30 Seconds, but VA 

Did Not Ensure That 

Veterans Receive 

Responses through Its 

Text Messaging Service
Page 15 GAO-16-373 Veterans Crisis Line

standards for how quickly responders should answer online chat requests 

and expects to develop them before the end of fiscal year 2016.25

However, our tests of text messages revealed a potential area of concern. 

Four of our 14 test text messages did not receive a response from the 

VCL. Of the remaining 10 test text messages, 8 received responses 

within 2 minutes, and 2 received responses within 5 minutes.

VA officials stated that text messages are expected to be answered 

immediately, but, as with online chats, VA has not yet developed formal 

performance standards for how quickly responders should answer text 

messages. VA data indicate that VCL responders sent responses to 87 

percent of text messages within 2 minutes of initiation of the conversation 

in both fiscal years 2014 and 2015. VA officials said that VA plans to 

establish performance standards for answering text messages before the 

end of fiscal year 2016. VA officials noted and we observed during a site 

visit that some incoming texts were abusive in nature or were not related 

to a crisis situation.26 According to VA officials, in these situations, if this 

is the only text message waiting for a response, a VCL responder will 

send a response immediately. However, if other text messages are 

awaiting responses, VA will triage these text messages and reply to those 

with indications of crisis first. This triage process may have contributed to 

the number of our test texts that did not receive responses within 2 

minutes.

The VCL’s text messaging service provider offered several reasons for 

the possible nonresponses that we encountered in our test results. These 

included: (1) incompatibilities between some devices used to send text 

messages to the VCL and the software VA used to process the text 

messages, (2) occasional software malfunctions that freeze the text 

messaging interface at the VCL primary center, (3) inaudible audio 

prompts used to alert VCL primary center responders of incoming text 

messages, (4) attempts by people with bad intentions to disrupt the VCL’s 

text messaging service by overloading the system with a large number of 

 25VA officials explained that these performance standards will be developed using 

response-time data that will become available after an upgrade to the VCL’s 

telecommunications systems in mid-2016. Currently, retrospective data on online chat 

performance are available, but real-time data will be available after this 

telecommunications systems upgrade, according to VA officials.

26Our test text messages consisted of a simple greeting, such as “Hi” or “Hello.”
Page 16 GAO-16-373 Veterans Crisis Line

texts, and (5) incompatibilities between the web-browsers used by the 

VCL primary center and the text messaging software.

VA officials told us that they do not monitor and test the timeliness and 

performance of the VCL text messaging system, but rather rely solely on 

the VCL’s text messaging service provider for such monitoring and 

testing. They said that the provider had not reported any issues with this 

system. According to the provider, no routine testing of the VCL’s text 

messaging system is conducted. Standards for internal control in the 

federal government state that ongoing monitoring should occur in the 

course of normal operations, be performed continually, and be ingrained 

in the agency’s operations.27 Without routinely testing its text messaging 

system, or ensuring that its provider tests the system, VA cannot ensure 

that it is identifying limitations with its text messaging service and 

resolving them to provide consistent, reliable service to veterans.

 27See GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-

21.3.1 (Washington, D.C.: November 1999).

VA Has Taken Steps 

to Improve Its 

Monitoring of VCL 

Primary Center 

Performance but Has 

Not Established 

Targets and Time 

Frames for VCL Key 

Performance 

Indicators
Page 17 GAO-16-373 Veterans Crisis Line

VA has sought to enhance its capabilities for overseeing VCL primary 

center operations through a number of activities—including establishing a 

call center evaluation team, implementing revised performance standards 

for VCL primary center responders, implementing silent monitoring of 

VCL primary center responders, and analyzing VCL caller complaints.28

Establishment of a call center evaluation team. In October 2013, VA 

established a permanent VCL call center evaluation team that is 

responsible for monitoring the performance of the VCL primary center.29

The call center evaluation team analyzes VCL data, including information 

on the number of calls received and the number of calls routed to backup 

call centers from the primary center. VA officials told us that they use 

these data to inform management decisions about VCL operations. For 

example, these data were used as part of its decision to implement 

staggered shifts for VCL primary center responders in an attempt to 

increase the number of calls answered at the VCL primary center.

Implementation of revised performance standards for VCL primary 

center responders. In October 2015, VA implemented new performance 

standards for all VCL primary center responders that will be used to 

assess their performance in fiscal year 2016. According to VA officials, 

these performance standards include several measures of responder 

performance—such as demonstrating crisis-intervention skills, identifying 

callers’ needs, and addressing those needs in an appropriate manner 

using VA approved resources. VA officials told us that by the summer of 

2016 VCL primary center supervisors will have access to real-time 

information on VCL primary center responders’ performance against 

these standards and can track their workload and performance 

periodically. These officials explained that they anticipate these 

performance standards will be reviewed and revised as needed for the 

fiscal year 2017 performance year.

Silent monitoring of VCL primary center responders. In February 

2016, VA officials reported that they were beginning silent monitoring of 

 28VCL also tracks and analyzes complaints about the services of VCL backup call centers 

as a part of this effort.

29According to VA officials, this team was initially staffed with VA employees detailed from 

other areas of the department in December 2012. Permanent staff for call center 

evaluation were hired in October 2013.

VA Established a Call 

Center Evaluation Team, 

Implemented Revised 

Responder Performance 

Standards, and Analyzed 

VCL Caller Complaints
Page 18 GAO-16-373 Veterans Crisis Line

all VCL responders using recently developed standard operating 

procedures, standard data collection forms, and standard feedback 

protocols. These officials explained that the VCL primary center silent 

monitoring would begin in mid-February 2016 with four VA medical 

center–based suicide-prevention coordinators completing silent 

monitoring of 15 to 20 calls a week to the VCL primary center through 

March 2016.30 These officials explained that six full-time silent monitors 

had been hired as part of the VCL quality assurance staff and would 

begin conducting silent monitoring of VCL primary center calls in April 

2016 once their training had been completed. During the initial rollout, the 

four VA medical center–based suicide-prevention coordinators will 

remotely access VCL primary center calls, complete the standard data 

collection form, and send the information to the observed VCL primary 

center responders’ supervisors for feedback delivery. Once the six fulltime silent monitors begin completing these activities, they will complete 

all call monitoring and deliver feedback to VCL primary center responders 

and will coordinate with VCL primary center supervisors on an as-needed 

basis. VA officials explained in February 2016 that they were unsure how 

many VCL primary center calls these six full-time silent monitors would be 

able to observe and will clarify this expectation once these silent monitors 

begin their duties in April 2016.

Analysis of VCL caller complaints. In October 2014, VA created a 

mechanism for tracking complaints it receives from VCL callers and 

external parties, such as members of Congress and veterans, about the 

performance of the VCL primary and backup call centers. Complaints can 

be about services provided by either the VCL primary center or one of the 

VCL backup call centers. In fiscal year 2015, the VCL received over 200 

complaints from veterans and others regarding call center operations. 

These complaints included issues with VCL primary center and backup 

call center customer service and wait times to reach a responder. 

According to VA officials, each complaint is investigated to validate its 

legitimacy and determine the cause of any confirmed performance 

concerns. This validation process includes speaking with the complainant 

and VA staff, as applicable. The results and disposition of each complaint 

 30These four VA medical center-based suicide-prevention coordinators are serving on 

part-time details to complete these VCL primary center silent monitoring duties. 
Page 19 GAO-16-373 Veterans Crisis Line

are documented in VA’s complaint tracking database.31 For complaints 

that include details on specific responders, VA officials told us that they 

investigate complaints and use legitimate complaints as part of the 

performance evaluation process for VCL primary center responders. 

Specifically, these officials explained that when a complaint about a VCL 

primary center responder’s customer service is verified as accurate by a 

VA psychologist or supervisor after it is investigated, it can affect a VCL 

primary center responder’s annual performance appraisal. The 

investigation process also includes verifying any associated 

documentation of the activities at the source of the complaint.

In 2011, VA established key performance indicators to evaluate VCL 

primary center operations; however, we found these indicators did not 

have established measureable targets or time frames for their completion.

VCL key performance indicators lack measurable targets. We found 

that VA’s list of VCL key performance indicators did not include 

information on the targets the department had established to indicate their 

successful achievement. For example, VA included a key performance 

indicator for the percentage of calls answered by the VCL in this list but 

did not include information on what results would indicate success for (1) 

the VCL as a whole, (2) the VCL primary center, or (3) the VCL backup 

call centers. As another example, VA did not establish targets for the 

percentage of calls abandoned by callers prior to speaking with VCL 

responders. Measureable targets should include a clearly stated minimum 

performance target and a clearly stated ideal performance target.32 These 

targets should be quantifiable or otherwise measurable and indicate how 

well or at what level an agency or one of its components aspires to 

perform.33 Such measurable targets are important for ensuring that the 

 31According to VA officials, information on VCL complaints is housed on the VA Northeast 

Program Evaluation Center’s servers and, once entered, is spot-checked by VA staff to 

ensure accuracy.

32See GAO, VA Health Care: Additional Guidance, Training, and Oversight Needed to 

Improve Clinical Contract Monitoring, GAO-14-54 (Washington, D.C.: Oct. 2013).

33Consistent with the Government Performance and Results Act Modernization Act of 

2010, the Office of Management and Budget states that a performance goal should 

include a tangible, measurable objective or a quantifiable standard, value, or rate. See 

Office of Management and Budget, Preparation, Submission, and Execution of the 

Budget—Strategic Plans, Annual Performance Plans, Performance Reviews, and Annual 

Program Performance Reports. 

VCL Key Performance 

Indicators Lack 

Measureable Targets and 

Time Frames
Page 20 GAO-16-373 Veterans Crisis Line

VCL call center evaluation team can effectively measure VCL 

performance.

VCL key performance indicators lack time frames for their 

completion. We found that VA’s list of VCL key performance indicators 

did not include information on when the department expected the VCL to 

complete or meet the action covered by each key performance indicator. 

For example, for VA’s key performance indicator for the percentage of 

calls answered by the VCL, the department did not include a date by 

which it would expect the VCL to complete this action. As another 

example, VA did not establish dates by which it would meet targets yet to 

be established for the percentage of calls abandoned by callers prior to 

speaking with VCL responders. Time frames for action are a required 

element of performance indicators and are important to ensure that 

agencies can track their progress and prioritize goals.34

Guidance provided by the Office of Management and Budget states that 

performance goals—similar to VA’s key performance indicators for the 

VCL—should include three elements: (1) a performance indicator, which 

is how the agency will track progress; (2) a target; and (3) a period.35 VA 

officials reported that they are currently implementing a comprehensive 

process improvement plan, discussed later in this report, that will help 

ensure the right structures and processes are in place, which they believe 

are logical precursors to examining VCL outcomes and establishing 

targets and time frames for performance indicators. Without establishing 

targets and time frames for the successful completion of its key 

performance indicators for the VCL, VA cannot effectively track and 

publicly report progress or results for its key performance indicators for 

accountability purposes.

 34Consistent with the Government Performance and Results Act Modernization Act of 

2010, the Office of Management and Budget defines a performance goal as a statement 

of the level of performance to be accomplished within a time frame. See Office of 

Management and Budget, Preparation, Submission, and Execution of the Budget—

Strategic Plans, Annual Performance Plans, Performance Reviews, and Annual Program 

Performance Reports.

35See Office of Management and Budget, Preparation, Submission, and Execution of the 

Budget—Strategic Plans, Annual Performance Plans, Performance Reviews, and Annual 

Program Performance Reports.
Page 21 GAO-16-373 Veterans Crisis Line

VA’s backup call coverage contract, awarded in October 2012 and in 

place at the time of our review, did not include detailed performance 

requirements in several key areas for the VCL backup call centers. Clear 

performance requirements for VCL backup call centers are important for 

defining VA’s expectations of these service partners. However, VA has 

taken steps to strengthen the performance requirements of this contract 

by modifying it in March 2015 and beginning the process of replacing it 

with a new contract.36

October 2012 backup call coverage contract. This contract provided a 

network of Lifeline local crisis centers that could serve as VCL backup call 

centers managed by a contractor.37 This contractor was responsible for 

overseeing and coordinating the services of VCL backup call centers that 

answer overflow calls from the VCL primary center. This contract as 

initially awarded included few details on the performance requirements for 

VCL backup call centers. For example, the contract did not include any 

 36According to VA officials, this new contract was awarded in April 2016.

37The backup call coverage contract in place at the time of our review was awarded in 

October 2012 with a 1-year base and two 1-year option periods (for a total of 3 years of 

coverage) and was set to expire in September 2015. However, according to VA officials, 

the contract was extended through May 2016 while the department was finalizing a new 

contract. VA officials reported that the new backup call coverage contract was awarded in 

April 2016.

VA Is Strengthening 

Requirements for 

VCL Backup Call 

Centers, but VA and 

SAMHSA Do Not 

Collect Information to 

Assess How Often 

and Why Callers Are 

Not Reaching the 

VCL

VA Is Enhancing 

Performance 

Requirements for Its 

Backup Call Coverage 

Contractor
Page 22 GAO-16-373 Veterans Crisis Line

information on several key aspects of VCL backup call center 

performance, including: (1) the percentage of VCL calls routed to each 

VCL backup call center that should be answered, (2) VA’s expectations 

on whether or not VCL backup call centers could use voice answering 

systems or caller queues for VCL calls, and (3) VA’s documentation 

requirements for VCL calls answered at the VCL backup call centers. 

Detailed performance requirements on these key aspects of VCL backup 

call center performance are necessary for VA to effectively oversee the 

performance of the contractor and the VCL backup call centers. By not 

specifying performance requirements for the contractor on these key 

performance issues, VA missed the opportunity to validate contractor and 

VCL backup call center performance and mitigate weaknesses in VCL 

call response. For example, representatives from one VCL backup call 

center provided data that showed that the backup call center answered 

about 50 percent of the VCL calls it received. However, without a 

performance requirement establishing a standard for the percentage of 

calls each VCL backup call center should answer, VA could not determine 

whether this was acceptable performance for a VCL backup call center. 

As of December 2015, this VCL backup call center reported that it had 

improved its performance and answered about 66 percent of calls it 

received from July 2015 to December 2015.

VA officials told us about several concerns with the performance of the 

backup call centers operating under the October 2012 contract based on 

their own observations and complaints reported to the VCL. These 

concerns included the inconsistency and incompleteness of VCL backup 

call centers’ responses to VCL callers, limited or missing documentation 

from records of VCL calls answered by VCL backup call center 

responders, limited information provided to VA that could be used to track 

VCL backup call center performance, and the use of voice answering 

systems or extended queues for VCL callers reaching some VCL backup 

call centers. For example, VA officials reported that some veterans did 

not receive complete suicide assessments when their calls were 

answered at VCL backup call centers. In addition, VA officials noted that 

they had observed some VCL backup call centers failing to follow VCL 

procedures, such as not calling a veteran who may be in crisis when a 

third-party caller requested that the responder contact the veteran. 

According to VA officials, these issues led to additional work for the VCL 

primary center, including staffing one to two responders per shift to review 

the call records submitted to the VCL primary center by backup call 

centers and to determine whether these calls required additional follow-up 

from the VCL primary center. These officials estimated that 25 to 30 

percent of backup call center call records warranted additional follow-up 
Page 23 GAO-16-373 Veterans Crisis Line

to the caller from a VCL primary center responder, including 

approximately 5 percent of backup call center call records that needed to 

be completely reworked by a VCL primary center responder.

March 2015 backup call coverage contract modification. Given these 

concerns, in March 2015 VA modified the October 2012 backup call 

coverage contract to add more explicit performance requirements for its 

backup call coverage contractor, which likely took effect more quickly 

than if the department had waited for a new contract to be awarded. 

These modified requirements included (1) the establishment of a 24-

hours-a-day, 7-days-a-week contractor-staffed emergency support line 

that VCL backup call centers could use to report problems, (2) a 

prohibition on VCL backup call centers’ use of voice answering systems, 

(3) a prohibition on VCL backup call centers placing VCL callers on hold 

before a responder conducted a risk assessment, (4) documentation of 

each VCL caller’s suicide risk assessment results, and (5) transmission of 

records for all VCL calls to the VCL primary center within 30 minutes of 

the call’s conclusion.

Development of new backup call coverage contract. In July 2015, VA 

began the process of replacing its backup call coverage contract by 

publishing a notice to solicit information from prospective contractors on 

their capability to satisfy the draft contract terms for the new contract; this 

new backup call coverage contract was awarded in April 2016.38 We 

found that these new proposed contract terms included the same 

performance requirement modifications that were made in March 2015, 

as well as additional performance requirements and better data reporting 

from the contractor that could be used to improve VA’s oversight of the 

VCL backup call centers. Specifically, the proposed contract terms added 

performance requirements to address VCL backup call center 

performance—including a requirement for 90 percent of VCL calls 

received by a VCL backup call center to be answered by a backup call 

center responder within 30 seconds and 100 percent to be answered by a 

backup call center responder within 2 minutes. In addition, the proposed 

contract terms include numerous data reporting requirements that could 

allow VA to routinely assess the performance of its VCL backup call 

centers and identify patterns of noncompliance with the contract’s 

 38This notice—referred to as a sources sought notice—included a draft performance work 

statement. In April 2016, VA officials reported that this contract was awarded to the 

previous backup call coverage contractor. 
Page 24 GAO-16-373 Veterans Crisis Line

performance requirements more efficiently and effectively than under the 

prior contract. The proposed terms for the new contract also state that VA 

will initially provide and approve all changes to training documentation 

and supporting materials provided to VCL backup call centers in order to 

promote the contractor’s ability to provide the same level of service that is 

being provided by the VCL primary center.

We found that when callers do not press “1” during the initial Lifeline 

greeting, their calls may take longer to answer than if the caller had 

pressed “1” and been routed to either the VCL primary center or a VCL 

backup call center.39 As previously discussed, VA and SAMHSA 

collaborated to link the toll-free numbers for both Lifeline and the VCL 

through an interagency agreement. The greeting instructs callers to press 

“1” to be connected to the VCL; if callers do not press “1,” they will be 

routed to one of SAMHSA’s 164 Lifeline local crisis centers. To mimic the 

experience of callers who do not press “1” to reach the VCL when 

prompted, we made 34 covert nongeneralizable test calls to the national 

toll-free number that connects callers to both Lifeline and the VCL during 

August 2015 and we did not press “1” to be directed to the VCL.40 For 23 

of these 34 calls, our call was answered in 30 seconds or less. For 11 of 

these calls, we waited more than 30 seconds for a responder to answer—

including 3 calls with wait times of 8, 9, and 18 minutes. Additionally, one 

of our test calls did not go through, and during another test call we were 

asked if we were safe and able to hold.41 VA’s policy prohibits VCL 

responders from placing callers on hold prior to completing a suicide 

assessment; Lifeline has its own policies and procedures.42

 39The initial greeting is about 30 seconds long and prompts the caller to press “1” to be 

connected to the VCL at the end of the greeting. If callers do not press”1,” the call is 

routed to one of SAMHSA’s 164 Lifeline local crisis centers based on the area code of the 

callers’ telephone numbers. 

40These 34 calls were a random but nongeneralizable sample.

41When asked if we were safe and could hold, we terminated this test call. 

42We did not review Lifeline’s policies and procedures as a part of this audit due to our 

focus on the VCL. We focused our review of Lifeline on those elements of their operations 

that interacted with the VCL or VA, such as the interagency agreement between VA and 

SAMHSA that governs the shared use of a single national toll-free number between the 

VCL and Lifeline. 

VA and SAMHSA Do Not 

Collect Information 

Needed to Assess How 

Often and Why Callers Do 

Not Reach the VCL
Page 25 GAO-16-373 Veterans Crisis Line

According to officials and representatives from VA, SAMHSA, and the 

VCL backup call centers, as well as our experience making test calls 

where we did not press “1,” there are several reasons why a veteran may 

not press “1” to be routed to the VCL, including

• an intentional desire to not connect with VA,

• failure to recognize the prompt to press “1” to be directed to the VCL,

• waiting too long to respond to the prompt to press “1” to be directed to 

the VCL, or

• calling from a rotary telephone that does not allow the caller to press 

“1” when prompted.

VA officials said they had not estimated the extent to which veterans 

intending to reach the VCL did not press “1” during the Lifeline greeting.43

These officials explained that their focus has been on ensuring that 

veterans who did reach the VCL received appropriate service from the 

VCL primary center and backup call centers. In addition, SAMHSA 

officials said that they also do not collect this information.44 These officials 

reported that SAMHSA does not require the collection of demographic 

information, including veteran status, for a local crisis center to participate 

in the Lifeline network. However, they noted that SAMHSA could request 

through its grantee that administers the Lifeline network that local crisis 

centers conduct a onetime collection of information to help determine how 

often and why veterans reach Lifeline local crisis centers. SAMHSA 

officials explained that they could work with the Lifeline grantee to explore 

optimal ways of collecting this information that would be (1) clinically 

appropriate, (2) a minimal burden to callers and Lifeline’s local crisis

centers, and (3) in compliance with the Office of Management and 

Budget’s paperwork reduction and information collection policies. The 

interagency agreement between VA and SAMHSA assigns SAMHSA 

responsibilities for monitoring the use of the national toll-free number—

 43According to SAMHSA officials, in 2014 about 383,000 callers abandoned their calls to 

Lifeline during the initial greeting used to direct callers to either Lifeline local crisis centers 

or the VCL. We did not assess the reasons these calls were abandoned.

44According to SAMHSA officials, the SAMHSA grantee responsible for administering 

Lifeline conducted a survey in 2014 that captured veteran-related data. However, 

SAMHSA had no involvement with this survey or the data collection activities of the 

Lifeline local crisis centers that provided the information because it was outside the scope 

of SAMHSA’s grant to the organization. Further, HHS stated that the SAMHSA grantee did 

not share the results of the survey with SAMHSA. We did not evaluate the results of this 

survey.
Page 26 GAO-16-373 Veterans Crisis Line

1-800-273-TALK (8255)—that is used to direct callers to both the VCL 

and Lifeline. These responsibilities include monitoring the use of the line, 

analyzing trends, and providing recommendations about projected needs 

and technical modifications needed to meet these projected needs. Using 

the information collected from the Lifeline local crisis centers on how often 

and why veterans reach Lifeline, as opposed to the VCL, VA and 

SAMHSA officials could then assess whether the extent to which this 

occurs merits further review and action.

Although the results of our test are not generalizable, substantial wait 

times for a few of our covert calls suggest that some callers may 

experience longer wait times to speak with a responder in the Lifeline 

network than they would in the VCL’s network. Without collecting 

information to examine how often and why veterans do not press “1” 

when prompted to reach the VCL, VA and SAMHSA cannot determine the 

extent veterans reach the Lifeline network when intending to reach the 

VCL and may experience longer wait times as a result. In addition, 

limitations in information on how often and why this occurs do not allow 

VA and SAMHSA to determine whether or not they should collaborate on 

plans to address the underlying causes of veterans not reaching the VCL. 

Standards for internal control in the federal government state that 

information should be communicated both internally and externally to 

enable the agency to carry out its responsibilities.45 For external 

communications, management should ensure there are adequate means 

of communicating with, and obtaining information from, external 

stakeholders that may have a significant impact on the agency achieving 

its goals.

 45GAO/AIMD-00-21.3.1.
Page 27 GAO-16-373 Veterans Crisis Line

In June 2014, VA assessed the operational state of the VCL and, based 

on its findings, designed a performance-improvement plan that outlined 

actions to address problems VA identified regarding the VCL’s workforce, 

processes, technology, and infrastructure.46 To implement this plan, in 

March 2015 VA began a series of rapid process-improvement events, 

such as improvements to VCL primary center responder training, 

designed to solve problems identified by VCL staff and stakeholders with 

actions that could be implemented within 60 to 90 days.47

According to VA officials and documentation provided by these officials, 

these rapid process-improvement events led to several changes at the 

VCL primary center in 2015 and 2016. As we previously noted, these 

changes include implementation of staggered shifts; development of 

silent monitoring procedures, and the hiring of dedicated staff to complete 

this monitoring; and new call-handling procedures previously discussed. 

They also include some remaining follow-up activities, such as completing 

the implementation of remaining planned quality-assurance activities in 

fiscal year 2016. These measures—if fully implemented—represent 

positive steps to improve VCL operations.

VA has developed additional plans to address other concerns with VCL 

operations. These plans address issues at the VCL primary center related 

to renovation of new space, upgrades to telecommunications, and the 

introduction of a caller queue.

 46See Department of Veterans Affairs, Health Resource Center: Assessment of Veteran 

Crisis Line & National Call Center for Homeless Veterans (June 2014). In 2014, the VHA 

National Call Center-Health Resource Center conducted a 3-day assessment of the VCL 

and provided insight on recommended actions to improve key operational areas, such as 

organizational structure and human capital; processes; technology, business, and 

infrastructure requirements; and physical space and workplace environment. According to 

VA officials, the National Call Center-Health Resource Center is one of VA’s largest call 

centers. It is dedicated to providing customer service and support to veterans, their 

beneficiaries and others regarding health benefits. According to VA officials, the National 

Call Center-Health Resource Center is a subject-matter expert in call center operations. 

47Rapid process-improvement events are a critical part of organizational improvements 

designed and implemented using Lean methodologies. Lean methodologies focus on 

developing system efficiencies by identifying and removing waste. These methodologies 

have been generally accepted by health care systems throughout the United States in the 

last 10 years. In March 2015, VA adopted Lean methodologies and techniques as a 

means of accomplishing strategic process and performance improvement. 

VA Has Developed 

and Begun 

Implementation of 

Plans to Improve VCL 

Operations
Page 28 GAO-16-373 Veterans Crisis Line

Renovation of new space for VCL primary center operations. We 

found that the VCL primary center responders are housed in two different 

buildings originally designed for patient care delivery. According to VA 

officials, these buildings do not reflect call center leading practices that 

recommend large, open rooms that provide supervisors greater access to 

the responders they oversee. However, in February 2016, VA officials 

reported that the department committed funding to relocate the VCL 

primary center operations to a renovated space on the VA medical center 

campus. The relocation is to be implemented in two phases. VA officials 

expect that the first phase, which includes moving administrative and 

monitoring staff, will be completed in June 2016; the second phase will 

relocate the rest of the VCL staff, including all responders. VA officials 

said they anticipate that the second phase will be completed in fiscal year 

2017. VA officials told us that they plan on using the National Call CenterHealth Resource Center’s large open-space layout as a model in 

designing the VCL primary center’s new space. According to VA officials, 

the National Call Center-Health Resource Center follows leading 

practices for call center operations as set by the International Call 

Management Institute.

Upgrade of VCL primary center telecommunication infrastructure.

VA officials told us that the VCL primary center uses the telephone 

infrastructure of the VA medical center rather than a separate telephone 

system that would be more conducive to operating a call center. 

According to a telephone infrastructure change justification that VA 

information-technology officials prepared, the VCL primary center’s 

existing telephone system does not meet the requirements for operating a 

call center of its size. This documentation indicates that improvements

are needed in several features of the VCL’s existing telephone system—

including call routing, call recording, data capture, and automatic callback. 

In February 2016, VA officials reported that planned improvements to the 

VCL primary center’s telephone system would be implemented by June 

2016; however, the VCL primary center will continue to operate using part 

of the VA medical center’s telephone system.

Introduction of VCL primary center caller queue. VA’s evaluation of 

the VCL conducted in 2014 noted that a possible option for improving 

VCL call response would be to implement a queue at the VCL primary 

center that would allow callers to wait a longer period for a VCL primary 
Page 29 GAO-16-373 Veterans Crisis Line

center responder before being sent to a VCL backup call center.48

Currently, VA allows VCL primary center responders 30 seconds to 

answer calls before routing them to VCL backup call centers for a 

response. In February 2016, VA officials told us that they are considering 

implementing this type of queue. According to these officials, they are 

considering allowing VCL calls to remain at the VCL primary center for up 

to 5 minutes and they explained that this 5-minute period was determined 

based on feedback they received from veterans on how long they would 

be willing to wait for a responder. These officials further explained that 

voice prompts would offer callers options as they waited in the queue to 

reach the next available VCL primary center responder or to be 

transferred to other VA call centers for concerns unrelated to crisis 

situations.

The VCL plays an important role in providing a means by which veterans 

and those concerned about them can discuss unique challenges and 

crises they face, and provides a way to access VA’s mental health care 

services. However, the rapid growth of the VCL in recent years has 

coincided with operational and planning challenges that constrain its 

ability to serve veterans in crisis in a timely and effective manner. To its 

credit, VA has taken some interim but noteworthy steps to address these 

challenges. Building on these steps, VA and SAMHSA need to take 

additional actions to provide reasonable assurance that the VCL’s 

mission to serve veterans and others in crisis situations is met.

As our testing demonstrates, VA has not yet achieved its goal of 

answering 90 percent of all VCL calls within 30 seconds at the VCL 

primary center, but its planned and recently implemented changes, such 

as staggered shifts and enhanced call-handling procedures, are intended 

to gain VA system efficiencies that will help the department meet its goal 

once additional responders are hired. However, VA has not applied the 

same level of attention to its text messaging service and does not 

regularly test the VCL’s text messaging system. Without doing so, VA 

cannot ensure that veterans are receiving timely responses from VCL 

responders to their text messages.

 48See Department of Veterans Affairs, Health Resource Center: Assessment of Veteran 

Crisis Line & National Call Center for Homeless Veterans.

Conclusions
Page 30 GAO-16-373 Veterans Crisis Line

In addition, while VA has taken a number of steps to improve its 

monitoring of the VCL, VA continues to experience challenges related to 

weaknesses in VCL key performance indicators—including a lack of 

measurable targets and time frames. If left unresolved, these weaknesses 

will likely have negative effects on VA’s ability to ensure the VCL is 

providing the best service possible to veterans.

Despite efforts to coordinate the operations of the VCL and Lifeline 

through an interagency agreement, VA and SAMHSA have not collected 

information necessary to determine how often and why veterans intending 

to reach the VCL reach Lifeline instead. As a result, neither VA nor 

SAMHSA can assess the extent this occurs and the underlying causes 

that may need to be addressed.

To improve the timeliness and quality of VCL responses to veterans and 

others, we recommend that the Secretary of Veterans Affairs direct the 

Under Secretary for Health to take the following two actions:

• regularly test the VCL’s text messaging system to identify issues and 

correct them promptly; and

• document clearly stated and measurable targets and time frames for 

key performance indicators needed to assess VCL performance.

We further recommend that under the applicable terms of their 

interagency agreement, the Secretary of Veterans Affairs and the 

Secretary of Health and Human Services direct the Under Secretary for 

Health and the Administrator of the Substance Abuse and Mental Health 

Services Administration (SAMHSA), respectively, to collaborate and take 

the following two actions:

• collect information on how often and why callers intending to reach 

the VCL instead reach Lifeline local crisis centers; and

• review the information collected and, if necessary, develop plans to 

address the underlying causes.

We provided a draft of this report to VA and HHS for review and 

comment. In their written comments, summarized below and reprinted in 

appendixes II and III, both agencies concurred with our

recommendations. VA and HHS described ongoing or planned actions 

Recommendations for 

Executive Action

Agency Comments 

and Our Evaluation
Page 31 GAO-16-373 Veterans Crisis Line

and provided a timeline for addressing our recommendations. HHS also 

provided technical comments, which we incorporated as appropriate.

In response to our first recommendation, to regularly test the VCL’s text 

messaging system to identify issues and correct them promptly, VA said 

that the VCL’s analytics department will develop and implement a more 

robust and proactive system to test daily the VCL’s text messaging 

service by July 2016. In the interim, VA stated that it has a process for 

identifying, addressing, and troubleshooting problems that utilizes e-mail 

templates to notify its contract text service provider of issues or errors that 

require a response to troubleshoot the error.

In response to our second recommendation, to document clearly stated 

and measurable targets and time frames for key performance indicators 

needed to assess VCL performance, VA said that it is in the process of 

developing a monthly scorecard with elements assessing call center, 

staffing, quality-assurance, and crisis-response metrics with specific 

performance targets. VA estimates that by October 2016 it would 

establish targets and time frames for its performance indicators.

In response to our third recommendation, to collaborate with SAMHSA to 

collect information on how often and why callers intending to reach the 

VCL instead reach Lifeline local crisis centers, VA said that the VCL’s 

newly formed Clinical Advisory Board would foster collaboration amongst 

capable experts and leverage their collective expertise in facilitating an 

improved experience for callers, greater operational efficiencies, and 

increased access to the VCL for veterans in crisis. VA noted that the 

Clinical Advisory Board included members of SAMHSA, the VA Suicide 

Prevention Office, and other VA clinical offices. VA estimates that it would 

collect sufficient data, conduct a collaborative analysis with SAMHSA, 

and complete reporting to both agencies on this issue by October 2016.

HHS said that in response to this recommendation it would review ways 

to collect data on callers intending to reach the VCL but instead reaching

Lifeline local crisis centers.

In response to our fourth recommendation, to collaborate with SAMHSA 

to review the information collected and, if necessary, develop plans to 

address the underlying causes for callers intending to reach the VCL 

instead reaching Lifeline local crisis centers, VA said that the Clinical 

Advisory Board referenced above would evaluate this issue as a standing 

agenda item in its monthly meetings. VA said that the Clinical Advisory 

Board would establish a baseline regarding the frequency of this issue’s 

occurrence, monitor reported complaints about the press “1” functionality, 

and provide us with data from Clinical Advisory Board meetings to 
Page 32 GAO-16-373 Veterans Crisis Line

demonstrate action taken toward implementing our recommendation. VA 

expects to complete these actions by January 2017. HHS said that in 

response to this recommendation it would review the data collected as 

described above and, if necessary, address the underlying causes as 

appropriate.

These VA and HHS actions, if implemented effectively, would address the 

intent of our recommendations.

In its technical comments, HHS emphasized the distinction between the 

Lifeline network and the VCL, noting that the two programs operate with 

different policies, procedures, and resources. We revised the draft to 

more clearly reflect this distinction.

As agreed with your offices, unless you publicly announce the contents of 

this report earlier, we plan no further distribution until 30 days from the 

report date. At that time we will send copies of this report to the 

appropriate congressional committees, the Secretary of Veterans Affairs, 

the Secretary of Health and Human Services, and other interested 

parties. In addition, the report is available at no charge on the GAO 

website at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact 

Seto J. Bagdoyan at (202) 512-6722 or bagdoyans@gao.gov, or Randall 

B. Williamson at (202) 512-7114 or williamsonr@gao.gov. Contact points 

for our Offices of Congressional Relations and Public Affairs may be 

found on the last page of this report. GAO staff who made key 

contributions to this report are listed in appendix IV.

Seto J. Bagdoyan

Director, Forensic Audits and Investigative Service

Randall B. Williamson

Director, Health Care
Appendix I: Scope and Methodology

Page 33 GAO-16-373 Veterans Crisis Line

To determine the extent to which the Department of Veterans Affairs (VA) 

meets response-time goals for calls, online chats, and text messages 

received through the Veterans Crisis Line (VCL), we conducted several 

tests of VCL services during July and August 2015. These tests were 

designed to measure the timeliness of the VCL’s response to calls, online 

chats, and text messages.

We conducted a covert test of the VCL’s call response time using a 

generalizable sample of 119 test calls placed in July and August 2015. To 

develop this generalizable sample, we interviewed VA officials with 

knowledge about VCL primary and backup call center operations; 

obtained the VCL primary center’s historical call volume data in hourly 

increments for fiscal year 2013 through the end of the second quarter of 

fiscal year 2015; and generated a schedule of days and times during 

which our test calls would be made.1 This test call schedule was created 

by dividing the 62-day sample period into 496 primary sampling units, 

which we defined as 3-hour blocks of time.2 We then defined secondary 

sample units as 10-minute increments within each 3-hour block of time 

and selected a stratified two-stage random cluster sample of 144 10-

minute increments during which our test calls would be made.3 We 

selected the 144 10-minute increments by: (1) stratifying the primary 

sampling units into four strata—overnight, morning, afternoon, and 

evening—based on time of day; (2) identifying a stratified sample of 36 

primary sampling units that were allocated across the four strata based 

on call volume and our available resources; and (3) randomly selecting 

 1

The VCL’s primary center is located in Canandaigua, New York, and is operated by VA 

staff. VCL backup call coverage is provided by a contractor who manages the VCL backup 

call centers. Currently, there are five VCL backup call centers, located in Greenville, North 

Carolina; Miami, Florida; Portland, Oregon; Rochester, New York; and St. Louis, Missouri.

2

The total number of primary sampling units in the population was calculated by 

multiplying the eight 3-hour blocks of time that were present in each day by the 62-day 

target population. This resulted in 496 total primary sampling units. 

3

We omitted test calls from our sample when these calls were unsuccessful due to an 

error made by our test callers. As a result, our final sample consisted of 119 test calls. 

Appendix I: Scope and Methodology

Testing VCL 

Response Times

Tests of VCL Call 

Response Timeliness
Appendix I: Scope and Methodology

Page 34 GAO-16-373 Veterans Crisis Line

four 10-minute increments from each selected primary sampling unit.4

The results of this test can be used to estimate all VCL callers’ wait times 

for July and August 2015.5 Because we followed a probability procedure 

based on random selections, our sample is only one of a large number of 

samples that we might have drawn. Since each sample could have 

provided different estimates, we express our confidence in the precision 

of our particular sample’s results as a 95 percent confidence interval (i.e., 

a margin of error of within plus or minus a certain number of percentage 

points). This is the interval that would contain the actual population value 

for 95 percent of the samples we could have drawn. Percentage 

estimates from our analysis included in this report have a margin of error 

of within plus or minus 9 percentage points at the 95 percent confidence 

level. Estimates of the median wait time have a margin of error of within 

plus or minus 10 percent at the 95 percent confidence level.

When placing test calls, we used 20 telephone numbers with randomly 

selected area codes to mask the origin of the calls.6 Two analysts then 

independently measured and documented wait times by reviewing audio 

recordings of each test call. Wait times were measured from the time that 

the caller pressed “1” to reach the VCL and the time that a responder 

 4

We defined overnight as midnight to 5:59 a.m., morning as 6:00 a.m. to 11:59 a.m., 

afternoon as noon to 5:59 p.m., and evening as 6:00 p.m. to 11:59 p.m. We selected 

fewer primary sampling units from the overnight and morning strata than we did from the 

afternoon and evening strata. The VCL historically received more calls during the time 

frames of the afternoon and evening strata, so 10 primary sampling units were selected 

from each of those strata. Five primary sampling units were selected from the overnight 

stratum to minimize the frequency with which GAO personnel had to be staffed overnight. 

Five primary sampling units were selected from the morning stratum because the VCL 

historically received fewer calls during that time of day. This resulted in a selection of 30 

strata for our sample. Finally, we selected a total of 120 10-minute increments by 

randomly selecting 4 10-minute increments for each of the 30 primary sampling units that 

we selected for the sample. 

5

The results of these test calls are not generalizable to calls to the VCL outside of the July 

and August 2015 period. 

6

We developed the list of 20 randomly selected area codes by (1) generating a list of 

randomly selected three-digit numbers, (2) developing a list of the first 30 three-digit 

numbers that were active U.S. area codes, and (3) selecting the first 20 area codes that 

were available through the service that was used to establish the final, 10-digit telephone 

numbers. We used these 20 telephone numbers when placing our test calls so that VA 

could not detect through caller identification technology that the calls originated at GAO. 
Appendix I: Scope and Methodology

Page 35 GAO-16-373 Veterans Crisis Line

answered the test call.7 The final wait time for each test call was the 

lowest of the two wait times recorded by these analysts.

We successfully completed and measured the wait times for 119 test calls 

in 30 of the 36 selected primary sampling units. We did not complete calls 

in 25 of our selected 10-minute increments due to technical or scheduling 

issues. The resulting completion rate for our test calls was 83 percent 

(119 out of 144). The omitted test calls were distributed across all four 

strata and were unrelated to the time of day.

To test VA’s online chat and text message response timeliness, we 

reviewed VA’s procedures and training materials for operating both 

services. We then interviewed and observed VCL responders at the VCL 

primary center who responded to online chat and text messages. We also 

spoke with the VCL text messaging service provider to learn about the 

text messaging operations. To test the VCL’s online chat and text 

message response, we scheduled one covert test online chat or text 

message during each of the 30 primary sampling units used for the 

generalizable sample described above and recorded our wait times for a 

response.8 We measured the wait time for online chats and text 

messages as the elapsed time between when we sent the online chat or 

text message to the VCL and when we received a response from a 

responder. We initiated our test online chats through a link provided on 

the VCL’s website. We sent test text messages to the VCL through an 

Internet text messaging service provider in order to record our test data 

electronically. We removed one test text message attempt from the 

sample because of technical issues we experienced that may have 

prevented the message from reaching the VCL. As a result, our final 

samples consisted of 15 test online chats and 14 test text messages.

 7

We defined VCL call wait times as the difference (in seconds) between when a caller 

pressed “1” and when a live responder answered the test call at either the VCL primary 

center or a VCL backup call center. 

8

Although we used the primary sampling units selected for the generalizable sample of 

test calls to determine when the online chat or text message request was sent, we did not 

design the samples used to test VCL online chat and text message response times to be 

able to generalize our results to the population of all online chat and text message 

requests received by the VCL during July and August 2015. 

Tests of VCL Online Chat 

and Text Message 

Response
Appendix I: Scope and Methodology

Page 36 GAO-16-373 Veterans Crisis Line

We verified the reliability of VA’s reported VCL call data by interviewing 

officials responsible for managing them and reviewing reports that VA’s 

backup call coverage contractor provided to VA that documented the 

time, duration, and routing of every VCL call. The routing information 

included details on the call centers where each call was routed and 

identified the call center that ultimately answered each call. We were able 

to identify our test calls in these reports and confirmed that the data 

matched records we maintained for our test calls. This exercise also 

allowed us to confirm whether our test calls were answered at the VCL 

primary center or a VCL backup call center. On the basis of these actions, 

we found these data to be sufficiently reliable for the purposes of 

describing the quantity of requests for services reaching the VCL. We 

used these data to evaluate the timeliness of the VCL’s call response and 

compared the data to the department’s own goals.

To assess the effectiveness of the implementation of staggered shifts for 

responders at the VCL primary center, we compared VCL call data from 

September 6, 2015, through December 31, 2015, to that of September 1, 

2014, through December 31, 2014.9 We selected September 6, 2015, as 

the start date for our 2015 period of analysis because it was the first day 

that VA fully implemented staggered shifts at the VCL primary center. We 

chose the cutoff of December 31, 2015, because it corresponded to the 

most recent complete month of data available at the time of our analysis. 

We used call data from September 1, 2014, through December 31, 2014, 

because they reflected a comparable period from the year prior. We used 

these 2014 data as a comparison group to account for any seasonality 

patterns, variations, or fluctuations that might affect the demand for VCL 

services within a particular season, day of the week, or other periods. Our 

 9

Staggered shifts are work schedules that allow employees to start and stop their shifts at 

different times to help ensure more employees are working when demand for the work is 

highest and fewer employees are working when demand is lowest and help limit 

disruptions in service as responders begin and end their shifts.

Analysis of the VCL’s 

2013 through 2015 

Call Data

Data-Reliability Testing

Analysis of VCL Call Data 

Related to the 

Implementation of 

Staggered Shifts at the 

VCL Primary Center
Appendix I: Scope and Methodology

Page 37 GAO-16-373 Veterans Crisis Line

evaluation compared the average hourly call response percentages of the 

periods we examined and included analysis for the time of day using 

hourly intervals, day of the week, and holidays within each period. The 

average hourly response percentages are likely affected by several 

factors—such as call volume, staffing levels, and complexity of calls, for 

which we did not control. Our analysis examined differences by day of the 

week, time of day, and holidays, but did not control for the abovementioned or other factors that may affect the percentage of calls 

answered at the VCL primary center.

To determine whether callers attempting to reach the VCL who did not 

press “1” experienced longer wait times than those that did, we conducted 

a nongeneralizable test. The VCL is accessed by calling a single national 

toll-free number—1-800-273-TALK (8255)—shared by both the VCL and 

the National Suicide Prevention Lifeline (Lifeline). This toll-free number is 

managed by the Substance Abuse and Mental Health Services 

Administration (SAMHSA).10 To conduct our nongeneralizable test, we 

used a random sample of 34 covert test calls to conduct these tests 

where we mimicked the experience of VCL callers who do not follow the 

instructions of the voice prompt on this single national toll-free number to 

press “1” in order to reach the VCL. To do this, we placed two test calls 

where we did not press “1” as prompted to reach the VCL during each of 

the scheduled primary sampling units in August 2015. We recorded the 

wait times for each of the 34 test calls by calculating the amount of time 

that elapsed between the moment that an automated message informed 

us that the call was being transferred to a Lifeline local crisis center and 

when a responder answered our call. We masked the origin of these calls 

in a manner similar to that described for our generalizable sample of 119 

test calls placed to the VCL. Although the 34 test calls were randomly 

made, the results of these test calls are not generalizable due to the small 

number of calls included in our sample.

 10SAMHSA is an agency within the U.S. Department of Health and Human Services that 

leads public health efforts to advance the behavioral health of the nation. Currently, 

SAMHSA funds a cooperative agreement grant to administer Lifeline with the same entity 

that VA currently contracts with to provide VCL backup call coverage.

Test of Lifeline Local 

Crisis Center 

Response Times
Appendix II: Comments from the Department 

of Veterans Affairs

Page 38 GAO-16-373 Veterans Crisis Line

Appendix II: Comments from the Department 

of Veterans Affairs
Appendix II: Comments from the Department 

of Veterans Affairs

Page 39 GAO-16-373 Veterans Crisis Line
Appendix II: Comments from the Department 

of Veterans Affairs

Page 40 GAO-16-373 Veterans Crisis Line
Appendix II: Comments from the Department 

of Veterans Affairs

Page 41 GAO-16-373 Veterans Crisis Line
Appendix II: Comments from the Department 

of Veterans Affairs

Page 42 GAO-16-373 Veterans Crisis Line
Appendix II: Comments from the Department 

of Veterans Affairs

Page 43 GAO-16-373 Veterans Crisis Line
Appendix II: Comments from the Department 

of Veterans Affairs

Page 44 GAO-16-373 Veterans Crisis Line
Appendix III: Comments from the Department 

of Health and Human Services

Page 45 GAO-16-373 Veterans Crisis Line

Appendix III: Comments from the 

Department of Health and Human Services
Appendix III: Comments from the Department 

of Health and Human Services

Page 46 GAO-16-373 Veterans Crisis Line
Appendix IV: GAO Contacts and 

Staff Acknowledgments

Page 47 GAO-16-373 Veterans Crisis Line

Seto J. Bagdoyan, (202) 512-6722 or bagdoyans@gao.gov

Randall B. Williamson, (202) 512-7114 or williamsonr@gao.gov

In addition to the contacts named above, Gabrielle M. Fagan (Assistant 

Director), Marcia A. Mann (Assistant Director), James D. Ashley, Dean 

Campbell, Shaunessye D. Curry, Amber D. Gray, Katherine Nicole 

Laubacher, Olivia Lopez, Maria McMullen, Brynn P. Rovito, Amber H. 

Sinclair, and Shana B. Wallace made key contributions to this report. 

Members of our investigative staff also contributed to the report.

Appendix IV: GAO Contacts and Staff 

Acknowledgments

GAO Contacts

Staff 

Acknowledgments

(192456)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates. 

Listen to our Podcasts and read The Watchblog.

Visit GAO on the web at www.gao.gov.

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.