VA Health IT: Use of Acquisition Best Practices Can Improve Efforts to Implement a System to Support the Family Caregiver Program
Released: 2019-05-22T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: To provide greater support for caregivers of post-9/11 veterans, Congress and the President enacted legislation requiring VA to establish a program to assist caregivers with the rigors of caring for seriously injured veterans. In May 2011, the Veterans Health Administration (VHA), which operates VA's health care system, established the Family Caregiver Program at each of its VA medical centers across the United States. At that time, the department implemented an IT system, called CAT, to help support the program. Subsequently, the VA MISSION Act was enacted in June 2018, requiring VA to implement an IT system to fully support the Family Caregiver Program by October 1, 2018. Further, VA's Secretary is to certify the system by October 1, 2019.

GAO was asked to discuss its September 2014 report that examined how VHA is implementing the Family Caregiver Program. In addition, the statement includes relevant information VA provided on its actions toward addressing GAO's prior recommendation. The statement also discusses critical success factors related to IT acquisitions as identified in GAO's prior work. The reports cited throughout this statement include detailed information on the scope and methodology of GAO's prior reviews.

section_title: What GAO Found
paragraphs: In September 2014, GAO reported on the Department of Veterans Affairs' (VA) Program of Comprehensive Assistance for Family Caregivers (Family Caregiver Program) and found that the program office had limitations with its information technology (IT) system—the Caregiver Application Tracker (CAT). Specifically, the program did not have ready access to workload data that would allow it to monitor the effects of the program on VA medical centers' resources. VA has initiated various projects since 2015 to implement a new system, but has not yet been successful in its efforts. (See figure.) Specifically, in July 2015 VA initiated a project to improve the reliability of CAT's data, called CAT Rescue. However, the department reported in January 2017 that it had identified numerous defects during system testing. The project ended in April 2018 before any new system capabilities were implemented. A companion project was initiated in September 2015 to develop the Caregivers Tool (CareT), a new system intended to replace CAT. The CareT project was expected to use improved data from CAT Rescue, while also adding new system capabilities. However, the user acceptance testing of CareT identified the need for the department to develop more system capabilities than originally planned. Further, VA reported that implementing a system by October 1, 2018, as specified in the Maintaining Internal Systems and Strengthening Integrated Outside Networks Act of 2018 (MISSION Act), was not feasible. Subsequently, VA terminated CareT in February 2019. The department initiated another project in March 2019 to implement a new system, the Caregiver Record Management Application (CARMA). GAO has ongoing work to evaluate the department's efforts to implement an IT system to support the Family Caregiver Program as required by the MISSION Act.

GAO's prior work has determined that successfully overcoming IT acquisition challenges can best be achieved when critical success factors are applied. These factors can serve as a model of best practices that VA could apply to enhance the likelihood that the acquisition of a replacement IT system for the Family Caregiver Program will be successfully achieved. Examples of these critical success factors include, maintaining active engagement of program officials with stakeholders, involving end users and stakeholders in the development of requirements, and ensuring participation of end users in testing system functionality prior to formal end user acceptance testing.

section_title: What GAO Recommends
paragraphs: GAO recommended in 2014 that VA expedite the process for identifying and implementing an IT system that would fully support the Family Caregiver Program. VA concurred with the recommendation and subsequently began taking steps to implement a replacement system. The recommendation remains open.

section_title: Letter
paragraphs: Thank you for the opportunity to participate in today’s hearing regarding  the Department of Veterans Affairs’ (VA) efforts to implement an  information technology (IT) system to support the management and  execution of its Program of Comprehensive Assistance for Family  Caregivers (Family Caregiver Program).

To provide greater support for caregivers of post-9/11 veterans, Congress  and the President enacted legislation in May 2010 requiring VA to  establish a program to assist caregivers with the rigors of caring for  seriously injured veterans. In May 2011, the Veterans Health  Administration (VHA), which operates VA’s health care system,  established the Family Caregiver Program at each of its VA medical  centers (VAMC) across the United States.

At that time, the department implemented an IT system, called the  Caregiver Application Tracker (CAT), to help support the program.  However, we reported in September 2014 that CAT, which is still in use  today, had limitations and recommended that VA expedite the  implementation of a replacement system.

As you requested, my statement today summarizes findings from our  September 2014 report that discussed VA’s implementation of the Family  Caregiver Program. This statement also includes relevant information that  VA provided on its actions toward addressing our prior recommendation.  Further, my statement discusses critical success factors related to major  IT acquisitions identified in our prior work. We have previously reported  that these success factors could enhance the likelihood that an IT  acquisition will be successful. The reports cited throughout this statement  include detailed information on the scope and methodology of our prior  reviews.

We conducted the work on which this statement is based in accordance  with generally accepted government auditing standards. Those standards  require that we plan and perform the audit to obtain sufficient, appropriate  evidence to provide a reasonable basis for our findings and conclusions  based on our audit objectives. We believe that the evidence obtained  provides a reasonable basis for our findings and conclusions based on  our audit objectives.
subsections: section_title: Background
paragraphs: VHA’s Family Caregiver Program is designed to provide support and  services to family caregivers of post-9/11 veterans who have a serious  injury that was incurred or aggravated in the line of duty. The program  provides approved primary family caregivers with a monthly financial  stipend as well as training and other support services, such as counseling  and respite care.

The Family Caregiver Program has a series of eligibility requirements that  must be satisfied in order for family caregivers to be approved.

To meet the program’s initial eligibility criteria, the veteran seeking  caregiver assistance must have a serious injury that was incurred or  aggravated in the line of duty on or after September 11, 2001.  According to the program’s regulations, a serious injury is any injury,  including traumatic brain injury (TBI), psychological trauma, or other  mental disorder, that has been incurred or aggravated in the line of  duty and renders the veteran or servicemember in need of personal  care services.

The veteran must be in need of personal care services for a minimum  of 6 continuous months based on any one of the following clinical  eligibility criteria: (1) an inability to perform one or more activities of  daily living, such as bathing, dressing, or eating; (2) a need for  supervision or protection based on symptoms or residuals of  neurological or other impairment or injury such as TBI, post-traumatic  stress disorder, or other mental health disorders; (3) the existence of  a psychological trauma or a mental disorder that has been scored by  a licensed mental health professional, with a Global Assessment of  Functioning score of 30 or less, continuously during the 90-day  period immediately preceding the date on which VHA initially received  the application; or (4) the veteran has been rated 100 percent service  connected disabled for a qualifying serious injury and has been  awarded special monthly compensation that includes an aid and  attendance allowance.

To be considered competent to care for the veteran, family caregivers  must meet certain requirements including (1) having the ability to  communicate and follow details of the treatment plan and instructions  related to the care of the veteran; (2) not determined by VA to have  abused or neglected the veteran; (3) being at least 18 years of age;  and (4) either being a family member—such as a spouse, son or  daughter, parent, step-family member, or extended family member— or an unrelated person who lives or will live full-time with the veteran.

Family caregivers must also complete required training before being  approved for the program.
subsections: section_title: Family Caregiver Program Organizational Structure
paragraphs: VHA’s Caregiver Support Program office is responsible for developing  policy and providing guidance and oversight for the Family Caregiver  Program. It also directly administers the program’s stipend, provides  support services such as a telephone hotline and website, and arranges  coverage through the Civilian Health and Medical Program of the  Department of Veterans Affairs (CHAMPVA) for eligible caregivers if they  have no other coverage.

Further, the office provides funding to VAMCs to cover certain program  costs. These costs may include the salaries of the caregiver support  coordinators (CSC), who implement and administer the Family Caregiver  Program at the local VAMC level, and the costs VAMCs incur for having  their clinical staff, such as nurses, conduct the program’s required in- home visits to approved caregivers and their veterans.

CSCs are generally licensed social workers or registered nurses, and  they have both clinical and administrative responsibilities. Their clinical  responsibilities may include identifying and coordinating appropriate  interventions for caregivers or referrals to other VA or non-VA programs,  such as mental health treatment, respite care, or additional training and  education. Their administrative responsibilities may include responding to  inquiries about the program, overseeing the application process, entering  information about applications and approved caregivers into IT systems,  and facilitating the processing of appeals.

As of May 2014, there were 233 CSCs assigned to 140 VAMCs or health  care systems across the country. Additionally, each regional VISN office  has a VISN CSC lead for the program, who provides guidance to CSCs  and helps address their questions or concerns.

section_title: GAO Has Previously Reported on the Family Caregiver Program IT System Limitations
paragraphs: CAT, which was deployed in May 2011, is a web-based system that was  designed to facilitate the exchange of information about approved  caregivers between VAMCs and other VHA entities. Such entities include  the Health Administration Center, which processes the caregiver stipend  payments and administers CHAMPVA.

In 2014, we reported that the Caregiver Support Program office was not  able to easily retrieve data from CAT that would allow officials to better  assess workload trends at individual VAMCs—such as the length of time  applications are delayed or the timeliness of home visits—even though  these data were already captured in the system. Caregiver Support  Program officials only retrieved workload data on an ad hoc, as-needed  basis, which limited their ability to assess the scope and extent of  workload problems comprehensively at individual VAMCs and on a  system-wide basis. Program officials also expressed concern about the  reliability of the system’s data.

As we noted in our report, program officials also identified the need for a  more capable and flexible system that could interface with other  departmental systems. The officials told us that they had taken initial  steps to obtain another IT system to support the Family Caregiver  Program; however, the officials were not sure how long it would take to  implement the system. Accordingly, we recommended that VA expedite  the process for identifying and implementing a system that would fully  support the Family Caregiver Program.

VA concurred with our recommendation and subsequently began taking  actions in 2015 to implement a replacement system. These actions  included taking steps toward implementing short-term improvements to  CAT that were to be followed by the implementation of a long-term  replacement system. The recommendation continues to remain open.

section_title: Statute Directs VA to Implement an IT System to Support the Family Caregiver Program
paragraphs: The John S. McCain III, Daniel K. Akaka, and Samuel R. Johnson VA  Maintaining Internal Systems and Strengthening Integrated Outside  Networks Act of 2018 (VA MISSION Act), which was enacted in June  2018, included provisions directing VA to implement an IT system to  support the Family Caregiver Program and the incremental expansion of  program eligibility. Specifically, the act required VA to implement an IT  system to fully support the Family Caregiver Program by October 1, 2018.  According to the act, the system is to allow for data assessment and  comprehensive monitoring of the program. In particular, the system is to  have, among other things, the ability to (1) retrieve data to monitor  workload trends at the medical center and aggregate levels; (2) manage  an increased number of caregivers as the program expands; and (3)  integrate with other relevant IT systems at VHA.

The act also stated that VA was to submit an initial report to Congress  regarding the status of the planning, development, and deployment of this  system within 90 days of enactment of the VA MISSION Act, and that the  department is to submit a final report to Congress by October 1, 2019.  The final report is to include a certification by the VA Secretary that the  system has been implemented, along with a description of how the  Secretary is using the system to monitor the workload of the program.

section_title: VA Has Not Yet Implemented an IT System That Effectively Supports the Family Caregiver Program
paragraphs: Although we previously recommended that VA expedite implementation  of a replacement for CAT, and the MISSION Act directed the department  to implement an IT system to support the Family Caregiver Program, VA  has not yet been successful in its multiple efforts to implement such a  system. Specifically, VA has faced a number of difficulties in developing  and implementing short-term improvements as well as a long-term  replacement system for CAT.

In July 2015, VHA and the Office of Information and Technology (OIT)  initiated a joint acquisition project, called CAT Rescue, to update CAT  and improve the system’s data reliability. However, the department  reported in January 2017 that this project had experienced delays and  identified a large number of defects during system testing. VA terminated  the project in April 2018 before any new system capabilities were  implemented.

A companion project to CAT Rescue that VA initiated in September 2015  was to develop the Caregivers Tool (CareT), a new system intended to be  a long-term replacement for CAT. As envisioned, this system was to use  the improved data from CAT Rescue while also adding new system  capabilities. However, the user acceptance testing of CareT identified the  need for the department to develop more system capabilities than  originally planned. Further, the department determined that the time  period needed to perform additional system development would have  extended beyond the term of the development contract, which ended in  April 2017.

VA subsequently awarded a new CareT development contract in July  2017. However, after additional system development, the department  determined during user acceptance testing that the system was not  performing as expected and implementation of CareT was further  delayed. In October 2018, the department reported to congressional  committees that implementing a system to fully support the Family  Caregiver Program by the VA MISSION Act deadline was not feasible.  Subsequently, the department determined that CareT was not a viable  solution and VHA and OIT terminated work on the system in February  2019.

VHA and OIT began a third effort in March 2019 to acquire a replacement  system that is to be based on an existing commercial product. According  to OIT officials, the new IT solution, referred to as the Caregiver Record  Management Application (CARMA), is intended to replace CAT. However,  the department has not yet established a date for completing CARMA.

Thus, VA’s efforts to implement an IT system that supports the Family  Caregiver Program have been continuing with no end in sight. We have  ongoing work to further evaluate the status and progress of the  department’s efforts to implement a system to support the Family  Caregiver Program consistent with the VA MISSION Act requirements.  Figure 1 provides a timeline of the various IT projects that VA has  undertaken to support the program.

section_title: Critical Factors Underlying Successful IT Acquisitions
paragraphs: Our prior work has determined that successfully overcoming IT  acquisition challenges can best be achieved when critical success factors  are applied. Specifically, we reported in 2011 on common factors critical  to the success of IT acquisitions, based on seven agencies having each  identified the acquisition that best achieved the agency’s respective cost,  schedule, scope, and performance goals. These factors remain relevant  today and can serve as a model of best practices that agencies can apply  to enhance the likelihood that the acquisition of an IT system such as  CARMA will be successfully achieved.

Among the agencies’ seven IT investments, agency officials identified  nine factors as having been critical to the success of three or more of the  seven investments. These nine critical success factors are consistent with  leading industry practices for IT acquisition. The factors are:

Active engagement of program officials with stakeholders.

Qualified and experienced program staff.

Support of senior department and agency executives.

Involvement of end users and stakeholders in the development of  requirements.

Participation of end users in testing system functionality prior to formal  end user acceptance testing.

Consistency and stability of government and contractor staff.

Prioritization of requirements by program staff.

Regular communication maintained between program officials and the  prime contractor.

Sufficient funding.

Officials for all seven selected investments cited active engagement with  program stakeholders—individuals or groups (including, in some cases,  end users) with an interest in the success of the acquisition—as a critical  factor to the success of those investments. Agency officials stated that  stakeholders, among other things, reviewed contractor proposals during  the procurement process, regularly attended program management office  sponsored meetings, were working members of integrated project  teams, and were notified of problems and concerns as soon as  possible. Further, officials from two investments noted that actively  engaging with stakeholders created transparency and trust, and  increased the support from the stakeholders.

Additionally, officials for six of the seven selected investments indicated  that the knowledge and skills of the program staff were critical to the  success of the program. This included knowledge of acquisitions and  procurement processes, monitoring of contracts, large-scale  organizational transformation, Agile software development concepts,  and areas of program management such as earned value management  and technical monitoring.

Finally, officials for five of the seven selected investments identified  having the end users test and validate the system components prior to  formal end user acceptance testing for deployment as critical to the  success of their program. Similar to this factor, leading guidance  recommends testing selected products and product components  throughout the program life cycle. Testing of functionality by end users  prior to acceptance demonstrates, earlier rather than later in the program  life cycle, that the functionality will fulfill its intended use. If problems are  found during this testing, programs are typically positioned to make  changes that would be less costly and disruptive than ones made later in  the life cycle.

In conclusion, VA has invested considerable time in multiple efforts  toward improving and replacing its IT system to better serve the Family  Caregiver Program. However, even with these efforts, the department has  not yet implemented a system and the program is not prepared for  expansion. Going forward, it is important that VA take steps to improve its  efforts to implement a replacement IT system for the Family Caregiver  Program. In this regard, the department could benefit from applying  critical success factors we previously reported as leading to successful  federal IT acquisitions. These factors can serve as a model of best  practices that the department can apply to enhance the likelihood that its  effort to replace the IT system for the Family Caregiver Program will be  successful.

Chairs Lee and Brownley, Ranking Members Banks and Dunn, and  Members of the Subcommittees, this completes my prepared statement. I  would be pleased to respond to any questions that you may have.

section_title: GAO Contact and Staff Acknowledgments
paragraphs: If you or your staffs have any questions about this testimony, please  contact Carol C. Harris, Director, Information Technology Management  Issues, at (202) 512-4456 or harriscc@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this testimony statement. GAO staff who made key  contributions to this testimony are Mark Bird (Assistant Director),  Rebecca Eyler, Jacqueline Mai, Monica Perez-Nelson, Scott Pettis, and  Jennifer Stavros-Turner (Analyst in Charge).

This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.

Fast facts: The Veterans Administration established the Family Caregiver Program in 2011 to help families provide care to seriously injured veterans. It developed an IT system to help it run this program but the system has many problems.

We reported on these problems in 2014. For example, the system does not provide VA with data that would enable the agency to monitor how the Family Caregiver Program affects its medical centers' resources. We recommended the VA address these problems.

We testified about our prior and ongoing work related to this system. We also testified about steps VA has taken to fix the system and the problems with it that persist.

Question-summary annotations: questions: question: What limitations did GAO find with the Department of Veterans Affairs' IT system?
answer: In September 2014, GAO reported on the Department of Veterans Affairs' (VA) Program of Comprehensive Assistance for Family Caregivers (Family Caregiver Program) and found that the program office had limitations with its information technology (IT) system—the Caregiver Application Tracker (CAT). Specifically, the program did not have ready access to workload data that would allow it to monitor the effects of the program on VA medical centers' resources.
child_questions: question: What actions is VA taking to address these limitations?
answer: VA has initiated various projects since 2015 to implement a new system, but has not yet been successful in its efforts. (See figure.) Specifically, in July 2015 VA initiated a project to improve the reliability of CAT's data, called CAT Rescue. However, the department reported in January 2017 that it had identified numerous defects during system testing. The project ended in April 2018 before any new system capabilities were implemented. The department initiated another project in March 2019 to implement a new system, the Caregiver Record Management Application (CARMA).

question: What occured with the VA's CareT project?
answer: A companion project was initiated in September 2015 to develop the Caregivers Tool (CareT), a new system intended to replace CAT. The CareT project was expected to use improved data from CAT Rescue, while also adding new system capabilities. However, the user acceptance testing of CareT identified the need for the department to develop more system capabilities than originally planned. Further, VA reported that implementing a system by October 1, 2018, as specified in the Maintaining Internal Systems and Strengthening Integrated Outside Networks Act of 2018 (MISSION Act), was not feasible. Subsequently, VA terminated CareT in February 2019.

question: What are GAO's ongoing assessments?
answer: GAO has ongoing work to evaluate the department's efforts to implement an IT system to support the Family Caregiver Program as required by the MISSION Act.

questions: question: How does GAO suggest overcoming IT acquisition challenges?
answer: GAO's prior work has determined that successfully overcoming IT acquisition challenges can best be achieved when critical success factors are applied. These factors can serve as a model of best practices that VA could apply to enhance the likelihood that the acquisition of a replacement IT system for the Family Caregiver Program will be successfully achieved.
child_questions: question: How can VA use these suggestions in their acquisition of a replacement IT system?
answer: These factors can serve as a model of best practices that VA could apply to enhance the likelihood that the acquisition of a replacement IT system for the Family Caregiver Program will be successfully achieved.

question: What are examples of critical success factors?
answer: Examples of these critical success factors include, maintaining active engagement of program officials with stakeholders, involving end users and stakeholders in the development of requirements, and ensuring participation of end users in testing system functionality prior to formal end user acceptance testing.

questions: question: How has the government supported caregivers of veterans?
answer: To provide greater support for caregivers of post-9/11 veterans, Congress and the President enacted legislation requiring VA to establish a program to assist caregivers with the rigors of caring for seriously injured veterans.
child_questions: question: What does the Veterans Health Administration do?
answer: In May 2011, the Veterans Health Administration (VHA), which operates VA's health care system, established the Family Caregiver Program at each of its VA medical centers across the United States.
child_questions: question: What IT system did the department implement?
answer: At that time, the department implemented an IT system, called CAT, to help support the program.

question: What did the VA MISSION Act require VA to do?
answer: Subsequently, the VA MISSION Act was enacted in June 2018, requiring VA to implement an IT system to fully support the Family Caregiver Program by October 1, 2018. Further, VA's Secretary is to certify the system by October 1, 2019.

questions: question: What was GAO asked to discuss?
answer: GAO was asked to discuss its September 2014 report that examined how VHA is implementing the Family Caregiver Program.
child_questions: question: What else does this statement include?
answer: In addition, the statement includes relevant information VA provided on its actions toward addressing GAO's prior recommendation. The statement also discusses critical success factors related to IT acquisitions as identified in GAO's prior work.

question: How do the reports cited throughout this statement relate to GAO's work?
answer: The reports cited throughout this statement include detailed information on the scope and methodology of GAO's prior reviews.