Transitioning Veterans: Improved Oversight Needed to Enhance Implementation of Transition Assistance Program
Published: 2014-03-05T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Over the next few years, over a million military servicemembers are expected to transition to civilian life and some may face challenges such as finding employment. To help them, TAP provides departing servicemembers employment assistance and information on VA benefits, among other things. Begun in 2011, efforts to revamp TAP are underway based on the VOW to Hire Heroes Act of 2011 and the administration's recommendations. The act also mandated GAO to review TAP. This report addresses: 1) the status of TAP implementation; 2) the extent to which elements of effective implementation and evaluation of TAP have been addressed; and 3) any challenges that may remain. To do this GAO identified five elements of effective implementation and evaluation based on relevant federal laws and previously established GAO criteria for training programs; reviewed related GAO work; assessed reports, plans, and policies provided by agencies that administer TAP; interviewed officials from entities that support servicemembers and veterans; and conducted four nongeneralizable discussion groups with servicemembers who had taken TAP at three military installations.
section_title: What GAO Found
paragraphs: The Departments of Defense (DOD), Labor (DOL), and Veterans Affairs (VA) have implemented most of the key components of the Transition Assistance Program (TAP), a gateway to information and services available to servicemembers transitioning to civilian life. However, the agencies are still in the process of implementing other key components of TAP. While originally planned for October 2013, agencies now plan to implement virtually all components by the end of March 2014, with full implementation expected by June 2014.
Agencies' efforts are underway to adequately address three of five elements that GAO identified as important for effective implementation and evaluation of TAP:
1-Track attendance : DOD has systems to collect and report on attendance, which help measure the extent to which TAP achieves its attendance goals.
2-Ensure training quality : The agencies collect and plan to use participant feedback on instruction, content, and facilities to improve training. Each agency also plans to monitor its respective TAP components through site visits.
3-Assess career readiness : The agencies developed standards to assess servicemembers' career readiness. During a capstone assessment, commanders are expected to verify and document whether standards were met.
Agencies' efforts to address the remaining two elements are mixed:
4-Ensure participation and completion : DOD has assigned commanders the responsibility for overseeing participation and has required the services to schedule training and communicate its importance to servicemembers. While the Army and Air Force gauge participation at the command level, the Navy and Marines lack a similar oversight mechanism.
5-Measure performance and evaluate results : The agencies have established certain measures to assess program performance, but their TAP evaluation approach is incomplete. For example, the agencies have established measures to track program outputs, such as the percentage of servicemembers who have participated in TAP. However, the agencies' efforts to evaluate TAP results have focused on basic end-of-course evaluations and gauging servicemembers' readiness prior to separation instead of higher impact program evaluations, such as assessing the effectiveness of TAP on servicemembers 6 months after they have separated from the military. According to agency officials, such evaluations are being considered for certain components of TAP, but they could not provide GAO with a justification for including or excluding specific components of TAP in their evaluation planning efforts.
Based on GAO's prior work and according to officials from the agencies and organizations GAO spoke with, a key remaining challenge for TAP may be the unfavorable timing and location of program delivery for National Guard and Reserve members. Unlike active duty servicemembers, National Guard and Reserve members receive TAP services closer to their transition and in locations that are generally neither where they work nor live. As a result, they may be distracted and have less time to benefit from TAP services. DOD is not well positioned to verify these concerns because it is not collecting data about these members' experiences with the timing and location of TAP service delivery.
section_title: What GAO Recommends
paragraphs: GAO recommends that DOD improve oversight and implementation of TAP, including actions to gauge participation for all of the services and collect data about National Guard and Reserve members' experiences. DOD disagreed with two of GAO's three recommendations. GAO continues to believe that the recommendations are needed as discussed in the report.

section_title: Letter
paragraphs: Over the next 4 to 5 years, more than a million servicemembers are  expected to leave the military and transition into civilian life. Some of  these new veterans will face significant challenges, such as finding and  maintaining employment. To help them, the Transition Assistance  Program (TAP) has provided departing servicemembers counseling and  offered employment assistance and information on federal veteran  benefits, among other things.
Pub. L. No. 112-56, Title II, 125 Stat. 711, 712. After the VOW Act became law, the  unemployment rate for post-9/11 veterans improved—decreasing from 13.1 percent in  December 2011 to 7.3 percent in December 2013—but remained above the overall  unemployment rate for non-veterans (6.4 percent), according to the Bureau of Labor  Statistics. According to the Bureau of Labor Statistics, veterans between the ages of 18 to  24 maintain the highest rate of unemployment. In 2012, the unemployment rate for male  post-9/11 veterans age 18 to 24 was 20.0 percent, higher than the rate for nonveterans of  the same age group (16.4 percent).
Initiative Task Force (VEI Task Force) with the help of other federal  partners.
The VOW Act also mandated that GAO review TAP. report addresses:  1.  the status of efforts to implement changes to key components of TAP,  2.  the extent to which the agencies administering TAP have addressed  elements important for the effective implementation and evaluation of  TAP, and  3.  remaining challenges, if any, that the agencies administering TAP  may need to resolve.
Pub. L. No. 112-56, § 226, 125 Stat. 711, 719. On September 20 and 24 and November  12 and 20, 2013, we briefed the House and Senate Committees on Armed Services and  the House and Senate Committees on Veterans’ Affairs on our preliminary observations.  This report formally conveys the information provided during those briefings. interviews with officials from the agencies and services to determine if  these components were implemented at all TAP locations. We found the  data to be sufficiently reliable for the purposes of reporting on the status  of implementation.
To assess efforts to implement and evaluate the revamped TAP, we  identified five elements important for the effective implementation and  evaluation of TAP. The elements are: (1) systems to track attendance; (2)  processes for ensuring quality of instruction, content, and facilities; (3)  mechanisms to assess career readiness; (4) mechanisms to ensure  participation and completion; and (5) measures of performance and  processes to systematically evaluate the effectiveness or benefits of TAP.  We identified the elements based on a review of relevant federal laws,  including the VOW Act, and VEI Task Force recommendations for  changing TAP, as well as a GAO assessment guide that identifies  elements of effective training and development programs. We also  provided the elements to DOD and VA officials—as co-leaders of the VEI  Task Force—for review and comment, and they were generally in  agreement. Additional information about these elements and relevant  attributes is provided in appendix I. To determine the extent to which  agencies have addressed the five elements, we reviewed program  documentation from the agencies and discussed the policies and  mechanisms with knowledgeable officials. We compared the elements  with this documentation and results of our discussions with officials at  these agencies.
For example, we reviewed a January 2013 Defense Business Board report that identified  some of the remaining challenges associated with implementing and evaluating TAP. responsible for administering TAP and several veterans’ service  organizations. We also interviewed officials from federal agencies  responsible for administering TAP about any challenges they have faced  in implementing the revamped program. In addition, we interviewed  officials from organizations that support transitioning servicemembers and  veterans to obtain their perspectives on any challenges the agencies face  and what additional actions may be needed.
To increase our understanding and obtain servicemembers’ views of  TAP, we conducted three site visits during which we observed portions of  the revised TAP training sessions available at the time of our site visits.  We also conducted 4 discussion groups with 28 servicemembers from all  four services who had taken the training. We visited Joint Base Lewis- McChord, Washington (US Army/Air Force), Marine Corps Base  Quantico, and Naval Station Norfolk. We selected locations that had  implemented changes to TAP, and that provided for geographic diversity  and representation of all four services, but the information we collected  from these sites is not generalizable.
Because interagency partners were in the process of implementing  changes to TAP and the revamped program was not fully operational at  the time of our review, we were not able to determine whether the policies  and procedures were in place at all sites or if they were working as  intended. changes to TAP are yielding desired benefits or improvements in  outcomes.
For example, because DOD was in the process of, or planning to, monitor the program  content and delivery and collect data on attendance and outcomes, we did not test the  application of DOD’s policies and procedures related to these areas. In addition, we were  not able to review DOD’s plans to implement its military life cycle transition model because  it is in the early planning stage and is not due to be operational until October 2014. findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: Statutory and VEI Task Force Changes to TAP
paragraphs: Prior to the redesign initiated in 2011, TAP consisted of four core  components: (1) pre-separation counseling, (2) an employment  workshop, (3) an optional briefing on federal veteran benefits, and (4) the  Disabled Transition Assistance Program. Pre-separation counseling,  which includes VA benefits information, was required by law to be  provided prior to the VOW Act.
A number of revisions, new requirements, and components were added  to TAP by the VOW Act and the VEI Task Force. For example, the VOW  Act mandates that DOD require that transitioning servicemembers  participate in an employment workshop, with some exceptions. Included  among the VEI Task Force’s changes to the program are: an extended curriculum with segments on translating military skills to  civilian job requirements, financial planning, and individual counseling  and assessment with the goal of each servicemember developing an  Individual Transition Plan; an updated employment workshop and briefings on federal veteran  benefits divided into two sessions.embed information relevant for those who have or think they have a  service-connected disability rather than as part of a separate Disabled  Transition Assistance Program component;  The VA benefits I and II briefings a series of 2-day, career-specific tracks that focus on (1) pursuing  college education, (2) entering a technical skills training program, or  (3) starting a business. The track that a servicemember chooses  depends on his or her post-separation goals; a capstone event during which servicemembers are to demonstrate— and military service commanders verify—that they have met required  career-readiness standards. The standards pertain to employment,  education, and technical training, depending on the servicemember’s  post-separation goals; and a referral process—called a “warm handover”—to connect  servicemembers who do not meet the career readiness standards  with the appropriate partner agency (VA or DOL) to provide continued  support and services as veterans.
Additional details about the previous TAP compared to the revamped  program are shown in table 1.
Figure 1 shows each of the components in the order a servicemember  would currently participate in the redesigned program. The services  generally implement these core components at the installation level and  may include other components in addition to TAP.
Rather than TAP continuing to be an end-of-career event, DOD plans to  shift to a Military Life Cycle Transition Model after October 2014. This  model is intended to integrate transition preparation—counseling,  assessments, and access to resources to build skills or credentials— throughout the course of a servicemember’s military career.
section_title: Administration of the Revamped TAP
paragraphs: The VEI Task Force oversaw the design and development of the revised  TAP and was led by DOD and VA. Other agencies participating on the  VEI Task Force include DOL, the Department of Education, the Office of  Management and Budget, the Office of Personnel Management, and the  Small Business Administration (SBA). Members of the White House staff  and senior representatives from each service also participated. Each  agency is responsible for different activities. For example, DOD provides  guidance and monitors compliance with TAP provisions, and DOL  facilitates the employment workshop. Also, each service coordinates with  agencies on scheduling TAP workshops and briefings. The respective  roles and responsibilities are spelled out in a memorandum of  understanding (MOU), which the agencies signed in January 2014. A  new TAP governance structure, established in October 2013, steers  implementation of TAP and will modify the program, as needed, through  2016. The new governance structure is co-led by DOD in 2014 and co- chaired by VA and DOL.
section_title: Other Federal Employment Programs That Serve Transitioning Servicemembers and Veterans
paragraphs: With the draw down from the wars in Iraq and Afghanistan and as the  military makes ongoing and planned force structure reductions, many  servicemembers are projected to depart the military through 2017. TAP is  one of a number of federal programs to assist transitioning  servicemembers and veterans in developing job skills and securing  civilian employment. TAP serves as a gateway to additional information  and services that are available, either while servicemembers are on  active duty or after they have separated from the military. For example  the DOL employment workshop highlights many of the skills and  techniques helpful in obtaining employment. After completing the  workshop, servicemembers can benefit further by returning to the TAP  offices on installations, using services at local VA and DOL offices, or  using websites introduced to participants during TAP training.
Once servicemembers separate from the military, a number of other  federal programs offer assistance. These programs include five  employment and training programs overseen by DOL and VA that target  veterans.to veterans. For example, DOL provides grants to states to support state  workforce agency staff positions, Disabled Veterans’ Outreach Program  Specialists and Local Veterans Employment Representatives, who serve  veterans through the Jobs for Veterans State Grant Program. In addition,  VA provides employment services to certain veterans with disabilities  through the Vocational Rehabilitation and Employment Program. As of  2012, the program was offered in 56 regional offices and 169 satellite  offices. In addition, DOD helps National Guard and Reserve members  obtain civilian employment though its operation of other programs,  including the Yellow Ribbon Reintegration Program and Employer  Support of the Guard and Reserve. For example, the Yellow Ribbon  Reintegration Program serves National Guard and Reserve members and  their families by hosting events that provide information on employment  opportunities, health care, education/training opportunities, finances, and  legal benefits.
section_title: Agencies Report That Most Locations Have Implemented Key TAP Components, but Some Locations Are Experiencing Delays
paragraphs: As of December 2, 2013, DOD, DOL, and VA have implemented changes  to the program’s key components at most of the 206 military installations  that provide TAP. However, a few program components have not yet  been fully implemented by the agencies. For example, the agencies are  still using the previous version of the VA benefits briefing at a number of  locations and are offering the career technical training track at fewer than  half of the TAP locations. Although some agencies had planned to fully  implement the revamped TAP at all locations by October 1, 2013, they  missed their targeted time frame. According to the revised plan, agencies  now expect to implement virtually all components by the end of March  2014, with full implementation planned by June 2014. The planned start  dates and the status of agencies’ efforts to implement the key  components at the 206 TAP locations are summarized in figure 2.
Pre-separation counseling and DOD Core Curriculum: According to DOD  officials, eligible servicemembers were participating in pre-separation  counseling and the modules in the new DOD core curriculum by  November 21, 2012 at all TAP locations. As noted previously, part of the  DOD core curriculum includes a module on translating military skills to  civilian job requirements. Current military crosswalks map the majority of  military occupations to a single civilian occupation. Based on those  crosswalks and supplemented by analyses from Army and Navy  credentialing websites, tools like DOL’s My Next Move and My Next Move  for Veterans can suggest multiple occupations for career exploration. To  enhance the existing electronic tools used for the crosswalk, DOL  contracted with an organization to identify equivalencies between military  and civilian jobs, as required by the VOW Act. According to DOL officials,  the results of the military equivalencies study will enhance the military- civilian crosswalk by enabling a mapping of a single military occupation  into multiple civilian occupations based on an analysis of embedded skill  sets in addition to the similarity of tasks performed. This will be done for a  selected set of military occupations that represent 59 percent of current  active duty servicemembers. DOL officials said they plan to complete the  update of the electronic tools beginning in 2014.
DOL Employment Workshop: Effective November 21, 2012, DOD was  generally required to mandate participation in the program with some  exceptions. According to DOD and DOL officials, the agencies met this  participation requirement in the VOW Act by offering eligible  servicemembers, with some exceptions, the previous or revised version  of the DOL employment workshops at domestic and overseas locations  by November 21, 2012. Moreover, as required by the act, the DOL  employment workshops are being conducted by contractors at all TAP  locations.offered at all TAP locations, according to DOD and DOL officials.
As of the spring of 2013, the revised workshop was being  VA Benefits Briefing: All departing servicemembers are generally required  to be provided the VA benefits briefing. Officials from DOD and VA stated  that this participation requirement was met by offering servicemembers  the previous version of the VA benefits briefing while implementing a  phased rollout of the revised VA benefits I and II briefings. As of  December 2, 2013, the revised VA benefits I and II briefings were  unavailable at about 8 percent of TAP locations. VA officials said that the  revised VA benefits briefings are offered at all domestic locations.  However, the revised briefings are not currently offered in all overseas  locations, such as Air Force locations in Germany, South Korea, and  Japan. Although VA planned to implement the revised benefits briefings  by October 1, 2013, full implementation is now expected no later than  March 31, 2014, according to VA officials. DOD and VA officials said that  VA faced challenges, such as training enough personnel to facilitate the  revised briefings at these overseas locations during the extended furlough  and delays with negotiating agreements with foreign nations hosting U.S.  military forces.
Career-specific Tracks: Since late 2012 the agencies had been planning  to fully implement these tracks by October 2013. However, as of  December 2, 2013, the tracks were not fully implemented, particularly at  overseas locations:
Entrepreneurial Track: SBA offers the entrepreneurial track at least  quarterly at all domestic locations, but this only meets 12 percent of  the estimated total demand for the track, according to SBA officials.  Moreover, the track is not offered at the majority of overseas  locations. After reviewing a draft of this report, SBA officials stated  that the track can be extended to meet domestic and overseas  demand given additional funding recently provided in the fiscal year  2014 budget. Full implementation at all TAP locations is expected by  the end of fiscal year 2014, according to SBA officials.
Career Technical Training Track: VA is offering the career technical  training track at about 43 percent of the TAP locations. In particular,  many overseas locations lack this track. According to VA officials, the  delays were mainly due to efforts to incorporate substantial feedback  received from participants during the pilot phase. VA completely  revised the curriculum and piloted it in the summer of 2013. As a  result, VA delayed training the facilitators until the curriculum was  finalized and began its roll out in September 2013. According to the  services’ implementation plans, full implementation of this track is  expected by April 30, 2014.
Higher Education Track: DOD is offering the higher education track at  about 72 percent of TAP locations. According to DOD officials, they  plan to offer the track at all locations, but sequestration and other  resource constraints as well as delays in hiring and training facilitators  slowed the roll out of this track. Full implementation is expected by  April 9, 2014, according to the services’ implementation plans.
For servicemembers lacking access to the entrepreneurial track, content  from the track is available online. The online version, however, is a poor  substitute for the classroom-based track, according to SBA officials.  Unlike the technical training and higher education tracks, the  entrepreneurial track does not have associated readiness standards.  According to SBA officials, the readiness standards for the other tracks  were developed prior to the inclusion of this track in TAP. While they  considered creating associated standards, SBA officials decided that the  track’s main purpose is to help participating servicemembers determine  whether or not starting a business in general or their specific ideas for a  business is right for them, so standards were not needed.
If the higher education or career technical training tracks are not available  for servicemembers who wish to attend an institution of higher education  or seek technical training, other options are available to meet the  readiness standards associated with the tracks. For servicemembers in  remote locations or who are rapidly separating from the military, one  option is to access a virtual TAP curriculum, according to a draft DOD  policy under consideration. However, classroom instruction is the  preferred method. Another option for servicemembers located overseas is  to take the track at a domestic location when they return to the United  States, according to DOD officials. Moreover, according to DOD policy, a  servicemember could meet the standards outside of the track by  completing the required documents or activities associated with those  standards, such as by completing a comparison of academic institution  choices and a college or university application. While servicemembers  may meet the readiness standards without taking the tracks, they would  miss instruction, for example, on resources to cope with challenges  transitioning into college as nontraditional students (those who are older  or with family obligations).
Capstone Event: The services, working with DOD, DOL, and VA, are  finalizing implementation of a “capstone event”—a final check to verify  that servicemembers have met TAP requirements—and a referral  process, known as a “warm handover,” for those servicemembers who do  not meet the requirements. Although as of December 2013 the services  were holding capstone events at most locations, according to the  services’ implementation plans, the details for how the capstone event  and warm handover will work are still being finalized by DOD, DOL, VA,  and the services. Specifically, the agencies and services must clarify the  roles and responsibilities of servicemembers, TAP staff, commanders,  and the partner agencies as well as develop policies and guidance that  underpin this effort, according to a December 2013 DOD report about  implementation of the capstone event at TAP locations.
In addition to implementing the key components of TAP at military  installations, the services continue to update physical infrastructure at a  number of locations to provide an optimal experience for servicemembers  participating in TAP components. General standards for infrastructure are  set forth in DOD policy. Approximately 6 percent of locations lack  computer availability and according to officials from DOD, the Navy and  Marine Corps, other infrastructure is still being put in place at a number of  their domestic and overseas locations. According to officials from DOD,  they expect all of the TAP locations to meet infrastructure standards by  March 31, 2014.
Military Life Cycle Model: According to the agencies’ implementation plan,  the envisioned end state for the redesigned TAP involves integrating  transition preparation throughout the course of a servicemember’s career.  The agencies refer to this end state as the military life cycle transition  model. The services plan to fully implement the military life cycle by  October 2014. Under this new transition model, for example, the Army  intends for all new servicemembers to receive counseling and initiate an  individual development plan regarding their military career goals within 30  days of reporting to their first permanent duty station. Overall, DOD  intends such counseling and planning to continue throughout a  servicemember’s military career at various “touchpoints”, such as when   For example, servicemembers will be expected to  they are promoted.create plans for achieving their military and post-military goals for  employment, education, or starting their own business. Further,  servicemembers are to be made aware of the career readiness standards  they must meet long before they separate.
section_title: Agencies’ Efforts for TAP Adequately Address Three of Five Key Elements Associated with Effective Program Implementation and Evaluation
paragraphs: Agencies have efforts underway to address three of the five key elements  associated with effective program implementation and evaluation for TAP.  However, agencies’ efforts to address the remaining two elements are  mixed.
subsections: section_title: Agencies Are Adequately Addressing Three Elements
subsections: section_title: Element 1: Tracking Attendance
paragraphs: Effective training programs have systems to track data, and we found  that DOD and the services have systems to collect and report on  attendance rates and are taking steps to improve the reliability of these  data. DOD developed a DOD-wide system to track attendance for all TAP  components, which has been in use since October 2012. This system is  populated with attendance data from the services. The Army and Air  Force each input TAP attendance data into their own systems, which they  then transmit to the DOD system, while the Navy and the Marines input  TAP attendance data directly into the DOD system because they do not  have service-level systems that allow for tracking individual TAP  attendance.
DOD and the services are taking steps to improve the reliability of these  data. According to DOD officials, accuracy will improve now that the  capstone event is in place because servicemember attendance at each of  the three required TAP training components will be verified at this event.
According to DOD officials, the services are taking other steps to improve  the accuracy of TAP attendance data, as well. For example, the Navy  plans to replace the paper data collection systems that exist at some  installations with electronic attendance tracking, which may be less prone  to data-entry error.
DOD tracks attendance to measure progress toward its performance goal  of eligible servicemembers attending the mandatory components each  year, beginning in fiscal year 2013. In February 2014, DOD reduced the  fiscal year 2014 performance goal from 90 percent to 85 percent.  According to DOD officials, the expected attendance rate is less than 100  percent because some servicemembers do not complete all required TAP  training for various reasons. For example, some may separate quickly,  such as for medical or disciplinary reasons, and not have an opportunity  to fully take TAP.
For fiscal year 2013, DOD expects an attendance rate of about 75  percent, primarily because servicemembers that transitioned in that year  may have taken several TAP components in fiscal year 2012, prior to  when DOD and the services began tracking these rates. DOD anticipates  meeting their goal in fiscal year 2014.
section_title: Element 2: Ensuring Training Quality
paragraphs: Effective training programs incorporate feedback into training efforts,and each agency involved in TAP plans to use this type of information, as  well as monitor their respective components, to improve TAP. To  measure participants’ reactions to each TAP component they attend,  including the career-specific tracks, DOD launched a participant  assessment in April 2013. The assessment seeks feedback on the quality  of the instruction, content, and facilities.knowledge of the information presented in each component of the  training.
It also measures participants’  According to agency officials, the agencies plan to use the results from  the assessment to monitor the performance and outcomes for the  redesigned TAP, assess trends, determine areas of improvement, and  modify TAP components as appropriate. DOD is leading this effort. To  help facilitate this effort, DOD officials said that they plan to analyze  feedback at the DOD, service, and installation levels and share this  information with partner agencies and the four services in a quarterly  report. For example, the participant assessment collects demographic  information that will allow for a comparison of the responses of  servicemembers in different military branches, locations, and groups,  such as enlisted personnel and officers. DOD performed a similar  analysis in mid-2013 for responses to two questions in the participant  assessment. This analysis provided DOD with information that could be  used to understand how the program’s usefulness is perceived by  different populations. Moreover, according to DOD’s TAP evaluation plan,  DOD plans to convene teams with the partner agencies on an as-needed  basis to make recommendations to address challenges, concerns, and  areas for improvement. According to VA and DOL officials, they also plan  to analyze the feedback to improve their respective components.
The agencies also monitor their respective TAP components through site  visits or plan to do so. According to DOD’s monitoring plan, DOD visits  installations to monitor the TAP components it is responsible for as well  as overall TAP implementation. According to DOL officials, their staff  intends to perform annual monitoring visits to each domestic installation  where the employment workshop is provided, but not at overseas  installations. They said that at overseas locations they plan to rely on  feedback from participant assessments, the company hired to facilitate  the workshops, and the military staff at those DOD sites who will monitor  the facilitators’ instruction and whether they are engaging and  knowledgeable. VA officials said that they have not completed their  monitoring plan for the VA benefits I and II briefings and its career  technical training track. However, they expect to have a quality assurance  plan completed by March 2014. Also, in comments provided after  reviewing a draft of this report, VA officials stated that VA is participating  in the DOD site visits to installations to monitor the TAP components.  These are joint, agency-administered, on-site staff assistance visits led by  DOD. Also, VA currently has contract staff at each military installation to  monitor implementation of the VA benefits I and II briefings and the career  technical training track. that TAP participants have completed products, such as resumes and  job applications, which demonstrate that they are career ready.
Created a capstone event to verify that standards have been met; the  capstone event can be completed one-on-one, in large groups, or in  small groups. For example, the Marines model is one-on-one.
Assigned the task of verifying career readiness to commanders and  identified steps to ensure that commanders or their designees are  properly trained to assess an individual servicemember’s career  readiness. To document whether or not a servicemember is career  ready, DOD developed a new form called the Individual Transition  Plan (ITP) Checklist in which each one of the career readiness  standards is listed. We have reprinted this form in appendix II.
Developed procedures for providing a warm handover, or referral, for  servicemembers not meeting the career readiness standards.  Generally, the warm handover is to be a confirmed person-to-person  contact.
According to the December 2013 DOD capstone report, during the recent  pilot of the capstone event, several implementation challenges emerged.  For example, confusion exists among servicemembers, TAP staff,  commanders, and the agencies as to their roles and responsibilities in the  capstone event. According to the DOD capstone report, to address this  challenge, the agencies are taking actions, such as the services  implementing a training program for their TAP staff and planning efforts to  educate commanders and servicemembers on the requirements,  purpose, and importance of the capstone event.
As noted previously, according to DOD guidance, if a servicemember has  not met the career readiness standards at the time of the capstone event,  they are to be referred to the appropriate curriculum or services before  they transition. If they do not meet the standards before separating from  the service, the services plan to provide them with a “warm handover”.  The delivery of the warm handover may vary at domestic and overseas  locations because DOL and VA have limited capacity at overseas   According to the DOD capstone report, DOL staff will likely  locations.not be present at capstone events overseas and will conduct the warm  handover in other ways, and VA staff will be located only at larger  overseas installations. For example, servicemembers overseas will be  able to contact DOL’s call center within established hours, which, due to  time differences, may limit opportunities to make a person-to-person  contact.
section_title: Agencies’ Efforts to Address the Remaining Two Elements are Mixed
subsections: section_title: Element 4: Ensuring Participation and Completion
paragraphs: Effective training programs encourage participation and hold both  individuals and their leaders responsible. While DOD has taken steps that  are consistent with most attributes of effective training programs, two  services currently lack an oversight mechanism at the commander level  to help ensure participation. DOD’s efforts underway include: (1)  prioritizing training for servicemembers based on agreed-upon goals and  priorities, (2) encouraging servicemembers to buy in to training goals, and  (3) communicating the importance of training. For example, DOD and  the services communicate the importance of TAP training by providing  information on their websites, including how TAP aids in a successful  transition, and through other communications, such as brochures with  similar information. In addition, DOD has assigned unit commanders the  responsibility of ensuring that eligible servicemembers have full access to  and successfully complete required TAP components.
However, only the Army and the Air Force possess the capability to  gauge the rate at which servicemembers under an individual unit  commander participate in TAP. Specifically, the Army and Air Force  provide commanders and their leaders information on their unit’s  participation levels. In contrast, the Navy and the Marines do not have  such systems. According to Navy officials, they obtained funding and plan  to develop such a system by late June 2014 and plan to start using the  data for oversight no later than fiscal year 2015. Marine officials said that  because they have a long-standing culture of requiring servicemembers  to attend TAP training, such an oversight mechanism was not  necessary. In our 2002 and 2005 reviews of TAP, we found that  servicemembers sometimes faced difficulties being released from military  duties to attend TAP because of the priority accorded their military  mission or the lack of supervisory support for TAP.on the pilot of the capstone event, DOD reported that ensuring  servicemember participation in capstone events was a challenge for most  of the services. According to DOD, lack of servicemembers’ awareness of  this requirement and lack of commanders’ support may have hampered  participation in capstone events. Without routine information on  servicemembers’ participation by commander, it may be difficult to hold  accountable those directly responsible for ensuring participation.
section_title: Element 5: Measuring Performance and Evaluating Results
paragraphs: Outputs are the direct products and services delivered by a program, and outcomes are  the results of those products and services. See GAO, Performance Measurement and  Evaluation: Definitions and Relationships, GAO-11-646SP (Washington, D.C.: May 2011). basic end-of-course evaluations to higher level impact evaluations.agencies plan to evaluate TAP at lower levels. Their evaluation plan  includes (1) gauging participant reaction to all TAP components through  end-of-course evaluations and (2) determining whether servicemembers  met the career readiness standards prior to separation. Higher level  evaluations are also important to help gauge the effectiveness of TAP,  and the agencies have not demonstrated a strategic approach to planning  such evaluations. Since the program is administered by multiple  agencies, planning an evaluation is more challenging than if it were  administered by a single agency. In December 2011, the agencies stated,  in an internal report, that they would develop a methodology to assess the  effectiveness of TAP by engaging new veterans approximately six months  after they have separated from the military. Despite the modular nature of  TAP, this report did not specify whether this type of higher level  evaluation would assess all of TAP or certain components. DOL and VA  are considering higher level evaluations for the employment workshop  and VA benefits briefings. For example, DOL is considering different  methodologies it could use to conduct an impact evaluation of the  employment workshop. However, the agencies could not provide a  rationale for not using higher-level evaluations to assess either TAP  overall or some of the other TAP components and career-specific tracks.  Best practices call for the development of a written evaluation strategy,  which the agencies prepared for lower evaluation levels, but not for higher  levels. Without a written evaluation strategy that identifies the TAP  components to be evaluated, and includes the appropriate level and  timing of the evaluation and methods, the agencies may miss important  opportunities to obtain information needed to make fully informed  decisions on whether and how to modify TAP and may either over invest  or under invest in evaluations.
section_title: DOD Lacks a Process to Assess the TAP Experience of National Guard and Reserve Members
paragraphs: Based on GAO’s prior work and interviews with stakeholders, we  identified a key remaining challenge; namely, that DOD lacks a process to  assess the TAP experience of National Guard and Reserve members.  DOD’s policy generally requires all eligible servicemembers, including  members of the National Guard and Reserve, to be exposed to the entire  TAP experience even if their circumstances differ. Unlike active duty  servicemembers, many members of the National Guard and Reserve hold  civilian occupations, and federal law protects their employment rights  when they return from active duty. If they can document their civilian  employment or acceptance to an institution of higher education or  accredited technical training, members of the National Guard and  Reserve and other eligible servicemembers can be exempted from  attending the DOL employment workshop. Nevertheless, they are  generally required to complete pre-separation counseling, the VA benefits  I and II briefings and, under DOD policy, participate in the capstone  event. Further, the policy states that they must attend the DOD core  curriculum and applicable career specific tracks if they cannot meet the  career readiness standards associated with these two components.
However, according to many stakeholders that we talked with, including  officials from the services and organizations that support members of the  National Guard and Reserve and other servicemembers, DOD has not  resolved long-standing concerns that eligible members of the National  Guard and Reserve attend TAP offerings in locations and on a timetable  that diminish their experience. Specifically, given that eligible National Guard and Reserve members generally demobilize at locations where  they neither work nor live, they may be distracted by their desire to return  home, which can affect how much the training benefits them. Separating  servicemembers need to be aware of the benefits and services and know  how to access them to take advantage of them, such as the Post-9/11 GI  Bill. In a January 2013 report, the Defense Business Board Task Group  recommended that eligible National Guard and Reserve members be  afforded the opportunity to demobilize and transition in their home unit’s  geographical area. According to stakeholders, eligible National Guard  and Reserve members also typically have less time to complete the  program because they often demobilize more quickly than active duty  servicemembers separate. For example, according to DOD’s 2011  Handbook, the Reserve demobilization timeline makes scheduling a pre- separation counseling appointment not later than 90 days prior to leaving  active duty impractical for Reserve members.
According to officials from DOD and the services, they took steps to help  meet the needs of National Guard and Reserve members eligible for  TAP. For example, the agencies tailored the content of TAP components  to better suit the needs of National Guard and Reserve members after   However, only one  analyzing the results of feedback from pilots of TAP.of the steps taken directly addresses the concerns of stakeholders related  to the location and timing of TAP. Specifically, eligible members of the  Army National Guard are allowed to participate in the DOL employment  workshop and the capstone event in their home unit’s geographical area,  according to DOD and Army officials. These eligible Army National Guard  members will remain on active duty while participating in these two  components.
Nonetheless, DOD may not be well positioned to determine whether its  actions successfully address the long-standing challenges in designing  transition services for the National Guard and Reserve. This is because  while DOD collects participant feedback through the online assessment,  this tool does not ask whether the timing and location met the needs of  servicemembers, including the National Guard and Reserve. Moreover,  DOD’s planned survey to active duty servicemembers contains questions  about the timing of TAP, but DOD has not drafted similar questions for its  survey to the National Guard and Reserve. According to OMB, being  able to track and measure specific program data can help agencies  diagnose problems, identify drivers of future performance, evaluate risk,  support collaboration, and inform follow-up actions. Without a  systematic process targeted to identify any remaining long-standing  concerns, DOD will not be able to make fully informed decisions about the  extent to which eligible members of the National Guard and Reserve reap  the full benefits of TAP.
section_title: Conclusions
paragraphs: Helping servicemembers successfully transition to civilian life after their  service ends is an important government goal. The agencies undertook  an ambitious effort to redesign the 1990s-era program to provide  servicemembers with training and skills to successfully transition to  civilian life. The increase in the number of servicemembers attending the  revised TAP components, along with the significant coordination effort  among multiple agencies and military services, pose significant  implementation challenges. Efforts to implement the redesigned program  continue to evolve. The importance of serving the ongoing and projected  wave of servicemembers departing the military increases the urgency to  fully implement TAP components as well as to finalize related policies and  procedures. To the extent that the program remains behind schedule in  implementing TAP, some transitioning servicemembers may be denied  the full benefit of the revamped program. Because TAP is not fully  implemented, the full impact of the revamped TAP, particularly the warm  handover—a key effort to serve those deemed at greater risk of not being  career ready—remains to be seen.
The revamped TAP exhibits elements important for the effective  implementation and evaluation of the program. Yet, we found room for  improvement in several key areas. In particular, while federal law  generally requires DOD to mandate participation in the employment  workshop and DOD policy requires that all eligible transitioning  servicemembers participate in TAP overall, the Navy and the Marines  lack the ability to provide unit commanders and their leaders with  information on participation levels of servicemembers under their  command. Without the capability to gauge and report participation in TAP  components by unit commander—those directly responsible for ensuring  servicemember participation—leaders may find that holding them  accountable is difficult.
While the administration cites the revamped TAP as a key strategy of its  crosscutting goal to improve the career readiness of veterans, it may not  be positioned to determine the extent to which the program prepared  veterans to pursue their post-separation goals. Without a written  evaluation strategy that identifies the TAP components to be evaluated  and includes the appropriate level and timing of the evaluation methods,  the agencies may miss important opportunities to obtain information  needed to make fully informed decisions on whether and how to modify  TAP and may either over invest or under invest in evaluations.
The circumstances of National Guard and Reserve members differ from  those of other active duty servicemembers. Many stakeholders have  raised concerns that these circumstances diminish eligible National  Guard and Reserve members’ TAP experience. Absent efforts to  systematically collect information about eligible National Guard and  Reserve members’ experiences under the revamped program, DOD may  not be well positioned to determine whether there are problems with how  TAP is provided to these groups.
section_title: Recommendations for Executive Action
paragraphs: Based on our review, we recommend that the Secretary of Defense take  the following actions:  1.  To better ensure servicemember participation in and completion of  TAP, direct the Under Secretary for Personnel and Readiness to  require that all services provide unit commanders and their leaders  information on TAP participation levels of servicemembers under their  command, similar to that provided by the Army and Air Force. Such  information could be used to help hold leaders accountable for  ensuring TAP participation and completion.  2.  To provide information on the extent to which the revamped TAP is  effective, direct the Under Secretary for Personnel and Readiness to  work with the partner agencies to develop a written strategy for  determining which components and tracks to evaluate and the most  appropriate evaluation methods. This strategy should include a plan to  use the results of evaluations to modify or redesign the program, as  appropriate.  3.  To ensure that decisions about the participation of eligible members of  the National Guard and Reserves in TAP are fully informed, direct the  Under Secretary for Personnel and Readiness to systematically  collect information on any challenges facing demobilizing members of  the National Guard and Reserves regarding the logistics of the timing  and location to attend TAP. For example, agencies might add  questions to their online assessment tool specific to eligible members  of the National Guard and Reserves who participate in TAP.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to DOD, VA, DOL, and SBA for  comment. In its comments, DOD agreed with one recommendation and  disagreed with the other two recommendations. VA generally agreed with  our findings. Written comments from DOD and VA are reproduced in  appendices III and IV. DOL and SBA did not provide comments, but all  four agencies provided technical comments, which we incorporated into  the final report as appropriate.
DOD agreed with our recommendation to work with partner agencies to  develop a written strategy for determining which components and tracks  to evaluate and the most appropriate evaluation methods. DOD stated  that it will continue to support interagency collaboration, which, as of  January 31, 2014, has been formalized in a memorandum of  understanding among the agencies administering TAP.
DOD disagreed with our recommendation to require that all of the  services provide unit commanders and their leaders information on TAP  participation levels of servicemembers under their command. DOD stated  that we justified the need for a mechanism to ensure a servicemember’s  completion of TAP by commander based on concerns that without such a  mechanism, commanders will not support the release of servicemembers  to attend TAP. DOD also said that these concerns are based on  observations that preceded full implementation of the capstone event. In  October 2013, DOD launched the capstone as a mandatory process by  which commanders verify TAP participation and positively affirm that  servicemembers have or have not met career readiness standards. In  addition, DOD said that capstone event implementation was  accompanied by a communications campaign to inform both commanders  and the services’ TAP providers of this key requirement. Finally, DOD  said that capstone event completion is monitored departmentwide to  ensure compliance.
We agree with DOD that departmentwide monitoring of capstone event  completion is an important element in helping ensure compliance.  However, such monitoring does not provide routine information to  commanders and their leaders and not all TAP locations will be monitored  routinely. As noted in our report, based on the pilot of the capstone event  last fall, DOD reported that ensuring servicemember participation in  capstone events was a challenge for most of the services (with the  exception of the Air Force), adding that this challenge is possibly due in  part to lack of commanders’ support. Commander support for the program  has been a long-standing challenge for the program. Therefore, we  continue to believe that our recommendation is needed because it would  establish an accountability mechanism for TAP that mirrors the level at  which responsibility has been assigned. DOD has assigned unit  commanders the responsibility of ensuring that eligible servicemembers  have full access to and successfully complete required TAP components.  Providing information to unit commanders and their leaders on TAP  participation levels of servicemembers under their command—similar to  that provided by the Army and Air Force—could thus promote  accountability and oversight. Servicemember participation in TAP is  generally required by law and DOD policy, and also relates to a Cross- Agency Priority Goal, which reinforces the need for an appropriate  accountability mechanism.
DOD also stated that the Navy is funding IT system upgrades to provide  the ability to analyze and report program compliance at the command  level. DOD added that the Marine Corps has mandated participation since  the program's inception, and Marine Corps commanders leverage the  capabilities of the personnel system to identify eligible Marines and  schedule their TAP attendance. DOD stated that despite the limitations of  a data tracking system that underreports compliance figures, the Marine  Corps had the highest compliance rate of all services, as of December  2013. DOD said it recommends that department-wide compliance data be  allowed to normalize to show true compliance percentages before the  services are required to fund and implement expensive systems for the  sole purpose of providing an additional mechanism for commander  accountability. However, we continue to believe that our recommendation  is needed. Three of the services either have or are working to develop the  kind of accountability mechanism that we are recommending, but the  Marine Corps does not plan to develop such a system. While DOD  reports that the Marine Corps has the highest TAP compliance rate of all  services, DOD did not provide any data on these rates. Finally, in  response to DOD’s recommendation to wait until compliance data  normalize, we believe that this would not be appropriate. We do not see  any advantage to delaying the implementation of appropriate  accountability mechanisms to provide assurance that the ongoing and  projected wave of servicemembers departing the military receive the  expected level of services from TAP.
DOD disagreed with our recommendation to systematically collect  information on challenges facing demobilizing members of the National  Guard and Reserve regarding the timing and location to attend TAP.  Specifically, DOD stated that it has long understood that the National  Guard and Reserves operate under schedules and logistical constraints  that differ from those of active duty servicemembers. DOD stated that  several processes (which we note in our report) are already in place to  identity and rectify any misalignments between TAP services and the  needs of eligible National Guard and Reserve members, including regular  coordinating councils that include representation from the National Guard  and Reserve and TAP managers, as well as a participant assessment  that provides ample opportunity for eligible National Guard and Reserve  members to voice concerns. In addition, DOD highlighted the  implementation of the military life cycle transition model in which DOD  plans to provide the services, including to the National Guard and  Reserve, with the latitude to optimize placement of certain elements of  TAP throughout a servicemember's military service. According to DOD,  the military life cycle transition model may help address some of the  challenges related to the timing and location of program delivery. As we  note in the report, full implementation of the military life cycle transition  model is planned by October 2014.
Nonetheless, at this time DOD is not fully positioned to know whether or  not the revamped program addresses the long-standing challenges facing  eligible members of the National Guard and Reserve in taking TAP. As  we describe in the report, all eligible servicemembers, including the  National Guard and Reserve members, have an opportunity to provide  feedback about the instruction, content, and facilities for TAP. However,  the participant assessment does not ask questions specific to the  National Guard and Reserve experience even though they face different  circumstances than active duty servicemembers. Given these differences,  we continue to believe that unless DOD systematically collects  information on any challenges facing eligible members of the National  Guard and Reserve, DOD is at risk of not fully knowing whether the  revamped TAP addresses long-standing challenges. In addition, the  move to a military life cycle transition model could enable active duty  servicemembers and members of the National Guard and Reserve to  benefit from transition-related assistance throughout their military service.  Recognizing that the military life cycle transition model is not implemented  and specific policies and plans are not completed, many unknowns  remain about how it will work in general and how National Guard and  Reserve members will fare specifically. During our review, we asked DOD  how this program would work in practice, including for National Guard and  Reserve members, but DOD did not provide us details.
We are sending copies of this report to appropriate congressional  committees; the Secretary of Defense; the Secretary of Veterans Affairs,  the Secretary of Labor; the Acting Administrator of the Small Business  Administration; and other interested parties. In addition, this report will be  available at no charge on GAO’s website at http://www.gao.gov.
If you or your staff members have any questions regarding this report,  please contact me at (202) 512-7215 or sherrilla@gao.gov. Contact  points for our Offices of Congressional Relations and Public Affairs may  be found on the last page of this report. GAO staff who made major  contributions to this report are listed in appendix V.
section_title: Appendix I: Technical Appendix
paragraphs: This appendix summarizes our work to assess efforts to implement and  evaluate the revamped TAP.
subsections: section_title: Elements and Relevant Attributes and the Extent to Which They Have Been Addressed
paragraphs: For each of the five elements that we identified as important for the  effective implementation and evaluation of TAP, we identified relevant  attributes that we used to determine the extent to which the elements  were addressed in the revamped TAP (see table 2).
To determine the extent to which agencies have addressed each element  and attribute, we made our assessments in two steps. First, an analyst  reviewed all of the evidence. Based on that review, the analyst  documented evidence that conformed to the elements and attributes and  made and documented a judgment about the status of these efforts. Then  a second analyst separately reviewed the assessment and documented  their agreement or disagreement. The two analysts discussed any  differences in their assessments and made changes based on their verbal  resolution of those differences.
We assessed the status of the agencies’ efforts to address each relevant  attribute as “completed”, “underway”, or “not completed”. Based on this  analysis, we then determined the status of efforts to address each  respective attribute overall. For example, for the status of efforts to  address an attribute overall to be considered “completed”, efforts to  address each relevant attribute had to be completed. For the status of  efforts to be considered “underway”, it had to be documented that efforts  to address each relevant attribute were progressing toward being  finalized, or were a mix of underway efforts and finalized efforts. For the  status of efforts to be considered “not completed”, efforts to address at  least one relevant attribute were not determined to be finalized or  underway.
We considered the agency to be “adequately addressing” the element if  the overall status of the efforts to address the relevant attributes was  determined to be completed or underway. However, if some attributes  related to an element had efforts completed or underway to address  them, but one or more other attributes did not have efforts completed or  underway to address it (them), then we consider the overall efforts to  address the element to be “mixed”.
We used the following general decision rules for characterizing the status  of efforts to address each attribute as “completed”, “underway”, and “not  completed”:  “Completed” means interagency partners (or relevant agency) have a  system, policy, or procedure in place to address a relevant attribute. “Underway” means interagency partners (or relevant agency) have an  authoritative internal or public document that demonstrate progress  toward implementing actions to address the attribute (e.g., agencies  have draft policies, operating procedures, or guidance, or they have  interim systems in place and/or are developing systems). “Not completed” means that interagency partners (or relevant agency)  have shown little to no action toward implementing actions to address  the attribute, meaning they lack an authoritative internal or public  document (e.g., no draft policies, operating procedures, or guidance  or interim systems).
As we conducted this analysis, we kept in mind how far along agencies  (mainly DOD and the military services) are in implementing each  component of TAP (informed by objective 1). Because agencies were in  the process of implementing the revamped TAP and the program was not  fully operational at the time of our review, we were not able to determine  whether the policies and procedures are in place at all sites or if they are  working as intended. Similarly, we were not able to comment on whether  the changes to TAP are yielding desired benefits or improvements in  outcomes.
section_title: Appendix II: DOD Form 2958, Individual Transition Plan (ITP) Checklist
section_title: Appendix III: Comments from the Department of Defense
section_title: Appendix IV: Comments from the Department of Veterans Affairs
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
paragraphs: Andrew Sherrill, (202) 512-7215 or sherrilla@gao.gov.
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, individuals making key  contributions to this report were Patrick Dibattista, Assistant Director;  Julianne Hartman Cutts; and James Whitcomb. In addition, key support  was provided by James Bennett, Rachael Chamberlin, David Chrisinger,  Brett Fallavollita, Alex Galuten, Kathy Leslie, David Moser, and Michael  Silver.