Transitioning Veterans: Improved Oversight Needed to Enhance Implementation of Transition Assistance Program
Released: 2014-03-05T00:00:00

Letter

Background

Statutory and VEI Task Force Changes to TAP

Prior to the redesign initiated in 2011, TAP consisted of four core components: (1) pre-separation counseling, (2) an employment workshop, (3) an optional briefing on federal veteran benefits, and (4) the Disabled Transition Assistance Program. Pre-separation counseling, which includes VA benefits information, was required by law to be provided prior to the VOW Act.

A number of revisions, new requirements, and components were added to TAP by the VOW Act and the VEI Task Force. For example, the VOW Act mandates that DOD require that transitioning servicemembers participate in an employment workshop, with some exceptions. Included among the VEI Task Force’s changes to the program are: an extended curriculum with segments on translating military skills to civilian job requirements, financial planning, and individual counseling and assessment with the goal of each servicemember developing an Individual Transition Plan; an updated employment workshop and briefings on federal veteran benefits divided into two sessions.embed information relevant for those who have or think they have a service-connected disability rather than as part of a separate Disabled Transition Assistance Program component; The VA benefits I and II briefings a series of 2-day, career-specific tracks that focus on (1) pursuing college education, (2) entering a technical skills training program, or (3) starting a business. The track that a servicemember chooses depends on his or her post-separation goals; a capstone event during which servicemembers are to demonstrate— and military service commanders verify—that they have met required career-readiness standards. The standards pertain to employment, education, and technical training, depending on the servicemember’s post-separation goals; and a referral process—called a “warm handover”—to connect servicemembers who do not meet the career readiness standards with the appropriate partner agency (VA or DOL) to provide continued support and services as veterans.

Additional details about the previous TAP compared to the revamped program are shown in table 1.

Figure 1 shows each of the components in the order a servicemember would currently participate in the redesigned program. The services generally implement these core components at the installation level and may include other components in addition to TAP.

Rather than TAP continuing to be an end-of-career event, DOD plans to shift to a Military Life Cycle Transition Model after October 2014. This model is intended to integrate transition preparation—counseling, assessments, and access to resources to build skills or credentials— throughout the course of a servicemember’s military career.

Administration of the Revamped TAP

The VEI Task Force oversaw the design and development of the revised TAP and was led by DOD and VA. Other agencies participating on the VEI Task Force include DOL, the Department of Education, the Office of Management and Budget, the Office of Personnel Management, and the Small Business Administration (SBA). Members of the White House staff and senior representatives from each service also participated. Each agency is responsible for different activities. For example, DOD provides guidance and monitors compliance with TAP provisions, and DOL facilitates the employment workshop. Also, each service coordinates with agencies on scheduling TAP workshops and briefings. The respective roles and responsibilities are spelled out in a memorandum of understanding (MOU), which the agencies signed in January 2014. A new TAP governance structure, established in October 2013, steers implementation of TAP and will modify the program, as needed, through 2016. The new governance structure is co-led by DOD in 2014 and co- chaired by VA and DOL.

Other Federal Employment Programs That Serve Transitioning Servicemembers and Veterans

With the draw down from the wars in Iraq and Afghanistan and as the military makes ongoing and planned force structure reductions, many servicemembers are projected to depart the military through 2017. TAP is one of a number of federal programs to assist transitioning servicemembers and veterans in developing job skills and securing civilian employment. TAP serves as a gateway to additional information and services that are available, either while servicemembers are on active duty or after they have separated from the military. For example the DOL employment workshop highlights many of the skills and techniques helpful in obtaining employment. After completing the workshop, servicemembers can benefit further by returning to the TAP offices on installations, using services at local VA and DOL offices, or using websites introduced to participants during TAP training.

Once servicemembers separate from the military, a number of other federal programs offer assistance. These programs include five employment and training programs overseen by DOL and VA that target veterans.to veterans. For example, DOL provides grants to states to support state workforce agency staff positions, Disabled Veterans’ Outreach Program Specialists and Local Veterans Employment Representatives, who serve veterans through the Jobs for Veterans State Grant Program. In addition, VA provides employment services to certain veterans with disabilities through the Vocational Rehabilitation and Employment Program. As of 2012, the program was offered in 56 regional offices and 169 satellite offices. In addition, DOD helps National Guard and Reserve members obtain civilian employment though its operation of other programs, including the Yellow Ribbon Reintegration Program and Employer Support of the Guard and Reserve. For example, the Yellow Ribbon Reintegration Program serves National Guard and Reserve members and their families by hosting events that provide information on employment opportunities, health care, education/training opportunities, finances, and legal benefits.

Agencies Report That Most Locations Have Implemented Key TAP Components, but Some Locations Are Experiencing Delays

As of December 2, 2013, DOD, DOL, and VA have implemented changes to the program’s key components at most of the 206 military installations that provide TAP. However, a few program components have not yet been fully implemented by the agencies. For example, the agencies are still using the previous version of the VA benefits briefing at a number of locations and are offering the career technical training track at fewer than half of the TAP locations. Although some agencies had planned to fully implement the revamped TAP at all locations by October 1, 2013, they missed their targeted time frame. According to the revised plan, agencies now expect to implement virtually all components by the end of March 2014, with full implementation planned by June 2014. The planned start dates and the status of agencies’ efforts to implement the key components at the 206 TAP locations are summarized in figure 2.

Pre-separation counseling and DOD Core Curriculum: According to DOD officials, eligible servicemembers were participating in pre-separation counseling and the modules in the new DOD core curriculum by November 21, 2012 at all TAP locations. As noted previously, part of the DOD core curriculum includes a module on translating military skills to civilian job requirements. Current military crosswalks map the majority of military occupations to a single civilian occupation. Based on those crosswalks and supplemented by analyses from Army and Navy credentialing websites, tools like DOL’s My Next Move and My Next Move for Veterans can suggest multiple occupations for career exploration. To enhance the existing electronic tools used for the crosswalk, DOL contracted with an organization to identify equivalencies between military and civilian jobs, as required by the VOW Act. According to DOL officials, the results of the military equivalencies study will enhance the military- civilian crosswalk by enabling a mapping of a single military occupation into multiple civilian occupations based on an analysis of embedded skill sets in addition to the similarity of tasks performed. This will be done for a selected set of military occupations that represent 59 percent of current active duty servicemembers. DOL officials said they plan to complete the update of the electronic tools beginning in 2014.

DOL Employment Workshop: Effective November 21, 2012, DOD was generally required to mandate participation in the program with some exceptions. According to DOD and DOL officials, the agencies met this participation requirement in the VOW Act by offering eligible servicemembers, with some exceptions, the previous or revised version of the DOL employment workshops at domestic and overseas locations by November 21, 2012. Moreover, as required by the act, the DOL employment workshops are being conducted by contractors at all TAP locations.offered at all TAP locations, according to DOD and DOL officials.

As of the spring of 2013, the revised workshop was being VA Benefits Briefing: All departing servicemembers are generally required to be provided the VA benefits briefing. Officials from DOD and VA stated that this participation requirement was met by offering servicemembers the previous version of the VA benefits briefing while implementing a phased rollout of the revised VA benefits I and II briefings. As of December 2, 2013, the revised VA benefits I and II briefings were unavailable at about 8 percent of TAP locations. VA officials said that the revised VA benefits briefings are offered at all domestic locations. However, the revised briefings are not currently offered in all overseas locations, such as Air Force locations in Germany, South Korea, and Japan. Although VA planned to implement the revised benefits briefings by October 1, 2013, full implementation is now expected no later than March 31, 2014, according to VA officials. DOD and VA officials said that VA faced challenges, such as training enough personnel to facilitate the revised briefings at these overseas locations during the extended furlough and delays with negotiating agreements with foreign nations hosting U.S. military forces.

Career-specific Tracks: Since late 2012 the agencies had been planning to fully implement these tracks by October 2013. However, as of December 2, 2013, the tracks were not fully implemented, particularly at overseas locations:

Entrepreneurial Track: SBA offers the entrepreneurial track at least quarterly at all domestic locations, but this only meets 12 percent of the estimated total demand for the track, according to SBA officials. Moreover, the track is not offered at the majority of overseas locations. After reviewing a draft of this report, SBA officials stated that the track can be extended to meet domestic and overseas demand given additional funding recently provided in the fiscal year 2014 budget. Full implementation at all TAP locations is expected by the end of fiscal year 2014, according to SBA officials.

Career Technical Training Track: VA is offering the career technical training track at about 43 percent of the TAP locations. In particular, many overseas locations lack this track. According to VA officials, the delays were mainly due to efforts to incorporate substantial feedback received from participants during the pilot phase. VA completely revised the curriculum and piloted it in the summer of 2013. As a result, VA delayed training the facilitators until the curriculum was finalized and began its roll out in September 2013. According to the services’ implementation plans, full implementation of this track is expected by April 30, 2014.

Higher Education Track: DOD is offering the higher education track at about 72 percent of TAP locations. According to DOD officials, they plan to offer the track at all locations, but sequestration and other resource constraints as well as delays in hiring and training facilitators slowed the roll out of this track. Full implementation is expected by April 9, 2014, according to the services’ implementation plans.

For servicemembers lacking access to the entrepreneurial track, content from the track is available online. The online version, however, is a poor substitute for the classroom-based track, according to SBA officials. Unlike the technical training and higher education tracks, the entrepreneurial track does not have associated readiness standards. According to SBA officials, the readiness standards for the other tracks were developed prior to the inclusion of this track in TAP. While they considered creating associated standards, SBA officials decided that the track’s main purpose is to help participating servicemembers determine whether or not starting a business in general or their specific ideas for a business is right for them, so standards were not needed.

If the higher education or career technical training tracks are not available for servicemembers who wish to attend an institution of higher education or seek technical training, other options are available to meet the readiness standards associated with the tracks. For servicemembers in remote locations or who are rapidly separating from the military, one option is to access a virtual TAP curriculum, according to a draft DOD policy under consideration. However, classroom instruction is the preferred method. Another option for servicemembers located overseas is to take the track at a domestic location when they return to the United States, according to DOD officials. Moreover, according to DOD policy, a servicemember could meet the standards outside of the track by completing the required documents or activities associated with those standards, such as by completing a comparison of academic institution choices and a college or university application. While servicemembers may meet the readiness standards without taking the tracks, they would miss instruction, for example, on resources to cope with challenges transitioning into college as nontraditional students (those who are older or with family obligations).

Capstone Event: The services, working with DOD, DOL, and VA, are finalizing implementation of a “capstone event”—a final check to verify that servicemembers have met TAP requirements—and a referral process, known as a “warm handover,” for those servicemembers who do not meet the requirements. Although as of December 2013 the services were holding capstone events at most locations, according to the services’ implementation plans, the details for how the capstone event and warm handover will work are still being finalized by DOD, DOL, VA, and the services. Specifically, the agencies and services must clarify the roles and responsibilities of servicemembers, TAP staff, commanders, and the partner agencies as well as develop policies and guidance that underpin this effort, according to a December 2013 DOD report about implementation of the capstone event at TAP locations.

In addition to implementing the key components of TAP at military installations, the services continue to update physical infrastructure at a number of locations to provide an optimal experience for servicemembers participating in TAP components. General standards for infrastructure are set forth in DOD policy. Approximately 6 percent of locations lack computer availability and according to officials from DOD, the Navy and Marine Corps, other infrastructure is still being put in place at a number of their domestic and overseas locations. According to officials from DOD, they expect all of the TAP locations to meet infrastructure standards by March 31, 2014.

Military Life Cycle Model: According to the agencies’ implementation plan, the envisioned end state for the redesigned TAP involves integrating transition preparation throughout the course of a servicemember’s career. The agencies refer to this end state as the military life cycle transition model. The services plan to fully implement the military life cycle by October 2014. Under this new transition model, for example, the Army intends for all new servicemembers to receive counseling and initiate an individual development plan regarding their military career goals within 30 days of reporting to their first permanent duty station. Overall, DOD intends such counseling and planning to continue throughout a servicemember’s military career at various “touchpoints”, such as when For example, servicemembers will be expected to they are promoted.create plans for achieving their military and post-military goals for employment, education, or starting their own business. Further, servicemembers are to be made aware of the career readiness standards they must meet long before they separate.

Agencies’ Efforts for TAP Adequately Address Three of Five Key Elements Associated with Effective Program Implementation and Evaluation

Agencies have efforts underway to address three of the five key elements associated with effective program implementation and evaluation for TAP. However, agencies’ efforts to address the remaining two elements are mixed.

Agencies Are Adequately Addressing Three Elements

Element 1: Tracking Attendance

Effective training programs have systems to track data, and we found that DOD and the services have systems to collect and report on attendance rates and are taking steps to improve the reliability of these data. DOD developed a DOD-wide system to track attendance for all TAP components, which has been in use since October 2012. This system is populated with attendance data from the services. The Army and Air Force each input TAP attendance data into their own systems, which they then transmit to the DOD system, while the Navy and the Marines input TAP attendance data directly into the DOD system because they do not have service-level systems that allow for tracking individual TAP attendance.

DOD and the services are taking steps to improve the reliability of these data. According to DOD officials, accuracy will improve now that the capstone event is in place because servicemember attendance at each of the three required TAP training components will be verified at this event.

According to DOD officials, the services are taking other steps to improve the accuracy of TAP attendance data, as well. For example, the Navy plans to replace the paper data collection systems that exist at some installations with electronic attendance tracking, which may be less prone to data-entry error.

DOD tracks attendance to measure progress toward its performance goal of eligible servicemembers attending the mandatory components each year, beginning in fiscal year 2013. In February 2014, DOD reduced the fiscal year 2014 performance goal from 90 percent to 85 percent. According to DOD officials, the expected attendance rate is less than 100 percent because some servicemembers do not complete all required TAP training for various reasons. For example, some may separate quickly, such as for medical or disciplinary reasons, and not have an opportunity to fully take TAP.

For fiscal year 2013, DOD expects an attendance rate of about 75 percent, primarily because servicemembers that transitioned in that year may have taken several TAP components in fiscal year 2012, prior to when DOD and the services began tracking these rates. DOD anticipates meeting their goal in fiscal year 2014.

Element 2: Ensuring Training Quality

Effective training programs incorporate feedback into training efforts,and each agency involved in TAP plans to use this type of information, as well as monitor their respective components, to improve TAP. To measure participants’ reactions to each TAP component they attend, including the career-specific tracks, DOD launched a participant assessment in April 2013. The assessment seeks feedback on the quality of the instruction, content, and facilities.knowledge of the information presented in each component of the training.

It also measures participants’ According to agency officials, the agencies plan to use the results from the assessment to monitor the performance and outcomes for the redesigned TAP, assess trends, determine areas of improvement, and modify TAP components as appropriate. DOD is leading this effort. To help facilitate this effort, DOD officials said that they plan to analyze feedback at the DOD, service, and installation levels and share this information with partner agencies and the four services in a quarterly report. For example, the participant assessment collects demographic information that will allow for a comparison of the responses of servicemembers in different military branches, locations, and groups, such as enlisted personnel and officers. DOD performed a similar analysis in mid-2013 for responses to two questions in the participant assessment. This analysis provided DOD with information that could be used to understand how the program’s usefulness is perceived by different populations. Moreover, according to DOD’s TAP evaluation plan, DOD plans to convene teams with the partner agencies on an as-needed basis to make recommendations to address challenges, concerns, and areas for improvement. According to VA and DOL officials, they also plan to analyze the feedback to improve their respective components.

The agencies also monitor their respective TAP components through site visits or plan to do so. According to DOD’s monitoring plan, DOD visits installations to monitor the TAP components it is responsible for as well as overall TAP implementation. According to DOL officials, their staff intends to perform annual monitoring visits to each domestic installation where the employment workshop is provided, but not at overseas installations. They said that at overseas locations they plan to rely on feedback from participant assessments, the company hired to facilitate the workshops, and the military staff at those DOD sites who will monitor the facilitators’ instruction and whether they are engaging and knowledgeable. VA officials said that they have not completed their monitoring plan for the VA benefits I and II briefings and its career technical training track. However, they expect to have a quality assurance plan completed by March 2014. Also, in comments provided after reviewing a draft of this report, VA officials stated that VA is participating in the DOD site visits to installations to monitor the TAP components. These are joint, agency-administered, on-site staff assistance visits led by DOD. Also, VA currently has contract staff at each military installation to monitor implementation of the VA benefits I and II briefings and the career technical training track. that TAP participants have completed products, such as resumes and job applications, which demonstrate that they are career ready.

Created a capstone event to verify that standards have been met; the capstone event can be completed one-on-one, in large groups, or in small groups. For example, the Marines model is one-on-one.

Assigned the task of verifying career readiness to commanders and identified steps to ensure that commanders or their designees are properly trained to assess an individual servicemember’s career readiness. To document whether or not a servicemember is career ready, DOD developed a new form called the Individual Transition Plan (ITP) Checklist in which each one of the career readiness standards is listed. We have reprinted this form in appendix II.

Developed procedures for providing a warm handover, or referral, for servicemembers not meeting the career readiness standards. Generally, the warm handover is to be a confirmed person-to-person contact.

According to the December 2013 DOD capstone report, during the recent pilot of the capstone event, several implementation challenges emerged. For example, confusion exists among servicemembers, TAP staff, commanders, and the agencies as to their roles and responsibilities in the capstone event. According to the DOD capstone report, to address this challenge, the agencies are taking actions, such as the services implementing a training program for their TAP staff and planning efforts to educate commanders and servicemembers on the requirements, purpose, and importance of the capstone event.

As noted previously, according to DOD guidance, if a servicemember has not met the career readiness standards at the time of the capstone event, they are to be referred to the appropriate curriculum or services before they transition. If they do not meet the standards before separating from the service, the services plan to provide them with a “warm handover”. The delivery of the warm handover may vary at domestic and overseas locations because DOL and VA have limited capacity at overseas According to the DOD capstone report, DOL staff will likely locations.not be present at capstone events overseas and will conduct the warm handover in other ways, and VA staff will be located only at larger overseas installations. For example, servicemembers overseas will be able to contact DOL’s call center within established hours, which, due to time differences, may limit opportunities to make a person-to-person contact.

Agencies’ Efforts to Address the Remaining Two Elements are Mixed

Element 4: Ensuring Participation and Completion

Effective training programs encourage participation and hold both individuals and their leaders responsible. While DOD has taken steps that are consistent with most attributes of effective training programs, two services currently lack an oversight mechanism at the commander level to help ensure participation. DOD’s efforts underway include: (1) prioritizing training for servicemembers based on agreed-upon goals and priorities, (2) encouraging servicemembers to buy in to training goals, and (3) communicating the importance of training. For example, DOD and the services communicate the importance of TAP training by providing information on their websites, including how TAP aids in a successful transition, and through other communications, such as brochures with similar information. In addition, DOD has assigned unit commanders the responsibility of ensuring that eligible servicemembers have full access to and successfully complete required TAP components.

However, only the Army and the Air Force possess the capability to gauge the rate at which servicemembers under an individual unit commander participate in TAP. Specifically, the Army and Air Force provide commanders and their leaders information on their unit’s participation levels. In contrast, the Navy and the Marines do not have such systems. According to Navy officials, they obtained funding and plan to develop such a system by late June 2014 and plan to start using the data for oversight no later than fiscal year 2015. Marine officials said that because they have a long-standing culture of requiring servicemembers to attend TAP training, such an oversight mechanism was not necessary. In our 2002 and 2005 reviews of TAP, we found that servicemembers sometimes faced difficulties being released from military duties to attend TAP because of the priority accorded their military mission or the lack of supervisory support for TAP.on the pilot of the capstone event, DOD reported that ensuring servicemember participation in capstone events was a challenge for most of the services. According to DOD, lack of servicemembers’ awareness of this requirement and lack of commanders’ support may have hampered participation in capstone events. Without routine information on servicemembers’ participation by commander, it may be difficult to hold accountable those directly responsible for ensuring participation.

Element 5: Measuring Performance and Evaluating Results

Outputs are the direct products and services delivered by a program, and outcomes are the results of those products and services. See GAO, Performance Measurement and Evaluation: Definitions and Relationships, GAO-11-646SP (Washington, D.C.: May 2011). basic end-of-course evaluations to higher level impact evaluations.agencies plan to evaluate TAP at lower levels. Their evaluation plan includes (1) gauging participant reaction to all TAP components through end-of-course evaluations and (2) determining whether servicemembers met the career readiness standards prior to separation. Higher level evaluations are also important to help gauge the effectiveness of TAP, and the agencies have not demonstrated a strategic approach to planning such evaluations. Since the program is administered by multiple agencies, planning an evaluation is more challenging than if it were administered by a single agency. In December 2011, the agencies stated, in an internal report, that they would develop a methodology to assess the effectiveness of TAP by engaging new veterans approximately six months after they have separated from the military. Despite the modular nature of TAP, this report did not specify whether this type of higher level evaluation would assess all of TAP or certain components. DOL and VA are considering higher level evaluations for the employment workshop and VA benefits briefings. For example, DOL is considering different methodologies it could use to conduct an impact evaluation of the employment workshop. However, the agencies could not provide a rationale for not using higher-level evaluations to assess either TAP overall or some of the other TAP components and career-specific tracks. Best practices call for the development of a written evaluation strategy, which the agencies prepared for lower evaluation levels, but not for higher levels. Without a written evaluation strategy that identifies the TAP components to be evaluated, and includes the appropriate level and timing of the evaluation and methods, the agencies may miss important opportunities to obtain information needed to make fully informed decisions on whether and how to modify TAP and may either over invest or under invest in evaluations.

DOD Lacks a Process to Assess the TAP Experience of National Guard and Reserve Members

Based on GAO’s prior work and interviews with stakeholders, we identified a key remaining challenge; namely, that DOD lacks a process to assess the TAP experience of National Guard and Reserve members. DOD’s policy generally requires all eligible servicemembers, including members of the National Guard and Reserve, to be exposed to the entire TAP experience even if their circumstances differ. Unlike active duty servicemembers, many members of the National Guard and Reserve hold civilian occupations, and federal law protects their employment rights when they return from active duty. If they can document their civilian employment or acceptance to an institution of higher education or accredited technical training, members of the National Guard and Reserve and other eligible servicemembers can be exempted from attending the DOL employment workshop. Nevertheless, they are generally required to complete pre-separation counseling, the VA benefits I and II briefings and, under DOD policy, participate in the capstone event. Further, the policy states that they must attend the DOD core curriculum and applicable career specific tracks if they cannot meet the career readiness standards associated with these two components.

However, according to many stakeholders that we talked with, including officials from the services and organizations that support members of the National Guard and Reserve and other servicemembers, DOD has not resolved long-standing concerns that eligible members of the National Guard and Reserve attend TAP offerings in locations and on a timetable that diminish their experience. Specifically, given that eligible National Guard and Reserve members generally demobilize at locations where they neither work nor live, they may be distracted by their desire to return home, which can affect how much the training benefits them. Separating servicemembers need to be aware of the benefits and services and know how to access them to take advantage of them, such as the Post-9/11 GI Bill. In a January 2013 report, the Defense Business Board Task Group recommended that eligible National Guard and Reserve members be afforded the opportunity to demobilize and transition in their home unit’s geographical area. According to stakeholders, eligible National Guard and Reserve members also typically have less time to complete the program because they often demobilize more quickly than active duty servicemembers separate. For example, according to DOD’s 2011 Handbook, the Reserve demobilization timeline makes scheduling a pre- separation counseling appointment not later than 90 days prior to leaving active duty impractical for Reserve members.

According to officials from DOD and the services, they took steps to help meet the needs of National Guard and Reserve members eligible for TAP. For example, the agencies tailored the content of TAP components to better suit the needs of National Guard and Reserve members after However, only one analyzing the results of feedback from pilots of TAP.of the steps taken directly addresses the concerns of stakeholders related to the location and timing of TAP. Specifically, eligible members of the Army National Guard are allowed to participate in the DOL employment workshop and the capstone event in their home unit’s geographical area, according to DOD and Army officials. These eligible Army National Guard members will remain on active duty while participating in these two components.

Nonetheless, DOD may not be well positioned to determine whether its actions successfully address the long-standing challenges in designing transition services for the National Guard and Reserve. This is because while DOD collects participant feedback through the online assessment, this tool does not ask whether the timing and location met the needs of servicemembers, including the National Guard and Reserve. Moreover, DOD’s planned survey to active duty servicemembers contains questions about the timing of TAP, but DOD has not drafted similar questions for its survey to the National Guard and Reserve. According to OMB, being able to track and measure specific program data can help agencies diagnose problems, identify drivers of future performance, evaluate risk, support collaboration, and inform follow-up actions. Without a systematic process targeted to identify any remaining long-standing concerns, DOD will not be able to make fully informed decisions about the extent to which eligible members of the National Guard and Reserve reap the full benefits of TAP.

Conclusions

Helping servicemembers successfully transition to civilian life after their service ends is an important government goal. The agencies undertook an ambitious effort to redesign the 1990s-era program to provide servicemembers with training and skills to successfully transition to civilian life. The increase in the number of servicemembers attending the revised TAP components, along with the significant coordination effort among multiple agencies and military services, pose significant implementation challenges. Efforts to implement the redesigned program continue to evolve. The importance of serving the ongoing and projected wave of servicemembers departing the military increases the urgency to fully implement TAP components as well as to finalize related policies and procedures. To the extent that the program remains behind schedule in implementing TAP, some transitioning servicemembers may be denied the full benefit of the revamped program. Because TAP is not fully implemented, the full impact of the revamped TAP, particularly the warm handover—a key effort to serve those deemed at greater risk of not being career ready—remains to be seen.

The revamped TAP exhibits elements important for the effective implementation and evaluation of the program. Yet, we found room for improvement in several key areas. In particular, while federal law generally requires DOD to mandate participation in the employment workshop and DOD policy requires that all eligible transitioning servicemembers participate in TAP overall, the Navy and the Marines lack the ability to provide unit commanders and their leaders with information on participation levels of servicemembers under their command. Without the capability to gauge and report participation in TAP components by unit commander—those directly responsible for ensuring servicemember participation—leaders may find that holding them accountable is difficult.

While the administration cites the revamped TAP as a key strategy of its crosscutting goal to improve the career readiness of veterans, it may not be positioned to determine the extent to which the program prepared veterans to pursue their post-separation goals. Without a written evaluation strategy that identifies the TAP components to be evaluated and includes the appropriate level and timing of the evaluation methods, the agencies may miss important opportunities to obtain information needed to make fully informed decisions on whether and how to modify TAP and may either over invest or under invest in evaluations.

The circumstances of National Guard and Reserve members differ from those of other active duty servicemembers. Many stakeholders have raised concerns that these circumstances diminish eligible National Guard and Reserve members’ TAP experience. Absent efforts to systematically collect information about eligible National Guard and Reserve members’ experiences under the revamped program, DOD may not be well positioned to determine whether there are problems with how TAP is provided to these groups.

Recommendations for Executive Action

Based on our review, we recommend that the Secretary of Defense take the following actions: 1. To better ensure servicemember participation in and completion of TAP, direct the Under Secretary for Personnel and Readiness to require that all services provide unit commanders and their leaders information on TAP participation levels of servicemembers under their command, similar to that provided by the Army and Air Force. Such information could be used to help hold leaders accountable for ensuring TAP participation and completion. 2. To provide information on the extent to which the revamped TAP is effective, direct the Under Secretary for Personnel and Readiness to work with the partner agencies to develop a written strategy for determining which components and tracks to evaluate and the most appropriate evaluation methods. This strategy should include a plan to use the results of evaluations to modify or redesign the program, as appropriate. 3. To ensure that decisions about the participation of eligible members of the National Guard and Reserves in TAP are fully informed, direct the Under Secretary for Personnel and Readiness to systematically collect information on any challenges facing demobilizing members of the National Guard and Reserves regarding the logistics of the timing and location to attend TAP. For example, agencies might add questions to their online assessment tool specific to eligible members of the National Guard and Reserves who participate in TAP.

Agency Comments and Our Evaluation

We provided a draft of this report to DOD, VA, DOL, and SBA for comment. In its comments, DOD agreed with one recommendation and disagreed with the other two recommendations. VA generally agreed with our findings. Written comments from DOD and VA are reproduced in appendices III and IV. DOL and SBA did not provide comments, but all four agencies provided technical comments, which we incorporated into the final report as appropriate.

DOD agreed with our recommendation to work with partner agencies to develop a written strategy for determining which components and tracks to evaluate and the most appropriate evaluation methods. DOD stated that it will continue to support interagency collaboration, which, as of January 31, 2014, has been formalized in a memorandum of understanding among the agencies administering TAP.

DOD disagreed with our recommendation to require that all of the services provide unit commanders and their leaders information on TAP participation levels of servicemembers under their command. DOD stated that we justified the need for a mechanism to ensure a servicemember’s completion of TAP by commander based on concerns that without such a mechanism, commanders will not support the release of servicemembers to attend TAP. DOD also said that these concerns are based on observations that preceded full implementation of the capstone event. In October 2013, DOD launched the capstone as a mandatory process by which commanders verify TAP participation and positively affirm that servicemembers have or have not met career readiness standards. In addition, DOD said that capstone event implementation was accompanied by a communications campaign to inform both commanders and the services’ TAP providers of this key requirement. Finally, DOD said that capstone event completion is monitored departmentwide to ensure compliance.

We agree with DOD that departmentwide monitoring of capstone event completion is an important element in helping ensure compliance. However, such monitoring does not provide routine information to commanders and their leaders and not all TAP locations will be monitored routinely. As noted in our report, based on the pilot of the capstone event last fall, DOD reported that ensuring servicemember participation in capstone events was a challenge for most of the services (with the exception of the Air Force), adding that this challenge is possibly due in part to lack of commanders’ support. Commander support for the program has been a long-standing challenge for the program. Therefore, we continue to believe that our recommendation is needed because it would establish an accountability mechanism for TAP that mirrors the level at which responsibility has been assigned. DOD has assigned unit commanders the responsibility of ensuring that eligible servicemembers have full access to and successfully complete required TAP components. Providing information to unit commanders and their leaders on TAP participation levels of servicemembers under their command—similar to that provided by the Army and Air Force—could thus promote accountability and oversight. Servicemember participation in TAP is generally required by law and DOD policy, and also relates to a Cross- Agency Priority Goal, which reinforces the need for an appropriate accountability mechanism.

DOD also stated that the Navy is funding IT system upgrades to provide the ability to analyze and report program compliance at the command level. DOD added that the Marine Corps has mandated participation since the program's inception, and Marine Corps commanders leverage the capabilities of the personnel system to identify eligible Marines and schedule their TAP attendance. DOD stated that despite the limitations of a data tracking system that underreports compliance figures, the Marine Corps had the highest compliance rate of all services, as of December 2013. DOD said it recommends that department-wide compliance data be allowed to normalize to show true compliance percentages before the services are required to fund and implement expensive systems for the sole purpose of providing an additional mechanism for commander accountability. However, we continue to believe that our recommendation is needed. Three of the services either have or are working to develop the kind of accountability mechanism that we are recommending, but the Marine Corps does not plan to develop such a system. While DOD reports that the Marine Corps has the highest TAP compliance rate of all services, DOD did not provide any data on these rates. Finally, in response to DOD’s recommendation to wait until compliance data normalize, we believe that this would not be appropriate. We do not see any advantage to delaying the implementation of appropriate accountability mechanisms to provide assurance that the ongoing and projected wave of servicemembers departing the military receive the expected level of services from TAP.

DOD disagreed with our recommendation to systematically collect information on challenges facing demobilizing members of the National Guard and Reserve regarding the timing and location to attend TAP. Specifically, DOD stated that it has long understood that the National Guard and Reserves operate under schedules and logistical constraints that differ from those of active duty servicemembers. DOD stated that several processes (which we note in our report) are already in place to identity and rectify any misalignments between TAP services and the needs of eligible National Guard and Reserve members, including regular coordinating councils that include representation from the National Guard and Reserve and TAP managers, as well as a participant assessment that provides ample opportunity for eligible National Guard and Reserve members to voice concerns. In addition, DOD highlighted the implementation of the military life cycle transition model in which DOD plans to provide the services, including to the National Guard and Reserve, with the latitude to optimize placement of certain elements of TAP throughout a servicemember's military service. According to DOD, the military life cycle transition model may help address some of the challenges related to the timing and location of program delivery. As we note in the report, full implementation of the military life cycle transition model is planned by October 2014.

Nonetheless, at this time DOD is not fully positioned to know whether or not the revamped program addresses the long-standing challenges facing eligible members of the National Guard and Reserve in taking TAP. As we describe in the report, all eligible servicemembers, including the National Guard and Reserve members, have an opportunity to provide feedback about the instruction, content, and facilities for TAP. However, the participant assessment does not ask questions specific to the National Guard and Reserve experience even though they face different circumstances than active duty servicemembers. Given these differences, we continue to believe that unless DOD systematically collects information on any challenges facing eligible members of the National Guard and Reserve, DOD is at risk of not fully knowing whether the revamped TAP addresses long-standing challenges. In addition, the move to a military life cycle transition model could enable active duty servicemembers and members of the National Guard and Reserve to benefit from transition-related assistance throughout their military service. Recognizing that the military life cycle transition model is not implemented and specific policies and plans are not completed, many unknowns remain about how it will work in general and how National Guard and Reserve members will fare specifically. During our review, we asked DOD how this program would work in practice, including for National Guard and Reserve members, but DOD did not provide us details.

We are sending copies of this report to appropriate congressional committees; the Secretary of Defense; the Secretary of Veterans Affairs, the Secretary of Labor; the Acting Administrator of the Small Business Administration; and other interested parties. In addition, this report will be available at no charge on GAO’s website at http://www.gao.gov.

If you or your staff members have any questions regarding this report, please contact me at (202) 512-7215 or sherrilla@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix V.

Appendix I: Technical Appendix

This appendix summarizes our work to assess efforts to implement and evaluate the revamped TAP.

Elements and Relevant Attributes and the Extent to Which They Have Been Addressed

For each of the five elements that we identified as important for the effective implementation and evaluation of TAP, we identified relevant attributes that we used to determine the extent to which the elements were addressed in the revamped TAP (see table 2).

To determine the extent to which agencies have addressed each element and attribute, we made our assessments in two steps. First, an analyst reviewed all of the evidence. Based on that review, the analyst documented evidence that conformed to the elements and attributes and made and documented a judgment about the status of these efforts. Then a second analyst separately reviewed the assessment and documented their agreement or disagreement. The two analysts discussed any differences in their assessments and made changes based on their verbal resolution of those differences.

We assessed the status of the agencies’ efforts to address each relevant attribute as “completed”, “underway”, or “not completed”. Based on this analysis, we then determined the status of efforts to address each respective attribute overall. For example, for the status of efforts to address an attribute overall to be considered “completed”, efforts to address each relevant attribute had to be completed. For the status of efforts to be considered “underway”, it had to be documented that efforts to address each relevant attribute were progressing toward being finalized, or were a mix of underway efforts and finalized efforts. For the status of efforts to be considered “not completed”, efforts to address at least one relevant attribute were not determined to be finalized or underway.

We considered the agency to be “adequately addressing” the element if the overall status of the efforts to address the relevant attributes was determined to be completed or underway. However, if some attributes related to an element had efforts completed or underway to address them, but one or more other attributes did not have efforts completed or underway to address it (them), then we consider the overall efforts to address the element to be “mixed”.

We used the following general decision rules for characterizing the status of efforts to address each attribute as “completed”, “underway”, and “not completed”: “Completed” means interagency partners (or relevant agency) have a system, policy, or procedure in place to address a relevant attribute. “Underway” means interagency partners (or relevant agency) have an authoritative internal or public document that demonstrate progress toward implementing actions to address the attribute (e.g., agencies have draft policies, operating procedures, or guidance, or they have interim systems in place and/or are developing systems). “Not completed” means that interagency partners (or relevant agency) have shown little to no action toward implementing actions to address the attribute, meaning they lack an authoritative internal or public document (e.g., no draft policies, operating procedures, or guidance or interim systems).

As we conducted this analysis, we kept in mind how far along agencies (mainly DOD and the military services) are in implementing each component of TAP (informed by objective 1). Because agencies were in the process of implementing the revamped TAP and the program was not fully operational at the time of our review, we were not able to determine whether the policies and procedures are in place at all sites or if they are working as intended. Similarly, we were not able to comment on whether the changes to TAP are yielding desired benefits or improvements in outcomes.

Appendix II: DOD Form 2958, Individual Transition Plan (ITP) Checklist

Appendix III: Comments from the Department of Defense

Appendix IV: Comments from the Department of Veterans Affairs

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact

Andrew Sherrill, (202) 512-7215 or sherrilla@gao.gov.

Staff Acknowledgments

In addition to the contact named above, individuals making key contributions to this report were Patrick Dibattista, Assistant Director; Julianne Hartman Cutts; and James Whitcomb. In addition, key support was provided by James Bennett, Rachael Chamberlin, David Chrisinger, Brett Fallavollita, Alex Galuten, Kathy Leslie, David Moser, and Michael Silver.