Military Personnel: DOD Should Develop a Plan to Evaluate the Effectiveness of Its Career Intermission Pilot Program
Released: 2015-10-27T00:00:00

Letter

Background

The NDAA for fiscal year 2009 initially authorized CIPP as a pilot program through December 31, 2012, establishing basic eligibility criteria for participants, providing guidelines for implementing the program, and establishing congressional reporting requirements. Specifically:

For each calendar year from 2009 through 2012, up to 20 officers and 20 enlisted servicemembers per military service are authorized to leave active duty for a period not to exceed 3 years. For each month of sabbatical taken, servicemembers must complete two months of obligated service upon their return to active duty.

Servicemembers who have completed their initial active duty service agreement and are not currently receiving a critical skills retention bonus are eligible to participate.

During their sabbatical, all servicemembers are required to serve in the Individual Ready Reserve and are required to undergo such inactive duty training as shall be required by the Secretary involved in order to ensure that the servicemember retains sufficient proficiency in the military skills, professional qualifications and physical readiness.

During sabbaticals, servicemembers receive two-thirtieths of their salary (i.e. 2-days pay per month) and maintain full health benefits for themselves and their dependents. In addition, DOD provides participants and their dependents with a paid relocation within the United States. For example, if servicemembers are taking a sabbatical to attend school, DOD will pay for them to move to the location of their educational program. At the end of the servicemember’s sabbatical, DOD will pay the costs to relocate the servicemember to his or her next assignment.

The NDAA for fiscal year 2015 kept these NDAA fiscal year 2009 guidelines and extended the program, allowing for servicemembers to start sabbaticals through December 31, 2019, returning to active duty no later than December 31, 2022. Appendix I shows when each military service implemented CIPP and the number of participants approved by each service as of July 2015.The fiscal year 2015 NDAA also extended the date for DOD to provide a final report to Congress—from March 1, 2016, to March 1, 2023, and it added additional reporting elements. DOD is now required to report the following:

A description of the number of applicants for the pilot program and the criteria used to select individuals for participation in the pilot program.

An evaluation of whether the authorities of the pilot programs provided an effective means to enhance the retention of members of the armed forces possessing critical skills, talents, and leadership abilities.

An evaluation of whether the career progression in the armed forces of individuals who participate in the pilot program has been or will be adversely affected; and the usefulness of the pilot program in responding to the personal and professional needs of individual members of the armed forces.

A description of reasons why servicemembers choose to participate in the pilot.

A description of the servicemembers, if any, who did not return to active duty at the conclusion of their sabbatical, and a statement of the reasons why these servicemembers did not return.

A statement about whether servicemembers were required to perform training as part of their participation in the pilot program, and if so, a description of the servicemembers who were required to perform training, the reasons they were required to perform training, and how often they were required to perform training.

A description of the costs to each military department of each pilot program.

Recommendations for legislative or administrative action as the Secretary concerned considers appropriate for the modification or continuation of the pilot programs.

CIPP Participation Has Been Lower Than Authorized Limits and the Military Services Have Identified Factors That Could Affect Participation, but DOD Has Not Established a Plan for Evaluating the Effect of CIPP on Retention

Participation in CIPP has remained below statutorily authorized limits, and officials have identified factors that could be affecting CIPP participation, but DOD has not established a plan for evaluating whether CIPP is an effective means to retain servicemembers. The rate of DOD-wide participation in CIPP has been at less than half the authorized limit of 160 participants per calendar year, and officials from each of the services stated that factors including statutory requirements, service-specific limitations, military culture, and personal financial constraints could be affecting participation. Additionally, although DOD officials stated that they would like to make CIPP a permanent program, and the services are required to provide a final report to Congress on its effectiveness, costs, and retention not later than March 1, 2023, DOD has not established a plan for evaluating the effect of the pilot program on retention of servicemembers.

CIPP Participation Has Remained Below Authorized Limits

Since Congress authorized CIPP in fiscal year 2009, participation has remained below authorized limits. As shown in figure 1, DOD is authorized to enroll up to 160 servicemembers per year in the program (up to 40 participants for each of the four services); but DOD-wide, the highest number of participants approved for CIPP was 76, in calendar year 2014.

From 2009 through 2012, only Navy personnel were participating in CIPP, but in 2013, the Marine Corps approved its first applicant, and in 2014, personnel from all four services were participating in the pilot. Some of the services have had participation levels closer to the authorized limits. For example, in 2014, of the 76 participants approved, 30 were Navy and 35 were Air Force. However, the Army and Marine Corps were below authorized limits, with 9 servicemembers approved from the Army and 2 from the Marine Corps.

Service Officials Have Identified Factors That May Affect Participation

Service officials identified four factors that may affect participation in CIPP—statutory requirements, service-specific limitations, military culture, and financial constraints.

Statutory Requirements—According to the CIPP authorizing statute, servicemembers are not eligible to participate in the program during the period of their initial active duty service agreement or if they are currently receiving a critical skills retention bonus. These eligibility criteria reduce the population eligible to apply for CIPP. For example, according to Navy officials, as of July 2015, almost 134,000 Navy servicemembers were ineligible to participate in CIPP because they were in their initial active duty service agreement period. According to a DOD budget analysis document, the initial service agreement for a Navy sailor typically occurs from 18 to 33 years of age, when professional goals compete most strongly with personal goals such as family planning. For example, one participant who responded to our questionnaire stated that she used CIPP after completing her initial service obligation to start her family. However, she would have preferred to take a sabbatical during her initial service obligation period when she was younger.

According to a DOD budget analysis document, for the Navy, retention at a servicemember’s first career reenlistment point is the most difficult to achieve. However, if servicemembers elected to participate in CIPP during their first service obligation period, they in effect would be electing retention during this critical timeframe. According to Navy officials, if these servicemembers were able to participate in CIPP, the CIPP-obligated service requirement would extend each servicemember’s existing period of obligated service, which could enhance retention.

Another statutory requirement caps the annual number of participants at 40 (20 officers and 20 enlisted) servicemembers per service. An Army official and some Navy officials were of the opinion that the limitations on the number of participants may reduce participation, stating that servicemembers may be hesitant to apply since so few people were selected annually. Proposed language in the fiscal year 2016 NDAA, if enacted, would repeal the prohibition on participation in CIPP by servicemembers who are in their initial obligated service period or who are receiving a critical military skills retention bonus, and it would eliminate program participation caps.

Service-specific limitations—Each military service has established selection processes and eligibility requirements that supplement the statutory requirements established by the NDAA for fiscal year 2009. For example, the Air Force rates applicants in various categories— such as job performance, leadership, experience, job responsibility, and education. As a result, according to Air Force officials, the most competitive applicants were prioritized for participation in CIPP, and less competitive applicants were disapproved for participation in the program. Further, service-specific guidance includes limitations on participation by servicemembers in certain career fields, such as Army medical personnel and some officers in the Navy Chaplain Corps and Judge Advocate General’s (JAG) Corps, as well as certain enlisted nuclear personnel. According to Navy and Air Force officials, additional career fields that require sustained proficiency (such as operating weapons systems or piloting aircraft), while not restricted from participation in CIPP, may have restrictions on breaks in service. For example, Navy officials stated that officers in the submarine community must receive a waiver to go longer than 3 years without a sea tour, and if officers exceed 5 years without a sea tour they can no longer work in the submarine community—this could occur if an officer took a 3-year sabbatical followed by a 3-year shore tour. According to Navy officials, if individuals in these communities participate in CIPP, measures are taken to ensure that they do not exceed timeframes that would result in the loss of their ability to serve in their community. Additionally, Navy and Air Force officials stated that pilots who do not have a minimum number of flight hours within a certain time period are no longer certified to operate their aircraft, and are required to complete additional training to be recertified. A Navy official stated that pilots are not disqualified from their position; however, additional training further extends the officer’s time out of operational service, which may affect the officer’s promotion potential.

Military culture—Officials from each service also stated that participation may be influenced by military culture, and that servicemembers have the perception that a break in service may have a negative effect on upward advancement. Specifically, officials from all the services stated that servicemembers may not trust assurances as to how a break in service would be viewed by promotion boards. For example, one participant was concerned that “a break in service would be viewed as taking an off-ramp, an easy path, taking out of the fast lane,” but upon returning from sabbatical has been reassured by knowledge of other participants who have returned from their sabbaticals and received promotions. Another participant reported being “told explicitly by my chain of command [before entering the program] that my career would suffer”; and another reported that upon returning from sabbatical the servicemember would “meet people, sadly even some senior leaders, who are not familiar with the program and assume I have decided to prioritize family over career or assume I do not want to competitive for advancement.” CIPP authorizing language includes provisions designed to mitigate any potential negative effect of a sabbatical on career advancement, but according to Army and Navy officials, until more CIPP participants return from sabbaticals and demonstrate career advancement, servicemembers may be hesitant to participate.

Financial constraints—The salary that servicemembers receive during the sabbatical period is equivalent to approximately 2 days of pay per month. Additionally, according to DOD policy, service members may not receive special or incentive pay or bonus payments while on sabbatical. Officials from the Army and the Navy stated that participation in CIPP likely will remain limited because servicemembers need financial resources to support themselves and their families during the sabbatical. One of the CIPP participants who responded to our questionnaire emphasized the need to have another source of income while participating. Another participant reported the opinion that CIPP “gives a options that are not available in any other program. However, the deal is not that great for the member—mainly because of the monetary hit. Since a member is coming back, I think it is possible to allow a person to receive some pay while participating in CIPP.”

DOD Has Not Established a Plan for Evaluating Whether CIPP Is an Effective Means to Retain Servicemembers

In February 2009, OUSD(P&R) issued a directive-type memorandum that authorized—but did not require—the Secretary of each military department to implement CIPP. According to the memorandum, if the services did implement CIPP, they were required to develop a method to evaluate the program. Specifically, the memorandum stated that the services should “have the appropriate oversight, analytical rigor, and proper evaluation methodologies” to evaluate the pilot. In September 2015, OUSD(P&R) reissued the memorandum and, among other things, included a requirement for each service to report to OUSD(P&R) annually on the status and effectiveness of the program. This report is to include information on the demographics of CIPP applicants, criteria used for selecting applicants, an assessment of the effectiveness of the program, and recommendations for legislative or administrative actions for the modification or continuation of the CIPP. However, neither DOD nor the services have developed a plan for evaluating the extent to which the pilot program is an effective means to retain servicemembers. The updated memorandum also clarifies DOD’s policy on servicemember benefits while on sabbatical and includes a requirement for each service to report to OUSD(P&R) on June 1st of each year on the program’s progression.

More specifically, based on the revised guidance, beginning June 1, 2016, the services will be required to provide OUSD(P&R) an evaluation of whether: the authorities for CIPP provide an effective means to enhance the retention of participant servicemembers possessing critical skills, talents, and leadership; the career progression of participant servicemembers has been or will be adversely affected; and

CIPP is useful in responding to the personal and professional needs of individual servicemembers.

These reporting elements are also required in the services’ final report to Congress, due March 2023. Interim reports on the implementation and current status of the pilot programs are due in 2017and 2019. DOD has proposed expansion of the pilot, and the proposed fiscal year 2016 NDAA includes language that will remove the pilot’s participation cap and some restrictions on participation. Additionally, DOD officials stated that CIPP should be made available permanently; however, without an evaluation of the program, the basis for DOD’s proposed changes to the program is unclear.

We have identified key features that should be included in pilot program evaluation plans, and along with private professional auditing and evaluation organizations, we have found that a well-developed and documented evaluation plan can help ensure that agency evaluations generate performance information needed to make effective program and policy decisions. Well-developed evaluation plans include key features such as: well-defined, clear, and measurable objectives; criteria or standards for determining pilot-program performance; clearly articulated methodology, including sound sampling methods, determination of appropriate sample size for the evaluation design, and a strategy for comparing the pilot results with other efforts; a clear plan that details the type and source of data necessary to evaluate the pilot, methods for data collection, and the timing and frequency of data collection; and a detailed data-analysis plan to track the program’s performance and evaluate the final results of the project.

Although the services are required to evaluate the effectiveness of CIPP, currently they do not have any plans for evaluating the program. Without a plan for evaluating the pilot that includes these key features, there will be limited assurance that the evaluations conducted will provide the information needed to make decisions about the future of CIPP.

Moreover, the establishment of a plan including key features such as well-defined, clear, and measurable objectives and standards for determining pilot-program performance may aid in addressing some of the challenges posed by the pilot’s timeline. Prior to the establishment of the June 2016 OUSD(P&R) reporting requirement, officials from all four services raised concerns about their ability to evaluate the effectiveness of the program so soon after implementation. Specifically, Marine Corps and Army officials stated that it is too early to determine the program’s effect on retention, and that it can take several years after a participant starts a sabbatical to determine whether the program contributed to retention. Marine Corps officials stated that if a participant took the maximum 3-year sabbatical followed by a 6-year obligated service period, it could take up to 9 years to determine whether the individual would decide to stay in the armed services beyond his or her period of obligated service. As of July 2015, of the 133 program participants, 5 have completed the obligated service period. Putting plans in place for how the pilot will be evaluated can guide the services on the data they need to collect as the pilot progresses, and can better position them to assess the pilot’s performance.

Navy Officials Stated That CIPP Has Provided an Option to Respond to Personal Needs of Servicemembers

According to Navy officials, CIPP has provided an option for the Navy to respond to the personal needs of servicemembers, and they believe the program has helped to retain servicemembers who otherwise might have left the military. Additionally, a DOD budget analysis document states that the Navy will retain a servicemember for a longer time period by using a combination of monetary and non-monetary incentives than would have been possible using only a single incentive. According to this document, in the Navy’s experience, financial incentives alone have not been adequate to retain certain categories of servicemembers, such as nuclear-trained surface warfare officers and senior nuclear-trained enlisted sailors serving on submarines and aircraft carriers. Navy CIPP participants have come from a range of career fields, including aviators, engineers, medical personnel, nuclear-trained surface warfare officers, and others.

Navy officials stated that they are not using CIPP to address any specific critical skills, but that a servicemember’s occupation is given consideration during the CIPP approval process. According to the Navy’s 2011 interim report to Congress, CIPP applicants need certain qualifications, including a record of demonstrating strong and sustained performance in challenging positions, leadership, professional skills, resourcefulness, ability or potential to contribute to and succeed in the Navy, and exemplary personal behavior and integrity. For example, according to a Navy CIPP document, a Petty Officer Second Class was identified by the JAG Corps as a servicemember who displayed the aptitude, work ethic, and talent needed to serve as an attorney. This individual was encouraged to take a sabbatical to complete her degree, earn a Juris Doctorate, and apply for a commission in the JAG Corps upon return from the sabbatical. After a 36-month sabbatical this servicemember earned a commission in the JAG Corps and became an attorney in the Navy.

In addition, officials stated that a career sabbatical may help to address the work-life balance that cannot be achieved through other human- capital programs. For example, one participant who responded to our questionnaire reported: “ believe provides a suitable option for work/life balance that helps offset goals/issues that cannot be addressed while on active duty and gives sailors an option besides getting out entirely.” In particular, officials stated that they are concerned that the Navy’s recurring sea-tour requirement may result in the loss of servicemembers with short-term personal needs or skill sets that are in demand in the private sector. For example, another respondent reported: “ is a great option for sailors who need to take a break from the arduous duty and demands of the Navy. Additionally, it can give sailors who are thinking about leaving the Navy the experience of what it is like to be in the civilian sector.”

The Navy collects information from participants, both when they start their sabbatical and when they return, about the extent to which CIPP was a factor in the participants’ choice to stay in the Navy; whether participants intend to make the Navy their career; whether participants would recommend CIPP to other servicemembers’ and whether CIPP has negatively affected their career. Also, a Navy CIPP document provided examples of participants who fared well with their career milestones following their return to active duty. For example, according to CIPP program managers, one officer was selected for promotion following sabbatical, and two other officers were selected for administrative screening boards upon their return. Our questionnaire asked CIPP participants if since returning to active duty, they have been told or otherwise experienced something specific that indicated CIPP participation might affect their career advancement. The responses were mixed. We received examples expressing the view that use of a sabbatical for educational purposes was positive because the education received while on sabbatical was beneficial for career advancement. Conversely, there were negative examples reporting that Navy chain of command views the break in service as a “lack of commitment,” or “leaving community while others continued to work.”

Conclusions

Congress authorized CIPP as a pilot program to help the services offer greater flexibility in career paths for servicemembers with the hope of increasing the retention of personnel with critical skills. All of the military services have implemented CIPP, and DOD officials have stated that the program should become permanent. Beginning in June 2016, the services will be required to evaluate and report annually on the effectiveness of the pilot. However, they do not have a plan to guide these evaluation efforts and help determine the extent to which the pilot program is an effective means to retain servicemembers. Without a plan that includes key features for evaluating CIPP’s value as a retention tool, DOD will be unlikely to determine the extent to which CIPP is achieving its intended purpose and thereby inform decision makers as to whether it should become a permanent program.

Recommendation for Executive Action

To assist DOD in determining whether CIPP is meeting its intended purpose of enhancing retention and providing greater flexibility in the career path of servicemembers, we recommend that the Secretary of Defense direct the Under Secretary of Defense for Personnel and Readiness, in collaboration with the service secretaries, develop and implement a plan to evaluate the pilot that includes key features such as well-defined, clear, and measurable objectives and standards for determining pilot-program performance.

Agency Comments

We provided a draft of this report to DOD for review and comment. In written comments, which are reprinted in their entirety in Appendix II, DOD concurred with our recommendation. DOD noted that they recognize the importance of developing well-defined measures to evaluate the effectiveness and utility of CIPP. DOD also provided technical comments, which we have incorporated in the report where appropriate.

We are sending copies of this report to the appropriate congressional committees, the Secretary of Defense, the Under Secretary of Defense for Personnel and Readiness, the Chairman of the Joint Chiefs of Staff, and the Secretaries of the military departments. The report also is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-3604 or farrellb@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III.

Appendix I: Status of the Career Intermission Pilot Program (CIPP) Implementation

The Navy implemented CIPP in 2009, followed by the Marine Corps in 2013, and the Air Force and Army in 2014; as of July 2015, the services had approved 161 servicemembers to participate in CIPP. The Navy has approved the highest number of participants, and as of July 2015, 37 participants have completed sabbaticals and returned to active duty. Table 1 shows the number and demographics of CIPP participants for each military service.

Navy—From 2009 to July 2015, 130 Navy servicemembers applied to participate in CIPP and 111 were approved, 11 were disapproved, 6 withdrew their applications before a final decision had been made, and 2 applications are pending. Of the 111 approved, 18 declined the offer. As of July 2015, 37 had completed sabbaticals. Of these 37, one separated before completing obligated service and 5 have completed their CIPP-related obligated service. Of these 5, one has since left active duty for the Navy Reserves, and one has since separated from the Navy. Participants used the program for several purposes, including pursuing higher education, supporting family (care for ailing parents or caring for young children), and staggering career timelines for dual-military spouses.

Air Force—In 2014, 46 Air Force servicemembers applied to participate in CIPP and 35 applicants were approved (1 was removed from the program for quality reasons arising after selection to the program). Of the remaining 34 selected, 4 declined the offer, and 30 accepted. As of July 2015, 23 participants had begun a sabbatical. The Air Force disapproved 11 applicants because they did not meet basic eligibility requirements or, according to Air Force officials, did not have competitive performance ratings. Participants plan to use their sabbaticals to, among other things, pursue education, care for a family member or start a family, and realign assignment timing or date of rank with an active-duty spouse to facilitate joint spouse assignment.

Army—In 2014, 10 Army servicemembers applied to participate in CIPP and 1 was determined to be ineligible due to remaining service obligation and 9 were approved. Of the 9 selected, 3 declined the offer in favor of other personnel actions, and 6 accepted. The 6 participants were expected to begin sabbaticals in summer 2015. Participants plan to use their sabbaticals to pursue higher education, address family and medical issues, travel, and align assignment cycle with an active-duty spouse.

Marine Corps—In 2013, 3 Marines applied and were approved, but one subsequently withdrew the application. In 2014, 2 applied and were approved, but one withdrew. In 2015, 2 applied, 1 was accepted, and 1 was determined to be ineligible. As of July 2015, 3 of the 4 total participants were on sabbatical. Applicants requested the sabbaticals to move with a spouse and attend graduate school, to focus on family and children, or to attend seminary.

Appendix II: Comments from the Department of Defense

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Kimberly Seay (Assistant Director), Vijay Barnabas, Tim Carr, Amie Lesser, Felicia Lopez, Richard Powelson, Tida Reveley, and Michael Silver made major contributions to this report.