Force Structure: Performance Measures Needed to Better Implement the Recommendations of the National Commission on the Structure of the Air Force
Released: 2016-05-06T00:00:00

Letter

Background

The National Defense Authorization Act for Fiscal Year 2013 established the National Commission on the Structure of the Air Force. The act required the commission to undertake a comprehensive study of the structure of the Air Force to determine whether, and how, the structure should be modified to best fulfill current and anticipated mission requirements for the Air Force in a manner consistent with available resources. The commission was to give particular consideration to evaluating a structure that achieved certain things, including an appropriate balance between the active and reserve component of the Air Force.

In January 2014, the commission submitted its report to the President and the House and Senate Armed Services Committees with 42 recommendations that varied in size, scope, and complexity. For example, one relatively straightforward recommendation was to discontinue the use of non-disclosure agreements in the corporate (budget) process. In contrast, one large and complex recommendation was to integrate the headquarters staffs of the components. In addition, the commission’s recommendations were addressed to different entities—the President (one recommendation), Congress (five recommendations), Secretary of Defense (four recommendations), and Secretary of the Air Force (32 recommendations). For example, one recommendation was for Congress to allow the closing or “warm basing” of some installations. Section 1055 of the Carl Levin and Howard P. “Buck” McKeon National Defense Authorization Act for Fiscal Year 2015 requires that the Secretary of the Air Force submit to the congressional defense committees a yearly report on the Air Force response to the commission’s recommendations. In February 2015, the Air Force submitted its first response to the commission’s recommendations; and, submitted its second response in February 2016.

Organizations Involved in Implementing the Commission’s Recommendations

In January 2013, the Air Force created the Total Force Task Force to identify options for integrating the active and reserve component to meet current and future Air Force requirements. The Task Force transitioned into the Total Force Continuum office within the Headquarters Air Force Strategic Plans and Programs Directorate. The Total Force Continuum office is led by general officers from each component, who, according to Air Force officials, manage and oversee the process for implementing the commission’s recommendations. Further, officials within the Total Force Continuum office work with other Air Force organizations to implement the commission’s recommendations. For example, according to Air Force documentation on the status of the commission’s recommendation, the team working to develop a cost model for calculating military personnel costs included representatives from the office of the Assistant Secretary of the Air Force for Financial Management and Comptroller; Air Force Reserve Command; and the office of the Director, Air Force Studies, Analyses, and Assessments.

On July 3, 2014, the Air Force established the Executive Committee, which is chaired by the Air Force Assistant Vice Chief of Staff, and includes senior leaders from Air Force headquarters offices and the reserve component. According to Air Force officials from the Total Force Continuum office, the Executive Committee tracks and acts on the commission’s recommendations, and provides regular updates to the Secretary of the Air Force and Chief of Staff of the Air Force.

Original Process for Implementing the Commission’s Recommendations

The Total Force Continuum office established a team for each commission recommendation comprised of representatives from each component. According to Air Force officials, each team is led by a Colonel, Lieutenant Colonel, or civilian equivalent and is responsible for reviewing the recommendation and proposing implementation actions. The teams present their proposed actions to the one-star general officers within the Total Force Continuum office. According to Air Force officials, upon the general officers’ approval, the team leader and general officers brief the Executive Committee. The Executive Committee may approve the proposed action or may direct the team to further study or modify the proposed actions. Semi-annually, the Executive Committee briefs the Secretary of the Air Force and Chief of Staff of the Air Force on the proposed implementation actions. According to Air Force officials, the Secretary and Chief of Staff may approve implementation of the proposed action, close the recommendation, or direct the team to do more work. Figure 1 illustrates this process.

Air Force Strategic Master Plan Human Capital Annex

In May 2015, the Air Force issued its Strategic Master Plan, which is intended, in part, to align activities across the Air Force in the areas of human capital, strategic posture, capabilities, and science and technology. The Strategic Master Plan contains annexes for each of these areas, which are intended to translate the Strategic Master Plan’s goals and objectives into tangible actions and priorities. The Strategic Master Plan has an overall goal for considering all components as a Total Force. The Human Capital Annex expands upon the concept of a total force and specifies goals, objectives, and timeframes for recruiting and training, retention, component integration, and other topics. Twenty-one of the commission’s recommendations are aligned with four of the six objectives in the Human Capital Annex, as part of the revised approach and as illustrated in appendix III to this report. For example, the Annex’s goals for career progression are similar to the commission’s recommendations on promotions, continuum of service, and multiple career track options. Also, the Annex’s goals for increasing integration among the components are similar to the commission’s recommendations for integrating headquarters’ staffs, integrating personnel systems, and making transition between components easier.

The Air Force Has Made Limited Progress Developing Plans to Implement the Commission’s Recommendations and Is Revising Its Management Approach

As of February 2016, two years after the commission issued its report, the Air Force closed six recommendations—five were implemented and the Air Force did not agree with the sixth. Thus, the Air Force still has to implement 36 of the 42 recommendations. The Air Force has only developed partial implementation plans for three of these 36 recommendations, which remain open.

The Air Force Encountered Challenges as It Began Implementing the Commission’s Recommendations According to Officials

Air Force officials from the Total Force Continuum office explained that the Air Force experienced challenges internal and external to the service that affected implementation of the commission’s recommendations. According to Air Force officials, internal challenges included: expected completion dates not identified; difficulties in coordinating implementation efforts across offices; and inter-related recommendations. First, neither the commission nor the Air Force established time frames for completing implementation of all the commission’s recommendations. The lack of an overall implementation time frame, or time frames for most individual recommendations, may have conveyed the idea that implementation was open-ended, since there was generally no clear end-date to work toward, according to Total Force Continuum Office officials. Second, the Air Force experienced challenges in coordinating implementation efforts across components and directorates. According to Air Force officials, extensive coordination and cooperation across the Air Force components and Air Force headquarters’ directorates is needed to implement many commission recommendations. This coordination did not consistently occur because, according to Air Force officials, the level of the team leaders who manage each recommendation—typically Colonels or Lieutenant Colonels—lacked the authority to task personnel outside their office which meant that cooperation and coordination were generally dependent on persuasion rather than direction. According to Air Force officials, although the Total Force Continuum office manages the process to implement the commission’s recommendations, the team leaders and members generally belong to other organizations. For example, the team working on the recommendation to develop an integrated pay and personnel system includes representatives from the office of the Deputy Chief of Staff for Manpower, Personnel and Services (team leader), Air Force Personnel Center, Air Force Reserves, National Guard Bureau, and the office of the Assistant Secretary of the Air Force for Financial Management and Comptroller. In addition, implementation team roles have been an additional duty, rather than the sole duty for these team leaders and members. Finally, successful implementation of some recommendations depends on implementation of related recommendations according to the February 2015 Report on Recommendations of the National Commission on the Structure of the Air Force and Air Force officials. For example, a pilot program to integrate active and reserve component forces into an “Integrated Wing” needs to be established and tested before 6 other recommendations can be fully implemented, according to Air Force officials. The results of this pilot program will inform decisions on how to implement a range of recommendations such as policies and procedures concerning filling key deputy positions, and considerations for personnel awards, decorations and promotions.

In addition to these internal challenges, Air Force officials explained that they also encountered external challenges. Air Force officials stated that while they can take actions to make some progress in implementing many of the commission’s recommendations, full implementation of some recommendations depends on support from external agencies or Congress. For example, one recommendation is for the President to direct the Departments of Defense and Homeland Security to develop national requirements for Homeland Security and Disaster Assistance. Also, Air Force officials have identified a number of legislative actions necessary to fully implement seven commission recommendations. For example, they said that legislative action is necessary to fully implement one recommendation to develop a pilot project for “continuum of service”, that is, the ability for personnel to transition more seamlessly among the components. To fully implement another recommendation related to instructor pilots, the Air Force identified legislative action as necessary to permit reserve personnel and dual status military technicians to train active duty pilots as a primary duty.

The Air Force Is Revising Its Approach to Manage and Oversee Recommendation Implementation

The Air Force is revising its approach to manage and oversee implementation of the remaining 36 commission recommendations, and it expected to fully initiate this approach in March 2016. According to Air Force officials, the Air Force Assistant Vice Chief of Staff directed the use of a new approach that strategically groups related recommendations to facilitate management, oversight, and coordination. To do this, the Air Force aligned the commission’s recommendations with objectives in the Air Force Human Capital Annex of its Strategic Master Plan. According to Air Force officials, categorizing the recommendations under the objectives in the Human Capital Annex, will help the Air Force synchronize related efforts and thereby minimize the potential for overlapping efforts. The Air Force also designated a General Officer and/or civilian equivalent at the Senior Executive Service level to manage and oversee implementation of recommendations in each group. The General Officer or civilian equivalent at the Senior Executive Service level will periodically report the progress of his or her group of recommendations to the governance structure for the Human Capital Annex and the Executive Committee. Recommendations that did not align with any of the objectives were grouped under the Total Force Continuum office. Half (21) of the commission’s recommendations fall within the Total Force Continuum group, including recommendations related to increasing the number of reserve component instructor pilots, fielding equipment concurrently among active and reserve component, and identifying homeland security and disaster assistance requirements. The Total Force Continuum office will periodically report the progress on the recommendations to the Executive Committee, according to Air Force officials.

According to Air Force officials, the intent of this new structure is to provide the needed level of urgency, oversight to ensure accountability, and authority to provide direction across directorates and components. In addition, the Air Force intends to use implementation templates for each group of recommendations as well as each individual recommendation. The templates’ instructions explain that the structure is adapted from the structure for implementation-type plans discussed in an Air Force instruction. Upon reviewing the templates, we noted that they contain a requirement for milestones and are to specify the tasks that need to be completed to implement each recommendation. According to Executive Committee minutes and Air Force officials, recommendation milestones will be driven by, and aligned with, the Human Capital annex objectives’ milestones. Under the revised approach, recommendation implementation remains an additional duty for the team leaders and members.

Table 1 summarizes the Air Force’s approach to implementing the commission’s recommendations before and after the Air Force began to develop its revised approach.

Air Force’s Revised Approach Requires Tasks and Milestones, but Does Not Require Performance Measures Containing Key Attributes

The Air Force’s revised approach requires that tasks and milestones be developed to manage implementation of the commission’s recommendations, which is consistent with leading practices on program management and our prior work on business process reengineering. However, the revised approach does not require performance measures. Our prior work has demonstrated that using performance measures facilitates assessment of how goals are being achieved and can also identify areas for improvement, if needed.

Air Force officials explained that their revised approach requires aligning actions on related recommendations and identifying tasks and milestones for each recommendation. Leading practices for program management advocate grouping related projects in a coordinated way to maximize benefits, track actual against planned milestones, and to identify interrelationships among projects, as well as to monitor project performance to identify any needed modifications. In addition, our prior work and leading practices have shown that an implementation plan, consisting of tasks, milestones, and performance measures that contain key attributes can help organizations gauge progress toward achieving their desired results and can help leaders identify when corrective actions are needed.

We determined that the Air Force has made progress incorporating leading practices for milestones and tasks, but has not incorporated—and did not yet have plans to require—performance measures that include key attributes. To understand the extent to which the Air Force had previously incorporated leading practices, we reviewed implementation plans for 3 of the 36 recommendations that have not yet been implemented. These plans were developed before the Air Force adopted its revised management approach. We evaluated these three plans because, at the time we conducted our analysis, the Air Force had not developed any plans under its revised approach. Based on our analysis of these Air Force plans, we found that the Air Force had developed strategic milestones and partially identified tasks to achieve objectives in the 3 implementation plans it developed before adopting its revised management and oversight approach. In the implementation plans for the 3 recommendations we reviewed, the plans were not presented in a single document and the documents did not always identify milestones for all interim tasks.

Our prior work suggests that, ideally, objectives and measures should be described in a single document, such as an implementation plan, that defines how results can be measured. According to our Business Process Reengineering Assessment Guide, agencies undergoing business transformations should develop a detailed implementation plan that lays out what needs to be done to achieve implementation of new processes by identifying milestones and specifying timetables for all actions so that progress can be closely monitored. We have also reported that developing and using specific milestones to guide and gauge progress toward achieving an agency’s desired results informs management of the rate of progress toward achieving goals and whether adjustments need to be made to maintain progress within given time frames.

Also, our prior work on performance measurement and federal internal controls discusses using performance measures to assess performance over time. In 2012, we reported that federal agencies engaging in large projects can use performance measures to determine how well they are achieving their goals and to identify areas for improvement, if needed. We have found that by developing and tracking performance against a baseline for all measures, agencies can better evaluate progress and determine whether or not goals are being achieved. Through our prior work on performance measurement, we have identified key attributes of performance measures that can help managers monitor progress toward achieving program goals and priorities. Previous GAO work also indicates that agencies successful in measuring performance had performance measures that demonstrate results, are limited in number, cover multiple priorities, and provide useful information for decision making. Table 2 below summarizes selected attributes of performance measures and lists potential adverse consequences if attributes are missing.

We analyzed the three implementation plans developed under the Air Force’s original approach, to assess the extent to which the performance measures contained in the plans contained the key attributes discussed above. We found that the plans’ performance measures contained one or more of the seven relevant key attributes, but they did not incorporate all seven attributes. For example, the performance measures did not consistently include clear, measurable, objective measures and a baseline assessment of current performance. Moreover, because the commission’s recommendations were addressed individually and not grouped together at the time the measures were developed, we could not easily determine whether measures for related recommendations collectively addressed the Air Force’s priorities in a balanced manner.

The revised management approach, discussed in the Executive Committee’s December 2015 meeting, includes tasks and milestones, but not performance measures. According to Air Force officials, the milestones for each group are required to be developed according to a standardized template that the Total Force Continuum office developed. The implementation plan template contains an overall plan for the group of recommendations and annexes detailing plans for each recommendation within the group. According to Air Force officials, the information in the plan is to include: a discussion of the commission’s recommendation including relationships to other commission recommendations; strategic milestones and tasks with completion dates, assumptions, and constraints; and a project schedule with tasks, offices of primary responsibility, number of days for task completion and implementation dates. However, the template does not provide for performance measures. The milestones are due at the March 2016 Executive Committee meeting. According to Air Force officials, the milestones and the proposed template apply the basic principles contained in Air Force guidance on business case analysis procedures, which include linking tasks to specific, achievable milestones. According to Air Force officials, as of December 2015, the Air Force requires the use of project management charts that include program objectives and milestones specifying tasks that will have to be completed to implement each recommendation. Under the revised approach, project schedules will link milestones and tasks with responsible offices and identify due dates for completion of both strategic milestones and tactical tasks in a single document. According to Air Force officials, each month, the Executive Committee will review each group’s status and progress in achieving their tasks and milestones. In addition, at each monthly meeting, the Executive Committee will review in detail all the recommendations within one of the groups. Therefore, the Executive Committee will review each individual recommendation and any associated issues, tasks, and milestones at least twice a year.

Because the new, revised approach and the implementation plan template do not require development of performance measures, the Air Force will continue to lack critical information to oversee implementation of the commission’s recommendations and will not have full visibility to track progress. Moreover, by not requiring performance measures that contain key attributes in its implementation plans, the Air Force may be missing critical information that can be used to identify areas needing attention.

The Air Force Has Developed a Process to Assess the Mix of Reserve and Active Forces, Which It Uses to Inform Budget Planning

The Air Force developed a process to identify potential active and reserve component force mix options intended to better address and balance risks, costs, and sustainability. These options are presented to Air Force senior leaders for their consideration, and the leaders’ decisions inform the first phase of the multi-year budget development or Planning, Programming, Budgeting, and Execution process.

The Process to Develop Force Mix Options Combines Data Analysis and Stakeholder Input

Noting that the active and reserve component train to the same standards, the commission’s report discussed the advantages of shifting positions from the active to the reserve component, including saving money that could be used for readiness and investment. The Air Force developed its force mix option process to evaluate the mix of active and reserve component forces across the Air Force. As part of this process, the Air Force developed customized, classified data analyses for 44 aircraft types and mission areas. For example, the Air Force analyzed: bombers; the various aircraft used for the personnel recovery mission; and civil engineering, logistics, and medical services forces. The Air Force completed its initial analysis of all primary mission areas in December 2015, but it plans to periodically re-evaluate each analysis, because inputs to the analysis—such as requirements, cost data, and assumptions—change periodically.

The Total Force Continuum office within the Headquarters Air Force Directorate of Strategic Plans manages the force mix option process. The process combines data analysis and stakeholder inputs. According to officials from the Total Force Continuum office, the data analysis portion of the process uses authoritative data from established sources across the Air Force. For example major commands such as Air Combat Command provide flying hours data and the office of the Assistant Secretary of the Air Force for Financial Management and Comptroller provides cost data. Relevant stakeholders from across the Air Force (including Air Force Reserves, Air National Guard, various Air Force Headquarters directorates, and major commands), also have the opportunity to provide their comments, ideas, and suggestions at several points throughout the process. Using the data analysis, stakeholders develop and refine force mix options. The key components of developing force mix options— assumptions, data analysis, and stakeholder input— are discussed below.

Assumptions: According to officials who manage the process, the data analysis contains key assumptions concerning demand, readiness, and operational tempo. Demand is derived from Department of Defense (DOD)-approved planning scenarios. In building this analysis, the Air Force focuses on the surge period to determine whether or not it has enough forces, but it focuses on the “post-surge” period (i.e., the period of time after cessation of major combat operations when there is a continuous demand for forces to rotate in and out of the area for several years) to determine its appropriate mix of its forces. The Air Force assumes that all units are ready and available to perform their missions and also assumes that it will be able to comply with operational deployment guidance. The DOD goal for operational deployment to dwell ratio for active component forces is 1:2 or greater, meaning one deployment period would be followed by a non-deployed period that is at least twice as long. For the reserve component, the Air Force assumes it can meet a 1:5 mobilization to dwell ratio, meaning that one mobilization period would be followed by a period five times as long when the unit is not mobilized. Since the data in the workbooks is inter-connected, any change in the assumptions could affect the output. For example, if the reserve component’s mobilization to dwell ratio changes (e.g., from 1:5 to 1:10), then more active component forces may be needed to meet the demand. Also, if units are less than fully ready, then any gap between the supply of forces and the scenario’s rotational demand may increase, thereby increasing risk, according to Air Force officials. These assumptions are explained to senior leaders before the analysis results are presented.

Data analysis: For each aircraft type and mission area, the Air Force builds a classified, customized Excel workbook that contains a series of interconnected spreadsheets that contain hundreds of cells. Although each workbook is customized for each aircraft or mission area, they generally contain the same or similar types of data including: unit manpower by component; flying hours and cost per flying hour; direct and indirect personnel costs; and information on the current number of units; comparisons of unit supply and demand.

According to officials who manage the process, much of the data entered into the Excel workbooks comes from standard data sources. For example, flying hours data come from the approved training program for each aircraft type, and manpower data comes from unit manning documents. Stakeholders review the data content and analysis and provide inputs and corrections. However, there are limitations on how the information in the workbooks should be used, according to Air Force officials. For example, workbook information on location is generic and should not be used for basing decisions.

Stakeholder input: Stakeholders use the data analysis to develop force mix options. The Excel workbooks and spreadsheets contain the data analysis but do not automatically generate force mix options or predict combat effectiveness. Instead, Air Force personnel use the data analysis in the workbooks to identify and assess the options—(i.e., to see how a change in force mix may affect the capacity to meet the combatant commander’s continuing demand for forces rotating into and out of an area after cessation of major combat operations). Stakeholders assess the advantages and disadvantages of various options including any capacity gaps that could result and relative differences in costs. When the analyses are presented to decision makers, the decision makers may direct additional analysis or assess additional options. Figure 2 illustrates the Air Force’s force mix option process based on knowledgeable Air Force officials’ descriptions.

Force Mix Options Are Provided for Senior Leaders’ Consideration and Budget Planning

During the process, a number of force mix options are presented to Air Force senior leaders at decision meetings. The information presented to the senior leaders includes an explanation of key assumptions and a summary chart that contains force mix options, the proposed option, and views of stakeholders including the major commands, Air Force Reserves, and Air National Guard. Figure 3 below is a notional example of an output summary chart that is presented to senior leaders. From left to right, the figure shows: current forces and forces for force mix options. For each pair of bars, the first bar represents the capacity to rotate forces for normal operations. The second bar represents the capacity of active and reserve component forces to meet the combatant commander’s continuing demand for force rotations following major combat operations (labeled as “post-surge”). The figure also illustrates the difference, if any, between the supply of forces and the “post-surge” demand. Finally, the circle at the top of each pair of bars represents the relative cost of the option.

According to officials who manage the process, stakeholders’ views are presented and discussed at senior leader decision meetings, and stakeholders can raise any issue of importance to them including viability and risk. For example, stakeholders have raised issues such as difficulties recruiting reserve component forces in remote locations.

These officials explained that the force mix option process identifies inputs into the planning phase of the budget development process. For example, they said, the force mix option process supported consideration of moving some strategic airlift into the reserve component as the budget for fiscal year 2017 was developed and also informed a decision to increase the reserve component positions for civil engineering. However, it is not clear at this time how many of the proposed force mix options from all the analyses completed by December 2015 may ultimately be implemented since the budget development and execution cycle spans up to four years. Therefore, it is not clear at this time how much the overall mix of active and reserve component forces may change over time.

Conclusions

The Air Force has made progress by recently taking actions to improve the management and oversight of its implementation of the commission’s recommendations. These actions are aimed toward addressing the challenges the Air Force has experienced in implementing the recommendations such as coordination across offices, linking efforts on related recommendations, and setting deadlines for completing implementation. The Air Force’s revised approach includes aspects of leading program management and performance measurement practices—by using a template that requires the identification of tasks and milestones. However, the revised approach is new and unproven and does not require that performance measures be developed for each commission recommendation in order to assess progress and effects. While the three implementation plans developed under the original approach included performance measures, our analysis found that those measures lacked key attributes of performance measures. Without complete implementation plans that include performance measures, Air Force leaders may lack key information they could use to monitor progress and assess whether performance is meeting expectations for the 36 recommendations that are still open.

Recommendation for Executive Action

To facilitate implementation of the commission’s recommendations and provide managers with information to gauge progress and identify areas that may need attention, we recommend that the Secretary of the Air Force in coordination with the Chief of Staff of the Air Force direct the Assistant Vice Chief of Staff of the Air Force to develop complete implementation plans that include performance measures for all 36 commission recommendations that remain open.

Agency Comments

We provided a draft of this report to the Department of Defense (DOD) for review and comment. DOD’s comments, provided by the Air Force, are reproduced in appendix IV. The Air Force also provided technical comments which we have incorporated as appropriate.

The Air Force agreed with our recommendation to develop complete implementation plans that include performance measures for the 36 remaining open National Commission on the Structure of the Air Force (commission) recommendations. In its comments, the Air Force agreed that performance measures with the key attributes described in our report could provide valuable information useful in tracking the Air Force’s progress on the recommendations and identifying needed corrective actions. The Air Force estimated a completion date of March 2017 for developing performance measures.

We are sending copies of this report to the appropriate congressional committees and the Secretary of the Air Force. The report also is available at no charge on GAO’s website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-3489 or pendletonj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix V.

Appendix I: List of Recommendations in the National Commission on the Structure of the Air Force Report and the Status of Each Recommendation

Status (as of February 2016) Open cost” approach for calculating military personnel costs. 2. Budgeting Flexibility: Congress should allow DOD increased flexibility in applying budget cuts across budget categories, including installations. 3. Resourcing the Reserve Components: To ensure the Air Force leverages full capacity of all components of the force, the Air Force should plan, program, and budget for increased reliance on the reserve components.

Infrastructure: The Air Force should consider, and Congress should allow, the closing or warm basing of some installations. 5. Air Force Reserve Command: Congress should disestablish the Air Force Reserve 6. Staff Integration: The Air Force should integrate the existing staffs of all components. 7. Air Force Reserve (AFR) Unit Integration: The Chief of Staff of the Air Force should direct the integration of AFR units into corresponding active component organizations. 8. Full-Time and Part-Time Mix: The combination of full-time and part-time positions should be 9. Air National Guard (ANG) Unit Integration: The Chief of Staff of the Air Force should direct the integration of Air Force units into corresponding ANG organizations. 10. ANG Unit Size: The Chief of Staff of the Air Force, in coordination with the Director of the ANG, should change wing-level organizations to group organizations. 11. Concurrent Fielding of Equipment: As the Air Force acquires new equipment, force integration plans should adhere to the principle of proportional and concurrent fielding cross the components. 12. Policy Revisions: Integrating units will require manpower and personnel policy revisions. 13. Designated Operational Capability (DOC) Statements: The Air Force should discontinue the practice of separate DOC documents for active and reserve units of the same type and place the integrated units under single DOC statements. 14. Key Leadership Positions: The Air Force should ensure that integrated units are filled competitively by qualified airmen irrespective of component, but key deputy positions should always be filled by an “opposite” component member. 15. Effective Control Measures: The Air Force must establish effective control measures to ensure that both active and reserve component airmen have adequate paths and opportunities for advancement and career development. 16. Awards, Decorations, and Promotions: The integrated chain of command must take special care in managing personnel issues such as awards and decorations, promotions, and assignment opportunities.

Recommendation 17. Professional Military Education Positions: Commander, Air University should develop a new baseline for its student and instructor positions to achieve a proportionate representation of the components on faculty and student body by fiscal year 2018.

Status (as of February 2016) Open 18. Total Force Competency Standard: The Air Force should develop a Total Force competency standard across all specialties and career fields before the end of fiscal year 2016. 19. Access to Non-Resident Education: The Air Force should ensure that revised curriculum and competency standards are achievable by making non-resident education programs equally accessible to personnel of all components. 20. Increase ARC (Air Reserve Component) Capacity: The Air Force should increase its utilization of the reserve component by increasing the routine employment of reserve component units and individuals to meet recurring rotational requirements. 21. Operational ARC Funding: The Air Force should include in all future budget submissions a specific funding line for operational support by the reserve component. 22. Council of Governors: The Secretary of Defense should revise its agreement with the Council of Governors to enable Air Force leadership to consult directly with the Council when requested, including discussion of pre-decisional information 23. Non-Disclosure Agreements: The Air Force should discontinue the use of non-disclosure agreements in the corporate process. 24. State Adjutants General: The Air Force should continue to advance current informal means for engaging with The Adjutants General. 25. Cyberspace Airmen: The Air Force should fill much of the demand for Cyberspace career 26. Space Domain: The Air Force should build more reserve component opportunities in the 27. GIISR (Global Integrated Intelligence, Surveillance, and Reconnaissance) Billets: The Air Force should integrate all of its new GIISR units, and the preponderance of new billets should be for the reserve component. 28. Special Operations: The Air Force should increase reserve component presence in Special Operations through greater integration. 29. ICBM (Inter Continental Ballistic Missile) Mission: As a pilot program, the Air Force should expand reserve component contributions to the ICBM mission. 30. Instructor Pilots: The Air Force should replace some of the 1,800 active instructor pilots with prior-service volunteers from the Air Reserve Component who would not rotate back to operational squadrons. 31. Homeland Security and Disaster Assistance: The President should direct the Departments of Defense and Homeland Security to develop national requirements for Homeland Security and Disaster Assistance. 32. Homeland Defense and DSCA (Defense Support to Civil Authorities): DOD Air Force should treat Homeland Defense and Defense Support to Civil Authorities as real priorities. 33. Duty Statuses: Congress should reduce the number of separate duty statuses from more 34. Integrated Personnel Management: The Air Force should unify personnel management for all three components under a single integrated organization.

Recommendation Air Force Integrated Pay and Personnel System (AF-IPPS): The Air Force should accelerate the development of an integrated pay and personnel system.

PERSTEMPO (Personnel Tempo) Metric: The Air Force should use a single metric to measure the personnel tempo and stress on its active and reserve forces.

Non-Deployment PERSTEMPO: DOD should update the definition of a non-deployment PERSTEMPO event for the reserve component to include situations where the reserve component member is away from a civilian job or attendance at school.

PERSTEMPO and AF-IPPS: The Air Force should include PERSTEMPO accounting in AF-IPPS.

Continuum of Service: The Air Force should develop a pilot project for the implementation of Continuum of Service.

Active Duty Service Commitments: The Air Force should revise the rules for current active duty service commitments to enable members to meet the commitment in some combination of active, guard, and reserve service.

Multiple Career Track Options: The Air Force should develop a new service construct consisting of multiple career track options. “Up or Out”: Congress should amend restrictive aspects of current statutes that mandate “up or out” career management policies.

Appendix II: Objectives, Scope, and Methodology

This report (1) evaluates the extent to which the Air Force has made progress in implementing the commission’s recommendations and (2) describes how the Air Force has assessed the potential for increasing the proportion of reserve to active component forces as discussed in the commission’s report.

To evaluate the extent to which the Air Force has made progress in implementing the commission’s recommendations, we reviewed Air Force documents, such as briefings to the Executive Committee on the status of the commission’s recommendations, the Executive Committee Charter, and a draft template designed by the Air Force’s Total Force Continuum office for describing milestones, tasks, and other details related to implementing each group of recommendations and individual recommendations. We interviewed Air Force officials to understand their approach to manage and provide oversight for implementing the commission’s recommendations. We also reviewed Executive Committee minutes that documented discussions on the status of implementing the commission’s recommendations and documented decisions to close recommendations as implemented. We interviewed Air Force officials and team leaders to understand what actions they had taken to implement selected recommendations.

We selected a non-probability sample of seven of the 36 open recommendations by identifying those with the following attributes: (1) the Air Force had identified implementing the recommendations as facing challenges; (2) the reason for limited progress was not clear based on a review of Air Force status briefings; (3) at least one recommendation had a team leader from the reserve component; and (4) multiple recommendations where the implementation was led by the same team. While the descriptions of implementing the non-probability sample of seven selected recommendations cannot be projected to all 36 open recommendations, they do illustrate the Air Force’s original process for implementing the commission’s recommendations.

In addition, we analyzed the implementation plans that had been developed under the Air Force’s original approach for 3, inter-related recommendations to determine the extent to which these documents incorporated leading practices such as milestones, tasks, and performance measures. We evaluated these plans because, at the time we conducted our analysis, the Air Force had not developed any plans under its revised approach. Also, Air Force officials said that plans for these 3 recommendations were the only implementation plans that had been developed under the original approach. The documents constituting the implementation plan for the 3, inter-related recommendations were not contained in one consolidated document but instead consisted of multiple documents, such as action plans and other guidance-type documents that were identified by Air Force officials as implementation plans. For purposes of our analysis, we refer to these multiple documents for each of the 3 recommendations as “implementation plans”.

We compared the implementation plans for the 3 recommendations with leading practices for program management that included the use of milestones and tasks. We derived these practices by reviewing a combination of Air Force guidance, industry practices, and our prior work. We also compared the implementation plans with leading practices on performance measures which we derived from a combination of federal internal control standards, prior GAO work on performance measurement and planning; the Government Performance and Results Act (GPRA)—as updated by the GPRA Modernization Act of 2010; related guidance from the Office of Management and Budget; and Air Force guidance on business case analyses. Our prior work identified ten attributes of effective performance measures. Of the ten attributes associated with effective performance measures, we selected seven attributes against which to evaluate performance measures in the Air Force’s three implementation plans. We excluded the attributes for “government-wide priorities”, “core program activities,” and “linkage”, since we judged these attributes less relevant to the Air Force’s plans. Excluding these attributes would still yield a sound assessment. The seven attributes we selected would provide comprehensive information, over time, on how well the Air Force was progressing with its plans to implement the recommendations and identify areas for increased focus.

In our scorecard analyses, two GAO analysts independently conducted analyses of performance measures, milestones, and tasks described in each implementation plan. Any disagreements between the two assessments were discussed and reconciled by a third analyst. In our scorecard analysis of milestones and tasks, we determined that implementation plans included milestones when we were able to identify milestones in the plans. We determined that the implementation plans “addressed” inclusion of tasks tied to milestones if each task had a milestone associated with it. An implementation plan “partially addressed” inclusion of tasks tied to milestones if some, but not all of the tasks had an associated milestone. Finally, an implementation plan “did not address” inclusion of tasks tied to milestones if we determined that none of the tasks had a milestone tied to it. In our scorecard analysis of performance measures, we determined that a performance measure “addressed” an attribute when it included all elements of the attribute, even if it lacked specificity and details and could thus be improved upon. A performance measure “partially addressed” an attribute when it included more than one, but not all, elements of the attribute.

Consequently, our designation of “partially addressed,” may include a wide variation of, from one to six, demonstrated attributes. A performance measure “did not address” an attribute when it did not include or discuss any elements of the attribute, and/or any implicit references were either too vague or general to permit assessment. For the attribute “limited overlap” we determined that the attribute was “not applicable” if the implementation of the commission recommendation was not dependent on implementation of another recommendation. To supplement this analysis and gain further insight into issues of strategic import, we also interviewed cognizant officials from the Air Force team implementing the 3, inter-related recommendations that have implementation plans, team leaders and members for seven recommendations, representatives from the Air Force Reserves, Air National Guard, and Air Force headquarters staff from the Total Force Continuum office within the Headquarters Air Force Strategic Plans and Programs Directorate, and the office of the Assistant Secretary of the Air Force for Financial Management and Comptroller.

To describe how the Air Force has assessed the potential for increasing the proportion of reserve to active component forces as discussed in the commission’s report, we identified the scope, content, and process for the Air Force’s High Velocity Analysis (referred to in the report as the force mix option process) which is the process the Air Force developed to identify and assess force mix options. We first identified the scope of the Air Force analysis by reviewing documentation such as the Air Force analysis schedule and interviewing Air Force officials (from the Strategic Plans and Programs Directorate and the office of the Assistant Secretary of the Air Force, Financial Management and Comptroller). This work provided information on how individual analyses were developed and which aircraft and mission areas were analyzed. Since the Air Force had developed 55 Excel workbooks for analyzing 44 aircraft types and mission areas as of December 2015, we selected a non-probability sample to serve as illustrative examples and learn how the workbooks were built, identify data inputs and their sources, and identify who verifies data inputs, how data are verified, and how the workbooks are used to develop force mix options. From the universe of 44 aircraft types and mission areas, we selected three analyses that had the following attributes: the analysis was complete and not on-going; one that was for a combat mission, one that was for a combat support mission; and one that had a “deploy in place” mission. The three analyses we selected were: bombers, personnel recovery, and intercontinental ballistic mission forces. We did not trace a sample of the data in each workbook back to its original source documents or verify workbook formulas since we were not assessing the accuracy or reliability of the data or analyses. We did review documentation showing that the Air Force has steps in its process for stakeholders to review and modify workbook inputs. While the details of the three analyses we sampled cannot be projected to all 55 workbooks, the sample did enable us to describe the overall force mix option process, including the type of information presented to Air Force leadership.

Next, we reviewed examples of documentation of the results of the force mix option process such as examples of briefings presented to senior Air Force leadership to determine whether assumptions and limitations were presented to decision makers. Finally, we interviewed Air Force officials to understand how the Air Force has generally used the results to inform the budget development process and we reviewed an Air Force report and minutes from an Air Force leadership meeting to identify examples.

We conducted this performance audit from June 2015 to May 2016 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix III: Air Force Alignment of Commission Recommendations with Human Capital Annex Objectives

The Air Force aligned 21 of the commission’s 42 recommendations with four of the six objectives in the Air Force Human Capital Annex of the Strategic Master Plan. The Human Capital Annex objectives are: (1) Attracting and Recruiting; (2) Developing the Force; (3) Talent Management (4) Retaining Ready, Resilient Airmen and Families; (5) Agile, Inclusive and Innovative Institutions; and (6) One Air Force. According to Air Force officials, none of the commission’s recommendations aligned with two of the Human Capital objectives— Attracting and Recruiting or Retaining Ready, Resilient Airmen and Families. The Air Force categorized recommendations that do not fall within the Human Capital Annex objectives into a separate group that will be managed by the general officers from the Total Force Continuum Office. Table 3 below lists the commission’s recommendations that are aligned with four of the Human Capital Annex objectives as well as those that will be managed by the Total Force Continuum office.

Appendix IV: Comments from the Department of Defense

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, the following staff members make key contributions to this report: Michael Ferren, Assistant Director, Brenda M. Waterfield, Krislin M. Bolling, Grant Sutton, Barbara Wooten, Patricia Farrell Donahue, Anne Stevens, and Mike Shaughnessy.