Force Structure: Performance Measures Needed to Better Implement the Recommendations of the National Commission on the Structure of the Air Force
Published: 2016-05-06T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In January 2014, the National Commission on the Structure of the Air Force (commission) issued its report, which included 42 recommendations for improving how the Air Force manages its total force. The report also discussed the feasibility of shifting 36,600 personnel from the active to the reserve component and estimated that doing so could save $2 billion annually.
Senate Report 114-49 included a provision for GAO to review matters related to the Air Force's efforts to implement the commission's recommendations. This report (1) evaluates the extent to which the Air Force has made progress in implementing the commission's recommendations and (2) describes how the Air Force has assessed the potential for increasing the proportion of reserve to active component forces. GAO reviewed documentation of the Air Force's efforts and compared Air Force implementation plans with leading practices for program management derived from the public and private sectors and GAO's prior work. GAO also reviewed documentation and interviewed officials in order to describe the Air Force's process for assessing its active and reserve component mix.
section_title: What GAO Found
paragraphs: As of February 2016, the Air Force had made limited progress in implementing the commission's recommendations—it had closed 6 recommendations and had taken action to revise its approach for managing implementation of the remaining 36 open recommendations. Air Force officials encountered challenges as they began implementing the commission's recommendations which the revised approach may address. For example, the Air Force had difficulty coordinating across components and offices and coordinating among teams working on inter-related recommendations. Under the revised approach, the Air Force has grouped related recommendations together and placed responsibility for each group under senior officials to improve coordination. According to Air Force officials, the revised approach requires development of milestones and tasks for each recommendation but does not require development of performance measures. Federal internal control standards, leading program management practices, and GAO's prior work have shown that performance measures which contain key attributes—such as baseline and trend data—can help managers monitor progress toward achieving program goals and identify areas for corrective actions. Since the revised approach was not fully in place as of January 2016, the Air Force had not developed complete implementation plans with milestones, tasks, and performance measures to monitor and oversee progress on the remaining 36 open recommendations. Under its original management approach, the Air Force had developed implementation plans for 3 recommendations. These plans generally contained milestones and tasks but were incomplete, since they did not consistently include performance measures that were clear, measurable, or contained a baseline from which implementation progress could be measured. While the Air Force's revised approach includes some positive steps, it is new, its effectiveness is unknown, and it does not require performance measures to gauge progress. Without complete implementation plans that include performance measures which reflect the key attributes, the Air Force will continue to lack important information to monitor progress and assess whether performance is meeting expectations for the 36 recommendations that are still open.
Several of the commission's recommendations related to the feasibility of shifting a portion of the active to the reserve component forces. The Air Force has assessed potential changes to its force mix using a process it developed for this purpose. The process combines quantitative and qualitative analysis with stakeholder input and judgment to identify options for changing the mix of active and reserve component forces. These options are then presented to senior Air Force leaders for their consideration, and the leaders' decisions inform the planning phase of the budget development process. To support its process, the Air Force has developed customized, classified data analyses for 44 aircraft types and mission areas. The Air Force finished these analyses in December 2015, and officials said the results informed planning for the fiscal year 2018 budget. However, since the budget development and execution cycle spans up to four years, it is not clear at this time how many of the proposed force mix options may ultimately be implemented.
section_title: What GAO Recommends
paragraphs: GAO recommends that, for the 36 remaining open commission recommendations, the Air Force develop complete implementation plans that include performance measures. The Air Force agreed with GAO's recommendation.

section_title: Letter
paragraphs: Following disagreements over the Air Force’s proposals to reduce aircraft  and Air National Guard end strength, the National Defense Authorization  Act for Fiscal Year 2013 established the National Commission on the  Structure of the Air Force (hereafter referred to as the commission). The  act required the commission to conduct a study to determine whether,  and how, the Air Force structure should be modified to best fulfill current  and anticipated mission requirements in a manner consistent with  available resources. In January 2014, the commission issued its final  report, which included 42 recommendations. The commission’s report  also discussed the feasibility of shifting 36,600 personnel from the active  component into the reserve component—the Air National Guard and Air  Force Reserve—and estimated that doing so could save $2 billion  annually in manpower costs with no reduction in total force end strength.  The Air Force agreed with 41 of the 42 commission’s recommendations.  The recommendations focused on a range of topics from personnel  policies and systems to determining the appropriate balance between the  active and reserve component.
Section 1055 of the Carl Levin and Howard P. “Buck” McKeon National  Defense Authorization Act for Fiscal Year 2015 required that the Air Force  provide the congressional defense committees with annual reports— following submittal of the President’s Budgets for fiscal years 2016  through 2019—on the response of the Air Force to the 42  recommendations. Section 1055 required that the first report set forth  specific milestones for Air Force review of the recommendations, and  preliminary implementation plans for recommendations that do not require  further review. The Air Force issued its initial report to the congressional  defense committees in February 2015. However, in its report  accompanying a bill for the National Defense Authorization Act for Fiscal  Year 2016, the Senate Armed Services Committee expressed concern  that the Air Force had not provided discernible milestones or preliminary  implementation plans. The committee report also noted that the Air Force  report did not include analysis from the process it used to develop the  active and reserve component force mix options in response to some of  the commission’s recommendations. The committee report included a  provision that we review the Air Force’s methodology and effectiveness in  its effort to plan for and implement the commission’s recommendations.  Our report (1) evaluates the extent to which the Air Force has made  progress in implementing the commission’s recommendations and (2)  describes how the Air Force has assessed the potential for increasing the  proportion of reserve to active component forces, as discussed in the  commission’s report.
To evaluate the extent to which the Air Force has made progress in  implementing the commission’s recommendations, we reviewed related  Air Force documents, such as briefings to the Executive Committee on  the status of the commission’s recommendations, and the Executive  Committee Charter. We interviewed Air Force officials to understand their  approach to managing and providing oversight for implementing the  commission’s recommendations. We also reviewed Executive Committee  minutes and interviewed Air Force officials about their efforts to  implement selected recommendations. We judgmentally selected a  sample of open recommendations that, as a group, had the following  attributes: Air Force officials had identified implementing the  recommendations as facing challenges; the reason for limited progress  was not clear based on a review of an Air Force status briefing; included  at least one recommendation with a team leader from the reserve  component; and included multiple recommendations where the  implementation was led by the same team. In addition, we compared the  implementation plans that had been developed as of September 2015 to  leading practices on program management including identifying  milestones, tasks, and performance measures. We identified these  selected leading practices on milestones, tasks, and performance  measures from a combination of prior GAO work on performance  measurement and planning, the Government Performance and Results  Act (GPRA)—as updated by the GPRA Modernization Act of 2010— related guidance from the Office of Management and Budget, Air Force  guidance on business case analyses, and federal internal control  standards. In December 2015, Air Force officials confirmed that no  additional implementation plans had been developed since September  2015.
To describe how the Air Force has assessed the potential for increasing  the proportion of reserve to active component forces as discussed in the  commission’s report, we identified the scope of the Air Force’s analysis by  reviewing documentation such as the Air Force analysis schedule and by  interviewing Air Force officials. We reviewed documentation of the results  of the force mix option process such as briefings presented to senior Air  Force leaders and the minutes of a decision forum meeting. We also  interviewed Air Force officials to understand how they conduct and use  their force mix option process—which the Air Force calls its High Velocity  Analysis—to identify and assess force mix options. Finally, we  interviewed Air Force officials regarding how the results of this process  had been used to inform budget development. See appendix II for a  detailed description of our scope and methodology.
We conducted this performance audit from June 2015 to May 2016 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: The National Defense Authorization Act for Fiscal Year 2013 established  the National Commission on the Structure of the Air Force. The act  required the commission to undertake a comprehensive study of the  structure of the Air Force to determine whether, and how, the structure  should be modified to best fulfill current and anticipated mission  requirements for the Air Force in a manner consistent with available  resources. The commission was to give particular consideration to  evaluating a structure that achieved certain things, including an  appropriate balance between the active and reserve component of the Air  Force.
In January 2014, the commission submitted its report to the President and  the House and Senate Armed Services Committees with 42  recommendations that varied in size, scope, and complexity. For  example, one relatively straightforward recommendation was to  discontinue the use of non-disclosure agreements in the corporate  (budget) process. In contrast, one large and complex recommendation  was to integrate the headquarters staffs of the components. In addition,  the commission’s recommendations were addressed to different  entities—the President (one recommendation), Congress (five  recommendations), Secretary of Defense (four recommendations), and  Secretary of the Air Force (32 recommendations). For example, one  recommendation was for Congress to allow the closing or “warm  basing” of some installations. Section 1055 of the Carl Levin and  Howard P. “Buck” McKeon National Defense Authorization Act for Fiscal  Year 2015 requires that the Secretary of the Air Force submit to the  congressional defense committees a yearly report on the Air Force  response to the commission’s recommendations. In February 2015, the  Air Force submitted its first response to the commission’s  recommendations; and, submitted its second response in February  2016.
subsections: section_title: Organizations Involved in Implementing the Commission’s Recommendations
paragraphs: In January 2013, the Air Force created the Total Force Task Force to  identify options for integrating the active and reserve component to meet  current and future Air Force requirements. The Task Force transitioned  into the Total Force Continuum office within the Headquarters Air Force  Strategic Plans and Programs Directorate. The Total Force Continuum  office is led by general officers from each component, who, according to  Air Force officials, manage and oversee the process for implementing the  commission’s recommendations. Further, officials within the Total Force  Continuum office work with other Air Force organizations to implement  the commission’s recommendations. For example, according to Air Force  documentation on the status of the commission’s recommendation, the  team working to develop a cost model for calculating military personnel  costs included representatives from the office of the Assistant Secretary  of the Air Force for Financial Management and Comptroller; Air Force  Reserve Command; and the office of the Director, Air Force Studies,  Analyses, and Assessments.
On July 3, 2014, the Air Force established the Executive Committee,  which is chaired by the Air Force Assistant Vice Chief of Staff, and  includes senior leaders from Air Force headquarters offices and the  reserve component. According to Air Force officials from the Total Force  Continuum office, the Executive Committee tracks and acts on the  commission’s recommendations, and provides regular updates to the  Secretary of the Air Force and Chief of Staff of the Air Force.
section_title: Original Process for Implementing the Commission’s Recommendations
paragraphs: The Total Force Continuum office established a team for each  commission recommendation comprised of representatives from each  component. According to Air Force officials, each team is led by a  Colonel, Lieutenant Colonel, or civilian equivalent and is responsible for  reviewing the recommendation and proposing implementation actions.  The teams present their proposed actions to the one-star general officers  within the Total Force Continuum office. According to Air Force officials,  upon the general officers’ approval, the team leader and general officers  brief the Executive Committee. The Executive Committee may approve  the proposed action or may direct the team to further study or modify the  proposed actions. Semi-annually, the Executive Committee briefs the  Secretary of the Air Force and Chief of Staff of the Air Force on the  proposed implementation actions. According to Air Force officials, the  Secretary and Chief of Staff may approve implementation of the proposed  action, close the recommendation, or direct the team to do more work.  Figure 1 illustrates this process.
section_title: Air Force Strategic Master Plan Human Capital Annex
paragraphs: In May 2015, the Air Force issued its Strategic Master Plan, which is  intended, in part, to align activities across the Air Force in the areas of  human capital, strategic posture, capabilities, and science and  technology. The Strategic Master Plan contains annexes for each of  these areas, which are intended to translate the Strategic Master Plan’s  goals and objectives into tangible actions and priorities. The Strategic  Master Plan has an overall goal for considering all components as a Total  Force. The Human Capital Annex expands upon the concept of a total  force and specifies goals, objectives, and timeframes for recruiting and  training, retention, component integration, and other topics. Twenty-one  of the commission’s recommendations are aligned with four of the six  objectives in the Human Capital Annex, as part of the revised approach  and as illustrated in appendix III to this report. For example, the Annex’s  goals for career progression are similar to the commission’s  recommendations on promotions, continuum of service, and multiple  career track options. Also, the Annex’s goals for increasing integration  among the components are similar to the commission’s recommendations  for integrating headquarters’ staffs, integrating personnel systems, and  making transition between components easier.
section_title: The Air Force Has Made Limited Progress Developing Plans to Implement the Commission’s Recommendations and Is Revising Its Management Approach
paragraphs: As of February 2016, two years after the commission issued its report, the  Air Force closed six recommendations—five were implemented and the  Air Force did not agree with the sixth. Thus, the Air Force still has to  implement 36 of the 42 recommendations. The Air Force has only  developed partial implementation plans for three of these 36  recommendations, which remain open.
subsections: section_title: The Air Force Encountered Challenges as It Began Implementing the Commission’s Recommendations According to Officials
paragraphs: Air Force officials from the Total Force Continuum office explained that  the Air Force experienced challenges internal and external to the service  that affected implementation of the commission’s recommendations.  According to Air Force officials, internal challenges included: expected  completion dates not identified; difficulties in coordinating implementation  efforts across offices; and inter-related recommendations. First, neither  the commission nor the Air Force established time frames for completing  implementation of all the commission’s recommendations. The lack of an  overall implementation time frame, or time frames for most individual  recommendations, may have conveyed the idea that implementation was  open-ended, since there was generally no clear end-date to work toward,  according to Total Force Continuum Office officials. Second, the Air Force  experienced challenges in coordinating implementation efforts across  components and directorates. According to Air Force officials, extensive  coordination and cooperation across the Air Force components and Air  Force headquarters’ directorates is needed to implement many  commission recommendations. This coordination did not consistently  occur because, according to Air Force officials, the level of the team  leaders who manage each recommendation—typically Colonels or  Lieutenant Colonels—lacked the authority to task personnel outside their  office which meant that cooperation and coordination were generally  dependent on persuasion rather than direction. According to Air Force  officials, although the Total Force Continuum office manages the process  to implement the commission’s recommendations, the team leaders and  members generally belong to other organizations. For example, the team  working on the recommendation to develop an integrated pay and  personnel system includes representatives from the office of the Deputy  Chief of Staff for Manpower, Personnel and Services (team leader), Air  Force Personnel Center, Air Force Reserves, National Guard Bureau,  and the office of the Assistant Secretary of the Air Force for Financial  Management and Comptroller. In addition, implementation team roles  have been an additional duty, rather than the sole duty for these team  leaders and members. Finally, successful implementation of some  recommendations depends on implementation of related  recommendations according to the February 2015 Report on  Recommendations of the National Commission on the Structure of the Air  Force and Air Force officials. For example, a pilot program to integrate  active and reserve component forces into an “Integrated Wing” needs to  be established and tested before 6 other recommendations can be fully  implemented, according to Air Force officials. The results of this pilot  program will inform decisions on how to implement a range of  recommendations such as policies and procedures concerning filling key  deputy positions, and considerations for personnel awards, decorations  and promotions.
In addition to these internal challenges, Air Force officials explained that  they also encountered external challenges. Air Force officials stated that  while they can take actions to make some progress in implementing many  of the commission’s recommendations, full implementation of some  recommendations depends on support from external agencies or  Congress. For example, one recommendation is for the President to  direct the Departments of Defense and Homeland Security to develop  national requirements for Homeland Security and Disaster Assistance.  Also, Air Force officials have identified a number of legislative actions  necessary to fully implement seven commission recommendations. For  example, they said that legislative action is necessary to fully implement  one recommendation to develop a pilot project for “continuum of service”,  that is, the ability for personnel to transition more seamlessly among the  components. To fully implement another recommendation related to  instructor pilots, the Air Force identified legislative action as necessary to  permit reserve personnel and dual status military technicians to train  active duty pilots as a primary duty.
section_title: The Air Force Is Revising Its Approach to Manage and Oversee Recommendation Implementation
paragraphs: The Air Force is revising its approach to manage and oversee  implementation of the remaining 36 commission recommendations, and it  expected to fully initiate this approach in March 2016. According to Air  Force officials, the Air Force Assistant Vice Chief of Staff directed the use  of a new approach that strategically groups related recommendations to  facilitate management, oversight, and coordination. To do this, the Air  Force aligned the commission’s recommendations with objectives in the  Air Force Human Capital Annex of its Strategic Master Plan. According  to Air Force officials, categorizing the recommendations under the  objectives in the Human Capital Annex, will help the Air Force  synchronize related efforts and thereby minimize the potential for  overlapping efforts. The Air Force also designated a General Officer  and/or civilian equivalent at the Senior Executive Service level to manage  and oversee implementation of recommendations in each group. The  General Officer or civilian equivalent at the Senior Executive Service level  will periodically report the progress of his or her group of  recommendations to the governance structure for the Human Capital  Annex and the Executive Committee. Recommendations that did not  align with any of the objectives were grouped under the Total Force  Continuum office. Half (21) of the commission’s recommendations fall  within the Total Force Continuum group, including recommendations  related to increasing the number of reserve component instructor pilots,  fielding equipment concurrently among active and reserve component,  and identifying homeland security and disaster assistance requirements.  The Total Force Continuum office will periodically report the progress on  the recommendations to the Executive Committee, according to Air Force  officials.
According to Air Force officials, the intent of this new structure is to  provide the needed level of urgency, oversight to ensure accountability,  and authority to provide direction across directorates and components. In  addition, the Air Force intends to use implementation templates for each  group of recommendations as well as each individual recommendation.  The templates’ instructions explain that the structure is adapted from the  structure for implementation-type plans discussed in an Air Force  instruction. Upon reviewing the templates, we noted that they contain a  requirement for milestones and are to specify the tasks that need to be  completed to implement each recommendation. According to Executive  Committee minutes and Air Force officials, recommendation milestones  will be driven by, and aligned with, the Human Capital annex objectives’  milestones. Under the revised approach, recommendation  implementation remains an additional duty for the team leaders and  members.
Table 1 summarizes the Air Force’s approach to implementing the  commission’s recommendations before and after the Air Force began to  develop its revised approach.
section_title: Air Force’s Revised Approach Requires Tasks and Milestones, but Does Not Require Performance Measures Containing Key Attributes
paragraphs: The Air Force’s revised approach requires that tasks and milestones be  developed to manage implementation of the commission’s  recommendations, which is consistent with leading practices on program  management and our prior work on business process reengineering.  However, the revised approach does not require performance measures.  Our prior work has demonstrated that using performance measures  facilitates assessment of how goals are being achieved and can also  identify areas for improvement, if needed.
Air Force officials explained that their revised approach requires aligning  actions on related recommendations and identifying tasks and milestones  for each recommendation. Leading practices for program management  advocate grouping related projects in a coordinated way to maximize  benefits, track actual against planned milestones, and to identify  interrelationships among projects, as well as to monitor project  performance to identify any needed modifications. In addition, our prior  work and leading practices have shown that an implementation plan,  consisting of tasks, milestones, and performance measures that contain  key attributes can help organizations gauge progress toward achieving  their desired results and can help leaders identify when corrective actions  are needed.
We determined that the Air Force has made progress incorporating  leading practices for milestones and tasks, but has not incorporated—and  did not yet have plans to require—performance measures that include  key attributes. To understand the extent to which the Air Force had  previously incorporated leading practices, we reviewed implementation  plans for 3 of the 36 recommendations that have not yet been  implemented. These plans were developed before the Air Force adopted  its revised management approach. We evaluated these three plans  because, at the time we conducted our analysis, the Air Force had not  developed any plans under its revised approach. Based on our analysis  of these Air Force plans, we found that the Air Force had developed  strategic milestones and partially identified tasks to achieve objectives in  the 3 implementation plans it developed before adopting its revised  management and oversight approach. In the implementation plans for the  3 recommendations we reviewed, the plans were not presented in a  single document and the documents did not always identify milestones for  all interim tasks.
Our prior work suggests that, ideally, objectives and measures should be  described in a single document, such as an implementation plan, that  defines how results can be measured. According to our Business Process  Reengineering Assessment Guide, agencies undergoing business  transformations should develop a detailed implementation plan that lays  out what needs to be done to achieve implementation of new processes  by identifying milestones and specifying timetables for all actions so that  progress can be closely monitored. We have also reported that  developing and using specific milestones to guide and gauge progress  toward achieving an agency’s desired results informs management of the  rate of progress toward achieving goals and whether adjustments need to  be made to maintain progress within given time frames.
Also, our prior work on performance measurement and federal internal  controls discusses using performance measures to assess performance  over time. In 2012, we reported that federal agencies engaging in large  projects can use performance measures to determine how well they are  achieving their goals and to identify areas for improvement, if needed.  We have found that by developing and tracking performance against a  baseline for all measures, agencies can better evaluate progress and  determine whether or not goals are being achieved. Through our prior  work on performance measurement, we have identified key attributes of  performance measures that can help managers monitor progress toward  achieving program goals and priorities. Previous GAO work also  indicates that agencies successful in measuring performance had  performance measures that demonstrate results, are limited in number,  cover multiple priorities, and provide useful information for decision  making. Table 2 below summarizes selected attributes of performance  measures and lists potential adverse consequences if attributes are  missing.
We analyzed the three implementation plans developed under the Air  Force’s original approach, to assess the extent to which the performance  measures contained in the plans contained the key attributes discussed  above. We found that the plans’ performance measures contained one or  more of the seven relevant key attributes, but they did not incorporate all  seven attributes. For example, the performance measures did not  consistently include clear, measurable, objective measures and a  baseline assessment of current performance. Moreover, because the  commission’s recommendations were addressed individually and not  grouped together at the time the measures were developed, we could not  easily determine whether measures for related recommendations  collectively addressed the Air Force’s priorities in a balanced manner.
The revised management approach, discussed in the Executive  Committee’s December 2015 meeting, includes tasks and milestones, but  not performance measures. According to Air Force officials, the  milestones for each group are required to be developed according to a  standardized template that the Total Force Continuum office developed.  The implementation plan template contains an overall plan for the group  of recommendations and annexes detailing plans for each  recommendation within the group. According to Air Force officials, the  information in the plan is to include: a discussion of the commission’s  recommendation including relationships to other commission  recommendations; strategic milestones and tasks with completion dates,  assumptions, and constraints; and a project schedule with tasks, offices  of primary responsibility, number of days for task completion and  implementation dates. However, the template does not provide for  performance measures. The milestones are due at the March 2016  Executive Committee meeting. According to Air Force officials, the  milestones and the proposed template apply the basic principles  contained in Air Force guidance on business case analysis procedures,  which include linking tasks to specific, achievable milestones. According  to Air Force officials, as of December 2015, the Air Force requires the use  of project management charts that include program objectives and  milestones specifying tasks that will have to be completed to implement  each recommendation. Under the revised approach, project schedules  will link milestones and tasks with responsible offices and identify due  dates for completion of both strategic milestones and tactical tasks in a  single document. According to Air Force officials, each month, the  Executive Committee will review each group’s status and progress in  achieving their tasks and milestones. In addition, at each monthly  meeting, the Executive Committee will review in detail all the  recommendations within one of the groups. Therefore, the Executive  Committee will review each individual recommendation and any  associated issues, tasks, and milestones at least twice a year.
Because the new, revised approach and the implementation plan  template do not require development of performance measures, the Air  Force will continue to lack critical information to oversee implementation  of the commission’s recommendations and will not have full visibility to  track progress. Moreover, by not requiring performance measures that  contain key attributes in its implementation plans, the Air Force may be  missing critical information that can be used to identify areas needing  attention.
section_title: The Air Force Has Developed a Process to Assess the Mix of Reserve and Active Forces, Which It Uses to Inform Budget Planning
paragraphs: The Air Force developed a process to identify potential active and reserve  component force mix options intended to better address and balance  risks, costs, and sustainability. These options are presented to Air Force  senior leaders for their consideration, and the leaders’ decisions inform  the first phase of the multi-year budget development or Planning,  Programming, Budgeting, and Execution process.
subsections: section_title: The Process to Develop Force Mix Options Combines Data Analysis and Stakeholder Input
paragraphs: Noting that the active and reserve component train to the same  standards, the commission’s report discussed the advantages of shifting  positions from the active to the reserve component, including saving  money that could be used for readiness and investment. The Air Force  developed its force mix option process to evaluate the mix of active and  reserve component forces across the Air Force. As part of this process,  the Air Force developed customized, classified data analyses for 44  aircraft types and mission areas. For example, the Air Force analyzed:  bombers; the various aircraft used for the personnel recovery mission;  and civil engineering, logistics, and medical services forces. The Air  Force completed its initial analysis of all primary mission areas in  December 2015, but it plans to periodically re-evaluate each analysis,  because inputs to the analysis—such as requirements, cost data, and  assumptions—change periodically.
The Total Force Continuum office within the Headquarters Air Force  Directorate of Strategic Plans manages the force mix option process. The  process combines data analysis and stakeholder inputs. According to  officials from the Total Force Continuum office, the data analysis portion  of the process uses authoritative data from established sources across  the Air Force. For example major commands such as Air Combat  Command provide flying hours data and the office of the Assistant  Secretary of the Air Force for Financial Management and Comptroller  provides cost data. Relevant stakeholders from across the Air Force  (including Air Force Reserves, Air National Guard, various Air Force  Headquarters directorates, and major commands), also have the  opportunity to provide their comments, ideas, and suggestions at several  points throughout the process. Using the data analysis, stakeholders  develop and refine force mix options. The key components of developing  force mix options— assumptions, data analysis, and stakeholder input— are discussed below.
Assumptions: According to officials who manage the process, the data  analysis contains key assumptions concerning demand, readiness, and  operational tempo. Demand is derived from Department of Defense  (DOD)-approved planning scenarios. In building this analysis, the Air  Force focuses on the surge period to determine whether or not it has  enough forces, but it focuses on the “post-surge” period (i.e., the period of  time after cessation of major combat operations when there is a  continuous demand for forces to rotate in and out of the area for several  years) to determine its appropriate mix of its forces. The Air Force  assumes that all units are ready and available to perform their missions  and also assumes that it will be able to comply with operational  deployment guidance. The DOD goal for operational deployment to dwell  ratio for active component forces is 1:2 or greater, meaning one  deployment period would be followed by a non-deployed period that is at  least twice as long. For the reserve component, the Air Force assumes it  can meet a 1:5 mobilization to dwell ratio, meaning that one mobilization  period would be followed by a period five times as long when the unit is  not mobilized. Since the data in the workbooks is inter-connected, any  change in the assumptions could affect the output. For example, if the  reserve component’s mobilization to dwell ratio changes (e.g., from 1:5 to  1:10), then more active component forces may be needed to meet the  demand. Also, if units are less than fully ready, then any gap between the  supply of forces and the scenario’s rotational demand may increase,  thereby increasing risk, according to Air Force officials. These  assumptions are explained to senior leaders before the analysis results  are presented.
Data analysis: For each aircraft type and mission area, the Air Force  builds a classified, customized Excel workbook that contains a series of  interconnected spreadsheets that contain hundreds of cells. Although  each workbook is customized for each aircraft or mission area, they  generally contain the same or similar types of data including: unit manpower by component; flying hours and cost per flying hour; direct and indirect personnel costs; and information on the current number of units;  comparisons of unit supply and demand.
According to officials who manage the process, much of the data entered  into the Excel workbooks comes from standard data sources. For  example, flying hours data come from the approved training program for  each aircraft type, and manpower data comes from unit manning  documents. Stakeholders review the data content and analysis and  provide inputs and corrections. However, there are limitations on how the  information in the workbooks should be used, according to Air Force  officials. For example, workbook information on location is generic and  should not be used for basing decisions.
Stakeholder input: Stakeholders use the data analysis to develop force  mix options. The Excel workbooks and spreadsheets contain the data  analysis but do not automatically generate force mix options or predict  combat effectiveness. Instead, Air Force personnel use the data analysis  in the workbooks to identify and assess the options—(i.e., to see how a  change in force mix may affect the capacity to meet the combatant  commander’s continuing demand for forces rotating into and out of an  area after cessation of major combat operations). Stakeholders assess  the advantages and disadvantages of various options including any  capacity gaps that could result and relative differences in costs. When the  analyses are presented to decision makers, the decision makers may  direct additional analysis or assess additional options. Figure 2 illustrates  the Air Force’s force mix option process based on knowledgeable Air  Force officials’ descriptions.
section_title: Force Mix Options Are Provided for Senior Leaders’ Consideration and Budget Planning
paragraphs: During the process, a number of force mix options are presented to Air  Force senior leaders at decision meetings. The information presented to  the senior leaders includes an explanation of key assumptions and a  summary chart that contains force mix options, the proposed option, and  views of stakeholders including the major commands, Air Force  Reserves, and Air National Guard. Figure 3 below is a notional example  of an output summary chart that is presented to senior leaders. From left  to right, the figure shows: current forces and forces for force mix options.  For each pair of bars, the first bar represents the capacity to rotate forces  for normal operations. The second bar represents the capacity of active  and reserve component forces to meet the combatant commander’s  continuing demand for force rotations following major combat operations  (labeled as “post-surge”). The figure also illustrates the difference, if any,  between the supply of forces and the “post-surge” demand. Finally, the  circle at the top of each pair of bars represents the relative cost of the  option.
According to officials who manage the process, stakeholders’ views are  presented and discussed at senior leader decision meetings, and  stakeholders can raise any issue of importance to them including viability  and risk. For example, stakeholders have raised issues such as  difficulties recruiting reserve component forces in remote locations.
These officials explained that the force mix option process identifies  inputs into the planning phase of the budget development process. For  example, they said, the force mix option process supported consideration  of moving some strategic airlift into the reserve component as the budget  for fiscal year 2017 was developed and also informed a decision to  increase the reserve component positions for civil engineering. However,  it is not clear at this time how many of the proposed force mix options  from all the analyses completed by December 2015 may ultimately be  implemented since the budget development and execution cycle spans  up to four years. Therefore, it is not clear at this time how much the  overall mix of active and reserve component forces may change over  time.
section_title: Conclusions
paragraphs: The Air Force has made progress by recently taking actions to improve  the management and oversight of its implementation of the commission’s  recommendations. These actions are aimed toward addressing the  challenges the Air Force has experienced in implementing the  recommendations such as coordination across offices, linking efforts on  related recommendations, and setting deadlines for completing  implementation. The Air Force’s revised approach includes aspects of  leading program management and performance measurement  practices—by using a template that requires the identification of tasks and  milestones. However, the revised approach is new and unproven and  does not require that performance measures be developed for each  commission recommendation in order to assess progress and effects.  While the three implementation plans developed under the original  approach included performance measures, our analysis found that those  measures lacked key attributes of performance measures. Without  complete implementation plans that include performance measures, Air  Force leaders may lack key information they could use to monitor  progress and assess whether performance is meeting expectations for  the 36 recommendations that are still open.
section_title: Recommendation for Executive Action
paragraphs: To facilitate implementation of the commission’s recommendations and  provide managers with information to gauge progress and identify areas  that may need attention, we recommend that the Secretary of the Air  Force in coordination with the Chief of Staff of the Air Force direct the  Assistant Vice Chief of Staff of the Air Force to develop complete  implementation plans that include performance measures for all 36  commission recommendations that remain open.
section_title: Agency Comments
paragraphs: We provided a draft of this report to the Department of Defense (DOD) for  review and comment.  DOD’s comments, provided by the Air Force, are  reproduced in appendix IV.  The Air Force also provided technical  comments which we have incorporated as appropriate.
The Air Force agreed with our recommendation to develop complete  implementation plans that include performance measures for the 36  remaining open National Commission on the Structure of the Air Force  (commission) recommendations.  In its comments, the Air Force agreed  that performance measures with the key attributes described in our report  could provide valuable information useful in tracking the Air Force’s  progress on the recommendations and identifying needed corrective  actions.  The Air Force estimated a completion date of March 2017 for  developing performance measures.
We are sending copies of this report to the appropriate congressional  committees and the Secretary of the Air Force. The report also is  available at no charge on GAO’s website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-3489 or pendletonj@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. Key contributors to this report are listed in  appendix V.
section_title: Appendix I: List of Recommendations in the National Commission on the Structure of the Air Force Report and the Status of Each Recommendation
paragraphs: Status  (as of February 2016)  Open  cost” approach for calculating military personnel costs.   2.  Budgeting Flexibility: Congress should allow DOD increased flexibility in applying budget cuts  across budget categories, including installations.  3.  Resourcing the Reserve Components: To ensure the Air Force leverages full capacity of all  components of the force, the Air Force should plan, program, and budget for increased  reliance on the reserve components.
Infrastructure: The Air Force should consider, and Congress should allow, the closing or  warm basing of some installations.   5.  Air Force Reserve Command: Congress should disestablish the Air Force Reserve  6.  Staff Integration: The Air Force should integrate the existing staffs of all components.   7.  Air Force Reserve (AFR) Unit Integration: The Chief of Staff of the Air Force should direct the  integration of AFR units into corresponding active component organizations.  8.  Full-Time and Part-Time Mix: The combination of full-time and part-time positions should be  9.  Air National Guard (ANG) Unit Integration: The Chief of Staff of the Air Force should direct  the integration of Air Force units into corresponding ANG organizations.   10.  ANG Unit Size: The Chief of Staff of the Air Force, in coordination with the Director of the  ANG, should change wing-level organizations to group organizations.  11.  Concurrent Fielding of Equipment: As the Air Force acquires new equipment, force  integration plans should adhere to the principle of proportional and concurrent fielding cross  the components.   12.  Policy Revisions: Integrating units will require manpower and personnel policy revisions.   13.  Designated Operational Capability (DOC) Statements: The Air Force should discontinue the  practice of separate DOC documents for active and reserve units of the same type and place  the integrated units under single DOC statements.  14.  Key Leadership Positions: The Air Force should ensure that integrated units are filled  competitively by qualified airmen irrespective of component, but key deputy positions should  always be filled by an “opposite” component member.  15.  Effective Control Measures: The Air Force must establish effective control measures to  ensure that both active and reserve component airmen have adequate paths and  opportunities for advancement and career development.  16.  Awards, Decorations, and Promotions: The integrated chain of command must take special  care in managing personnel issues such as awards and decorations, promotions, and  assignment opportunities.
Recommendation  17.  Professional Military Education Positions: Commander, Air University should develop a new  baseline for its student and instructor positions to achieve a proportionate representation of  the components on faculty and student body by fiscal year 2018.
Status  (as of February 2016)  Open  18.  Total Force Competency Standard: The Air Force should develop a Total Force competency  standard across all specialties and career fields before the end of fiscal year 2016.   19.  Access to Non-Resident Education: The Air Force should ensure that revised curriculum and  competency standards are achievable by making non-resident education programs equally  accessible to personnel of all components.  20.  Increase ARC (Air Reserve Component) Capacity: The Air Force should increase its  utilization of the reserve component by increasing the routine employment of reserve  component units and individuals to meet recurring rotational requirements.  21.  Operational ARC Funding: The Air Force should include in all future budget submissions a  specific funding line for operational support by the reserve component.   22.  Council of Governors: The Secretary of Defense should revise its agreement with the Council  of Governors to enable Air Force leadership to consult directly with the Council when  requested, including discussion of pre-decisional information  23.  Non-Disclosure Agreements: The Air Force should discontinue the use of non-disclosure  agreements in the corporate process.  24.  State Adjutants General: The Air Force should continue to advance current informal means  for engaging with The Adjutants General.  25.  Cyberspace Airmen: The Air Force should fill much of the demand for Cyberspace career  26.  Space Domain: The Air Force should build more reserve component opportunities in the  27.  GIISR (Global Integrated Intelligence, Surveillance, and Reconnaissance) Billets: The Air  Force should integrate all of its new GIISR units, and the preponderance of new billets should  be for the reserve component.  28.  Special Operations: The Air Force should increase reserve component presence in Special  Operations through greater integration.  29.  ICBM (Inter Continental Ballistic Missile) Mission: As a pilot program, the Air Force should  expand reserve component contributions to the ICBM mission.  30.  Instructor Pilots: The Air Force should replace some of the 1,800 active instructor pilots with  prior-service volunteers from the Air Reserve Component who would not rotate back to  operational squadrons.  31.  Homeland Security and Disaster Assistance: The President should direct the Departments of  Defense and Homeland Security to develop national requirements for Homeland Security and  Disaster Assistance.  32.  Homeland Defense and DSCA (Defense Support to Civil Authorities): DOD Air Force should  treat Homeland Defense and Defense Support to Civil Authorities as real priorities.  33.  Duty Statuses: Congress should reduce the number of separate duty statuses from more  34.  Integrated Personnel Management: The Air Force should unify personnel management for all  three components under a single integrated organization.
Recommendation  Air Force Integrated Pay and Personnel System (AF-IPPS): The Air Force should accelerate the  development of an integrated pay and personnel system.
PERSTEMPO (Personnel Tempo) Metric: The Air Force should use a single metric to measure  the personnel tempo and stress on its active and reserve forces.
Non-Deployment PERSTEMPO: DOD should update the definition of a non-deployment  PERSTEMPO event for the reserve component to include situations where the reserve  component member is away from a civilian job or attendance at school.
PERSTEMPO and AF-IPPS: The Air Force should include PERSTEMPO accounting in AF-IPPS.
Continuum of Service: The Air Force should develop a pilot project for the implementation of  Continuum of Service.
Active Duty Service Commitments: The Air Force should revise the rules for current active duty  service commitments to enable members to meet the commitment in some combination of active,  guard, and reserve service.
Multiple Career Track Options: The Air Force should develop a new service construct consisting  of multiple career track options.   “Up or Out”: Congress should amend restrictive aspects of current statutes that mandate “up or  out” career management policies.
section_title: Appendix II: Objectives, Scope, and Methodology
paragraphs: This report (1) evaluates the extent to which the Air Force has made  progress in implementing the commission’s recommendations and (2)  describes how the Air Force has assessed the potential for increasing the  proportion of reserve to active component forces as discussed in the  commission’s report.
To evaluate the extent to which the Air Force has made progress in  implementing the commission’s recommendations, we reviewed Air Force  documents, such as briefings to the Executive Committee on the status of  the commission’s recommendations, the Executive Committee Charter,  and a draft template designed by the Air Force’s Total Force Continuum  office for describing milestones, tasks, and other details related to  implementing each group of recommendations and individual  recommendations. We interviewed Air Force officials to understand their  approach to manage and provide oversight for implementing the  commission’s recommendations. We also reviewed Executive Committee  minutes that documented discussions on the status of implementing the  commission’s recommendations and documented decisions to close  recommendations as implemented. We interviewed Air Force officials and  team leaders to understand what actions they had taken to implement  selected recommendations.
We selected a non-probability sample of seven of the 36 open  recommendations by identifying those with the following attributes: (1) the  Air Force had identified implementing the recommendations as facing  challenges; (2) the reason for limited progress was not clear based on a  review of Air Force status briefings; (3) at least one recommendation had  a team leader from the reserve component; and (4) multiple  recommendations where the implementation was led by the same team.  While the descriptions of implementing the non-probability sample of  seven selected recommendations cannot be projected to all 36 open  recommendations, they do illustrate the Air Force’s original process for  implementing the commission’s recommendations.
In addition, we analyzed the implementation plans that had been  developed under the Air Force’s original approach for 3, inter-related  recommendations to determine the extent to which these documents  incorporated leading practices such as milestones, tasks, and  performance measures. We evaluated these plans because, at the time  we conducted our analysis, the Air Force had not developed any plans  under its revised approach. Also, Air Force officials said that plans for  these 3 recommendations were the only implementation plans that had  been developed under the original approach. The documents constituting  the implementation plan for the 3, inter-related recommendations were  not contained in one consolidated document but instead consisted of  multiple documents, such as action plans and other guidance-type  documents that were identified by Air Force officials as implementation  plans. For purposes of our analysis, we refer to these multiple documents  for each of the 3 recommendations as “implementation plans”.
We compared the implementation plans for the 3 recommendations with  leading practices for program management that included the use of  milestones and tasks. We derived these practices by reviewing a  combination of Air Force guidance, industry practices, and our prior  work. We also compared the implementation plans with leading practices  on performance measures which we derived from a combination of  federal internal control standards, prior GAO work on performance  measurement and planning; the Government Performance and Results  Act (GPRA)—as updated by the GPRA Modernization Act of 2010;  related guidance from the Office of Management and Budget; and Air  Force guidance on business case analyses. Our prior work identified ten  attributes of effective performance measures. Of the ten attributes  associated with effective performance measures, we selected seven  attributes against which to evaluate performance measures in the Air  Force’s three implementation plans. We excluded the attributes for  “government-wide priorities”, “core program activities,” and “linkage”,  since we judged these attributes less relevant to the Air Force’s plans.  Excluding these attributes would still yield a sound assessment. The  seven attributes we selected would provide comprehensive information,  over time, on how well the Air Force was progressing with its plans to  implement the recommendations and identify areas for increased focus.
In our scorecard analyses, two GAO analysts independently conducted  analyses of performance measures, milestones, and tasks described in  each implementation plan. Any disagreements between the two  assessments were discussed and reconciled by a third analyst. In our  scorecard analysis of milestones and tasks, we determined that  implementation plans included milestones when we were able to identify  milestones in the plans. We determined that the implementation plans  “addressed” inclusion of tasks tied to milestones if each task had a  milestone associated with it. An implementation plan “partially addressed”  inclusion of tasks tied to milestones if some, but not all of the tasks had  an associated milestone. Finally, an implementation plan “did not  address” inclusion of tasks tied to milestones if we determined that none  of the tasks had a milestone tied to it. In our scorecard analysis of  performance measures, we determined that a performance measure  “addressed” an attribute when it included all elements of the attribute,  even if it lacked specificity and details and could thus be improved upon.  A performance measure “partially addressed” an attribute when it  included more than one, but not all, elements of the attribute.
Consequently, our designation of “partially addressed,” may include a  wide variation of, from one to six, demonstrated attributes. A performance  measure “did not address” an attribute when it did not include or discuss  any elements of the attribute, and/or any implicit references were either  too vague or general to permit assessment. For the attribute “limited  overlap” we determined that the attribute was “not applicable” if the  implementation of the commission recommendation was not dependent  on implementation of another recommendation. To supplement this  analysis and gain further insight into issues of strategic import, we also  interviewed cognizant officials from the Air Force team implementing the  3, inter-related recommendations that have implementation plans, team  leaders and members for seven recommendations, representatives from  the Air Force Reserves, Air National Guard, and Air Force headquarters  staff from the Total Force Continuum office within the Headquarters Air  Force Strategic Plans and Programs Directorate, and the office of the  Assistant Secretary of the Air Force for Financial Management and  Comptroller.
To describe how the Air Force has assessed the potential for increasing  the proportion of reserve to active component forces as discussed in the  commission’s report, we identified the scope, content, and process for the  Air Force’s High Velocity Analysis (referred to in the report as the force  mix option process) which is the process the Air Force developed to  identify and assess force mix options. We first identified the scope of the  Air Force analysis by reviewing documentation such as the Air Force  analysis schedule and interviewing Air Force officials (from the Strategic  Plans and Programs Directorate and the office of the Assistant Secretary  of the Air Force, Financial Management and Comptroller). This work  provided information on how individual analyses were developed and  which aircraft and mission areas were analyzed. Since the Air Force had  developed 55 Excel workbooks for analyzing 44 aircraft types and  mission areas as of December 2015, we selected a non-probability  sample to serve as illustrative examples and learn how the workbooks  were built, identify data inputs and their sources, and identify who verifies  data inputs, how data are verified, and how the workbooks are used to  develop force mix options. From the universe of 44 aircraft types and  mission areas, we selected three analyses that had the following  attributes: the analysis was complete and not on-going; one that was for a  combat mission, one that was for a combat support mission; and one that  had a “deploy in place” mission. The three analyses we selected were:  bombers, personnel recovery, and intercontinental ballistic mission  forces. We did not trace a sample of the data in each workbook back to  its original source documents or verify workbook formulas since we were  not assessing the accuracy or reliability of the data or analyses. We did  review documentation showing that the Air Force has steps in its process  for stakeholders to review and modify workbook inputs. While the details  of the three analyses we sampled cannot be projected to all 55  workbooks, the sample did enable us to describe the overall force mix  option process, including the type of information presented to Air Force  leadership.
Next, we reviewed examples of documentation of the results of the force  mix option process such as examples of briefings presented to senior Air  Force leadership to determine whether assumptions and limitations were  presented to decision makers. Finally, we interviewed Air Force officials  to understand how the Air Force has generally used the results to inform  the budget development process and we reviewed an Air Force report  and minutes from an Air Force leadership meeting to identify examples.
We conducted this performance audit from June 2015 to May 2016 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix III: Air Force Alignment of Commission Recommendations with Human Capital Annex Objectives
paragraphs: The Air Force aligned 21 of the commission’s 42 recommendations with  four of the six objectives in the Air Force Human Capital Annex of the  Strategic Master Plan. The Human Capital Annex objectives are: (1)  Attracting and Recruiting; (2) Developing the Force; (3) Talent  Management (4) Retaining Ready, Resilient Airmen and Families; (5)  Agile, Inclusive and Innovative Institutions; and (6) One Air Force.  According to Air Force officials, none of the commission’s  recommendations aligned with two of the Human Capital objectives— Attracting and Recruiting or Retaining Ready, Resilient Airmen and  Families. The Air Force categorized recommendations that do not fall  within the Human Capital Annex objectives into a separate group that will  be managed by the general officers from the Total Force Continuum  Office. Table 3 below lists the commission’s recommendations that are  aligned with four of the Human Capital Annex objectives as well as those  that will be managed by the Total Force Continuum office.
section_title: Appendix IV: Comments from the Department of Defense
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, the following staff members make  key contributions to this report: Michael Ferren, Assistant Director,  Brenda M. Waterfield, Krislin M. Bolling, Grant Sutton, Barbara Wooten,  Patricia Farrell Donahue, Anne Stevens, and Mike Shaughnessy.