Military Readiness: DOD Needs a Clear and Defined Process for Setting Aircraft Availability Goals in the New Security Environment
Published: 2003-04-07T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The attacks on September 11, 2001, show that threats to U.S. security can now come from any number of terrorist groups, at any number of locations, and in wholly unexpected ways. As a result, the Department of Defense (DOD) is shifting to a new defense strategy focused on dealing with uncertainty by acting quickly across a wide range of combat conditions. One key ingredient of the new strategy is the availability of aircraft to carry out their missions. Key measures of availability include the percentage of time an aircraft can perform at least one or all of its assigned missions, termed the "mission capable" (MC) and "full mission capable" (FMC) rates, respectively. GAO examined whether key DOD aircraft have been able to meet MC and FMC goals in recent years, and DOD's process for setting aircraft availability goals.
section_title: What GAO Found
paragraphs: Less than one-half of the 49 key active-duty aircraft models that GAO reviewed met their MC or FMC goals during fiscal years 1998-2002. The levels of mission capability varied by military service and type of aircraft, and the levels at which the goals were set also varied widely, even among the same type of aircraft. However, the MC and FMC goals for each model changed little over time. Since 1998, only 11 of 49 aircraft models (22 percent) experienced a change to their goals. Seven of the changes were to raise the goals to higher levels. Difficulties in meeting the goals are caused by a complex combination of logistical and operational factors. Despite their importance, DOD does not have a clear and defined process for setting aircraft availability goals. The goal-setting process is largely undefined and undocumented, and there is widespread uncertainty among the military services over how the goals were established, who is responsible for setting them, and the continuing adequacy of MC and FMC goals as measures of aircraft availability. Uncertainty and the lack of documentation in setting the goals ultimately obscures basic perceptions of readiness and operational effectiveness, undermines congressional confidence in the basis for DOD's funding requests, and brings into question the appropriateness of those goals to the new defense strategy. DOD guidance does not define the availability goals that the services must establish or require any objective methodology for setting them. Nor does it require the services to identify one office as the coordinating agent for goal setting or to document the basis for the goals chosen. DOD officials told GAO that the guidance has not been updated since 1990 to reflect the new security environment of increased deployments and other changes since the end of the Cold War.

section_title: Letter
paragraphs: The terrorist attacks on September 11, 2001, clearly demonstrated that the  U.S. security landscape has changed. The familiar Cold War threats of  large-scale wars between nation states in predictable areas such as the  Koreas and the Middle East have been joined by a broad array of new  threats characterized by surprise and uncertainty. Attacks on U.S. security  can now come from any number of terrorist groups such as al Qaeda, in  any number of locations, and in wholly unexpected ways. As a result of the  changed security environment, the Department of Defense (DOD) has  rethought defense strategy and is shifting to a “capabilities-based”  approach focused on contending with uncertainty by enhancing its ability  to act quickly and decisively across a wide range of combat conditions and  locations.
One key ingredient of the new strategy is the availability of aircraft to  carry out their assigned missions. DOD requires each military service to  establish availability goals for aircraft and other major weapon systems,  and measures of the degree to which those goals are met. Key measures  include the percentage of time that an aircraft can perform at least one or  all of its assigned missions, termed the “mission capable” (MC) and “full  mission capable” (FMC) rates, respectively. MC and FMC goals and rates  are fundamental indicators of readiness expectations. They are also used  by DOD as indicators of maintenance and supply effectiveness and are  made available to the Congress for its general oversight of DOD.  Moreover, the level at which the goals are set also influences large  amounts of military spending for aircraft procurements, spare parts  inventories, and other resources needed to meet the goals. However, our  recent reports have identified problems in meeting MC goals among  certain aircraft. For example, we reported in June 2000 that many of the  cargo aircraft needed to meet wartime airlift requirements were not  meeting MC goals.
Concerned that the new capabilities-based strategy may be difficult to  carry out if aircraft are experiencing problems in meeting existing  availability goals, you requested that we examine DOD’s structure for  establishing MC and FMC goals for aircraft in the Air Force, Army, Navy,  and Marine Corps. This report addresses (1) whether key active-duty  aircraft have been able to meet existing MC and FMC goals, (2) the causes  of any difficulties in meeting those goals, and (3) whether DOD has a clear  and defined process for setting aircraft availability goals.  We performed  our review from February through November 2002 in accordance with  generally accepted government auditing standards.  Appendix II describes  the scope and methodology of our work.
subsections: section_title: Background
paragraphs: DOD aircraft are used to perform a variety of different missions. However,  for the purpose of this report, we have grouped them into five basic  categories: (1) various models of fighter/attack aircraft, such as the F/A-18  Hornet, provide air superiority or close air support of ground forces; (2)  bombers, such as the B-1 Lancer, provide long- and short-range delivery of  heavy munitions; (3) electronic command and control aircraft, such as the  E-3 Sentry, provide airspace and battlefield reconnaissance, command,  and control services; (4) tankers and cargo aircraft, such as the KC-135  Stratotanker and the C-5 Galaxy, respectively, provide air refueling  services and the ability to carry troops and equipment anywhere in the  world; and (5) helicopters, with their ability to hover as well as conduct  long- and short-range operations, are used for a variety of missions,  including transportation of troops and equipment, air assault and  reconnaissance, and search and rescue operations. Our review included a  total of 49 different aircraft models (over 5,600 individual aircraft in 2002)  in these five categories. These aircraft were considered by the services to  be their key active-duty operational aircraft. Table 1 lists these aircraft  models, along with the military service using them, and their MC and FMC  goals for fiscal year 2002.
DOD Instruction 3110.5, dated September 1990, requires all military  services to establish quantitative availability goals and corresponding  condition status measurements for these aircraft and other mission- essential systems and equipment. The goals established must estimate the  maximum aircraft performance that is achievable on the basis of the  aircraft’s design characteristics and planned peacetime usage, and  assuming full funding and optimal operation of the peacetime manpower  and logistic support systems. Military personnel, civilian contractors, or  both may perform the required maintenance under these systems. The  instruction prescribes a basic set of condition status measures, including  FMC, partial MC, and MC, that each service must use to describe the  capability of systems or equipment. FMC indicates that an aircraft has all  of the mission-essential systems and equipment it needs to perform all of  its missions installed and operating safely. Mission-essential systems are  those required to perform primary functions such as fire control, bombing,  communications, electronic countermeasures, or radar. Partial MC  indicates that an aircraft has the operable mission-essential equipment it  needs to perform at least one of its missions, but not all. For example, an  aircraft expected to be able to carry troops into combat during wartime in  all weather conditions, as well as to be able to fly humanitarian missions  during peacetime, would be considered partial MC if some of its  equipment were broken and it could fly only humanitarian missions in  clear weather. MC consists of the sum of the partial MC and FMC  measures; that is, the number of MC aircraft is equivalent to the sum of the  aircraft rated partial MC and the aircraft rated FMC. This report focuses  on MC and FMC goals because the Army, Navy/Marines, and parts of the  Air Force do not establish separate partial MC goals.
section_title: DOD Aircraft Experienced Widespread Problems in Meeting MC and FMC Goals
paragraphs: Many of DOD’s key aircraft have been unable to meet their MC and FMC  goals since at least 1998. For example, during fiscal years 1998-2002, only  23-35 percent of the 49 aircraft models we reviewed were able to meet  their MC goals, and 31-49 percent met their FMC goals. In most cases, the  actual rates were at least 5 percentage points below the goals. Average MC  and FMC rates varied by service and type of aircraft. For example, the  Army and Air Force had the highest average MC rates, followed by the  Marines and the Navy. These rates have increased slightly since fiscal year  2001 in all services except the Navy. Among aircraft types, the average MC  rates varied from 60 to 80 percent. Average MC rates were the highest for  helicopters, followed by cargo aircraft and tankers, fighter/attack aircraft,  bombers, and electronic command/control aircraft. While the rates have  fluctuated, MC and FMC goals have generally remained constant over  time. Since 1998, only 11 of 49 aircraft models (22 percent) experienced a  change to their goals—and 7 of these changes were to raise the goals.
subsections: section_title: Less Than One-Half of the Aircraft Models Met Goals
paragraphs: DOD’s key, high-demand aircraft have experienced widespread difficulties  in meeting MC and FMC goals since at least 1998. (Appendix I provides a  full listing of MC and FMC goals, rates, and other information by year for  each aircraft model we reviewed.) For example, during fiscal years 1998- 2002, the percentage of aircraft models meeting their MC goals never  exceeded 35 percent. (See fig. 1.) During this period, the rates for the  individual aircraft models were more than 5 percentage points below their  MC goals in 62 percent of the cases. The percentage of aircraft models  meeting FMC goals during the same period ranged from 31 to 49  percentage points, and 71 percent of the cases were more than 5  percentage points below the goals.
At the service level, Army aircraft generally met their MC goals the most  frequently, followed by the Marine Corps, Air Force, and Navy. (See fig. 2.)  The same rank order held for FMC goals.
As previously shown in table 1, the level at which the goals were set  showed little consistency, varying widely even among the same type of  aircraft. For example, MC goals for the bombers in our review ranged from  50 to 80 percent, and MC goals for the fighters, from 65 to 83 percent.
section_title: Actual MC Rates Varied by Service and Type of Aircraft
paragraphs: Actual MC rates also varied between services and the various aircraft  types. MC and FMC rates are based on the ratio of the number of hours an  aircraft was actually available to the total number of hours it could have  been available. The Navy/Marines and Air Force reduce the latter figure by  the amount of time an aircraft was away for scheduled depot maintenance,  while the Army does not make this adjustment. We computed the average  rates by service and aircraft type from service data on the total number of  hours each aircraft model was MC and FMC, and the total hours each  aircraft model was available each year.
The average annual MC and FMC rates for the services as a whole are  shown in figures 3 and 4. The Army and the Air Force had the highest  average MC rates, at 77-83 percent during fiscal years 1998-2002; followed  by the Marines, at about 71-75 percent; and the Navy, at 61-67 percent. A  similar pattern follows for the average FMC rates for the services.
When grouped by type of aircraft, average annual MC rates were highest  for helicopters (76-80 percent), cargo/tankers (75-79 percent), and  fighter/attack aircraft (75-77 percent). Average annual MC rates for  bombers (64-69 percent) and electronic command/control aircraft (60-67  percent) were somewhat lower. Average FMC rates showed similar rank  orders. (See figs. 5 and 6.)
section_title: Goals Generally Remained Constant over Time
paragraphs: MC and FMC goals have generally remained constant over time. Since  1998, only 11 of 49 aircraft models (22 percent) experienced a change to  their MC goals, FMC goals, or both. Seven models had their goals raised,  and three had their goals lowered. One model’s MC goal was changed but  then returned to its initial level. Ten of the 11 changes were for aircraft  operated by the Air Force. The remaining change was for a Marine Corps  aircraft. (See app. I for additional details.)
In fiscal year 2002, for example, the Air Force raised the MC goal for its E- 8 Joint Stars electronic command and control aircraft from 73 to 75  percent. According to officials, the E-8 is a relatively new (3-year-old)  aircraft that is slowly increasing its performance level as it matures and  Air Force maintenance personnel understand the aircraft better. The  increase in the MC rate was based on an analysis of actual E-8 MC rates  that were showing an upward trend in performance. The Air Force is the  only service that routinely conducts formal reviews of its goals. Air Force  officials told us that they generally try to keep the goals high because it is  difficult to stop the goals from dropping further once they begin to be  lowered. Moreover, officials believed that contractors need to be held to  high standards to keep spare parts inventories and other aspects of  maintenance at high levels. In another case, the MC goal for the Marine  Corps’ F/A-18D Hornet fighter was raised from 60 to 75 percent, and its  FMC goal, from 46 to 58 percent at the beginning of fiscal year 2000.  According to Navy documents, this increase was due to a change in the  aircraft’s assigned mission.
While most of the goals were either unchanged or increased, the Air  Force’s Air Combat Command developed a set of interim goals in fiscal  year 2000 for some of the fighters, bombers, and electronic  command/control aircraft under its command. These interim goals were  lower than its official MC goals. In 1999, the Command determined that  problems with suppliers and manpower shortages were undercutting its  ability to meet MC goals and lowering unit morale. To combat this  problem, the Command developed the interim goals listed in table 2. In  2002, the Command returned to using the pre-2000 goals for all but six  aircraft (A-10, E-3, F-15 C/D, F-15 E, RC-135, and U-2). According to  Command officials, the lower goals applied only to their units. Goals for  suppliers remained at official levels to keep spare parts inventories high.  Neither the other services nor the Air Force’s other major commands  responsible for aircraft operations have developed interim goals.
section_title: Mission Capable Problems Caused by a Combination of Factors
paragraphs: According to DOD officials, difficulties in meeting MC and FMC goals are  caused by a complex combination of interrelated logistical and operational  factors, with no dominating single problem. The complexity of aircraft  design, the lack of availability and experience of maintenance personnel,  aircraft age and usage patterns, shortages of spare parts, depot  maintenance systems and other operational factors, and perceived funding  shortages were all identified as causes of difficulties in meeting the goals.  As indicated below, our work found that some indicated factors were valid  causes, while the impact of others was less certain.
subsections: section_title: Aircraft Design Considered Key
paragraphs: Officials believe that the complexity of military aircraft affects its  availability, and thus its ability to meet MC goals. Military aircraft are  designed to handle a specific set of missions and provide a specific set of  capabilities over a projected useful lifespan. According to officials, each  aircraft can be inherently complex and maintenance intensive, or,  depending upon the missions and capabilities it was designed to provide,  simple and easy to maintain. For example, the B-2 bomber had the lowest  MC rates (32-44 percent) of any aircraft we reviewed. However, according  to Air Combat Command officials, one reason for these low rates is the  complex design of the aircraft. The B-2 is a very advanced aircraft with  low observable (stealthy) characteristics using new composite materials,  and Air Force personnel are still learning how to maintain the aircraft. In  contrast, the B-52 bomber had some of the highest MC rates (76-84  percent) of all the aircraft we reviewed. According to Air Force officials,  the B-52 is a relatively simple and flexible design intended for ease of  maintenance and durability.
section_title: Availability and Experience of Maintenance Personnel Have an Impact
paragraphs: Service officials also frequently linked shortages of the total number of  maintenance personnel, as well as their experience level, to the failure to  meet MC goals. Navy officials told us that the growing sophistication of  their aircraft in general requires maintenance personnel to take longer to  learn the complex computer and electronic skills needed to handle the  aircraft. However, high demand for these skills in the private sector makes  it difficult to retain personnel with these maintenance skills, leading to  turnover and increasing the difficulty in meeting the MC goals. Similarly, a  recent study published in the Air Force Journal of Logistics found that  the number and experience level of maintenance personnel correlated  highly with the MC rates of F-16 aircraft. As the number of experienced  personnel assigned to an aircraft increased, the MC rates increased as  well. Army officials also cited shortages of experienced maintenance  personnel as a cause of lower MC and FMC rates. However, they also  stated that it may be possible to raise the rates by maximizing the time that  maintenance personnel actually spend maintaining the aircraft. For  example, one Army Audit Agency study in 1998 found that maintenance  personnel at one unit were spending about 70 percent of their time on  nonmaintenance activities such as administrative duties, training, and time  attending to personal duties.
Personnel management is an area that we have cited as a major  management challenge and program risk for DOD. For years, DOD has  been wrestling with shortages of key personnel because of retention  problems. In 1999 we reported that the majority of factors cited as sources  of dissatisfaction and reasons to leave the military were related to work  circumstances, such as the lack of spare parts and materials needed to  perform daily job requirements.
section_title: Aircraft Age and Usage Patterns Also Believed to Influence Availability
paragraphs: The advancing age and usage patterns of aircraft were other factors often  cited by service officials as reasons why aircraft did not meet MC goals.  DOD’s inventory of aircraft is getting older. The Congressional Budget  Office recently reported that from 1980 to 2000, the average age of active- duty Navy aircraft rose from 11 years to more than 16 years; Air Force  aircraft, from 13 to more than 20 years; and Army helicopters, from 10 to  over 17 years. Logistics officials told us that aging influences on MC rates  typically follow a cyclical pattern over the life of an aircraft. When aircraft  are initially introduced, they go through a “shake down” period and have  low MC rates as new equipment and supply systems stabilize and  maintenance personnel learn to understand the aircraft. Eventually, MC  rates begin to rise and then stabilize at a higher working level. However, as  more and more flying time is accrued over the passing years, problems due  to materials and parts fatigue, corrosion, and obsolescence increase, and  MC rates begin to fall again. Modernization programs are then instituted to  replace worn and obsolete equipment, and the pattern begins again.
Although age may affect MC rates, we found no statistical evidence that  age alone explains difficulties in meeting MC goals. For example, our  analysis of average aircraft ages and 2002 MC rates found no indication  that older aircraft have the lowest MC rates. (See table 3.) With an  average age of 40 years, the B-52 is the second oldest aircraft in DOD’s  inventory. However, its MC rate of 81 for 2002 and historical MC rates  consistently in the upper 70s and low 80s rank it among the highest  performers we reviewed. According to Air Force officials at the Air  Combat Command, in addition to their simplicity, B-52s have a relatively  low number of actual flight hours, averaging about 16,000 hours each  despite their age. These officials believed that accrued flight hours are a  more appropriate measure of wear and tear than chronological age.
Moreover, according to these officials, the B-52 was originally scheduled  to retire in the mid-1990s. However, because of its durability and  flexibility, the Air Force decided to retain the aircraft until the average age  reaches 32,000 hours, projected at about 2040.
Logistics officials also believe that MC rates are affected by usage patterns  and whether the aircraft is operated under the conditions for which it was  designed. Officials told us that the large increase in deployments in recent  years has caused many DOD aircraft to be operated at rates higher than  expected during their design, thus accelerating aging problems. For  example, according to the Air Force Journal of Logistics study, F-15  fighters sent to Saudi Arabia in 1997 were flown at over three times their  normal rate.
section_title: Spare Parts Inventories Critical
paragraphs: Shortages of spare parts have been recognized by us and others for years  as a major contributor to lower-than-expected MC rates. As a result, we  have also cited DOD inventory management as a major management  challenge and program risk since 1990. Service officials continued to cite  spare parts shortages as a frequent cause of difficulties in meeting MC  goals. Spare parts shortages are caused by a number of problems,  including underestimates of demand, and contracting and other problems  associated with aging aircraft or small aircraft fleets.
We have reported on DOD’s problems in estimating aircraft spare parts  requirements for years. For example, in 1999 and again in 2001, we  reported that shortages of spare parts caused by inaccurate forecasting of  inventory requirements was degrading MC rates for key Air Force aircraft  such as the B-1B bomber, C-5 cargo planes, and F-16 fighters. In 2001 we  reported that key Navy aircraft were also having readiness problems  because of spare parts shortages resulting from underestimates of  demand. Officials continued to raise this issue as an underlying factor in  spare parts shortages. In addition, some officials also believed that the  higher operating tempos associated with increased deployments have  caused parts to fail quicker than expected, exacerbating weaknesses in  forecasting inventory requirements.
Air Force officials told us that aging aircraft, in particular, may experience  parts shortages and delays in repairs because original manufacturers may  no longer make required parts. To obtain a new part, officials must wait  for it to be manufactured. However, this may not be a high priority for the  commercial supplier because of the relatively low profit potential.  Alternatively, another company could make the part if the original  manufacturer were willing to give up its proprietary rights. However, this  can take longer and be more expensive than simply waiting for the original  manufacturer. Moreover, officials also told us that spare parts inventories  are sometimes reduced when aircraft are nearing the end of their  projected life. For example, Air Force officials said that in the mid-1990s  they began to shut down the spare parts supply for the B-52 because of its  anticipated retirement. This resulted in a depletion of inventories, the  canceling of contracts, and ultimately a drop in MC rates from 1997 to  2000. As a result of the decision to retain the B-52, the supply system is  recovering and MC rates are moving up.
Similarly, the size of the aircraft fleet can also influence spare parts  inventories and MC rates. According to officials, manufacturers may see  little profit in stocking large inventories of spare parts for a small fleet of  specialized military aircraft. Small fleets of aircraft can also suffer from  having their MC rates strongly influenced by the MC failures of just a few  aircraft. Large fleets of aircraft also have an advantage in having more  opportunities to remove serviceable parts from one aircraft and install  them in another—termed “cannibalizing”—thus helping to insulate their  MC rates from the impact of parts shortages. However, we recently  reported that while cannibalization is a widespread practice among the  services, it increases maintenance personnel workloads and lowers morale  and retention.
section_title: Maintenance Approach and Other Operational Factors May Affect MC Rates
paragraphs: Air Force and Navy officials cited changes to their maintenance  approaches as a significant cause of slower repair times and lowered MC  rates. In the mid-1990s the Air Force changed from a three-level  maintenance approach to a two-level approach. This change moved much  of the intermediate maintenance functions, such as the replacement or  emergency manufacture of parts, away from the air base level to  centralized maintenance depots. According to officials at both the Air  Combat Command and Air Mobility Command, these changes slowed the  pace of repairs significantly. Repair expertise was taken away from the  base level, and aircraft were shipped away from home base more often for  repairs. Moreover, officials believed that many experienced maintenance  people were lost as they refused to move to other locations associated  with the reorganizations. In this regard, our 1996 review of depot closures  noted that DOD’s outplacement program helped limit the number of  involuntary separations and that jobs were often available for employees  willing to relocate.
The Army continues to use a three-level maintenance system, as does the  Navy. However, Navy officials said they also changed their system in the  mid-1990s by introducing the integrated maintenance concept. This  approach, in contrast to the Air Force approach, increased the amount of  aircraft modernization and other work performed at the base level during  a time when funding for depot-level work was being reduced. However,  officials believed this change overloaded the base-level maintenance  systems and ultimately lowered reported MC rates.
From fiscal year 1988 to fiscal 2001, DOD reduced the number of major  depots from 38 to 19. During this same period, the maintenance workforce  was reduced by about 60 percent (from 156,000 to 64,500). These  reductions were the result of overall force structure reductions since the  end of the Cold War, as well as DOD’s desire to reduce costs by relying  more on the private sector for the performance of depot maintenance. We  have raised concerns that DOD’s downsizing of its depot infrastructure  and workforce was done without sound strategic planning and that  investments in facilities, equipment, and personnel in recent years have  not been sufficient to ensure the long-term viability of the depots.
Other operational factors can also affect MC rates. For example, from 1997  to 2000, the Air Force’s B-1 bomber had a major power system problem  that lowered MC rates by 12 points. To address the problem, the Air  Combat Command instituted a system of frequent video teleconferences  between the offices involved in the maintenance response to provide more  intensive management of the response. This approach worked, as the MC  rate climbed by 9 points by 2002. Management integration between the  operations and logistics sides of the organization was also viewed as key.  Good coordination between these two groups is essential because of the  complex and multifaceted causes of MC problems. Finally, Air Force  officials noted that some of the problems with Air Force MC rates could be  explained by a change in reporting procedures. During the mid-1990s, the  Air Force returned an aircraft to MC status after it was repaired but prior  to the actual check flight to ensure that it was operating correctly. Now,  the aircraft must pass the check flight before being classified as MC.  Officials believe that this change would tend to lower MC rates slightly.
section_title: Funding Levels Raise Concerns
paragraphs: Officials from all services cited underfunding of spare parts inventories,  maintenance depots, and other aspects of the maintenance and supply  systems as a key problem. For example, Army and Navy officials told us  that they often use remanufactured parts instead of new parts to save  money.
DOD reports in its Fiscal Year 2000 Performance Report that it has  increased funding for spare parts and depot maintenance requirements.  For example, the report indicates that funding for depot maintenance  increased from $5.58 billion to $7.01 billion from fiscal year 1997 to fiscal  1999 (most recent year that data are available). However, the report also  acknowledges an unfunded requirement of about $1.18 billion in fiscal  year 1999. Notwithstanding claims regarding the lack of funding for spare  parts, we recently reported that when provided additional funds for spare  parts, DOD was unable to confirm that those additional funds were used  for that purpose.
The pressures for more funding to maintain DOD’s aircraft may well go up  even more in coming years as the aircraft inventory continues to age. The  Congressional Budget Office estimates that spending for operations and  maintenance for aircraft increases by 1 to 3 percent for every additional  year of age.
section_title: DOD’S Goal-Setting Process Is Largely Undefined and Undocumented
paragraphs: Despite the importance of MC and FMC goals as measures of readiness  and logistical funding needs, we found widespread uncertainty over how  the services’ MC and FMC goals were established and who is responsible  for establishing them, as well as basic questions about the adequacy of  those goals as measures of aircraft availability. The services could not  explain and document how the original MC and FMC goals were set for  any of the aircraft in our review. Furthermore, some officials questioned  which goals are the best to use in reviewing aircraft availability: MC goals,  FMC goals, or perhaps a new type of goal. DOD’s instruction provides little  or no guidance on these and other key issues. DOD officials told us that  the instruction has not been updated to reflect the current environment of  increased deployments and other changes since the end of the Cold War.
subsections: section_title: Goals Are Important Indicators of Readiness, Operational Effectiveness, and Logistical Funding Needs
paragraphs: MC and FMC goals are used as fundamental measures of readiness  throughout DOD, used as indicators of operational effectiveness, and used  to help determine the size of spare parts inventories and other logistical  resources needed to maintain aircraft availability. As a result, the level at  which the goals are set can influence not only perceptions about  operations and readiness, but also millions of dollars in spending for  logistical operations.
In addition to the requirement to maintain MC and FMC data set forth by  DOD Instruction 3110.5, the services use MC and FMC measures as a  component of overall unit readiness determinations under DOD’s Global  Status of Resources and Training System. The System requires  commanders to rate their unit’s readiness at levels 1 (highest) through 5 on  the basis of a combination of their professional judgment and the  readiness ratings in four specific areas: personnel, training, equipment on  hand, and equipment condition. MC and FMC measures are used to  determine the ratings for equipment condition. For example, the Army  measures equipment condition (termed “serviceability” by the Army) for  aircraft by using the FMC rate. An FMC rate of 75 percent or more is  required for a level-1 readiness rating, the highest available. Congress also  requires DOD to include Status of Resources and Training System  information on the condition of equipment as well as specific information  on equipment that is not mission capable in its quarterly readiness reports  to Congress. These reports assist Congress in its general responsibilities  for overseeing DOD readiness and operations.
Similarly, according to DOD and service officials, MC and FMC goals are  used as management tools within DOD units to diagnose problems and  motivate personnel. For example, officials in the Air Combat Command  told us that their use of lower interim goals beginning in fiscal year 2000  was an attempt to raise unit morale that had suffered as a result of their  inability to meet the actual goals owing to shortages of personnel and  spare parts. In this regard, DOD’s instruction specifically calls for the  services to use the goals and condition status measurements, such as MC  and FMC, to review maintenance and supply effectiveness and to have  programs to identify and correct problems with systems and equipment.
Service officials told us that the goals also affect DOD’s funding levels  because the goals are used to help determine the size of spare parts  inventories and other logistical resources needed. Higher goals require  more money to maintain parts inventories and other resources needed to  achieve the goals. For example, officials told us that in the early 1990s, a  $100 million contract for logistics support for one Air Force aircraft  contained an MC goal of 90 percent. During this period, the contractor  kept supply bins full of parts and MC goals were met. However, in the mid- 1990s a new contractor was brought in, and the MC goal was dropped to  85 percent. According to Air Force officials, their decision to lower the MC  goal by 5 percentage points allowed the contractor to lower spare parts  inventories and reduced the price of the maintenance contract by $10  million. However, MC rates also dropped and eventually fell below the  new goal. The services have developed mathematical models to determine  the size and cost of the spare parts inventories needed to support various  levels of MC and FMC goals and other measures of aircraft availability. For  example, the Navy uses a model called “Readiness Based Sparing” that  takes a given FMC goal and determines the level of funding and spare  parts inventories needed to reach that goal. Such models are useful in the  case of spare parts inventories. However, we were not able to identify any  models in widespread operational use that integrated the other influences  on MC rates, such as maintenance personnel assigned, into an overall  model able to predict the impact of changes in those resources on MC and  FMC rates. Army and Air Force officials told us that they had recently  developed such integrated models, and they are currently in limited use to  test their validity. Navy officials told us that they did not yet have an  integrated model.
The potential amount of funds affected by the level at which MC and FMC  goals are set is large. Military service estimates of the spending of  operations and maintenance funds for aircraft spares and repair parts  were over $7 billion in fiscal year 2001. This figure does not include  spending from other sources such as procurement and working capital  funds.
section_title: Methods Used to Set Goals Unknown
paragraphs: Precisely how MC and FMC goals are established is unknown. DOD  officials said that a combined DOD and military service team establishes  operational requirements and MC goals during the acquisition process.  After approval, these requirements are recorded in the Operational  Requirements Document or other documents associated with the process.  According to officials, part of this process involves an engineering analysis  of the expected operational availability of the aircraft and the underlying  level of maintenance support elements needed. “Operational availability”  is an engineering term referring to the probability that equipment is not  down owing to failure. In comparison, MC and FMC goals represent the  expected percentage of time that an aircraft will be able to perform at  least one or all of its missions, respectively.
Service officials reviewed the acquisition documents for many of the  aircraft in our review, but were unable to explain and document how the  actual MC and FMC goals were chosen. According to officials, many of  these aircraft were acquired 20 to 30 years ago, under processes that have  changed over the years, and with no clear documentation of the basis for  the specific goal chosen. Moreover, there was often confusion over which  organizations were responsible for setting the goals.
For example, Navy officials pointed to a 1996 Center for Naval Analyses  study that attempted to determine how the MC and FMC goals for Navy  aircraft were originally computed. According to the study, however, “no  one knows the origin of the numbers or the method used to compute  them. Now, the numbers are routed to knowledgeable people for revision,  which are made without documenting the rationale for the changes.” In a  July 17, 2002, letter to us, the Navy further explained that it believed that  the MC goals were established in the early 1980s “to be in line with the  reported status quo for the day” with “no analytical rigor applied at the  time of their birth.” We requested a written explanation of how the goals  were set because, despite repeated referrals to various offices over several  months, no Navy official could explain how the goals were established or  identify the responsible office. According to Navy officials, there was  uncertainty between the program and policy offices as to who is  responsible for establishing the goals and who should answer our  questions.
Similarly, Army officials could not explain how their goals were set, and  two separate Army organizations believed the other was responsible for  setting the goals. The Army’s written response to our request for an  explanation of how the goals were set (dated July 31, 2002) was prepared  by officials from the Army’s Training and Doctrine Command and  forwarded to us by a letter from the Office of the Deputy Chief of Staff for  Logistics. The Deputy Chief of Staff’s letter states that MC goals for Army  aircraft are extracted from the System Readiness Objective contained in  the Operational Requirements Document established during an aircraft’s  acquisition, and that the Training and Doctrine Command is responsible  for establishing the System Readiness Objectives. However, the Training  and Doctrine Command’s letter states that it does not set System  Readiness Objectives and that the Deputy Chief of Staff for Logistics is  responsible for establishing readiness goals. Nonetheless, the Training and  Doctrine Command researched the operational requirements documents  for the Army aircraft in our review in an attempt to answer our question  about how the MC and FMC goals were set. The Command’s letter  identified the operational availability requirements for most of the aircraft  but did not explain how these requirements were set or make any  reference to the MC or FMC goals. Officials from the Office of the Deputy  Chief of Staff for Logistics told us that the Army is considering changing  the FMC goals for all its aircraft to 75 percent to match the requirement for  the highest-level readiness rating for equipment serviceability under the  Global Status of Resources and Training System’s criterion. They did not  know how the 75-percent-readiness-rating criterion was chosen.
Air Force officials also could not explain how the initial MC and FMC  goals for their aircraft were established. Officials from the Air Combat  Command—responsible for Air Force fighters, bombers, and electronic  command/control aircraft in our review—told us that they could find no  historical record of the process used to establish most of the goals.  Similarly, officials from the Air Mobility Command—responsible for the  cargo and tanker aircraft—stated that the Command was formed in 1992  out of elements from the Military Airlift and Strategic Air Commands and  did not know how the previous Commands had established the goals.  According to these officials, each of the major Commands that operate  aircraft and other major weapon systems in the Air Force is responsible  for establishing its own MC goals, and no one has published a standardized  methodology to use. Moreover, some of the documentation related to the  goals was lost when the Military Airlift and Strategic Air Commands were  deactivated. Similar to the Navy, however, officials from both Commands  believed that the goals were set on the basis of the historical performance  of similar aircraft and/or subjective Command judgments.
While Air Force officials could not explain how the initial goals were  established, they told us that their annual reviews of the goals are based  on a mix of historical trend analysis and requirements reviews. The Air  Force is the only service that conducts formal reviews of its goals each  year. According to officials from the Air Mobility and Air Combat  Commands, until 1997-98, reviews of the goals in both Commands were  based on an analysis of actual historical MC and FMC rates. For example,  analysts at the Air Mobility Command compared the goals with the actual  rates for the previous 2 years. Depending upon actual performance, the  goal could then be changed, sometimes on the basis of subjective  judgments. According to Air Combat Command officials, the MC goal for  the B-2 bomber was set in fiscal year 2000 using an analysis of historical  rates and command judgment. The first B-2 was delivered in 1993.
In 1997-98, the two Air Force Commands began to develop so-called  “requirements-based analyses” to review the standards. According to  officials at the Air Combat Command, for example, it was recognized that  the historical approach to reviewing the standards can perpetuate  relatively low standards because it simply accepts the low funding levels  and other problems that may lower MC rates without focusing on actual  mission needs. The new approach attempts to factor in wartime  operational requirements, peacetime flying hour requirements for pilot  training, and other such requirements. A mix of both approaches is  currently used by the commands to review the goals.
The services also differed in their treatment of other important aspects of  managing the goals, such as whether to vary the goals on the basis of an  aircraft’s deployment posture. The Navy was the only service to tier its  goals on the basis of its traditional practice of cyclical deployment  schedules on board its ships and aircraft carriers. Operational aircraft in  the Navy follow a cyclical pattern of deploying to sea on aircraft carriers  and other vessels for a set period of time, such as 6 months. Once the  deployed units are replaced, they experience a stand-down period during  which they recover from the rigors of deployment until it is time to begin  preparing for the next movement. The Navy varies the intensity of its  maintenance and its MC and FMC goals according to this pattern. Navy  aircraft more than 90 days away from a deployment have goals that are 5  percentage points lower than aircraft within 90 days of a deployment, and  aircraft actually deployed have goals that are 5 percentage points higher  than those within 90 days of deploying. In comparison, aircraft in the  Marine Corps and other services have a level approach to maintenance  where the goals do not vary, and maintenance is kept at a relatively  constant level. Navy officials believed that the cyclical approach to  maintenance could lower overall MC rates over time compared with the  level approach. This is because of the reduced maintenance attention  when the aircraft are not deployed.
section_title: Adequacy of MC and FMC Goals Questioned
paragraphs: Some officials questioned whether the MC and FMC goals are adequate  measures of an aircraft’s availability. For example, officials from the Air  Force’s Air Mobility Command stated that they focused on the MC goal  and not the FMC goal because their primary readiness objective is the  specific mission currently assigned, not every possible mission the aircraft  was designed for. Moreover, the Air Combat Command did not even  establish FMC goals. This Command was the only one we reviewed that  did not set FMC goals for its aircraft. Air Combat Command officials told  us that they could find no documentation to explain why the Command  did not establish FMC goals.
In contrast, Army officials stated that their units focus primarily on the  FMC goal because it is directly connected to readiness ratings under the  Status of Resources and Training System. Furthermore, Navy officials  stated that the military is moving away from the MC and FMC goals in  newer aircraft, such as the Joint Strike Fighter. This is because the MC and  FMC goals provide only a limited historical perspective and do not address  issues that are important to war-fighting commanders such as how often  an aircraft can fly missions over the course of a day and the probability  that the aircraft will complete its mission. The Joint Strike Fighter, for  example, is using a concept called “mission reliability” instead of MC and  FMC goals. Mission reliability is the probability that the Joint Strike  Fighter will complete its required operational mission without a failure.  According to Navy officials, the predictive value and information on flight  frequency and reliability provided by this new measure is very valuable to  war-fighting commanders and is better for mission-planning purposes than  the MC and FMC measures. Officials said that the mission-reliability  concept could be used throughout DOD’s inventory of aircraft.
section_title: DOD Instruction Provides Little or No Guidance on Key Issues
paragraphs: DOD Instruction 3110.5 provides only vague or no guidance on many of  the key issues raised in this report. For example, the instruction requires  each military service to establish availability goals for its mission-essential  systems and equipment, and a corresponding set of condition status  measures relative to those goals. The instruction specifically identifies MC,  FMC, and other specific capabilities as measures that the services must  maintain. However, it does not identify the specific goals that must be  established—MC, FMC, or any other—or the primary readiness objective  to be served. In this regard, the instruction states that the services should  assume planned peacetime usage in setting the goals. According to Air  Force officials, peacetime usage can be more taxing than wartime usage  because of the extra training and other requirements. Air Combat  Command officials told us that they believed that the instruction regarding  what goals—including the FMC goal—were required to be established was  unclear.
The instruction also provides little guidance on the methodology to be  used in setting the goals. It states that the services should provide  estimates of the maximum performance that is achievable, given the  design characteristics of the aircraft, and that full funding and optimal  operation of the logistics support system should be assumed. Service  officials said they believe that actual levels of funding, personnel, spare  parts inventories, and other key resources should be factored into the  process of setting the goals, since full funding has not been provided for  years. The instruction is silent on the issue of whether it is appropriate to  use historical trends of similar aircraft in determining the goals, as  opposed to a more analytical approach using actual requirements, for  example. The instruction is also silent on whether the aircraft availability  goals should vary on the basis of the aircraft’s deployment posture.  Moreover, it includes no requirement for the services to identify the  readiness and cost implications of setting the goals at different levels, to  help clarify the pros and cons of available choices and the guiding  principles used to decide on those choices.
Similarly, the instruction provides little organizational structure for the  goal-setting process in DOD. For example, it does not require the services  to identify one office as the coordinating organization for goal-setting and  other related activities. Furthermore, it does not require the services to  document the basis for the goals chosen or outline any of the basic  historical documentation that should be maintained for goal-setting and  other key activities during the process.
According to DOD officials from the office responsible for the instruction,  DOD Instruction 3110.5 dates back to the 1970s when readiness concerns  had reached a high point. The focus was on getting the services to set  benchmark readiness goals, and the instruction gave them latitude to  choose those goals, the methods for setting them, and the processes for  managing them. The instruction was revised in 1990. However, officials  told us that it has not been updated to reflect the current environment of  frequent deployments and other changes since the end of the Cold War,  and some now consider it a relic.
We performed our work from February through November 2002 in  accordance with generally accepted government auditing standards.  The  final publication of this report was delayed by the impact on DOD’s report  review and classification process of the terrorist attacks on September 11,  2001 and DOD’s preparations for potential conflict in Iraq.
section_title: Conclusions
paragraphs: While many of DOD’s key aircraft are not meeting MC and FMC goals, it is  difficult to determine how significant this problem is because of the  uncertainty and lack of documentation of the basis for the existing goals.  Moreover, without knowing the basis for the existing goals, it is also  difficult to know whether that basis is appropriate for the demands of the  new defense strategy.
DOD’s Instruction 3110.5 fails to clearly define the specific availability  goals that all services must establish. Without the perspective provided by  clear, consistent, and up-to-date goals, the perceptions of actual  performance are subject to continuing uncertainty and disagreement, and  confidence in the funding requests based on those perceptions is  undermined. Moreover, the lack of a standard methodology for the  services to use in setting the goals removes a safeguard for objectivity  from the process, risking the possibility that the methods used do not  realistically reflect actual requirements. This risk is increased when there  is uncertainty or disagreement over basic questions such as whether it is  appropriate to base the goals on a historical analysis or an analysis of  actual requirements, and whether full funding of logistical support systems  should be assumed in an era of reduced funding. Furthermore, the absence  of information on the readiness and cost implications of setting the goals  at different levels results in a lack of understanding of the pros and cons of  available choices and the guiding principles used to make those decisions.  Ultimately, inappropriately set goals can unnecessarily raise or lower the  cost of spare parts inventories and other logistical resources by millions of  dollars.
Also, DOD’s instruction requires the services neither to designate one  office to coordinate the establishment and maintenance of aircraft  availability goals, nor to document the basis for the goals chosen or other  key issues in the process. Clear responsibilities and requirements in these  areas are fundamental to the effective management of any performance  system. Without the transparency provided by adequate documentation of  the process, neither DOD nor the Congress can be reasonably assured that  the services have selected the optimal goals on the basis of preferred  principles.
section_title: Recommendations for Executive Action
paragraphs: To ensure that aircraft availability goals and their performance measures  are appropriate to the new defense strategy and based on a clear and  defined process, we recommend that (1) DOD and the services determine  whether different types of aircraft availability goals are needed, (2) as  appropriate, DOD and the services validate the basis for the existing MC  and FMC goals, and (3) the Secretary of Defense revise DOD Instruction  3110.5 to    clearly define the specific aircraft availability goals required to be  established by the military services and their accompanying  performance measures;    establish a standard methodology identifying objective principles of  analysis to be used by all services in setting the goals, including an  identification of the readiness and cost implications of setting the goals  at different levels; and  require each service to identify one office to act as a focal point for  coordinating the development of the goals and for maintaining a  documentary record of the basis for the goals chosen and other key  decisions in the goal-setting process.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of this report, DOD concurred or partially  concurred with all our recommendations. The department agreed to  determine whether different types of aircraft availability goals are needed,  including the option of tailoring such goals to unique military service and  mission requirements. DOD also agreed to validate the basis for the  existing goals, including the DOD Instruction 3110.5 requirement that full  funding of support systems be assumed in establishing availability goals.  In addition, DOD indicated that it would explore alternative  methodologies for setting goals, such as one based on unit deployment  cycles currently in use by the Navy.
DOD partially concurred with our recommendation for a series of  revisions to DOD Instruction 3110.5. It agreed with our recommendation  that the instruction be revised to require each service to designate a focal  point for the development and historical documentation of the goal-setting  process. However, DOD did not agree with the part of our  recommendation calling for it to include the performance measures  associated with the aircraft availability goals in the instruction. DOD  believed that that requirement implied that those performance measures  should be the sole or primary measure of the overall state of materiel  readiness. That was not our intent. Our recommendation is meant to  ensure that the goals and accompanying performance/status measures  selected are clearly defined in the instruction. As pointed out in the report,  this is not currently the case. We agree that determinations of overall  materiel readiness require the consideration of a variety of factors, such as  maintenance manning and supply fill rates, as well as metrics such as an  aircraft’s availability. However, we believe that the instruction should  continue its current requirement to include performance/condition status  measures relative to those goals. Clearly identifying the goals that are  sought and their performance measures in the instruction will help avoid  further uncertainty and disagreement over the level of basic aircraft  performance, and does not preclude the consideration of other metrics in  broader assessments of overall readiness.  For these reasons, we believe  no change to our recommendation is needed.
DOD also disagreed with the part of our recommendation calling for the  Secretary of Defense to revise the instruction to establish a standard  methodology identifying objective principles of analysis to be used in  setting the goals. It believed that the services should establish the detailed  analytical methodology because the types of goals and their basis may  vary by service, and the services have a better understanding of the  differences and complexities of their individual environments. We agree  with the need for some leeway at the service level to handle individual  differences between them. However, we continue to believe that all  services should adhere to a standardized set of overarching principles of  analysis in order to safeguard objectivity and transparency in the goal  setting process. Such principles could be identified in coordination with  the services during the department’s planned evaluation of the basis for  the current goals and alternative methodologies. The services could then  develop detailed methodologies consistent with these principles but  tailored to their individual environments.  Consequently, no change to our  recommendation is required.
The department’s comments are reprinted in appendix III. DOD also  provided technical comments, which we incorporated as appropriate.
As arranged with your office, unless you publicly announce its contents  earlier, we plan no further distribution of this report until 30 days after the  date of this letter. At that time, we will send copies to the appropriate  congressional committees; the Secretaries of Defense, the Army, the Navy,  and the Air Force; the Commandant of the Marine Corps; and the Director,  Office of Management and Budget. We will also make copies available to  others upon request. In addition, the report will be available at no charge  on the GAO Web site at http://www.gao.gov.
Please contact me at (757) 552-8100 if you or your staff have any questions  concerning this report. The major contributors to this report are listed in  appendix IV.
section_title: Appendix I: Mission Capable Goals and Rates, Fiscal Years 1991-2002
paragraphs: Legend: MC = mission capable, FMC = fully mission capable, FY= fiscal year, EC/C = electronic  command and control.
Fiscal year 2002 rates are as of February for the Army, Navy, and Marine Corps, and March 31 for  the Air Force.
Aircraft ages are as of September 30, 2001 for the Navy/Marines; December 31, 2001, for the Air  Force; and April 2002 for the Army.
Aircraft costs/flying hour are as of January 2001 for the Army, and September 30, 2001, for the Air  Force, Navy, and Marines.
section_title: Appendix II: Scope and Methodology
paragraphs: To identify Department of Defense (DOD) and service policies and  practices regarding mission capable (MC) goals and rates, we obtained  briefings, reviewed DOD and service regulations and prior reports by us  and others; and interviewed officials at the Office of the Secretary of  Defense: the Office of the Joint Chiefs of Staff; headquarters offices of the  Army, Navy/Marine Corps, and Air Force; and aviation commands and  other locations as appropriate.
To determine whether key DOD aircraft were meeting established MC and  FMC goals, we requested that each service identify its key active-duty  operational aircraft. We excluded reserve units from the scope of our  review, as well as active-duty training units and executive aircraft used to  transport officials on official business. The resultant list included a total of  46 different models of aircraft from the four military services, which we  categorized into five basic types: bombers, cargo/tanker aircraft,  electronic command/control aircraft, fighter/attack aircraft, and  helicopters. Three aircraft models (F/A-18A, F/A-18C, and EA-6B) were  used by both the Navy and Marine Corps. For our review, we counted the  Navy and Marine Corps versions of these aircraft as separate models,  resulting in a total of 49 aircraft models for review. We requested MC and  full mission capable (FMC) goal and rate data, aircraft age and cost, and  other data for these aircraft back to 1991 to provide a historical  perspective on goal and rate history. The Army and Air Force provided  comprehensive data from fiscal year 1991 to mid-fiscal year 2002.  However, the Navy and Marine Corps could provide data separated by  service only from fiscal year 1998 forward. These services changed their  reporting system in 1998 and were unable to provide comparable data for  prior years. As a result, we focused our report on the 5-year period  beginning in fiscal year 1998. However, we included the full array of Army  and Air Force data in appendix I. We used these data to conduct analyses  of whether the aircraft were meeting their goals. We also provided each  service with these databases for review, and they confirmed the results for  accuracy.
To identify the causes of difficulties in meeting MC and FMC goals, we  reviewed prior reports by us and others and conducted a variety of  comparative analyses of our data by service, aircraft type, model, age,  cost, and fiscal year. We then held discussions with each service to gain  their perspectives on the causes of observed difficulties in meeting the  goals.
To determine whether DOD has a clear and defined process for setting MC  and FMC goals, we reviewed DOD Instruction 3110.5 and other regulations  and conducted discussions with officials from the Office of the Secretary  of Defense and service headquarters in Washington, D.C., and with  officials from the headquarters of the Air Force’s Air Mobility and Air  Combat Commands; the Naval Air Systems Command; and Army Training  and Doctrine Command officials at Fort Rucker, Alabama. Because of the  difficulty in obtaining clear information on this issue, we also wrote formal  letters of inquiry to the Secretaries of the Army and Navy requesting  clarification of how the goals were established. Their responses to those  letters of inquiry were used in preparing our report.
We performed our work from February through November 2002 in  accordance with generally accepted government auditing standards.  The  final publication of this report was delayed by the impact on DOD’s report  review and classification process of the terrorist attacks on September 11,  2001, and DOD’s preparations for potential conflict in Iraq.
section_title: Appendix III: Comments from the Department of Defense
section_title: Appendix IV: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Staff Acknowledgments
paragraphs: In addition to those named above, Bernice Benta, Katherine Chenault, and  R.K. Wild made key contributions to this report.