Military Training: Potential to Use Lessons Learned to Avoid Past Mistakes is Largely Untapped
Published: 1995-08-09T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: GAO reviewed the effectiveness of the military's lessons learned programs in: (1) collecting all significant lessons learned information; (2) analyzing the information to identify recurring weaknesses; (3) disseminating the information to all potential users; and (4) implementing corrective actions and validating results.
section_title: What GAO Found
paragraphs: GAO found that the Marine Corps, the Air Force, and the Navy do not: (1) include all significant information from training exercises and operations in their lessons learned programs; and (2) analyze their lessons learned information to identify trends in performance weaknesses. In addition, GAO found that: (1) Marine Corps lessons learned data continue to highlight recurring deficiencies during major combined arms exercises in such areas as maneuver, fire support, engineering, chemical threat, intelligence, communications, and electronic countermeasures; (2) although the dissemination of lessons learned information is adequate, the Air Force does not make its information readily available to all potential users; (3) regardless of the availability and widespread distribution of lessons learned information, most services use this information on a limited basis because they lack the training in how to access the databases; and (4) the Army has made excellent use of trend analysis to develop a corrective action plan to address the highest priority areas.

section_title: Letter
paragraphs: This report discusses lessons learned programs in the military services and the Joint Staff. It discusses how well the services and the Joint Staff collect, analyze, disseminate, and follow up on lessons learned information from major training exercises and operations. It contains recommendations to the Secretary of Defense that are aimed at making better use of lessons learned information to avoid past mistakes.
We are sending copies of this report to the Chairmen and Ranking Minority Members, Senate and House Committees on Appropriations, Senate Committee on Armed Services, and House Committee on National Security; the Secretary of Defense; the Chairman, Joint Chiefs of Staff; the Director, Office of Management and Budget; and other interested parties.
Please contact me at (202) 512-5140 if you or your staff have any questions concerning this report. Major contributors to this report are listed in appendix III.
subsections: section_title: Introduction
paragraphs: The military services rely on major training exercises to assess their units’ strengths and weaknesses. These exercises generally take place at combat training centers that often enable units to train in an environment that closely parallels that of actual warfare. The primary centers used for conducting major exercises include (1) the Army’s combat training centers at Fort Irwin, California; Fort Polk, Louisiana; and Hohenfels, Germany; (2) the Marine Corps’ Air Ground Combat Center at Twenty-nine Palms, California; and (3) the Air Force’s Weapons and Tactics Center at Nellis Air Force Base, Nevada. The Navy conducts major training exercises at the Naval Strike Warfare Center in Fallon, Nevada, and during worldwide fleet operations. Joint military exercises are conducted at many worldwide locations, including Germany, South Korea, Egypt, and Central America.
The services use electronic instrumentation, observers, and subject matter experts to monitor and record the results of the exercises so that they can objectively document performance. Additional information on the services’ capabilities is obtained from the results of actual military operations, such as Operation Desert Storm.
The services document the results of military training exercises and operations in after-action reports, which include lessons learned information. The units use such information in preparing for operations and environments associated with their assigned combat missions and in tailoring training for anticipated future missions and events.
In addition, lessons learned information can help the services and the Joint Staff identify recurring weaknesses in key areas. The services and the Joint Staff can then publicize problem areas and deficiency trends, allowing others to benefit from their experiences, and institute corrective actions. According to senior military leaders, weaknesses can be addressed through changes to such areas as doctrine, training and education, tactics, leadership, and materiel.
Our prior work has revealed that the Army has not effectively used lessons learned information to eliminate recurring deficiencies and change doctrine, revise tactics, or develop improved training strategies. In September 1993, we reported that, although the Army was doing a good job of identifying lessons learned, it was not achieving the maximum benefit from the lessons in terms of changed doctrine or revised training practices because it lacked procedures for prioritizing the lessons and for tracking necessary changes in training and doctrine. In July 1986, we reported that Army assessments of exercise results identified many recurring deficiencies, yet the Army had not developed a system to identify causes of and solutions to problem areas.
subsections: section_title: Lessons Learned Programs Vary Among the Services
paragraphs: Each service and the Joint Staff has its own program for incorporating lessons learned information into its operations. The Department of Defense (DOD) has no regulations that establish policies for or require uniformity among the services’ lessons learned programs. Since no overall guidance exists, the services have taken different approaches to developing and operating their programs. However, even though the programs differ, the services and the Joint Staff use after-action reports as the primary source of information for their programs. As an example, Army lessons learned program guidance states that lessons learned programs should (1) effectively gather, analyze, disseminate, and use lessons learned information so that actions can be taken to correct deficiencies and (2) have a means for testing or validating whether the corrective action actually resolves the deficiency.
subsections: section_title: Army
paragraphs: The Army’s lessons learned program started in 1986. It is run by the Center for Army Lessons Learned, which is operated by the Training and Doctrine Command. The Center has a staff of about 25 civilian and military analysts that collect observations from exercises and operations, develop trends of deficiencies, and publish the results of its analyses in bulletins and newsletters that receive widespread distribution throughout the Army.
section_title: Marine Corps
paragraphs: The Marine Corps’ lessons learned program started in 1989 to centralize lessons learned information and address deficiencies identified in after-action reports. The program is managed by the Marine Corps Combat Development Command and staffed with four full-time personnel. The program collects, processes, and disseminates lessons learned information throughout the Marine Corps through the use of Compact Disc—Read Only Memory (CD-ROM) technology. The Marine Corps also prepares special lessons learned reports for users on specific subjects, such as the Hurricane Andrew disaster in Florida and Operation Restore Hope in Somalia.
section_title: Navy
paragraphs: The Navy’s lessons learned program was established in 1991 at its headquarters in Washington, D.C., and reorganized in 1993 under the direction of the newly established Naval Doctrine Command in Norfolk, Virginia. The program is run by two naval officers at the Doctrine Command and a civilian who manages the database. In addition, four of the Navy’s major commands serve as management sites for lessons learned information. These sites screen lessons learned information submitted by naval units and decide, within their respective area of authority, what information is appropriate for the Navy’s lessons learned database.
The Navy’s program collects, evaluates, and disseminates lessons learned information on operational and tactical issues. Similar to the Marine Corps, the Navy distributes its lessons learned information to users through CD-ROM technology.
section_title: Air Force
paragraphs: The Air Force’s lessons learned program is the only one of the services’ programs that is decentralized. As a result, each of the Air Force’s six major operational commands is responsible for developing and managing its own lessons learned program. Air Force regulations do not require that the commands’ programs be uniform, so each command can take different approaches to operating its program. In fact, the four major commands that we visited or contacted during our review all had different lessons learned programs. The programs were designed to account for, act on, and share lessons learned information throughout the command, but not throughout the Air Force. The staffing levels for the lessons learned programs varied from one to three individuals.
The Air Force also operates a limited lessons learned program at its headquarters. This program, which is staffed by two people, addresses lessons learned information that results only from the Air Force’s participation in joint exercises or operations or that affects more than one of the major commands’ missions.
section_title: Joint Staff
paragraphs: The Joint Staff established the Joint Center for Lessons Learned to maintain and manage a centralized database on lessons learned information from joint military operations and exercises. This information, which includes ways to improve practices or overcome problems, is disseminated periodically among the services. The Center is staffed with two military analysts and one contractor representative who are assisted, when necessary, by representatives of each military service.
section_title: Objectives, Scope, and Methodology
paragraphs: We reviewed the lessons learned programs in the military services and the Joint Staff to determine their effectiveness in (1) collecting all significant lessons learned information, (2) analyzing lessons to identify recurring weaknesses, (3) disseminating lessons to all potential users, and (4) implementing corrective action and validating results. To do so, we reviewed the regulations and program guidance related to the lessons learned programs within each service and the Joint Staff and the policies and systems that implement the regulations and guidance. We determined how the services and the Joint Staff obtain, document, and input lessons learned data from participants in exercises and operations into their lessons learned programs. We examined the extent to which the services and the Joint Staff analyzed lessons learned information to develop trends that could highlight recurring deficiencies. Furthermore, we examined the methods and mediums the services and the Joint Staff used to provide lessons learned information to their units and analyze outputs provided from the systems.
We reviewed individual lessons learned reports contained in service and Joint Staff databases that showed the results of exercises and operations. We used this information to identify recurring deficiencies, including those that could affect the success or outcome of an exercise or operation. We also examined whether the services and the Joint Staff had remedial action systems to address deficiencies. We determined whether remedial action systems had procedures for measuring the effectiveness of solutions that were developed for deficiencies.
We interviewed service officials who managed the lessons learned programs to obtain their views on the programs. We also obtained the views of Army, Air Force, and Marine Corps officials at combat training centers as they repeatedly observed the performance of large numbers of units. In addition, we interviewed the leadership of selected units that participated in large-scale training exercises to determine how they used lessons learned information to improve their performance and how they generated lessons learned from their training or operational experiences. (See app. II for a list of the military organizations we visited or contacted during our work.) Information about the Army’s lessons learned program, however, is based primarily on issues developed in our September 1993 report and limited follow-up discussions with officials at the Center for Army Lessons Learned.
We performed our review from December 1993 to December 1994 in accordance with generally accepted government auditing standards.
section_title: All Significant Lessons Learned Are Not Collected
paragraphs: The effectiveness of the services’ lessons learned programs varies considerably. The Marine Corps, Air Force, and Navy programs provide only limited assurance that significant lessons documented in combat training center analyses, fleet exercises, and after-action reports are included in their databases. This information is extremely important since, in several instances, it discloses weaknesses displayed by many units during their most important training exercises—those conducted at the services’ combat training centers. Some of the weaknesses, if not corrected, could have serious consequences on a real battlefield. Until the services take steps to ensure that all lessons learned information is collected in their databases, units will continue to miss a significant opportunity to avoid repeating past mistakes.
subsections: section_title: Marine Corps
paragraphs: The cornerstone of Marine Corps’ ground unit training is the combined arms exercise conducted at the Air Ground Combat Center, which provides extensive ground training to units about once every 2 years. From this training, Marine Corps evaluators and senior leaders assigned to the Center prepare several reports, which include lessons learned information, that are not included in the Marine Corps lessons learned database. This information could be extremely valuable to commanders preparing for a future combat training center rotation, since it documents tasks that commanders did not perform well and provides examples of successful practices used by others to avoid similar problems. Moreover, the reports summarize performance trends over time and include independent observers’ assessments of the performance of weapon systems and the effectiveness of doctrine. However, these reports are not routinely included in the Marine Corps’ lessons learned database.
Some of the weaknesses discussed in these reports, if not corrected, could have serious consequences on a real battlefield. For example, one report said that indirect fire was placed on or behind friendly forces. This happened because of improper coordination and a lack of situational awareness. Recurring weaknesses in breaching operations—a critical component of any large-scale maneuver operation—are other examples of significant lessons learned information that were not captured by the lessons learned database. For example, a 1993 report prepared by the Center stated that a breaching operation failed because it was not rehearsed and coordination between the engineers and maneuver units was poor. In 1994, the Center again reported that coordination problems contributed to the breaching force being committed before the support force was in place and before the enemy defending the obstacle could be suppressed. The report also said that several vehicles were destroyed during the breaching operation because they veered outside prescribed lanes and into minefields.
In another 1994 lessons learned report, the Center noted a weakness in the handling of intelligence information that sometimes led to erroneous conclusions by commanders about enemy intentions and force composition. This weakness was attributed to intelligence information that was seldom analyzed and incorporated in commanders’ battle plans. As a result, commanders were often forced to react to enemy initiatives rather than be proactive in shaping the battlefield.
After-action reports prepared by participating units upon completion of their combat training center exercises are another important source of lessons learned information that was not fully captured. Even though units are required to submit after-action reports that document their performance at the Center, officials that manage the lessons learned program told us that they had received only about half of the reports required. According to these officials, noncompliance with the requirement is primarily due to the lack of emphasis throughout the chain of command.
To address this problem, the Commander of the Marine Corps Combat Development Command sent a message in October 1994 to all major combat and support commands that stressed the importance of the lessons learned program and the need to improve after-action reporting. A lessons learned official told us that his office also periodically sent messages to units encouraging them to send in lessons learned information, but normally they did not follow up to determine compliance.
section_title: Air Force
paragraphs: The Air Force’s Weapons and Tactics Center is the Air Forces’s premier training center for tactical fighter units. It provides aircrew and support training in a simulated combat environment. Active duty tactical fighter units rotate through the Center about every 18 months and reserve units are expected to rotate about every 2 years. Training Center observers who oversee exercises capture lessons learned information about unit performance and enter the lessons in the Center’s lessons learned database.
The Center’s database includes lessons in a variety of functional areas such as command, control, and communications; planning; and friendly fire incidents or fratricide. The following are examples of lessons that could help other units avoid repeating mistakes:
Airborne Warning and Control System controllers or escort fighter pilots identified friendly aircraft as enemy forces because they were not familiar with the entire group of friendly aircraft on an air strike mission. Also, because friendly forces did not respond to threat information, the controllers were unsure which aircraft had been given the threat information and therefore were not able to focus their attention on other pressing matters.
Air strike missions were conducted under compressed time frames because of inadequate planning. Also, escort aircraft did not provide unrestricted airspace for aircraft delivering munitions because of a failure to communicate plans. These deficiencies resulted in the ineffective delivery of bombs and missiles on targets.
Recurring incidents of fratricide resulted from multiple causes, including the aircraft’s fuel tank configuration and color, and pilots’ failure to check or select the proper modes or codes in their electronic identification equipment.
Because the Air Force’s lessons learned program is decentralized, each major operational command manages its own lessons learned program. Similar to the Marine Corps, Air Force lessons learned managers at one major command we visited were not successful in obtaining after-action reports covering units’ participation in various exercises and operations. For example, the command collected only five lessons learned from the numerous exercises and operations conducted by its subordinate units during 1991. Command personnel told us that this happened because units did not have the proper software for collecting lessons learned information.
Also, one of the command’s subordinate composite air wings did not know that the command had a lessons learned program or that it was supposed to submit lessons learned information to the command. The wing, however, maintained a lessons learned database that contained significant information related to the operations of its composite wing. This information might have been useful to other Air Force composite wings with similar operations. The database included the following information:
Mission commanders should use F-15E and F-16 fighter aircraft to protect B-1 and B-52 bombers after they leave the target area, and the fighters should have sufficient fuel to cover slow-moving bombers.
Radar systems identified friendly aircraft as hostile, thus causing other aircraft to assume that friendly aircraft were threats and disrupting the air strike mission of a group of friendly aircraft.
KC-135 tanker aircraft spacing should allow sufficient time for the first tanker to become airborne before the second tanker releases its brakes for takeoff. This margin of time would allow the first tanker to abort its mission, if necessary, without causing the second aircraft to abort.
Conversely, another Air Force major command had established a process to monitor the submission of after-action reports by subordinate units. Procedures for the submission of reports were specified in command regulations and were emphasized in operations orders prepared for each individual exercise. In addition, a control center within the command monitored subordinate units’ participation in exercises and operations and followed up to ensure that required reports were submitted.
section_title: Navy
paragraphs: The Navy’s lessons learned program does not collect all of the significant lessons learned information that is recorded during fleet exercises. Units record observations about their performance during these exercises in lessons learned reports. Also, ship commanding officers, training instructors, and key exercise leaders discuss units’ performance in after-action debriefings. Lessons learned observations are submitted through the chain of command to the Navy’s lessons learned database. However, reports of after-action debriefings are not entered into the lessons learned database.
Reports of after-action debriefings document units’ performance in areas such as air and surface warfare and weapons usage. Naval fleet personnel told us that because these are performance debriefings, the results are not entered into lessons learned database. Debrief participants also can pass on the performance results to others under their command.
section_title: Army and Joint Staff
paragraphs: Army lessons learned information is collected from a variety of sources, including after-action reports and evaluations from independent observers. The information is widely distributed and is readily available to Army units. Additionally, information from training center exercises is published periodically by the Center for Army Lessons Learned to keep units informed of observations and trends. For example, in March 1994, the Center reported the following observations on tactical deployments:
Units routinely deploy without necessary intelligence field manuals and adequate quantities of materials needed to secure tactical equipment during shipping.
Leaders do not plan for medical treatment and evacuation in all phases of a deployment.
Communications capability is not introduced early enough in the deployment to ensure mission success.
The regional commanders in chief do not use independent observers to document units’ performance during joint exercises. The units are required, however, to submit after-action reports to the Joint Staff upon completion of an exercise. To ensure that it receives after-action reports for all major exercises, the Joint Staff tracks the reports received against quarterly schedules of military exercises to identify any missing reports.
section_title: Conclusions
paragraphs: Until the services take steps to ensure that all significant lessons learned information is included in their databases, they will not realize the full potential of these assessments to make necessary changes in doctrine, tactics, training, or materiel. As a result, units are likely to continue to miss an opportunity to avoid repeating past mistakes, many of which could have serious consequences on a real battlefield.
section_title: Recommendations
paragraphs: We recommend that the Secretary of Defense direct the Secretaries of the Navy and the Air Force to establish controls to ensure that all significant lessons learned information collected from combat training centers, fleet exercises, and other major training exercises are recorded in the services’ lessons learned databases.
section_title: Agency Comments and Our Evaluation
paragraphs: DOD generally agreed with our recommendation. According to DOD, the Marine Corps plans to collect trend information on unit performance at its combat training center and will include this information in its lessons learned database. The Navy has taken steps through such means as fleet operational orders, awareness messages, and increased training to ensure that lessons learned from major naval exercises are recorded in its database. DOD said that the Air Force records most lessons learned from combat training center exercises in a lessons learned database that is maintained at the combat training center. Because this database is not currently available to other Air Force major commands, we believe that the Air Force is missing an important opportunity to share these lessons learned with system users throughout the Air Force.
section_title: Lessons Learned Information Is Not Routinely Analyzed, and Mistakes Continue to Be Repeated
paragraphs: The services and the regional commanders in chief continue to repeat mistakes during military operations and major training exercises. For example, a recent Air Force lessons learned report stated that almost every problem occurring during Operation Restore Hope had been documented in a lessons learned report on previous exercises or contingencies. However, the Army is the only service that analyzes lessons learned information to identify recurring weaknesses. As a result, the other services and the Joint Staff cannot be assured that significant problems are identified and receive top-level management attention.
Two key steps must be completed if the Marine Corps, the Air Force, the Navy, and the Joint Staff are to identify and correct their most significant recurring deficiencies. The first step is to perform trend analyses of lessons learned information, which can highlight recurring weaknesses over a period of time. The second step is to rank the various problems on the basis of their significance. Completing these steps would allow the services and the Joint Staff to focus on correcting the highest priority problems.
subsections: section_title: Mistakes Continue to Be Repeated During Training Exercises and Operations
paragraphs: Over a number of years, lessons learned reports in each of the services and the Joint Staff have shown that many mistakes continue to be repeated in training exercises and military operations. These mistakes fall into different categories, including communications, fratricide, battlefield planning, reconnaissance, maneuver, combat engineering, chemical threat, fire support, and combat service support.
subsections: section_title: Marine Corps
paragraphs: In 1993, the Marine Corps published lessons learned information that summarized about 9 years of unit performance during combined arms exercises. This information disclosed recurring deficiencies, including (1) indirect fire being placed on or behind friendly forces; (2) inadequacies in several phases of obstacle breaching operations; (3) ineffective preparation of engagement areas, which is critical to stopping or slowing an enemy advance; and (4) units’ inability to integrate supporting arms and maneuver to destroy the enemy.
A lessons learned report from February 1990 also documented numerous training deficiencies that had been cited in previous reports. Significant deficiencies included ones in the areas of surveillance, target acquisition, and reconnaissance; camouflage/concealment; nuclear, biological, and chemical defense; electronic countermeasures; and communications.
Many of these deficiencies are still occurring today in combat training center exercises.
section_title: Air Force
paragraphs: A major recurring weakness that has been reported over a number of years is the inadequate communication of air tasking orders. These orders provide information to aircraft in the same or other military services and are necessary to coordinate a specific operational mission within an area of operation. According to lessons learned reports, inadequate communication of air tasking orders could result in serious consequences, including friendly fire losses.
Problems relating to air tasking orders were most recently reported during 1994 operations in Haiti, yet these problems were identified 4 years earlier during the Gulf War. A 1990-91 lessons learned report found that air tasking orders were inadequately transmitted among the services during Operations Desert Shield and Desert Storm. Likewise, a 1992 report said that the Air Force used a different communications system than the other services and lacked a standardized format for air tasking orders.
Other recurring deficiencies were illustrated in a 1993 report on lessons learned from Operation Restore Hope in Somalia. For example, the Air Force deployed an airlift communications system, which was to assist in air mobility operations, without qualified operators and training guides. The lack of trained, qualified operators resulted in delays in communicating mission-essential information and hampered the use of an important piece of communications equipment in an area where communications were important but limited. The report stated that this deficiency had occurred in 1990 and 1991 during Operations Desert Shield and Desert Storm. “Almost every problem occurring during Operation Restore Hope has already been documented in JULLS  as a result of previous exercises and contingencies. There appears to be a continuing trend of failure to fix problems already know  to exist. We end up paying again to achieve the same undesirable results.”
section_title: Navy
paragraphs: The Navy does not enter lessons learned information into its database if the lessons are similar to those that were previously reported and recorded. Therefore, Navy officials told us that, although it is difficult to identify recurring deficiencies through the lessons learned database, such problems do exist. The database showed that, at least as far back as 1989, (1) friendly force identification codes were not used properly and (2) several different air tasking order problems were experienced, including orders that contained inaccuracies regarding the capabilities of carriers and airwings, demonstrated improper planning to carry out air strikes, and went to several organizations that did not have a need for the orders.
section_title: Army
paragraphs: Since the establishment of the Army’s combat training centers in the 1980s, the Center for Army Lessons Learned has documented a number of recurring deficiencies in units’ performance. For example, a 1992 lessons learned report stated that the following problems continued to be repeated: (1) direct fire was not synchronized effectively; (2) reconnaissance and surveillance plans were not well coordinated, managed, or focused; (3) communications with higher headquarters were not properly planned and executed; (4) fire support plans did not support the scheme of maneuver; and (5) operations in a chemical environment were not satisfactory. Many of these same problems continue today.
section_title: Joint Staff
paragraphs: Regional commanders in chief have reported recurring deficiencies during training exercises and operations, including Just Cause in Panama (1989), Desert Shield and Desert Storm, and Restore Hope. According to a 1991 lessons learned report, one recurring problem was the lack of adequate training on the joint transportation planning and management system. This system schedules and manages strategic air and sea movements during peacetime and wartime. Joint Staff officials said that this problem had come up during almost every exercise since the early 1980s.
Another recurring problem has been inadequate training of personnel involved in the formation and operation of a Joint Task Force headquarters. For example, the task force headquarters for Operation Restore Hope, which included personnel from all services, was formed on an ad hoc basis after deployment. According to a lessons learned report from the operation, this situation resulted in inefficient planning, confusion, and a less-than-optimal deployment. Similar problems with this issue have been reported since the late 1980s.
section_title: Most Services Do Not Perform Trend Analyses or Prioritize Recurring Deficiencies
paragraphs: The Marine Corps, the Air Force, the Navy, and the Joint Staff do not analyze their lessons learned information to identify trends in performance weaknesses. Accordingly, it is difficult for them to differentiate the importance of correcting some deficiencies rather than others. On the other hand, the Army analyzes lessons learned information over time, which enables it to highlight the most pressing problem areas and focus on the highest priority areas.
subsections: section_title: Marine Corps, Air Force, Navy, and Joint Staff
paragraphs: Lessons learned program guidance does not require the Marine Corps, the Air Force, the Navy, or the Joint Staff to perform trend analyses. However, service and Joint Staff officials told us they believed trend analyses would be useful to them.
Marine Corps operations personnel in several units told us that trend analyses could highlight recurring deficiencies and that knowledge of these deficiencies would be especially useful in preparing for major training exercises because their units would have a better opportunity to overcome past mistakes. In 1993, the Marine Corps Combat Development Command developed a proposal to examine recurring operational and training deficiencies. Under the proposal, the Command’s Studies and Analysis Group would develop trends based on Marine Corps units’ performance over a number of exercises. The group could then identify recurring deficiencies and recommend corrective actions. The proposal was approved by the Commander of the Combat Development Command in November 1993 but has not been implemented. As of May 1995, a group analyst said that the delay in implementing the proposal was due to resource limitations and the inability to obtain more in-depth training data from the Air Ground Combat Center.
Air Force regulations do not require the major commands to develop trend analyses of lessons learned information. Nevertheless, the lessons learned program director at Air Force headquarters told us that one of the program’s most noted weaknesses was the lack of assigning priorities to performance deficiencies. According to this official, since trend analyses and prioritization are not being accomplished at the Air Force’s major commands, it is difficult for decisionmakers to differentiate the significance of problem areas.
The Navy’s lessons learned database does not contain the information necessary to perform trend analyses because the system screens out duplicate or similar deficiencies. Navy fleet operations personnel told us that they seldom used lessons learned information because of the high volume of unprioritized information in the database and the time constraints associated with their day-to-day operations.
Even though Joint Staff program guidance does not require trend analyses of lessons learned information, program officials said that information was available in their database to perform such analyses. However, they said that a shortage of resources precluded them from routinely analyzing the information.
Although Navy, Marine Corps, Air Force, and Joint Staff officials acknowledged that trend information was not routinely analyzed to highlight recurring deficiencies, they said that officials in leadership positions gained an awareness of the most significant problems through informal means such as conferences, meetings, and exercise planning discussions. In our view, the informal approach has not worked well, as recurring deficiencies have not been resolved. Moreover, reliance on an informal approach to problem solving does not provide for program continuity as military personnel are subject to periodic reassignment.
section_title: Army
paragraphs: The Center for Army Lessons Learned is responsible for identifying systemic training strengths and weaknesses of units that participate in major operations and exercises. After documenting lessons learned, the Center consolidates the information and analyzes trends and deficiencies. Under an ongoing Army proposal, these performance trends are expected to provide the basis for developing a priority issue list that ranks the importance of problems affecting war-fighting capabilities. According to an Army official, the priority issue list would enable Army leaders to establish clear priorities for those problems it deems most serious, identify the participants involved and establish accountability, and estimate the resources required to resolve problems. The Army expects to have this process in place by September 1995.
The Army has recently made excellent use of trend analyses. For example, the Army analyzed the extent of friendly fire incidents at its National Training Center from 1990 to 1993 and developed a corrective action plan to address this serious deficiency. Recent data shows that friendly fire-related incidents at the Center have decreased over 50 percent since 1990.
section_title: Conclusions
paragraphs: Military units continue to experience recurring deficiencies in exercises and operations, even though the services and the Joint Staff have lessons learned programs. This situation is unlikely to change markedly until the services and the Joint Staff begin to make better use of the wealth of lessons learned information contained in their databases. As it is now, the lessons are of limited value to military trainers because they provide no systematic insight to recurring deficiencies.
section_title: Recommendations
paragraphs: We recommend that the Secretary of Defense direct the Secretaries of the Navy and the Air Force and the Chairman of the Joint Chiefs of Staff to (1) analyze lessons learned information so that trend data can be developed to identify recurring deficiencies and (2) prioritize these recurring deficiencies so that limited resources can be concentrated on the most pressing problems.
To facilitate trend analyses in the Navy, we recommend that the Secretary of Defense direct the Secretary of the Navy to modify the Navy’s lessons learned program to retain all significant lessons learned from operations and exercises.
section_title: Agency Comments and Our Evaluation
paragraphs: DOD agreed with our recommendations as they applied to the Navy. It said that the Navy plans to implement a process, beginning in the first quarter of fiscal year 1996, to capture and retain all significant lessons learned from operations and exercises. Moreover, the Navy will analyze and identify trends in performance weaknesses through its newly established remedial action program. However, DOD said that trend analyses in the Air Force was unnecessary because the Air Force acted on deficiencies as they were identified. While this may be true for deficiencies recorded in the lessons learned database maintained at Air Force headquarters, DOD officials acknowledged this was not the case for the lessons learned that are recorded by the major commands. Until the Air Force undertakes trend analyses that systematically identifies and highlights recurring deficiencies in the major commands, there is no assurance that significant problems will be addressed and corrected.
DOD said the Joint Staff believes that trend analyses would be worthwhile, but that it is not sufficiently resourced to conduct such analyses at this time. Given the significance of the potential value that can be gained from such an analysis, for example, identifying matters that can make a difference between success or defeat on the battlefield, we believe that this is a matter that the Chairman of the Joint Chiefs of Staff should carefully review.
DOD did not agree with our conclusion that the Marine Corps does not analyze lessons learned information. DOD said that the Marine Corps analyzes lessons learned information through its remedial action and combat development processes. However, these processes address only those one-time deficiencies that the Marine Corps selects for remedial action. In the absence of a systematic process to analyze the lessons learned database to identify trends, the Marine Corps may be overlooking deficiencies of a recurring nature that warrant remedial action.
section_title: Many Factors Impede Units’ Ability to Use Lessons Learned Information
paragraphs: The Air Force does not routinely distribute lessons learned information throughout the Air Force. As a result, information from the major commands’ lessons learned databases is not reaching all potential users. The Joint Staff, the Army, the Navy, and the Marine Corps routinely distribute lessons learned information, and their users can access the information as needed. However, most of the services use this information only on a limited basis. The primary reason for this situation is that users lack the training necessary to access the high volume of information in the databases.
subsections: section_title: Dissemination of Lessons Learned Information Is Generally Adequate
paragraphs: The Air Force does not disseminate lessons learned information to its units on a routine basis because it does not have a centralized lessons learned program. Also, Air Force units only have access to lessons learned information from their own major command. Therefore, the units cannot benefit from the experiences of other Air Force units. Unit personnel told us that Air Force-wide lessons learned information would be beneficial in planning future exercises and operations.
Air Force units must specifically request lessons learned information from their major commands. If the information is available, it is sent to the units in the mail. However, units do not frequently request lessons learned information. For example, one major Air Force command maintained over 4,000 lessons learned reports in its database at command headquarters, yet in 1994 it received, on average, only 1 request for information per week from its subordinate units. Command officials told us that in 1993 they had received only about 30 requests for information. The official who managed this lessons learned program acknowledged that the dissemination of information was not very good and needed to be improved. Air Force personnel in one unit stated that their major command’s database was not very useful since it was not accessible to them.
Air Force lessons learned officials recognized the limitations of a decentralized lessons learned program, and they were attempting to improve access to program information. As of June 1995, the Air Force was developing a computer network that would provide access to lessons learned information throughout the Air Force. Once this capability is achieved, units within major commands throughout the Air Force should have better access to lessons learned information. One of the major commands that we visited plans to achieve this capability later in 1995. Another major command is testing the network. However, until this network becomes fully operational throughout the Air Force and is proven effective, units will continue to have limited access to important lessons learned information.
Navy lessons learned information is available to over 1,000 major and intermediate-level commands, specialized operational units, and individual ships. Until recently, Navy organizations had to request that they be included on the lessons learned distribution list to receive such information. As a result, all naval units may not have been receiving the information. In early 1995, the Navy took action to ensure that all commands, units, and ships were receiving lessons learned information.
The remaining services and the Joint Staff also provide access to their information. The Marine Corps distributes lessons learned information to over 500 organizations, principally units down through the battalion level. The Army periodically publishes this information in bulletins and newsletters that are sent to each Army specialty school and most other organizations throughout the Army. The Joint Staff routinely distributes lessons learned information to its major command organizations and to the other services such as the Navy, which publishes the Joint Staff database on CD-ROM along with its own lessons learned information.
section_title: Services’ Use of Available Lessons Learned Information Is Limited
paragraphs: Regardless of the availability and widespread distribution of lessons learned information, most services have used this information only on a limited basis. The principal reason for not making greater use of the information is the lack of training in how to easily access the databases.
subsections: section_title: Marine Corps
paragraphs: According to Marine Corps personnel, units do not use lessons learned information because users possess limited training and knowledge on how to access information in the system or how to process available information in a timely manner. For example, a unit representative told us that he had been in a headquarters organization for over 1 year, but knew of no one who had used the lessons learned database to obtain information. An officer from this unit attributed this fact to the users’ unfamiliarity with the information in the database and lack of training on how to use CD-ROM technology.
Lessons learned officials from the Marine Corps Combat Development Command recognized that users had problems with the CD-ROM technology needed to access the database and took steps in 1994 to expand training in this area. Specifically, these lessons learned officials began to regularly schedule visits to units to provide unit personnel with hands-on training on the operation of the lessons learned database and information on its benefits.
section_title: Navy
paragraphs: The Navy’s lessons learned database contains over 4,000 unprioritized reports. Accordingly, to use the system effectively, users must possess the skills needed to access the information and identify the most pressing problems. Some Navy fleet operations personnel told us that they seldom used lessons learned information because their operating tempo was extremely high and they had not been trained to use the system to quickly access specific lessons learned information. For example, several officers with submarine backgrounds said that they relied on other mechanisms for lessons learned to identify submarine-related lessons.
Some Atlantic Fleet staff officers said that they seldom used the Navy’s lessons learned database and felt no need to do so. They relied instead on more ad hoc systems to obtain lessons learned information. They specifically cited Navy message traffic, newsletters, bulletins, and discussions with their counterparts on other ships as sources of information. They also said that the lack of knowledge about the system and how to quickly access information hindered them from using the lessons learned database.
The manager of the Navy’s lessons learned database acknowledged that training for fleet personnel in the use of the system could be improved. He cited personnel turnover as a principal cause for some users’ unfamiliarity with the system. Further, he said that this situation was likely to continue until training became widespread.
section_title: Air Force
paragraphs: One Air Force unit that we visited did not use lessons learned information from its major command’s database because unit personnel did not know that a lessons learned database existed at the major command. It was for this reason that personnel at this unit told us they had never requested any lessons learned information from their major command. At another Air Force unit, personnel were aware that lessons learned information was maintained at their major command; nevertheless, they had used the database very little because they lacked knowledge of the database’s detailed information and because they had no quick, ready mechanism to access or obtain this information. Unit personnel had requested lessons learned information from their major command on several occasions, and it was provided to them through the mail. However, unit officials told us that requesting and obtaining information through the mail was time-consuming.
section_title: Army
paragraphs: Primary users of Army lessons learned information are the Training and Doctrine Command’s 18 schools, which develop training programs for Army personnel in their military specialties and tactical units. These schools are ultimately responsible for using lessons learned information to modify training and doctrine. Even though officials at several schools told us that they used lessons learned information to develop training plans and to update doctrine, they said that they did not keep track of how training and doctrine were modified based on this information. Likewise, the leadership of several Army units said that they used lessons learned information to prepare for major training events but did not keep track of how this information was used during training.
section_title: Conclusions
paragraphs: It is clear that the services are not maximizing the potential benefits of lessons learned information. For the most part, the dissemination of lessons learned information by the Joint Staff, the Army, the Navy, and the Marine Corps is adequate. The Air Force’s ongoing effort to establish a computer network that will provide access to lessons learned information throughout the Air Force could solve its dissemination problem. However, dissemination of lessons information is only the first step necessary to facilitate units’ use of the information. To better facilitate the use of lessons information, Air Force and Navy personnel need to possess skills necessary to access lessons in their services’ databases. The Marine Corps’ ongoing effort to provide unit personnel with the skills needed to access their lessons database is a step in the right direction.
section_title: Recommendations
paragraphs: We recommend that the Secretary of Defense direct the Secretaries of the Navy and the Air Force to provide training to key personnel in the use of lessons learned information and the technology for accessing and reviewing this information.
section_title: Agency Comments and Our Evaluation
paragraphs: DOD agreed with our recommendation. DOD said that the Navy had selected a more user friendly computer program to make the Navy lessons learned database more accessible to personnel and was working to incorporate lessons learned system training into various officer and selected enlisted schools. Also, DOD said that the Air Force is planning steps to ensure that its major commands provide training in the use of the lessons learned system. Moreover, the Air Force expects to improve the distribution of lessons learned information by implementing a wide area network throughout its major commands by the end of fiscal year 1996.
section_title: Lessons Learned Programs Lack Sufficient Follow-Up and Validation
paragraphs: Effective follow-up and validation are important parts of a lessons learned program since they are the only means for ensuring that problems have been corrected and are brought to closure. However, the Navy only recently implemented a follow-up process, and the Army does not expect to have a process in place to address training and doctrinal deficiencies until September 1995. The Marine Corps, the Joint Staff, and one of the Air Force commands that we visited seemed to have visibility over the status of corrective actions. Even though most of the services and the Joint Staff have requirements to validate corrective actions, not all of them have fully implemented procedures for this purpose.
subsections: section_title: Remedial Action Processes Vary
paragraphs: An important part of a lessons learned program is a remedial action process to track and follow up on actions taken to address problems. The remedial action process generally involves identifying problems, assigning responsibility for the problems, and monitoring corrective actions taken. However, one of the services does not have a remedial action process in place to address training and doctrinal issues, and another service only recently established one. The other services’ processes vary in effectiveness.
subsections: section_title: Marine Corps
paragraphs: Although the Marine Corps’ lessons learned program was established in 1989, the remedial action element of the program did not become operational until 1991. A Marine Corps lessons learned program official said that corrective actions are monitored primarily through the combat development process, which is a formal process that identifies battlefield requirements and develops combat capabilities. On the basis of our review of a sample of remedial action items, we found that the Marine Corps was able to successfully track the status of corrective actions through the combat development process.
section_title: Air Force and Joint Staff
paragraphs: The Air Force has directed each of its major commands to establish a remedial action element for its lessons learned program. However, the quality of remedial action processes in place at the major commands varies. For example, one of the commands we visited had only recently begun to systematically track corrective actions taken to address problem areas. A command official told us that before October 1994 the status of corrective actions could not be readily determined. According to the official, functional offices within the command were tasked to develop solutions to problems. However, the command had no systematic tracking system to determine the status of corrective actions. To correct this situation and improve its ability to track corrective actions, the command developed a spreadsheet to document the status of corrective actions.
In contrast, another major command we visited had implemented procedures to assign responsibility for solutions and systematically track the status of corrective actions. The office responsible for solving a problem is required to provide periodic status reports to the major command. On the basis of our review of a sample of lessons learned reports, we found that the command had visibility over the development and implementation of corrective actions.
The Joint Staff employs a similar remedial action process to that of the Air Force major command discussed previously. It assigns responsibility for developing solutions to problems of a joint nature to its own offices, or those within the services. These offices periodically report their progress to the Joint Staff, and the status of corrective actions is recorded as part of Joint Staff lessons learned reports. On the basis of our review of a sample of lessons learned reports, we found the Joint Staff had visibility over the status of corrective actions.
section_title: Navy and Army
paragraphs: The Navy did not establish a remedial action process for its lessons learned program until January 1995. Before that time, the lessons learned program was limited to providing information on operational issues for use by fleet personnel. As of May 1995, however, the Navy had not addressed any deficiencies through its remedial action process.
In September 1993, the Army’s Training and Doctrine Command began developing a remedial action process that would address lessons learned pertaining to training and doctrine deficiencies that it deemed most critical. Under this process, the Army plans to establish accountability for problem resolution and monitoring progress. The Army expects the process to be in place by September 1995.
section_title: Lessons Learned Programs Lack an Effective Validation Element
paragraphs: Validation of corrective actions (for example, testing the effectiveness of actions taken to correct deficiencies) can ensure that recurring deficiencies have been resolved and brought to closure. Validation can be accomplished by evaluating the effectiveness of potential solutions during a training exercise. However, the Navy does not require that its lessons learned program contain a validation element. The Army also does not formally validate solutions to deficiencies. However, the Army’s proposed enhancements to its lessons learned program would recognize the benefits of validation. Specifically, the Army plans to include a validation element in its remedial action process and test solutions to deficiencies through training exercises. As stated earlier, the Army expects the remedial action process to become operational by September 1995.
In contrast, the Marine Corps and the Air Force require validation. The Marine Corps requires that corrective actions be validated through its combat development process. Air Force guidance requires major commands to incorporate a validation element in their lessons learned program. However, only two of the four major commands we contacted had done so.
Joint Staff guidance states that validation is necessary to ensure the effectiveness of corrective actions taken to resolve problems. However, officials said that it is left to regional commanders in chief to determine whether corrective actions will be tested in training exercises. The Joint Staff permits open items to be closed by means other than testing, such as a determination by senior officials that all corrective actions were completed and that the actions taken had solved the problem. Joint Staff officials said that insufficient staffing was the principal reason for not taking a stronger oversight role in the validation process.
section_title: Conclusions
paragraphs: Without adequate follow-up and validation in remedial action processes, lessons learned programs can only be used to identify and distribute information about problems rather than to track and validate that solutions work. Until the services and the Joint Staff establish effective follow-up and validation procedures in their lessons learned programs, there will be little assurance that problems have been brought to closure and the possibility for repeating past mistakes will remain.
section_title: Recommendations
paragraphs: We recommend that the Secretary of Defense direct the Secretary of the Navy to incorporate a validation process into the Navy’s lessons learned program, the Secretary of the Air Force to take actions to ensure that each of the major commands complies with existing program guidance calling for the establishment of a validation process for their lessons learned programs, and the regional commanders in chief to ensure that solutions to deficiencies are tested in joint exercises or, if this is not appropriate, validated through alternative means.
section_title: Agency Comments and Our Evaluation
paragraphs: DOD agreed with our first two recommendations. DOD said that, as part of its lessons learned program, the Navy had established a remedial action program working group that will validate lessons learned. It also said that the Air Force would take action to ensure that the major commands establish a validation process for their lessons learned programs. Specifically, DOD said that Air Force headquarters will increase its oversight of lessons learned programs by monitoring the minutes of remedial action plan meetings conducted by the major commands and by assessing the commands’ compliance with program guidance.
A draft of this report recommended that the regional commanders in chief establish formal procedures to ensure that solutions to deficiencies are tested and validated. DOD said that this recommendation was unnecessary because current program guidance contained formal procedures to test corrective actions through the Joint Staff’s remedial action program. Although we agree that formal procedures for testing already exist, we found that commanders in chief seldom tested whether prior problems had been corrected in their exercises because (1) they were not required to do so and (2) they had insufficient time to analyze past problems before planning future exercises. We believe that testing solutions to problem areas in exercises is a vital part of assessing the capabilities of the regional commanders in chief to support national security strategies. Further, the failure to conduct such testing, when appropriate, reduces the effectiveness of collecting data on problems and, in our opinion, is a major reason contributing to recurring problems. Accordingly, we modified our recommendation to stress the importance of testing remedial actions and to recognize that, in some instances, it may be appropriate to close remedial action projects if their effectiveness can be demonstrated through alternative means.