Military Readiness: New Reporting System Is Intended to Address Long-Standing Problems, but Better Planning Is Needed
Published: 2003-03-28T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Department of Defense's (DOD) readiness assessment system was designed to assess the ability of units and joint forces to fight and meet the demands of the national security strategy. In 1998, GAO concluded that the readiness reports provided to Congress were vague and ineffective as oversight tools. Since that time, Congress added reporting requirements to enhance its oversight of military readiness. Therefore, the Chairman asked GAO to examine (1) the progress DOD made in resolving issues raised in the 1998 GAO report on both the unit-level readiness reporting system and the lack of specificity in DOD's Quarterly Readiness Reports to the Congress, (2) the extent to which DOD has complied with legislative reporting requirements enacted since 1997, and (3) DOD's plans to improve readiness reporting.
section_title: What GAO Found
paragraphs: Since 1998, DOD has made some progress in improving readiness reporting--particularly at the unit level--but some issues remain. For example, DOD uses readiness measures that vary 10 percentage points or more to determine readiness ratings and often does not report the precise measurements outside DOD. DOD included more information in its Quarterly Readiness Reports to the Congress. But quality issues remain--in that the reports do not specifically describe readiness problems, their effects on readiness, or remedial actions to correct problems. Nor do the reports contain information about funding programmed to address specific remedial actions. Although current law does not specifically require this information, Congress could use it for its oversight role. DOD complied with most, though not all, of the legislative readiness reporting requirements enacted by Congress in the National Defense Authorization Acts for Fiscal Years 1998-2002. For example, DOD (1) is now listing the individual units that have reported low readiness and reporting on the readiness of prepositioned equipment, as required by the fiscal year 1998 Act; (2) is reporting on 11 of 19 readiness indicators that commanders identified as important and that Congress required to be added to the quarterly reports in the fiscal year 1998 Act, but is not reporting on the other 8 readiness indicators; and (3) has not yet implemented a new comprehensive readiness reporting system as required in the fiscal year 1999 Act. As a result, Congress is not receiving all the information mandated by law. DOD issued a directive in June 2002 to establish a new comprehensive readiness reporting system that DOD officials said they plan to use to comply with the reporting requirements specified by Congress. The new system is intended to implement many of the recommendations included in a congressionally directed independent study for establishing such a system. However, the extent to which the new system will actually address the current system's shortcomings is unknown, because the new system is currently only a concept, and full capability is not scheduled until 2007. As of January 2003, DOD had not developed an implementation plan containing measurable performance goals, identification of resources, performance indicators, and an evaluation plan to assess progress in developing the new reporting system. Without such a plan, neither DOD nor the Congress will be able to fully assess whether the new system's development is on schedule and achieving desired results.

section_title: Letter
paragraphs: The Department of Defense’s (DOD) readiness assessment system was  designed to assess the ability of units and joint forces to fight and meet the  demands of the national security strategy. For more than a decade ending  in 1998, various audit and oversight organizations questioned the  thoroughness and reliability of DOD reports on the readiness of U.S.  forces. Since 1998, Congress has added reporting requirements to enhance  its oversight of military readiness. In doing so, Congress expressed  concern over contradictions between assessments of military unit  readiness in reports and observations made by military personnel in the  field.
DOD provides Congress a quarterly report that contains readiness  information from several sources: the unit-level readiness assessment  system; results of scenario-based assessments; and summaries of  information briefed to senior DOD officials. We reviewed DOD’s readiness  assessment and reporting system in 1998 and concluded that the readiness  reports provided Congress offered a vague description of readiness  problems and remedial actions and therefore were not effective as  oversight tools.
Considering the concerns raised in our 1998 report, reporting  requirements added by Congress since 1998, and the new national security  strategy, you asked us to provide an updated assessment of DOD’s  readiness reporting. As agreed with your office, we examined (1) the  progress DOD has made in resolving issues raised in our prior report on  both the unit level readiness reporting system and the lack of specificity in  the Department’s Quarterly Readiness Reports to the Congress, (2) the  extent to which DOD has complied with legislative reporting requirements  enacted since 1997, and (3) DOD’s plans to improve readiness reporting. In  conducting this analysis, we compared current reported readiness data  with legislative requirements and with data reported in 1998. We also  identified DOD initiatives for improving readiness reporting. We  conducted our review from June 2002 through January 2003 in accordance  with generally accepted government auditing standards. (For a complete  description of our methodology, see app. I.)
subsections: section_title: Background
paragraphs: DOD’s readiness assessment and reporting system was designed to assess  and report on military readiness at three levels—(1) the unit level; (2) the  joint force level; and (3) the aggregate, or strategic, level. Unit-level  readiness is assessed with the Global Status of Resources and Training  System (GSORTS), which is an automated system that assesses the extent  to which military units possess the required resources and training to  undertake their wartime missions. To address joint readiness, the  Chairman of the Joint Chiefs of Staff established the Joint Monthly  Readiness Review (now called the Joint Quarterly Readiness Review or  JQRR), that compiles readiness assessments from the combatant  commands, the combat support agencies, and the military services. The  Joint Staff and the services use these assessments to brief DOD’s  leadership on the Senior Readiness Oversight Council—an executive-level  forum for monitoring emerging readiness issues at the strategic level. The  briefings to the council are intended to present a view of readiness at the  aggregate force level. From these briefings to the council, DOD prepares a  legislatively mandated quarterly readiness report to Congress. Figure 1  provides an overview of DOD’s readiness assessment process.
We have issued several reports containing recommendations for  improving readiness reporting. In 1994, we recommended DOD develop a  more comprehensive readiness system to include 26 specific readiness  indicators. In 1998, we reported on shortcomings in DOD’s readiness  assessment system. At that time, we stated GSORTS’ limitations included  lack of precision in measurements, late reporting, subjective input, and  lack of standardization. Secondly, we reported that while the Quarterly  Readiness Reports to the Congress accurately reflected briefs to the  Senior Readiness Oversight Council, they lacked specific details on  deficiencies and remedial actions and thus did not meet the requirements  of 10 U.S.C. 482 (b). DOD concurred with our recommendation that the  Secretary of Defense take steps to better fulfill the legislative reporting  requirements under 10 U.S.C. 482 by providing (1) supporting data on key  readiness deficiencies and (2) specific information on planned remedial  actions. Finally, we reported that deficiencies identified as a result of the  Joint Monthly Readiness Reviews remained open because the solutions  require funding over the long term. In 2002, we issued a classified report  on DOD’s process for tracking the status of deficiencies identified in the  Joint Monthly Readiness Reviews. We made recommendations to improve  DOD’s deficiency status reporting system and for DOD to develop funding  estimates for correcting critical readiness deficiencies. In its comments,  DOD generally agreed with the report’s findings and recommendations.
section_title: Some Progress in Readiness Reporting, but Some Issues Remain
paragraphs: Although DOD has made progress in resolving readiness reporting issues  raised in our 1998 report, we found that some of the same issues still exist  today. For example, DOD has added information to its Quarterly  Readiness Reports to the Congress (hereafter referred to as the quarterly  reports). However, we found that the reports still contain vague  descriptions of readiness problems and remedial actions. Even though  some report annexes contain detailed data, the data as presented are not  “user friendly”—it is largely unevaluated and is not linked to readiness  issues mentioned in the report plus the report text does not explain how  the data relates to units’ readiness. Thus, as we reported in 1998, these  reports do not specifically describe readiness problems or remedial  actions as required under 10 U.S.C. 482 (b). We believe that this kind of  information would be useful for Congress to understand the significance  of the information in these reports for use in its oversight role.
subsections: section_title: Some Improvements in Unit-Level Reporting, but Some Issues Unchanged Since 1998
paragraphs: DOD has improved some aspects of its unit-level reporting system, the  Global Status of Resources and Training System (GSORTS). For example,  in 1998 GSORTS’ data were maintained in multiple databases and data  were not synchronized. As of September 2002, the data are reported to a  central site, and there is one database of record. Also in 1998, U.S. Army  GSORTS review procedures delayed submission of Army data, and all the  services’ data entry was manual. As of September 2002, Army reporting  procedures require reporting consistent with GSORTS’ requirements, and  all the services have automated data entry, which reduces errors. In 1998,  combat units only reported on readiness for wartime missions. As of  September 2002, combat units report on assigned mission readiness in  addition to wartime mission readiness.
Conversely, DOD has not resolved some issues we raised in 1998. For  example, readiness ratings are still reported in broad bands and actual  percentages of required resources are not externally reported. These  issues remain because the manual specifying readiness reporting rules has  not changed in these areas. The manual’s definition of readiness levels for  personnel has not changed since our 1998 report—it still defines readiness  levels in bands of 10 percentage points or more and does not require  external reporting of actual percentages. For example, the highest  personnel rating can range from 90 percent to 100 percent, and there is no  requirement to report the actual percentage outside of DOD. We have also  reported that GSORTS does not always reflect training and equipment  deficiencies. For example, we reported in April and June 2002 that  readiness data do not reflect the effect of training range restrictions on  unit readiness. We have also reported that GSORTS does not include  whether a unit’s chemical/biological equipment is usable.
In commenting on our analysis, the OUSD P&R office responsible for  readiness reporting stated that it recognized the imprecision of the current  measurements. According to that office, an effort to develop the planned  new readiness reporting system, which is discussed later in this report,  includes working with the DOD components to enhance and expand  readiness reporting.
section_title: Some Information Added to Quarterly Reports Since 1998, but Other Information Eliminated
paragraphs: Since our 1998 report, the quarterly reports improved in some areas, but  degraded in others. Although some information was added, we found that  some of the same quality issues remain—namely, that the reports do not  specifically describe readiness problems, their effects on readiness, or  remedial actions.
DOD has added information to the quarterly reports in response to  legislative direction. For example, DOD added information on the services’  cannibalization rates. Also, DOD added annual reports on infrastructure  and institutional training readiness. However, some information was  eliminated from the quarterly reports. For example, the law requires  results of joint readiness reviews to be reported to Congress. DOD  included these results until the July-September 2001 Quarterly Readiness  Report to the Congress. Since that report, four quarterly reports have been  issued without the joint force assessments. Defense officials responsible  for readiness reporting said that the joint readiness reviews were not  included because the scenarios were based on the former national security  strategy of two major wars. The officials stated they plan to include results  from the joint readiness reviews in future reports.
In commenting on our analysis the OUSD P&R office responsible for  readiness reporting stated that it continues to seek better ways to provide  concise, quality information.
section_title: Quarterly Reports Do Not Adequately Explain Readiness Issues
paragraphs: As we reported in 1998, we found that the quarterly reports still contain  broad statements of readiness issues and remedial actions, which are not  supported by detailed examples and are not related to data in the reports’  annexes. Among other things, the law requires the quarterly reports to  specifically describe each readiness problem and deficiency as well as  planned remedial actions. The reports did not specifically describe the  nature of each readiness problem or discuss the effects of each on unit  readiness. Also, the reports included only broad statements of remedial  actions that lacked details on timelines, objectives, or funding  requirements. For example, one report said that the Air Force continued to  experience shortages in critical job skills that affected the service’s ability  to train. The report did not refer the reader to data in annexes showing  training readiness ratings; it did not state which skills were short, which  training was not accomplished, or whether this shortage had or was  expected to affect units’ readiness ratings. Further, the report did not  explain the remedial actions taken or planned to reverse the skill shortage,  how long it would take to solve this problem, or what funding was  programmed to implement remedial actions. Defense readiness officials  agreed, stating that information in the quarterly reports is summarized to  the point that there are no details on readiness deficiencies, remedial  actions, or funding programmed to implement remedial actions. We  believe the Congress needs this type of information to understand the  significance of the information reported.
Although some of the quarterly report annexes contain voluminous data,  the data are not adequately explained or related to units’ readiness. The  law does not mandate specific explanations of these “readiness  indicators,” but we believe it is essential for Congress to understand the  significance of the information in these reports for use in its oversight role.
For example, DOD is required to report on the maintenance backlog.  Although the report provides the quantity of the backlog, it does not  explain the effect the backlog had on readiness. Specifically, the report did  not explain whether units’ readiness were affected because maintenance  was not accomplished when needed. In addition, DOD is required to report  on training commitments and deployments. The Expanded Quarterly  Readiness Report to Congress Implementation Plan dated February 1998  stated that “either an excessive or a reduced level of commitment could be  an indicator of potential readiness problems.” However, OUSD P&R did  not define what kind of “readiness problems” this data may indicate would  occur as a result of “excessive or reduced” levels of training and  deployments, such as degraded equipment or training. The data reported  are the amount of training away from home station and the amount of  deployments. However, these data are not explained or related to a unit’s  equipment or training ratings. Further, criteria have not been established  to distinguish between acceptable and unacceptable levels of the training  and deployment data reported. As a result, the reader does not know  whether the data reported indicate a problem or the extent of the problem.  In commenting on our analyses, OUSD P&R acknowledged “the  Department would be better served by providing more information as to  how various data relates to readiness.”
Generally, the quarterly reports also do not contain information on funding  programmed to implement specific remedial actions. For example, one  quarterly report included the statement that budgets were revised “to  address readiness and capabilities issues,” but no examples were  provided. Also, the report lacked statements explaining how this “budget  revision” would improve readiness. Although not required by law, we  believe it would prove useful for Congress to understand how DOD  addresses specific readiness problems.
In commenting on our analysis, OUSD P&R officials stated that they would  continue to work with the services to provide more fidelity with the  information presented in the quarterly report annexes. However, they also  said that detailed examples require significant staff effort throughout DOD  and that the added time for more detailed analysis could render the report  a historical document. They further said that complete information would  certainly be desired and agreed it is important for the Congress to  understand the significance of the information in the quarterly reports for  use in its oversight role.
section_title: DOD Has Not Fully Complied with All Legislative Requirements
paragraphs: DOD has complied with most, but not all, of the readiness reporting  requirements added by Congress in the National Defense Authorization  Acts for Fiscal Years 1998 through 2002. Congress added readiness  reporting requirements out of concern over contradictions between  assessment of military unit readiness in official readiness reports and  observations made by military personnel in the field. In a review of these  acts, we identified both recurring readiness reporting requirements that  were added to existing law and one-time reporting requirements related to  military readiness. We compared current readiness reporting to the  requirements in these acts to make an overall judgment on the extent of  compliance. We did not develop a total count of the number of reporting  requirements because the acts included a series of sections and  subsections that could be totaled in various ways. Because DOD is not  reporting on all the requirements added over the past several years, the  Congress is not receiving all the information mandated by law.
subsections: section_title: DOD Is Complying with Most of the New Requirements
paragraphs: Our analysis showed that DOD has complied with most of the  requirements added in the National Defense Authorization Acts for Fiscal  Years 1998-2002. For example, DOD took the following actions in response  to legislative requirements:    DOD is now reporting on the readiness of prepositioned equipment and is  listing individual units that have reported low readiness as required by the  National Defense Authorization Act for Fiscal Year 1998.
DOD is reporting on infrastructure and institutional training readiness as  required by the National Defense Authorization Act for Fiscal Year 1999.
DOD contracted for an independent study of requirements for a  comprehensive readiness reporting system and submitted the study report  to the Congress as required by the National Defense Authorization Act for  Fiscal Year 2000.
DOD has added quarterly information on the military services’  cannibalization rates as required by the National Defense Authorization  Act for Fiscal Year 2001.
section_title: DOD Is Not Reporting on All Items Required
paragraphs: DOD is reporting on some, though not all, of the items Congress required  be added to the quarterly readiness reports. For example, the National  Defense Authorization Act for Fiscal Year 1998 required 19 specific items  be reported that are consistent with our previously cited 1994 report on  readiness reporting. The 1994 report included a list of 26 readiness  indicators that DOD commanders said were important for a more  comprehensive assessment of readiness. A 1994 DOD-funded study by the  Logistics Management Institute found that 19 of the 26 indicators could  help DOD monitor critical aspects of readiness. The 19 items listed in the  National Defense Authorization Act for Fiscal Year 1998 are very similar to  those identified in the 1994 Logistics Management Institute study. DOD is  reporting on 11 of the 19 items and is not reporting on the other 8. The  eight items are (1) historical personnel strength data and trends, (2)  personnel status, (3) borrowed manpower, (4) personnel morale, (5)  operations tempo, (6) training funding, (7) deployed equipment, and (8)  condition of nonpacing equipment as required in the Act. In an  implementation plan setting forth how it planned to comply with reporting  on the 19 items, which was also required by the National Defense  Authorization Act for Fiscal Year 1998, DOD stated that it would not  report on these eight indicators for the following reasons:    Deployed equipment was considered part of the status of prepositioned  equipment indicator.
Historical personnel strength data and trends were available from the  annual Defense Manpower Requirements Report.
Training funding and operations tempo were believed to be represented  adequately in the budget requests as flying hours, steaming days, or  vehicle miles and were not considered good measures of readiness output.
Personnel strength status was considered to be part of the personnel  rating, but DOD agreed to investigate other ways to evaluate the effect of  service personnel working outside the specialty and grade for which they  were qualified.
Borrowed manpower data was only captured in a limited sector of the  deployable force and may not be meaningful until a better method is  developed to capture the data.
Personnel morale had no existing data sources.    The condition of nonpacing equipment had no reasonable measurement to  use as an indicator.
Notwithstanding the reasoning that DOD stated, these eight indicators  continue to be required by law, and we saw no indication in our work that  DOD is working to develop data for them.
Also, DOD is not complying with some of the requirements in the National  Defense Authorization Act for Fiscal Year 1999. Examples are as follows:    The act required DOD to establish and implement a comprehensive  readiness reporting system by April 2000. As of January 2003, DOD had  not implemented a new system, and officials said it is not expected to be  fully capable until 2007 or 7 years later than required.
The act also required DOD to develop implementing regulations for the  new readiness reporting system. DOD had not developed implementing  regulations as of January 2003.
The act required DOD to issue regulations for reporting changes in the  readiness of training or defense infrastructure establishments within 72  hours. Although DOD has provided some guidance, officials stated they  have not issued regulations because no mechanism exists for institutional  training or defense infrastructure establishments to report changes and  because these entities are not part of an established readiness reporting  system.
In commenting on our analyses, DOD officials acknowledged “the  Department is not in full compliance” and stated that they plan to achieve  compliance with the new readiness reporting system under development.  OUSD P&R officials said that the shortfalls in reporting are unwieldy  under the current system; OUSD P&R intends to correct these shortfalls  when the new system is functional. However, as noted above, DOD does  not plan to implement its new system until 2007. As of January 2003, DOD  also had not targeted incremental improvements in readiness reporting  during the period in which the new system is being developed. Until then,  Congress will receive less readiness information than it mandated by law.
section_title: Lack of Implementation Plan Could Hinder Development of New Readiness Reporting System
paragraphs: DOD issued a directive in June 2002 to establish a new readiness reporting  system. The Undersecretary of Defense for Personnel and Readiness is to  oversee the system to ensure the accuracy, completeness, and timeliness  of its information and data, its responsiveness, and its effective and  efficient use of modern practices and technologies. Officials in the OUSD  P&R readiness office responsible for developing the new system said that  they plan to use the new system to comply with the requirements in the  National Defense Authorization Acts and to address many of the  recommendations contained in a congressionally directed independent  study. However, as of January 2003, there are few details of what the new  system would include. Although the new system may have the potential to  improve readiness reporting, as of January 2003, it is only a concept  without detailed plans to guide development and monitor implementation.  As a result, the extent to which the new system will address existing  shortcomings is unknown.
subsections: section_title: DOD Directive Establishes a New Readiness Reporting System
paragraphs: The National Defense Authorization Act for Fiscal Year 1999 required DOD  to establish a comprehensive readiness reporting system. In doing so, the  Congress expressed concern about DOD’s lack of progress in developing a  more comprehensive readiness measurement system reflective of  operational realities. The Congress also noted that past assessments have  suffered from DOD’s inability to create and implement objective and  consistent readiness reporting criteria capable of providing a clear picture  to senior officials and the Congress. Subsequently, the August 2001  Defense Planning Guidance for Fiscal Years 2003-2007 called for the  development of a strategic plan for transforming DOD readiness reporting.
In June 2002, DOD issued a directive establishing the Department of  Defense Readiness Reporting System. The system will measure and  report on the readiness of military forces and the supporting infrastructure  to meet missions and goals assigned by the Secretary of Defense. All DOD  components will align their readiness reporting processes in accordance  with the directive.
The directive assigns oversight and implementing responsibility to the  Undersecretary of Defense for Personnel and Readiness. The  Undersecretary is responsible for developing, fielding, maintaining, and  funding the new system and scenario assessment tools. The  Undersecretary—in collaboration with the Joint Chiefs of Staff, Services,  Defense Agencies, and Combatant Commanders—is to issue implementing  instructions. The Chairman of the Joint Chiefs of Staff, the Service  Secretaries, the commanders of the combatant commands, and the heads  of other DOD components are each assigned responsibilities related to  readiness reporting.
OUSD P&R established a timetable to implement the new readiness  reporting system. OUSD P&R plans to achieve initial capability in 2004 and  reach full capability in 2007. OUSD P&R officials involved in developing  the system said that they have been briefing the concept for the new  reporting system since October 2002. As of January 2003 these officials  stated that they are continuing what they have termed the “concept  demonstration” phase, which began in October 2002. This phase consists  of briefing various offices within DOD, the Joint Staff, and the services to  build consensus and refine the new system’s concept. These officials also  said that the new system will incorporate many, but not all, of the  recommendations contained in a legislatively mandated independent study  of readiness reporting, which concluded that improvements were needed  to meet legislatively mandated readiness reporting requirements and  included numerous recommendations for what a new system should  include. For example, the study recommended that (1) DOD report on all  elements essential to readiness, such as depots, combat support agencies,  and Defense agencies; (2) reporting should be in terms of mission essential  tasks; and (3) DOD should measure the capability to carry out the full  range of National Security Strategy requirements—not just a unit’s  wartime mission.
section_title: An Implementation Plan Would Help Gauge Program Progress
paragraphs: We believe that successfully developing and implementing a large-scale  effort, such as DOD’s new readiness reporting system, requires an  implementation plan that includes measurable performance goals,  identification of resources, performance indicators, and an evaluation  plan. As discussed earlier, full implementation of DOD’s new readiness  reporting system is several years away, and much remains to be done. In  January 2003 the OUSD P&R office responsible for developing the new  system said that the new readiness reporting system is a large endeavor  that requires buy-in from many users and that the development of the  system will be challenging. This office also wrote that it had just been  given approval to develop the new readiness reporting system, was  targeting development of an implementing instruction in the March 2003  time frame, and had not developed an implementation plan to assess  progress in developing and implementing the new reporting system. The  directive establishing the new reporting system requires the  Undersecretary of Defense for Personnel and Readiness, in collaboration  with others, to issue implementing instructions for the new system.
DOD has experienced delays in implementing smaller readiness  improvements than envisioned in the new readiness reporting system. One  such effort involved development of an interface to query the existing  readiness data base (GSORTS). In a July 2002 report, the DOD Inspector  General reported that the planned implementation of this interface slipped  44 months, or just over 3.5 years. Also, history has shown it takes DOD  time to make changes in the readiness reporting system. As illustrated in  figure 2, DOD began reporting on specific readiness indicators 4 years  after it agreed with GAO recommendations to include them in readiness  reporting (see fig. 2).
Other DOD development efforts recognize the need for effective planning  to guide development. For example, DOD is working to transform military  training as directed by the Defense Planning Guidance for Fiscal Years  2003-07. A March 2002 Strategic Plan for Transforming DOD Training  developed by a different office within OUSD P&R discusses a training  transformation road map with major tasks subdivided into near-, mid-, and  long-term actions. The plan includes a list of near-term actions to be  completed by October 2003 and definition of mid- and long-term actions in  a comprehensive implementation plan that will identify specific tasks,  responsibilities, timelines, resources, and methods to assess completion  and measure success. The May 2002 Defense Planning Guidance update  for fiscal years 2004-2009 directs OUSD P&R, working with other DOD  components, to develop a comprehensive program to implement the  strategic training transformation plan and provide it to the Deputy  Secretary of Defense by April 1, 2003.
Since the directive for creating a new readiness reporting system  established broad policy with no specifics and since DOD has not  developed an implementation plan, the extent to which the new system  will address the current system’s shortcomings will remain unknown until  the new system is fully capable in 2007. Until then, readiness reporting will  continue to be based on the existing system.
Commenting on its plans for the new system, OUSD P&R said that it is in  the process of creating an Advanced Concept Technology Demonstration  (ACTD) structure for the new system and will produce all necessary  planning documents required within the established ACTD process.  However, this process is intended to provide decision makers an  opportunity to understand the potential of a new concept before an  acquisition decision. We do not believe the ACTD process will necessarily  result in an implementation plan to effectively monitor development and  assess whether the new system is being implemented on schedule and  achieving desired results.
DOD’s ACTD guidelines state the principal management tool for ACTDs is  a management plan, which provides a top-level description of the  objectives, critical events, schedule, funding, and measures of evaluation  for the project. We reported in December 2002 that these guidelines  contain advice and suggestions as opposed to formal directives and  regulations. DOD’s guidelines state that the ACTD should plan exercises  or demonstrations to provide an adequate basis for utility assessment. We  also reported in December 2002 that DOD lacks specific criteria to  evaluate demonstration results, which may cause acquisition decisions to  be based on too little knowledge. Therefore, we still believe an  implementation plan is necessary since the ACTD process does not require  a detailed implementation plan and does not always include specific  criteria to evaluate effectiveness.
section_title: Conclusions
paragraphs: While DOD has made some improvements in readiness reporting since  1998, some of the same issues remain unresolved today. Although DOD is  providing Congress more data than in 1998, the voluminous data are  neither evaluated nor explained. The quarterly reports do not link the  effects of “readiness issues” or deficiencies to changes in readiness at the  unit level. Also, as in 1998, the reports contain vague descriptions of  remedial actions not linked to specific deficiencies. Finally, the quarterly  reports do not discuss funding that is programmed to implement specific  remedial actions. As a result, the information available to Congress is not  as effective as it could be as an oversight tool.
Even though DOD directed development of a new readiness reporting  system, it has not yet developed an implementation plan identifying  objective and measurable performance goals, the resources and personnel  needed to achieve the goals, performance indicators, and an evaluation  plan to compare program results with goals, and milestones to guide  overall development of the new readiness system. Even though the new  system may have the potential to improve readiness reporting, without an  implementation plan little assurance exists that the new system will  actually improve readiness assessments by the time full capability is  planned in 2007. Without such a plan, it will also remain difficult to gauge  progress toward meeting the 2007 target date. This concern is reinforced  in light of the (1) years-long delays in implementing other readiness  reporting improvements and (2) the deficiencies in existing reporting that  OUSD P&R plans to rectify with the new system. Furthermore, without an  implementation plan neither senior DOD leadership nor the Congress will  be able to determine if the resources spent on this system are achieving  their desired results.
section_title: Recommendations for Executive Action
paragraphs: To improve the information available to Congress for its use in its  oversight role, we recommend that the Secretary of Defense direct the  OUSD P&R to improve the quality of information contained in the  quarterly reports. Specifically, we recommend that DOD’s reports explain  (in the unclassified section) the most critical readiness issues that are of  greatest concern to the department and the services. For each issue, we  recommend that DOD’s reports include    an analysis of the readiness deficiencies, including a clear explanation of  how the issue affects units’ readiness;    a statement of the specific remedial actions planned or implemented; and    clear statements of the funding programmed to implement each remedial  action.
To be able to assess progress in developing the new readiness system, we  recommend that the Secretary of Defense direct the OUSD P&R to develop  an implementation plan that identifies    performance goals that are objective, quantifiable, and measurable;  the cost and personnel resources needed to achieve the goals, including an  identification of the new system’s development and implementation costs  in the President’s Budget beginning in fiscal year 2005 and Future Years  Defense Plan;    performance indicators to measure outcomes;    an evaluation plan to compare program results with established goals; and    milestones to guide development to the planned 2007 full capability date.
To assist Congress in its oversight role, we recommend that the Secretary  of Defense give annual updates to the Congress on the new readiness  reporting system’s development to include    performance measures,    progress toward milestones,    comparison of progress with established goals, and  remedial actions, if needed, to maintain the implementation schedule.
section_title: Agency Comments and our Response
paragraphs: In written comments on a draft of this report, which are reprinted in  appendix II, the Department of Defense did not agree with our  recommendations.
In response to our recommendation that DOD improve the quality of  information contained in its quarterly readiness reports, DOD said that the  Quarterly Readiness Report to the Congress is one of the most  comprehensive and detailed reports submitted to the Congress that  discusses serious readiness issues and ways in which these issues are  being addressed. DOD further stated that the department presents  briefings on specific readiness issues to the Congress and that spending  more time and resources expanding the existing written report would be  counterproductive.
We recognize that the Quarterly Readiness Reports to the Congress  contain voluminous data. However, as discussed in this report, we found  that the quarterly reports’ annexes are large and mostly consist of charts  or other data that are not adequately explained and are not related to  units’ readiness. In some cases, criteria have not been established to  enable the reader to distinguish between acceptable and unacceptable  levels of the data reported. As a result, the reader cannot assess the  significance of the data because it is not at all clear whether the data  reported indicate a problem or the extent of the problem. Considering that  the quarterly reports contain inadequately explained data and that much of  the information is not “user friendly,” we continue to believe the quality of  information in the quarterly reports can be improved. In fact, we reviewed  all the quarterly reports provided to Congress since 1998 and found that  through the January-June 2001 report the reports did include an  unclassified summary of readiness issues for each service addressing four  topics—personnel, equipment, training, and enablers (critical units or  capabilities, such as specialized aircraft, essential to support operations).  However, the reports did not include supporting data or a discussion of  remedial actions. Since that time, these summaries have been eliminated  from the quarterly reports. For example, the unclassified narrative of the  last two reports available at the time we performed our work—January- March 2002 and April-June 2002—were less than two pages long and  neither discussed readiness issues nor ways in which these issues are  being addressed. One report discussed the new readiness reporting  system, and the other discussed a review of seven environmental laws.  Given that DOD has highlighted key issues in the past, we believe that  improving the quarterly reports would be beneficial if DOD were to focus  on the most critical readiness issues that are of greatest concern to the  services and includes supporting data and a discussion of remedial  actions. Therefore, we have modified our recommendation that DOD  improve the quality of readiness reporting to focus on readiness issues  deemed to be critical by the Secretary and the military services and to  provide more detailed data and analyses of those issues and the remedial  actions planned for each one.
DOD did not agree with our recommendations that it (1) develop an  implementation plan with, among other things, performance goals that are  objective, quantifiable, and measurable and (2) provide annual updates to  the Congress on the new readiness reporting system’s development. DOD  said that it had undertaken an initiative to develop better tools for  assessing readiness and that it intended to apprise Congress on its efforts  to develop tools for readiness assessment. DOD further stated that the  effort to improve readiness reporting is in its infancy, but that it has  established milestones, cost estimates, functional responsibilities, and  expected outcomes. DOD believes that further planning and a prescriptive  annual update to the Congress is unnecessary.
We agree that the new readiness reporting system may have the potential  to improve readiness reporting. However, as discussed in this report, the  directive establishing the new system contains very broad, high-level  statements of overall functional responsibilities and outcomes, but no  details on how these will be accomplished. Further, DOD has established  two milestones—initial capability in 2004 and full capability in 2007. DOD  does not have a road map explaining the steps needed to achieve full  capability by 2007, which is seven years after Congress mandated a new  system be in place. In addition, as discussed earlier in this report, DOD has  experienced delays in implementing much smaller readiness  improvements. While DOD has undertaken an initiative to develop better  tools for assessing readiness and intends to routinely and fully apprise the  Congress on its development efforts, tools are the mechanics for  evaluating readiness data. As such, tools are not the same thing as the  comprehensive readiness reporting system mandated by Congress that  DOD has said will include new metrics and will evaluate entities within  DOD that currently do not report readiness. Considering that Congress  expressed concern about DOD’s lack of progress in developing a  comprehensive system, that developing and implementing DOD’s planned  new system is scheduled to take 4 more years, and that delays have been  experienced in earlier efforts to make small improvements in readiness  reporting, we continue to believe that it is important for DOD to develop  an implementation plan to gauge progress in developing and implementing  the new readiness reporting system and to provide annual updates to the  Congress. Such a plan would be consistent with DOD’s approach to other  major initiatives such as transforming training. We have therefore retained  these two recommendations.
DOD also provided technical corrections and we have modified the report  where appropriate.
We are sending copies of this report to the Ranking Minority Member,  Subcommittee on Readiness, House Committee on Armed Services; the  Chairman and Ranking Minority Member, Subcommittee on Readiness and  Management Support, Senate Committee on Armed Services; other  interested congressional committees; Secretary of Defense; and the  Director, Office of Management and Budget. We will also make copies  available to others on request. In addition, the report will be available at  no charge on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions, please call me on (757) 552-8111 or  by E-mail at curtinn@gao.gov. Major contributors to this report were  Steven Sternlieb, Brenda Waterfield, James Lewis, Dawn Godfrey, and  Herbert Dunn.
section_title: Appendix I: Scope and Methodology
paragraphs: To assess the progress the Department of Defense (DOD) has made in  resolving issues raised in our prior report concerning both the unit level  readiness reporting system and the lack of specificity in DOD’s Quarterly  Readiness Reports to the Congress, we met with DOD officials and  reviewed regulations and quarterly reports. Specifically, we met with  officials of the Office of the Undersecretary of Defense for Personnel and  Readiness (OUSD P&R) responsible for readiness reporting, the Joint  Staff, and the military services to discuss their individual progress in each  of these areas. To assess progress regarding unit level readiness reporting,  we reviewed the Chairman of the Joint Chiefs of Staff manual governing  this system and the related service implementing instructions to determine  if these documents had changed since our 1998 report or if the manual and  service instructions continued to allow reporting in the same manner as  reflected in our earlier report. Through a comparison of the current and  prior documents, discussions with pertinent officials, and our analysis, we  determined whether the readiness reporting issues we raised in 1998 had  been resolved. We also reviewed the content of quarterly reports to assess  their quality and usefulness, and assess whether the problems we reported  in 1998 had been rectified. We discussed our analysis with OUSD P&R  officials and provided them with our analyses in order that they could fully  consider and comment on our methodology and conclusions. We did not  assess the accuracy of reported readiness data.
To determine the extent to which DOD has complied with legislative  reporting requirements enacted since our prior report, we compared a  complete listing of these requirements to DOD’s readiness reporting. First,  we identified the legislatively mandated readiness reporting requirements  enacted since our 1998 report. To accomplish this, we reviewed the  National Defense Authorization Acts for Fiscal Years 1998-2002 to list the  one-time and recurring reporting requirements related to military  readiness. We also requested congressional staff and OUSD P&R to review  the list, and officials from both offices agreed it was accurate. We did not  develop a total count of the number of reporting requirements because the  acts included a series of sections and subsections that could be totaled in  various ways. Once we obtained concurrence that this listing was  complete and accurate, we compared this list to current readiness  reporting to make an overall judgment on the extent of compliance.
To assess how DOD plans to improve readiness reporting, we reviewed the  June 2002 DOD directive establishing a new readiness reporting system  and a progress update briefing on the new system. We also obtained  readiness briefings from each of the services, OUSD P&R, and Joint Staff  officials. We performed several electronic searches of the Deputy Under  Secretary of Defense (Readiness) electronic Web site to determine the  status of readiness reporting. To assess how smoothly other readiness  improvements progressed, we reviewed DOD audit reports. We discussed  our findings with OUSD P&R officials and worked proactively with them  in conducting our analyses. Specifically, we provided them drafts of our  analyses for their comments and corrections.
We conducted our review from June 2002 through January 2003 in  accordance with generally accepted government auditing standards.
section_title: Appendix II: Comments from the Department of Defense